Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a prediction characterizing an environment. In one aspect, a method includes obtaining a respective observation characterizing a state of an environment for each time step in a sequence of multiple time steps, comprising, for each time step after a first time step in the sequence of time steps: processing a network input that comprises observations obtained for one or more preceding time steps to generate a plurality of acquisition decisions; obtaining an observation for the time step, wherein the observation includes data corresponding to modalities that are selected for acquisition at the time step, does not include data corresponding to modalities that are not selected for acquisition at the time step; and processing a model input that includes the observation for each time step in the sequence of time steps to generate the prediction.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a pathogenicity score characterizing a likelihood that a mutation to a protein is a pathogenic mutation, wherein the mutation modifies an amino acid sequence of the protein by replacing an original amino acid by a substitute amino acid at a mutation position in the amino acid sequence of the protein. In one aspect, a method comprises: generating a network input to a pathogenicity prediction neural network, wherein the network input comprises a multiple sequence alignment (MSA) representation that represents an MSA for the protein; processing the network input using the pathogenicity prediction neural network to generate a score distribution over a set of amino acids; and generating the pathogenicity score using the score distribution over the set of amino acids.
A method performed by one or more computers for obtaining an optimized algorithm that (i) is functionally equivalent to a target algorithm and (ii) optimizes one or more target properties when executed on a target set of one or more hardware devices. The method includes: initializing a target tensor representing the target algorithm; generating, using a neural network having a plurality of network parameters, a tensor decomposition of the target tensor that parametrizes a candidate algorithm; generating target property values for each of the target properties when executing the candidate algorithm on the target set of hardware devices; determining a benchmarking score for the tensor decomposition based on the target property values of the candidate algorithm; generating a training example from the tensor decomposition and the benchmarking score; and storing, in a training data store, the training example for use in updating the network parameters of the neural network.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling agents using reporter neural networks.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for using simulation-based inference to inferring a set of parameters such as measurements, from observations, e.g. real world observations. The method uses a score generation neural network to determine scores for individual observations or for groups of observations that are combined and used to iteratively adjust values of the parameters.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for simulating a state of an environment over a sequence of time steps. In one aspect, a method comprises, at each of one or more time steps: obtaining an environment mesh representing the state of the environment at the time step; generating a graph representing the state of the environment at the time step, comprising: determining that a first face of a first object mesh is within a collision distance of a second face of a second object mesh; and in response, instantiating a face-face edge in the graph that connects: (i) a first set of graph nodes in the graph that represent the first face in the first object mesh, and (ii) a second set of graph nodes in the graph that represent the second face in the second object mesh.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling an agent that is interacting with an environment. Implementations of the system use previously learned skills to explore states of the environment to collect and store training data, which is then used to train an action selection system. The system includes a set of skill action selection subsystems, each configured to select actions for the agent to perform for a respective skill. The set of skill action selection subsystems is used to explore states of the environment to collect the training data, keeping their individual action selection policies unchanged. A scheduler neural network selects the skill neural networks to use. The action selection system is trained on the stored training data.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network used to select actions to be performed by an agent interacting with an environment. Implementations of the described techniques can learn to explore the environment efficiently by storing and updating state embedding cluster centers based on observations characterizing states of the environment.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling agents. In particular, an agent can be controlled using an action selection neural network that performs in-context reinforcement learning when controlling an agent on a new task.
G06N 3/006 - Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
G06N 3/0442 - Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating an output sequence of discrete tokens using a diffusion model. In one aspect, a method includes generating, by using the diffusion model, a final latent representation of the sequence of discrete tokens that includes a determined value for each of a plurality of latent variables; applying a de-embedding matrix to the final latent representation of the output sequence of discrete tokens to generate a de-embedded final latent representation that includes, for each of the plurality of latent variables, a respective numeric score for each discrete token in a vocabulary of multiple discrete tokens; selecting, for each of the plurality of latent variables, a discrete token from among the multiple discrete tokens in the vocabulary that has a highest numeric score; and generating the output sequence of discrete tokens that includes the selected discrete tokens.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for A training a language model for performing a reasoning task. The system obtains a plurality of training examples. Each training example includes a respective sample query text sequence characterizing a respective sample query and a respective reference response text sequence that includes a reference final answer to the respective sample query. The system trains a reward model on the plurality of training examples. The reward model is configured to receive an input including a query text sequence characterizing a query and one or more reasoning steps that have been generated in response to the query and process the input to compute a reward score indicating how successful the one or more reasoning steps are in yielding a correct final answer to the query. The system trains the language model using the trained reward model.
A reinforcement learning system is proposed in which a policy model neural network is trained to control an agent to perform a task in successive time steps, by training a control system including the policy model neural network to select a respective action for each time step which gives a high value for a reward function based on the action, and which indicates the contribution of the action to solving the task. The reward function includes a term based on a progress value output by a progress model. The progress model generates the progress value upon receiving a first observation of the state of the environment at a time step before the performance of the action, and a second observation of the state of the environment at a time step following the performance of the action. The progress value is an estimate of the average time which an ensemble of experts who produced the demonstrations would have taken to transform the environment from how it appears in the first observation to how it appears in the second observation.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for processing a network input using a neural network that includes one or more regularized attention layers. In one aspect, a method comprises: receiving a layer input to a regularized attention layer, wherein the layer input to the regularized attention layer comprises a set of input embeddings; and applying a regularized attention operation over the set of input embeddings to generate a set of output embeddings, comprising: transforming intermediate attention scores using a set of shaping constants to generate a set of transformed attention scores, wherein: values of the shaping constants are initialized prior to training of the neural network and are not adjusted during the training of the neural network; and the values of the shaping constants are selected to regularize the set of output embeddings.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for enabling a user to conduct a dialogue. Implementations of the system learn when to rely on supporting evidence, obtained from an external search system via a search system interface, and are also able to generate replies for the user that align with the preferences of a previously trained response selection neural network. Implementations of the system can also use a previously trained rule violation detection neural network to generate replies that take account of previously learnt rules.
G06N 3/006 - Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling a facility through hierarchical reinforcement learning. In particular, the facility is controlled using a high-level controller neural network that makes high-level decisions and a low-level controller neural network that makes low-level controller decisions.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
16.
DATA-EFFICIENT REINFORCEMENT LEARNING WITH ADAPTIVE RETURN COMPUTATION SCHEMES
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for data-efficient reinforcement learning with adaptive return computation schemes.
G06N 3/006 - Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a policy neural network for use in controlling a robot. In particular, the policy neural network can be trained in simulation using images generated by a scene synthesis machine learning model.
G06N 3/0895 - Weakly supervised learning, e.g. semi-supervised or self-supervised learning
G06N 3/006 - Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling a robot manipulator that has a plurality of joints. One of the methods includes obtaining a control input that comprises one or more velocity values that specify a target velocity of a reference point in a given coordinate frame; determining a respective joint velocity for each of the plurality of joints by generating a solution to an optimization problem formulated from the control input; and controlling the robot manipulator, including causing the plurality of joints of the robot manipulator to move in accordance with the respective joint velocities to approximate the control input.
G05B 19/427 - Teaching successive positions by tracking the position of a joystick or handle to control the positioning servo of the tool head, master-slave control
19.
CONTROLLING AGENTS USING AMBIGUITY-SENSITIVE NEURAL NETWORKS AND RISK-SENSITIVE NEURAL NETWORKS
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling agents. In particular, an agent can be controlled using an action selection system that is risk-sensitive, ambiguity-sensitive, or both.
G06N 3/006 - Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
G06N 7/01 - Probabilistic graphical models, e.g. probabilistic networks
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a response to a query input using a selection-inference neural network.
Methods, systems, and computer readable storage media for performing operations comprising: obtaining a plurality of initial network inputs that have been classified as belonging to a corresponding ground truth class; processing each of the plurality of initial network inputs using a trained target neural network to generate a respective predicted network output for each initial network input, the respective predicted network output comprising a respective score for each of a plurality of classes, the plurality of classes comprising the ground truth class; identifying, based on the respective predicted network outputs and the ground truth class, a subset of the initial network inputs as having been misclassified by the trained target neural network; and determining, based on the subset of initial network inputs, one or more failure case latent representations, wherein each failure case latent representation is a latent representation that characterizes network inputs that belong to the ground truth class but that are likely to be misclassified by the trained target neural network.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for identifying agents in a system. According to one aspect, a method comprises: generating data defining a causal model of the system, comprising transmitting instructions to cause a plurality of interventions to be applied to the system, wherein each intervention modifies one or more variable elements in the system; processing the model of the system to identify one or more of the variable elements in the system as being decision elements, wherein each decision element represents an action selected by a respective agent in the system; and identifying one or more agents in the system based on the decision elements; and outputting data that identifies the agents in the system.
A computer-implemented method for determining, for a loss function which is a function of a parameter vector comprising a plurality of parameters, values for the parameters for which the parameter vector is a stationary point of the loss function. The method comprises determining initial values for the parameters; and repeatedly updating the parameters by: (a) determining at least one drift value indicative of discretization drift for a discrete update to the parameters based on the loss function; (b) determining at least one learning rate value by evaluating a learning rate function based on, and having an inverse relationship with, the at least one drift value; (c) determining respective updates to the parameters based upon a product of the at least one learning rate value and a gradient of the loss function with respect to the respective parameter for current values of the parameters; and (d) updating the parameters based upon the determined respective updates.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for optimizing a target algorithm using a state representation neural network.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a policy neural network that is used to control an agent. In particular, the policy neural network can be trained through model-free reinforcement learning with regularized Nash dynamics.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for simulating industrial facilities for control. One of the methods includes. at each of a plurality of time steps during a task episode: receiving, from a computer simulator of an industrial facility, measurements representing a current state of the facility; generating, from the measurements, an observation; providing the observation as input to a control policy for controlling the facility; receiving, as output, an action for controlling one or more setpoints of the facility; generating, from the action, one or more control inputs for the one or more setpoints of the facility; and providing, as input to the simulator, (i) the control inputs and (ii) current values for one or more configuration parameters of the simulator to cause the simulator to generate, as output, new measurements representing a new state of the facility.
G05B 13/02 - Adaptive control systems, i.e. systems automatically adjusting themselves to have a performance which is optimum according to some preassigned criterion electric
G05B 19/418 - Total factory control, i.e. centrally controlling a plurality of machines, e.g. direct or distributed numerical control (DNC), flexible manufacturing systems (FMS), integrated manufacturing systems (IMS), computer integrated manufacturing (CIM)
27.
Simulating Physical Environments with Discontinuous Dynamics Using Graph Neural Networks
This specification describes a simulation system that performs simulations of physical environments using a graph neural network. At each of one or more time steps in a sequence of time steps in a given time interval, the system can process a representation of a current state of the physical environment at the current time step using the graph neural network to generate a prediction of a next state of the physical environment at the next time step. Generally, the environment has discontinuous dynamics at one or more time points during the time interval.
G06F 30/20 - Design optimisation, verification or simulation
G06F 30/27 - Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
G06F 119/12 - Timing analysis or timing optimisation
28.
TRAINING CAMERA POLICY NEURAL NETWORKS THROUGH SELF PREDICTION
G06N 3/006 - Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
The invention describes a system and a method for controlling an agent interacting with an environment to perform a task, the method comprising, at each of a plurality of first time steps from a plurality of time steps: receiving an observation characterizing a state of the environment at the first time step; determining a goal representation for the first time step that characterizes a goal state of the environment to be reached by the agent; processing the observation and the goal representation using a low-level controller neural network to generate a low-level policy output that defines an action to be performed by the agent in response to the observation, wherein the low-level controller neural network comprises: a representation neural network configured to process the observation to generate an internal state representation of the observation, and a low-level policy head configured to process the state observation representation and the goal representation to generate the low-level policy output; and controlling the agent using the low-level policy output.
G06N 3/006 - Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
The invention describes the method performed by one or more computers and for training a base policy neural network that is configured to receive a base policy input comprising an observation of a state of an environment and to process the policy input to generate a base policy output that defines an action to be performed by an agent in response to the observation, the method comprising: generating training data for training the base policy neural network by controlling an agent using (i) the base policy neural network and (ii) an exploration strategy that maps, in accordance with a set of one or more parameters, base policy outputs generated by the base policy neural network to actions performed by the agent to interact with an environment, the generating comprising, at each of a plurality of time points: determining that criteria for updating the exploration strategy are satisfied at the time point; and in response to determining that the criteria are satisfied: generating a meta policy input that comprises data characterizing a performance of the base policy neural network in controlling the agent at the time point; processing the meta policy input using a meta policy to generate a meta policy output that specifies respective values for each of the set of one or more parameters that define the exploration strategy; and controlling the agent using the base policy neural network and in accordance with the exploration strategy defined by the respective values for the set of one or more parameters specified by the meta policy output.
G06N 3/008 - Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
47 ABSTRACT Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for simulating a state of a physical environment. In one aspect, a method performed by one or more computers for simulating the state of the physical environment is provided. The method includes, for each of multiple time steps: obtaining data defining a fine-resolution mesh and a coarse-resolution mesh that each characterize the state of the physical environment at the current time step, where the fine-resolution mesh has a higher resolution than the coarse-resolution mesh; processing data defining the fine- resolution mesh and the coarse-resolution mesh using a graph neural network that includes: (i) one or more fine-resolution update blocks, (ii) one or more coarse-resolution update blocks, and (iii) one or more up-sampling update blocks; and determining the state of the physical environment at a next time step using updated node embeddings for nodes in the fine-resolution mesh. DeepMind Technologies Limited F&R Ref.: 45288-0255WO1 PCT Application
G06F 30/23 - Design optimisation, verification or simulation using finite element methods [FEM] or finite difference methods [FDM]
G06F 30/27 - Design optimisation, verification or simulation using machine learning, e.g. artificial intelligence, neural networks, support vector machines [SVM] or training a model
An iterative method is proposed to train an action selection system of a reinforcement learning system, based on a reward function which defines a reward value for each action. The reward value includes an intrinsic reward term generated based on the outputs of two encoder models: an online encoder model and a target encoder model. The online encoder model is iteratively trained based on a loss function, and the target encoder model is updated to bring it closer to the online encoder model.
Systems, methods, and computer programs, for training and using a machine learning system to control an agent to perform a task. The machine learning system is trained using counterfactual internal states so that it can provide an output that explains the behavior of the system in causal terms, e.g. in terms of aspects of its environment that cause the system to select particular actions for the agent.
G06N 3/008 - Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
G06N 3/044 - Recurrent networks, e.g. Hopfield networks
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling a reinforcement learning agent in an environment to perform a task. In one aspect, a method comprises: maintaining a retrieval dataset that stores a plurality of history observations and, for each history observation, a respective associated context; receiving a current observation characterizing a current state of the environment; selecting one or more history observations from the plurality of history observations; processing, using an encoder neural network and in accordance with current values of encoder network parameters, an encoder network input comprising (i) the current observation and (ii) the one or more selected history observations and their respective associated context to generate a latent state representation for the current state of the environment; and using the latent state representation to determine an action to be performed by the agent in response to the current observation.
G06F 16/908 - Retrieval characterised by using metadata, e.g. metadata not derived from the content or metadata generated manually using metadata automatically derived from the content
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network to perform a machine learning task on one or more received inputs by using a hybrid training dataset with a semi-supervised learning technique. The hybrid training dataset includes multiple unlabeled training inputs and multiple labeled training inputs and, in some cases, more unlabeled training inputs than labeled training inputs.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for determining generalized eigenvectors that characterize a data set.
Systems, methods, and computer programs for learning to control an embodied agent to perform tasks. The techniques use internal, "intra-agent" speech when learning, and are thus able to perform tasks involving new objects without any direct experience of interacting with those objects, i.e. zero-shot. Implementations of the techniques use an image captioning neural network system to generate natural language captions used when training an action selection neural network system.
G06N 3/008 - Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
Systems and methods for encoding video, and for decoding video at an arbitrary temporal and/or spatial resolution. The techniques use a scene representation neural network that, in implementations, is configured to represent frames of a 2D or 3D video as a 3D model encoded in the parameters of the neural network.
G06N 3/04 - Architecture, e.g. interconnection topology
H04N 19/31 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the temporal domain
H04N 19/33 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using hierarchical techniques, e.g. scalability in the spatial domain
40.
NEGOTIATING CONTRACTS FOR AGENT COOPERATION IN MULTI-AGENT SYSTEMS
Methods, systems and apparatus, including computer programs encoded on computer storage media, for enabling agents to cooperate with one another in a way that improves their collective efficiency. The agents can modify their behavior by taking into account the behavior of other agents, so that a better overall result can be achieved than if each agent acted independently. This is done by enabling the agents to negotiate contracts with one another that restrict their respective actions.
G06Q 10/04 - Forecasting or optimisation specially adapted for administrative or management purposes, e.g. linear programming or "cutting stock problem"
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a response to a query input using a selection- inference neural network.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for compressing and decompressing data signals using sparse, meta-learned neural networks.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for privacy-sensitive training of a neural network. In one aspect, a method includes training a set of neural network parameters of the neural network on a set of training data over multiple training iterations to optimize an objective function. Each training iteration includes: sampling a batch of network inputs from the set of training data; determining a clipped gradient for each network input in the batch of network inputs; and updating the neural network parameters using the clipped gradients for the network inputs in the batch of network inputs.
A query processing system is described which receives a query input comprising an input token string and also at least one data item having a second, different modality, and generates a corresponding output token string.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using an epistemic machine learning model that improves the quality of outputs generated by a base machine learning model.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a machine learning model to perform a machine learning task. In one aspect, a method performed by one or more computer is described. The method includes: obtaining data defining a compute budget that characterizes an amount of computing resources allocated for training a machine learning model to perform a machine learning task; processing the data defining the compute budget using an allocation mapping, in accordance with a set of allocation mapping parameters, to generate an allocation tuple defining: (i) a target model size for the machine learning model, and (ii) a target amount of training data for training the machine learning model; instantiating the machine learning model, where the machine learning model has the target model size; and obtaining the target amount of training data for training the machine learning model.
Systems and methods for processing an image from a mobile device so that it appears to have been captured by a camera with particular characteristics, for example a digital SLR camera with particular settings. The system uses a trained image enhancement neural network. The image enhancement neural network can be trained without needing to rely on pairs of images of the same scene; some training methods are described.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling agents. In particular, an agent can be controlled using a hierarchical controller that includes a task policy neural network and a low-level controller neural network.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output sequences using language model neural networks. In particular, the output sequences include a response to an input query and inline evidence that includes a quote from a context document that supports the response.
A neural network system that is configured to learn a representation of data item, such as an image, audio, or text data item, through a self-supervised learning process. Implementations of the system couple two learning processes, an object discovery learning process and an object feature representation learning process. In implementations the object discovery learning process assists the object feature representation learning process in self-supervised learning of object feature representations, and the object feature representation learning process is used to improve the object discovery learning process.
A method is proposed to train an adaptive system to perform a video processing task, based on a database of compressed representations of video data items. The compressed representations were generated by a trained adaptive compressor unit.
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
H04N 19/172 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being an image region, e.g. an object the region being a picture, frame or field
H04N 19/177 - Methods or arrangements for coding, decoding, compressing or decompressing digital video signals using adaptive coding characterised by the coding unit, i.e. the structural portion or semantic portion of the video signal being the object or the subject of the adaptive coding the unit being a group of pictures [GOP]
52.
TRACKING QUERY POINTS IN VIDEOS USING NEURAL NETWORKS
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for tracking query points in videos using a point tracking neural network.
G06V 10/44 - Local feature extraction by analysis of parts of the pattern, e.g. by detecting edges, contours, loops, corners, strokes or intersections; Connectivity analysis, e.g. of connected components
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing network inputs using a neural network that implements partitioned attention.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling agents. In particular, an agent can be controlled using a policy neural network that has been trained to allow the agent to achieve cultural transmission after training.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating computer code using neural networks. One of the methods includes receiving description data describing a computer programming task; receiving a first set of inputs for the computer programming task; generating a plurality of candidate computer programs by sampling a plurality of output sequences from a set of one or more generative neural networks; for each candidate computer program in a subset of the candidate computer programs and for each input in the first set: executing the candidate computer program on the input to generate an output; and selecting, from the candidate computer programs, one or more computer programs as synthesized computer programs for performing the computer programming task based at least in part on the outputs generated by executing the candidate computer programs in the subset on the inputs in the first set of inputs.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a sequence of data elements that includes a respective data element at each position in a sequence of positions. In one aspect, a method includes: for each position after a first position in the sequence of positions: obtaining a current sequence of data element embeddings that includes a respective data element embedding of each data element at a position that precedes the current position, obtaining a sequence of latent embeddings, and processing: (i) the current sequence of data element embeddings, and (ii) the sequence of latent embeddings, using a neural network to generate the data element at the current position. The neural network includes a sequence of neural network blocks including: (i) a cross-attention block, (ii) one or more self-attention blocks, and (iii) an output block.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for using a neural network to generate a network output that characterizes an entity. In one aspect, a method includes: obtaining a representation of the entity as a set of data element embeddings, obtaining a set of latent embeddings, and processing: (i) the set of data element embeddings, and (ii) the set of latent embeddings, using the neural network to generate the network output. The neural network includes a sequence of neural network blocks including: (i) one or more local cross-attention blocks, and (ii) an output block. Each local cross-attention block partitions the set of latent embeddings and the set of data element embeddings into proper subsets, and updates each proper subset of the set of latent embeddings using attention over only the corresponding proper subset of the set of data element embeddings.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling a reinforcement learning agent in an environment. One of the methods may include maintaining data specifying a base policy set comprising a plurality of base policies for controlling the agent; receiving a current observation characterizing a current state of the environment; generating, for each of the plurality of base policies, one or more predicted future observations characterizing respective future states of the environment that are subsequent to the current state of the environment; using the predicted future observations generated for the plurality of base policies to determine a respective estimated value for each composite policy in a composite policy set with respect to the current state of the environment; and selecting an action using the respective estimated values for the composite policies.
A computer-implemented method for controlling a particular computer to execute a task is described. The method includes receiving a control input comprising a visual input, the visual input including one or more screen frames of a computer display that represent at least a current state of the particular computer; processing the control input using a neural network to generate one or more control outputs that are used to control the particular computer to execute the task, in which the one or more control outputs include an action type output that specifies at least one of a pointing device action or a keyboard action to be performed to control the particular computer; determining one or more actions from the one or more control outputs; and executing the one or more actions to control the particular computer.
G06F 3/01 - Input arrangements or combined input and output arrangements for interaction between user and computer
G06F 3/038 - Control and interface arrangements therefor, e.g. drivers or device-embedded control circuitry
G06N 3/0442 - Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for augmenting machine learning language models using search engine results. One of the methods includes obtaining question data representing a question; generating, from the question data, a search engine query for a search engine; obtaining a plurality of documents identified by the search engine in response to processing the search engine query; generating, from the plurality of documents, a plurality of conditioning inputs each representing at least a portion of one or more of the obtained documents; for each of a plurality of the generated conditioning inputs, processing a network input generated from (i) the question data and (ii) the conditioning input using a neural network to generate a network output representing a candidate answer to the question; and generating, from the network outputs representing respective candidate answers, answer data representing a final answer to the question.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for optimizing a set of design parameters. In one aspect, a method includes: obtaining a respective initial value for each design parameter, and iteratively optimizing current values of the design parameters over a sequence of optimization iterations. The method further includes, each optimization iteration: generating a representation of an initial state of an environment using the current values of the design parameters, processing an input including the representation of the initial state of the environment using a simulation neural network to generate an output that defines a simulation of the state of the environment over a sequence of one or more time steps, determining a reward, determining gradients of the reward with respect to the current values of the design parameters, and updating the current values of the design parameters using the gradients.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a data item using a diffusion neural network. In particular, the data item is generated by guiding a reverse diffusion process using a time-independent guidance neural network.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a final output sequence. In one aspect, a method comprises: receiving a current output sequence comprising one or more current output segments; receiving a set of reference segments and a respective reference segment embedding of each reference segment that has been generated using an embedding neural network; for each current output segment: processing the current output segment using the embedding neural network to generate a current output segment embedding of the current output segment; and selecting k most similar reference segments to the current output segment using the reference segment embeddings and the current output segment embedding; and processing the current output sequence and the k most similar reference segments for each current output segment to generate an additional output segment that follows the current output sequence in the final output sequence.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network having one or more conditional computation layers, where each conditional computation layer includes a gating sub-layer having multiple gating parameters and an expert sub-layer having multiple expert neural networks. In one aspect, a method comprises: sampling a batch of target output sequences that comprises a respective ground truth output token at each of multiple output positions; for each target output sequence, processing the target output sequence using the neural network to generate a network output that includes respective score distributions over the vocabulary of output tokens for the output positions in the target output sequence; and training each gating sub-layer using respective rewards for the gating sub-layer for the output positions through reinforcement learning to optimize a reinforcement learning objective function that measures an expected reward received by the gating sub-layer.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling agents. In particular, an interactive agent can be controlled based on multi-modal inputs that include both an observation image and a natural language text sequence.
G06N 3/0442 - Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
G06N 3/084 - Backpropagation, e.g. using gradient descent
G06N 3/0895 - Weakly supervised learning, e.g. semi-supervised or self-supervised learning
G06N 3/008 - Artificial life, i.e. computing arrangements simulating life based on physical entities controlled by simulated intelligence so as to replicate intelligent life forms, e.g. based on robots replicating pets or humans in their appearance or behaviour
G05D 1/00 - Control of position, course, altitude, or attitude of land, water, air, or space vehicles, e.g. automatic pilot
66.
DESIGNING PROTEINS BY JOINTLY MODELING SEQUENCE AND STRUCTURE
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for designing a protein by jointly generating an amino acid sequence and a structure of the protein. In one aspect, a method comprises: generating data defining the amino acid sequence and the structure of the protein using a protein design neural network, comprising, for a plurality of positions in the amino acid sequence: receiving the current representation of the protein as of the current position; processing the current representation of the protein using the protein design neural network to generate design data for the current position that comprises: (i) data identifying an amino acid at the current position, and (ii) a set of structure parameters for the current position; and updating the current representation of the protein using the design data for the current position.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for distributing a top k computation across multiple computing units of an integrated circuit. One of the methods includes computing, by each of the plurality of computing units and for each candidate vector in a respective subset of the candidate vectors assigned to the computing unit, a respective distance between the query vector and the candidate vector; initializing, by the integrated circuit, a cut-off distance value; determining, by the integrated circuit, a final cut-off distance value; and providing, by the integrated circuit and as an output of a top k computation for the query vector and the set of candidate vectors, the candidate vectors that have respective distances that satisfy the final cut-off distance value.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling a plurality of robots. One of the methods includes: obtaining state data representing a current state of the environment; generating, from the state data, graph data representing a graph of the current state of the environment; processing the graph data using a graph neural network to generate a graph output that comprises a respective updated feature representation for each of the robot nodes in the graph; and selecting, based on the graph output, a respective action to be performed by each of the robots.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for predicting a structure of a protein that comprises a plurality of amino acid chains using a protein structure prediction neural network, where each chain comprises a respective sequence of amino acids. In one aspect, a method comprises: receiving a network input for the protein structure prediction neural network, wherein the network input characterizes the protein; processing the network input characterizing the protein using the protein structure prediction neural network to generate a network output that characterizes a predicted structure of the protein; and determining the predicted structure of the protein based on the network output.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling agents. In particular, an agent can be controlled using a hierarchical controller that includes a high-level controller neural network, a mid-level controller neural network, and a low-level controller neural network.
G06N 3/006 - Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for controlling a reinforcement learning agent in an environment to perform a task using a retrieval-augmented action selection process. One of the methods includes receiving a current observation characterizing a current state of the environment; processing an encoder network input comprising the current observation to determine a policy neural network hidden state that corresponds to the current observation; maintaining a plurality of trajectories generated as a result of the reinforcement learning agent interacting with the environment; selecting one or more trajectories from the plurality of trajectories; updating the policy neural network hidden state using update data determined from the one or more selected trajectories; and processing the updated hidden state using a policy neural network to generate a policy output that specifies an action to be performed by the agent in response to the current observation.
G06N 3/006 - Artificial life, i.e. computing arrangements simulating life based on simulated virtual individual or collective life forms, e.g. social simulations or particle swarm optimisation [PSO]
G06N 3/0442 - Recurrent networks, e.g. Hopfield networks characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU]
Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for training a classification machine-learning model. The system obtains calibration training examples and prediction training examples, determines a threshold value based on the calibration training examples, generates data characterizing predicted confidence sets based on the threshold value and the prediction training examples, and update model parameters based at least on the predicted confidence sets.
G16H 50/20 - ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a reinforcement learning system to select actions to be performed by an agent interacting with an environment to perform a particular task. In one aspect, one of the methods includes obtaining a training sequence comprising a respective training observations at each of a plurality of time steps; obtaining demonstration data comprising one or more demonstration sequences; generating a new training sequence from the training sequence and the demonstration data; and training the goal-conditioned policy neural network on the new training sequence through reinforcement learning.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output sequences using a non-auto-regressive neural network.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for rating tasks and policies using conditional probability distributions derived from equilibrium-based solutions of games. One of the methods includes: determining, for each action selection policy in a pool of action selection policies, a respective performance measure of the action selection policy on each task in a pool of tasks, processing the performance measures of the action selection policies on the tasks to generate data defining a joint probability distribution over a set of action selection policy - task pairs, and processing the joint probability distribution over the set of action selection policy - task pairs to generate a respective rating for each action selection policy in the pool of action selection policies, where the respective rating for each action selection policy characterizes a utility of the action selection policy in performing tasks from the pool of tasks.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing inputs using a neural network system that includes one or more transformed activation function layers.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for selecting actions to be performed by an agent to interact with an environment using an action selection neural network. In one aspect, a method comprises, at each time step in a sequence of time steps: generating a current representation of a state of a task being performed by the agent in the environment as of the current time step as a sequence of data elements; autoregressively generating a sequence of data elements representing a current action to be performed by the agent at the current time step; and after autoregressively generating the sequence of data elements representing the current action, causing the agent to perform the current action at the current time step.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for controlling agents. In particular, an agent can be controlled to perform a task episode by switching the control policy that is used to control the agent at one or more time steps during the task episode.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for raining an agent neural network for use in controlling an agent to perform a plurality of tasks. One of the methods includes maintaining population data specifying a population of one or more candidate agent neural networks; and training each candidate agent neural network on a respective set of one or more tasks to update the parameter values of the parameters of the candidate agent neural networks in the population data, the training comprising, for each candidate agent neural network: obtaining data identifying a candidate task; obtaining data specifying a control policy for the candidate task; determining whether to train the candidate agent neural network on the candidate task; and in response to determining to train the candidate agent neural network on the candidate task, training the candidate agent neural network on the candidate task.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating control signals for controlling a magnetic field for confining plasma in a chamber of a magnetic confinement device. One of the methods includes, for each of a plurality of time steps, obtaining an observation characterizing a current state of the plasma in the chamber of the magnetic confinement device, processing an input including the observation using a plasma confinement neural network to generate a magnetic control output that characterizes control signals for controlling the magnetic field of the magnetic confinement device, and generating the control signals for controlling the magnetic field of the magnetic confinement device based on the magnetic control output.
Methods, systems and apparatus, including computer programs encoded on computer storage media. One of the methods includes obtaining a plurality of images of a macromolecule having a plurality of atoms, training a decoder neural network on the plurality of images, and after the training, generating a plurality of conformations for at least a portion of the macromolecule that each include respective three-dimensional coordinates of each of the plurality of atoms, wherein generating each conformation includes sampling a conformation latent representation from a prior distribution over conformation latent representations, processing a respective input including the sampled conformation latent representation using the decoder neural network to generate a conformation output that specifies three-dimensional coordinates of each of the plurality of atoms for the conformation, and generating the conformation from the conformation output.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing video data using an adaptive visual speech recognition model. One of the methods includes receiving a video that includes a plurality of video frames that depict a first speaker; obtaining a first embedding characterizing the first speaker; and processing a first input comprising (i) the video and (ii) the first embedding using a visual speech recognition neural network having a plurality of parameters, wherein the visual speech recognition neural network is configured to process the video and the first embedding in accordance with trained values of the parameters to generate a speech recognition output that defines a sequence of one or more words being spoken by the first speaker in the video.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for processing multi-modal inputs using language models. In particular, the inputs include an image, and the image is encoded by an image encoder neural network to generate a sequence of image embeddings representing the image. The sequence of image embeddings is provided as at least part of an input sequence to that is processed by a language model neural network.
G06V 10/80 - Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
G06V 10/82 - Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
84.
PREDICTING SPECTRAL REPRESENTATIONS FOR TRAINING SPEECH SYNTHESIS NEURAL NETWORKS
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network to perform speech synthesis. One of the methods includes obtaining a training data set for training a first neural network to process a spectral representation of an audio sample and to generate a prediction of the audio sample, wherein, after training, the first neural network obtains spectral representations of audio samples from a second neural network; for a plurality of audio samples in the training data set: generating a ground-truth spectral representation of the audio sample; and processing the ground-truth spectral representation using a third neural network to generate an updated spectral representation of the audio sample; and training the first neural network using the updated spectral representations, wherein the third neural network is configured to generate updated spectral representations that resemble spectral representations generated by the second neural network.
G10L 25/30 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique using neural networks
G06N 3/04 - Architecture, e.g. interconnection topology
G10L 13/10 - Prosody rules derived from text; Stress or intonation
G10L 13/033 - Voice editing, e.g. manipulating the voice of the synthesiser
85.
CONTINUAL LEARNING NEURAL NETWORK SYSTEM TRAINING FOR CLASSIFICATION TYPE TASKS
There is disclosed a computer-implemented method for training a neural network-based system. The method comprises receiving a training data item and target data associated with the training data item. The training data item is processed using an encoder to generate an encoding of the training data item. A subset of neural networks is selected from a plurality of neural networks stored in a memory based upon the encoding; wherein the plurality of neural networks are configured to process the encoding to generate output data indicative of a classification of an aspect of the training data item. The encoding is processed using the selected subset of neural networks to generate the output data. An update to the parameters of the selected subset of neural networks is determined based upon a loss function comprising a relationship between the generated output data and the target data associated with the training data item. The parameters of the selected subset of neural networks are updated based upon the determined update.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for performing a machine learning task on a network input to generate a network output. One of the systems includes an attention neural network comprising one or more hierarchical attention blocks, each hierarchical attention block configured to: receive an input sequence for the hierarchical attention block; maintain a 5 plurality of memory summary keys, each memory summary key corresponding to a respective one of a plurality of partitions of a sequence of memory block inputs; determine a proper subset of the plurality of memory summary keys; and generate an attended input sequence for the hierarchical attention block including applying an attention mechanism over the respective memory block inputs at the memory positions within the partitions of 10 the sequence of memory block inputs that correspond to the proper subset of the plurality of memory summary keys.
This specification describes a method performed by one or more data processing apparatus that includes: sampling a latent from a set of possible latents, selecting actions to be performed by an agent to interact with an environment over a sequence of time steps using an action selection neural network that is conditioned on the sampled latent, determining a respective reward received for each time step in the sequence of time steps using an ensemble of discriminator models, and training the action selection neural network based on the rewards using a reinforcement learning technique. Each discriminator model can process an observation to generate a respective prediction output that predicts which latent the action selection neural network was conditioned on to cause the environment to enter the state characterized by the observation.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for training a neural network that includes one or more graph neural network layers. In one aspect, a method comprises: generating data defining a graph, comprising: generating a respective final feature representation for each node, wherein, for each of one or more of the nodes, the respective final feature representation is a modified feature representation that is generated from a respective feature representation for the node using respective noise; processing the data defining the graph using one or more of the graph neural network layers of the neural network to generate a respective updated node embedding of each node; and processing, for each of one or more of the nodes having modified feature representations, the updated node embedding of the node to generate a respective de-noising prediction for the node.
A computer-implemented video generation neural network system, configured to determine a value for each of a set of object latent variables by sampling from a respective prior object latent distribution for the object latent variable. The system comprises a trained image frame decoder neural network configured to, for each pixel of each generated image frame and for each generated image frame time step process determined values of the object latent variables to determine parameters of a pixel distribution for each of the object latent variables, combine the pixel distributions for each of the object latent variables to determine a combined pixel distribution, and sample from the combined pixel distribution to determine a value for the pixel and for the time step.
This specification relates to methods for controlling agents to perform actions according to a goal (or option) comprising a sequence of local goals (or local options) and corresponding methods for training. As discussed herein, environment dynamics may be modelled sequentially by sampling latent variables, each latent variable relating to a local goal and being dependent on a previous latent variable. These latent variables are used to condition an action-selection policy neural network to select actions according to the local goal. This allows the agents to reach more diverse states than would be possible through a fixed latent variable or goal, thereby encouraging exploratory behavior. In addition, specific methods described herein model the sequence of latent variables through a simple linear and recurrent relationship that allows the system to be trained more efficiently. This avoids the need to learn a state-dependent higher level policy for selecting the latent variables which can be difficult to train in practice.
Computer implemented systems and methods for training an action selection policy neural network to select actions to be performed by an agent to control the agent to perform a task. The techniques are able to optimize multiple objectives one of which may be to stay close to a behavioral policy of a teacher. The behavioral policy of the teacher may be defined by a predetermined dataset of behaviors and the systems and methods may then learn offline. The described techniques provide a mechanism for explicitly defining a trade-off between the multiple objectives.
The actions of an agent in an environment are selected using a policy model neural network which implements a policy model defining, for any observed state of the environment characterized by an observation received by the policy model neural network, a state-action distribution over the set of possible actions the agent can perform. The policy model neural network is jointly trained with a cost model neural network which, upon receiving an observation characterizing the environment, outputs a reward vector. The reward vector comprises a corresponding reward value for every possible action. The training involves a sequence of iterations, in each of which (a) a cost model is derived based on the state-action distribution of a candidate policy model defined in one or more previous iterations, and subsequently (b) a candidate policy model is obtained based on reward vector(s) defined by the cost model obtained in the iteration.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for generating a network output using a neural network. In one aspect, a method comprises: obtaining: (i) a network input to a neural network, and (ii) a set of query embeddings; processing the network input using the neural network to generate a network output that comprises a respective dimension corresponding to each query embedding in the set of query embeddings, comprising: processing the network input using an encoder block of the neural network to generate a representation of the network input as a set of latent embeddings; and processing: (i) the set of latent embeddings, and (ii) the set of query embeddings, using a cross-attention block that generates each dimension of the network output by cross-attention of a corresponding query embedding over the set of latent embeddings.
Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for training a neural network for performing a task. The system maintains data specifying (i) a plurality of candidate neural networks and (ii) a partitioning of the plurality of candidate neural networks into a plurality of partitions. The system repeatedly performs operations, including: training each of the candidate neural networks; evaluating each candidate neural network using a respective fitness function for the partition; and for each partition, updating the respective values of the one or more hyperparameters for at least one of the candidate neural networks in the partition based on the respective fitness metrics of the candidate neural networks in the partition. After repeatedly performing the operations, the system selects, from the maintained data, the respective values of the network parameters of one of the candidate neural networks.
Systems and methods for training rate control neural networks through reinforcement learning. During training, reward values for training examples are generated from the current performance of the rate control neural network in encoding the video in the training example and the historical performance of the rate control neural network in encoding the video in the training example.
Methods, systems, and apparatus, including computer programs encoded on a computer storage medium, for unmasking a masked representation of a protein using a protein reconstruction neural network. In one aspect, a method comprises: receiving the masked representation of the protein; and processing the masked representation of the protein using the protein reconstruction neural network to generate a respective predicted embedding corresponding to one or more masked embeddings that are included in the masked representation of the protein, wherein a predicted embedding corresponding to a masked embedding in a representation of the amino acid sequence of the protein defines a prediction for an identity of an amino acid at a corresponding position in the amino acid sequence, wherein a predicted embedding corresponding to a masked embedding in a representation of the structure of the protein defines a prediction for a corresponding structural feature of the protein.
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating output signals using variable-rate discrete representations. One of the methods includes generating, using a generative neural network, an event sequence representing a run-length encoding of a discrete representation of the audio signal, the event sequence comprising a respective event at each of a plurality of event sequence time steps; generating the discrete representation of the audio signal from the event sequence using run-length decoding; and processing the discrete representation using a decoder neural network, wherein the decoder neural network is configured to process the discrete representation of the audio signal to generate the prediction of the audio signal.
G10L 25/30 - Speech or voice analysis techniques not restricted to a single one of groups characterised by the analysis technique using neural networks
Methods, systems, and apparatus, including computer programs encoded on computer storage media, for precipitation nowcasting using generative neural networks. One of the methods includes obtaining a context temporal sequence of a plurality of context radar fields characterizing a real-world location, each context radar field characterizing the weather in the real-world location at a corresponding preceding time point; sampling a set of one or more latent inputs by sampling values from a specified distribution; and for each sampled latent input, processing the context temporal sequence of radar fields and the sampled latent input using a generative neural network that has been configured through training to process the temporal sequence of radar fields to generate as output a predicted temporal sequence comprising a plurality of predicted radar fields, each predicted radar field in the predicted temporal sequence characterizing the predicted weather in the real-world location at a corresponding future time point.
A computer-implemented method for generating an output token sequence from an input token sequence. The method combines a look ahead tree search, such as a Monte Carlo tree search, with a sequence-to-sequence neural network system. The sequence-to-sequence neural network system has a policy output defining a next token probability distribution, and may include a value neural network providing a value output to evaluate a sequence. An initial partial output sequence is extended using the look ahead tree search guided by the policy output and, in implementations, the value output, of the sequence-to-sequence neural network system until a complete output sequence is obtained.
patents
