公开(公告)号：US20210124352A1

公开(公告)日：2021-04-29

申请号：US16667441

申请日：2019-10-29

申请人： LOON LLC

发明人： Salvatore J. Candido ,  Jun Gong ,  Marc Gendron-Bellemare

IPC分类号： G05D1/00 ,  G06N3/04 ,  G06N3/08 ,  G05D1/02 ,  B64B1/00

摘要： The technology relates to navigating aerial vehicles using deep reinforcement learning techniques to generate flight policies. An operational system for controlling flight of an aerial vehicle may include a computing system configured to process an input vector representing a state of the aerial vehicle and output an action, an operation-ready policies server configured to store a trained neural network encoding a learned flight policy, and a controller configured to control the aerial vehicle. The input vector may be processed using the trained neural network encoding the learned flight policy. A method for navigating an aerial vehicle may include selecting a trained neural network encoding a learned flight policy from an operation policies server, generating an input vector comprising a set of characteristics representing a state of the aerial vehicle, selecting an action, by the trained neural network, based on the input vector, converting the action into a set of commands, by a flight computer, the set of commands configured to cause the aerial vehicle to perform the action, and causing, by a controller, the aerial vehicle to perform the action using the set of commands.

公开(公告)号：US11015935B2

公开(公告)日：2021-05-25

申请号：US16222309

申请日：2018-12-17

申请人： LOON LLC

发明人： Salvatore J. Candido ,  Jun Gong ,  Jacob Buswell Roberts ,  Vincent Carroll

IPC分类号： G01C21/00 ,  G01C21/20 ,  B64C39/02 ,  H04L29/08 ,  G06N3/08

摘要： The technology relates to generating a flight map for an aircraft. For instance, this may include, running a plurality of simulations by placing a simulated aircraft within each cell of a grid representing areas of the earth and using predicted wind data. Each simulations identifies a cell in which each aircraft is located at the end of the simulation. A connection graph may be generated using any identified cells. The connection graph may be used to determine a flight map for an actual aircraft using a cost function and iterating from a destination cell to an initial cell. The flight map may be used to determine a route for the actual aircraft. In some examples, the flight map may be refined by running further simulations. The refined flight map may then be used to determine a route for the actual aircraft.

公开(公告)号：US20210123741A1

公开(公告)日：2021-04-29

申请号：US16667424

申请日：2019-10-29

申请人： LOON LLC

发明人： Salvatore J. Candido ,  Jun Gong ,  Marc Gendron-Bellemare ,  Marlos Cholodovskis Machado

IPC分类号： G01C21/20 ,  G06N3/08 ,  G06N3/04

摘要： The technology relates to navigating aerial vehicles using deep reinforcement learning techniques to generate flight policies. A computing system may include a simulator configured to produce simulations of a flight of the aerial vehicle in a region of an atmosphere, a replay buffer configured to store frames of the simulations, and a learning module having a deep reinforcement learning architecture configured to, by a reinforcement learning algorithm, process an input of a set of frames, and output a neural network encoding a learned flight policy. A meta-learning system may include stacks of learning systems, a coordinator configured to provide an instruction to the learning systems that includes a parameter and a start time, and an evaluation server configured to evaluate resulting rewards from learned flight policies generated by the learning systems.