公开(公告)号：US20210201464A1

公开(公告)日：2021-07-01

申请号：US16728532

申请日：2019-12-27

Applicant: Zoox, Inc.

Inventor： Sarah Tariq ,  James William Vaisey Philbin ,  Yi Xu

IPC: G06T7/00 ,  G05D1/02

Abstract: A sensor degradation detection and remediation system includes one or more sensors configured to collect image data from an environment. A combination of techniques may be used to detect degradations within regions of the image data captured by a sensor, including one or more of determining a level of the visual consistency between the associated image regions captured by different sensors, determining a level of opaqueness of the image regions, and/or measuring temporal movement of the image regions captured by a sensor over a period of time. Operations of a vehicle or other system may be controlled based at least in part on the detection of degradations of the image data captured by the sensors, including automated cleaning of a sensor surface, reducing a level of reliance on the image data received from the sensor, and/or changing a direction of travel of the vehicle.

公开(公告)号：US20210181757A1

公开(公告)日：2021-06-17

申请号：US16732274

申请日：2019-12-31

Applicant: Zoox, Inc.

Inventor： Kratarth Goel ,  James William Vaisey Philbin ,  Praveen Srinivasan ,  Sarah Tariq

IPC: G05D1/02 ,  G06T7/579 ,  G05D1/00 ,  G06T7/11 ,  G06N20/00

Abstract: A machine-learning (ML) architecture for determining three or more outputs, such as a two and/or three-dimensional region of interest, semantic segmentation, direction logits, depth data, and/or instance segmentation associated with an object in an image. The ML architecture may output these outputs at a rate of 30 or more frames per second on consumer grade hardware.

公开(公告)号：US20210103285A1

公开(公告)日：2021-04-08

申请号：US16591518

申请日：2019-10-02

Applicant: Zoox, Inc.

Inventor： James William Vaisey Philbin ,  Cooper Stokes Sloan ,  Noureldin Ehab Hendy ,  Nicholas George Charchut ,  Chuang Wang

IPC: G05D1/02 ,  G05D1/00 ,  B60W30/095 ,  B60W50/00

Abstract: A collision avoidance system may validate, reject, or replace a trajectory generated to control a vehicle. The collision avoidance system may comprise a secondary perception component that may receive sensor data, receive and/or determine a corridor associated with operation of a vehicle, classify a portion of the sensor data associated with the corridor as either ground or an object, determine a position and/or velocity of at least the nearest object, determine a threshold distance associated with the vehicle, and control the vehicle based at least in part on the position and/or velocity of the nearest object and the threshold distance.

公开(公告)号：US20190391578A1

公开(公告)日：2019-12-26

申请号：US16013748

申请日：2018-06-20

Applicant: Zoox, Inc.

Inventor： Sarah Tariq ,  James William Vaisey Philbin ,  Kratarth Goel

IPC: G05D1/00 ,  G05D1/02 ,  G06K9/32 ,  G06K9/62 ,  G06T3/40 ,  G06N3/08 ,  G06N7/00

Abstract: Techniques for utilizing multiple scales of images as input to machine learning (ML) models are discussed herein. Operations can include providing an image associated with a first scale to a first ML model. An output of the first ML model can include a first bounding box indicative of a first region of the image representing a first object, with the first bounding box falling within a first range of sizes. Next, a scaled image can be generated by scaling the image. The scaled image can be provided to a second ML model, which can output a second bounding box indicative of a second region of the image representing a second object, the second bounding falling within a second range of sizes. Thus, inputting a scaled image to a same ML model (or to different ML models) can result in different detected features in the images.

公开(公告)号：US20190272446A1

公开(公告)日：2019-09-05

申请号：US15910758

申请日：2018-03-02

Applicant: Zoox, Inc.

Inventor： Juhana Kangaspunta ,  Kai Zhenyu Wang ,  James William Vaisey Philbin

IPC: G06K9/62 ,  G01C21/32 ,  G06F17/30 ,  G06T7/10 ,  G05D1/02

Abstract: A system may automatically create training datasets for training a segmentation model to recognize features such as lanes on a road. The system may receive sensor data representative of a portion of an environment and map data from a map data store including existing map data for the portion of the environment that includes features present in that portion of the environment. The system may project or overlay the features onto the sensor data to create training datasets for training the segmentation model, which may be a neural network. The training datasets may be communicated to the segmentation model to train the segmentation model to segment data associated with similar features present in different sensor data. The trained segmentation model may be used to update the map data store, and may be used to segment sensor data obtained from other portions of the environment, such as portions not previously mapped.

公开(公告)号：US10176388B1

公开(公告)日：2019-01-08

申请号：US15409841

申请日：2017-01-19

Applicant: Zoox, Inc.

Inventor： Mahsa Ghafarianzadeh ,  James William Vaisey Philbin

IPC: G06K9/00 ,  G06T7/174

Abstract: Systems and methods for segmenting an image using a convolutional neural network are described herein. A convolutional neural network (CNN) comprises an encoder-decoder architecture, and may comprise one or more Long Short Term Memory (LSTM) layers between the encoder and decoder layers. The LSTM layers provide temporal information in addition to the spatial information of the encoder-decoder layers. A subset of a sequence of images is input into the encoder layer of the CNN and a corresponding sequence of segmented images is output from the decoder layer. In some embodiments, the one or more LSTM layers may be combined in such a way that the CNN is predictive, providing predicted output of segmented images. Though the CNN provides multiple outputs, the CNN may be trained from single images or by generation of noisy ground truth datasets. Segmenting may be performed for object segmentation or free space segmentation.