公开(公告)号：US20230099920A1

公开(公告)日：2023-03-30

申请号：US17994991

申请日：2022-11-28

Applicant: Waymo LLC

Inventor： Junhua Mao ,  Lo Po Tsui ,  Congcong Li ,  Edward Stephen Walker, JR.

IPC: G06N3/08 ,  G06N3/04

Abstract: Methods, systems, and apparatus, including computer programs encoded on computer storage media, for training a classifier to detect open vehicle doors. One of the methods includes obtaining a plurality of initial training examples, each initial training example comprising (i) a sensor sample from a collection of sensor samples and (ii) data classifying the sensor sample as characterizing a vehicle that has an open door; generating a plurality of additional training examples, comprising, for each initial training example: identifying, from the collection of sensor samples, one or more additional sensor samples that were captured less than a threshold amount of time before the sensor sample in the initial training example was captured; and training the machine learning classifier on first training data that includes the initial training examples and the additional training examples to generate updated weights for the machine learning classifier.

公开(公告)号：US10963734B1

公开(公告)日：2021-03-30

申请号：US16656974

申请日：2019-10-18

Applicant: Waymo LLC

Inventor： Jiajing Wang ,  Chi Zeng ,  Lorena Zalles ,  Lo Po Tsui ,  Heidi Lam

IPC: G06K9/62 ,  G05D1/02 ,  G05D1/00 ,  G06T7/73 ,  G06K9/00 ,  G06F16/35 ,  B60R1/00 ,  G06T7/521 ,  G01S17/931

Abstract: Aspects of the disclosure relate to facilitating review of labels. For instance, a first type of label for a first set of labels and a second type of label for a second set of labels may be received. The first set of labels may be generated by a first labeling source and may classify one or more objects captured by a sensor of a vehicle. The second set of labels may be generated by a second labeling source different from the first labeling source and may classify the one or more objects. A search is conducted for objects associated with both the first type of labels for the first set of labels and the second type of label for the second set of labels in order to identify search results. The histograms may be generated from the search results and histograms may be provided for display to a human operator.

公开(公告)号：US10699167B1

公开(公告)日：2020-06-30

申请号：US16165468

申请日：2018-10-19

Applicant: Waymo LLC

Inventor： Jonathan Baldwin Dowdall ,  Nafees Ahmed ,  Lo Po Tsui ,  Peter Morton

IPC: G06K9/62 ,  G06K9/00 ,  G06F3/0484 ,  G06F3/0481

Abstract: Aspects of the disclosure relate to generating a grid or a visual list to facilitate operator review of labels. The system receives a first set of labels generated by a first labeling source and a second set of labels generated by a second labeling source. The first set of labels and the second set of labels each classify one or more objects perceived in one or more scenes captured by a sensor of a vehicle, such that each of the one or more objects has a corresponding first label and a corresponding second label. The system determines discrepancies between the corresponding first label and the corresponding second label for each of the one or more objects, and generates a grid or a visual list using the determined discrepancies. The system provides the grid or the visual list for display to a human operator.

公开(公告)号：US20250103844A1

公开(公告)日：2025-03-27

申请号：US18973983

申请日：2024-12-09

Applicant: Waymo LLC

Inventor： Justin Thorsen ,  Changchang Wu ,  Alper Ayvaci ,  Tiffany Chen ,  Lo Po Tsui ,  Zhinan Xu ,  Chen Wu ,  Sean Rafferty

IPC: G06K19/067 ,  G09F3/00

Abstract: Aspects of the disclosure provide for automatically generating labels for sensor data. For instance, first sensor data for a vehicle may be identified. This first sensor data may have been captured by a first sensor of the vehicle at a first location during a first point in time and may be associated with a first label for an object. Second sensor data for the vehicle may be identified. The second sensor data may have been captured by a second sensor of the vehicle at a second location at a second point in time outside of the first point in time. The second location is different from the first location. A determination may be made as to whether the object is a static object. Based on the determination that the object is a static object, the first label may be used to automatically generate a second label for the second sensor data.

公开(公告)号：US20220391616A1

公开(公告)日：2022-12-08

申请号：US17341255

申请日：2021-06-07

Applicant: Waymo LLC

Inventor： Lo Po Tsui ,  Allen Minich ,  Megan Van Welie ,  Neil Barooah ,  Dragomir Anguelov

IPC: G06K9/00 ,  G06N20/00

Abstract: Methods, systems, and apparatus, including computer programs encoded on a computer storage medium that validates labels associated with sensor measurements of a scene in an environment. One of the methods includes receiving data representing a sensor measurement of a scene in an environment generated by one or more sensors. The sensor measurement can be associated with one or more labels, and each label can identify a portion of the sensor measurement that has been classified as measuring an object in the environment. For each of the labels, a determination can be made as to whether the label satisfies each of the validation criteria. Each validation criterion can measure whether one or more characteristics of the label are consistent with one or more characteristics of real-world objects in the environment. In response to determining that a particular label of the one or more labels does not satisfy one or more of the validation criteria, a notification can be generated indicating that the particular label is not a valid label for any real-world object in the scene of the environment.

公开(公告)号：US10891518B1

公开(公告)日：2021-01-12

申请号：US16220100

申请日：2018-12-14

Applicant: Waymo LLC

Inventor： Aditya Joshi ,  Ingrid Fiedler ,  Lo Po Tsui

IPC: G06K9/62 ,  G01S17/89 ,  G06K9/00 ,  G06N20/20 ,  G01S17/58 ,  G01S17/04

Abstract: Aspects of the disclosure relate to training a labeling model to automatically generate labels for objects detected in a vehicle's environment. In this regard, one or more computing devices may receive sensor data corresponding to a series of frames perceived by the vehicle, each frame being captured at a different time point during a trip of the vehicle. The computing devices may also receive bounding boxes generated by a first labeling model for objects detected in the series of frames. The computing devices may receive user inputs including an adjustment to at least one of the bounding boxes, the adjustment corrects a displacement of the at least one of the bounding boxes caused by a sensing inaccuracy. The computing devices may train a second labeling model using the sensor data, the bounding boxes, and the adjustment to increase accuracy of the second labeling model when automatically generating bounding boxes.

公开(公告)号：US12211269B2

公开(公告)日：2025-01-28

申请号：US17314925

申请日：2021-05-07

Applicant: Waymo LLC

Inventor： Yulai Shen ,  Henrik Kretzschmar ,  Jeffrey Sham ,  Jeffrey Carlson ,  Lo Po Tsui ,  Dragomir Anguelov

IPC: G06V20/20 ,  G06N20/00 ,  G06V10/20 ,  G06V20/40

Abstract: Methods, computer systems, and apparatus, including computer programs encoded on computer storage media, for generating and editing object track labels for objects detected in video data. One of the methods includes obtaining a video segment comprising multiple image frames associated with multiple time points; obtaining object track data specifying a set of object tracks; providing, for presentation to a user, a user interface for modifying the object track data, the user interface displaying object timeline representations of the object tracks; receiving one or more user inputs that indicate one or more modifications to the object timeline representations; updating the object timeline representations displayed in the timeline display area; and updating the object track data according to the updated object timeline representations.

公开(公告)号：US20230046289A1

公开(公告)日：2023-02-16

申请号：US17893376

申请日：2022-08-23

Applicant: WAYMO LLC

Inventor： Justin Thorsen ,  Changchang Wu ,  Alper Ayvaci ,  Tiffany Chen ,  Lo Po Tsui ,  Zhinan Xu ,  Chen Wu ,  Sean Rafferty

IPC: G06K19/067 ,  G09F3/00

Abstract: Aspects of the disclosure provide for automatically generating labels for sensor data. For instance, first sensor data, for a vehicle may be identified. This first sensor data may have been captured by a first sensor of the vehicle at a first location during a first point in time and may be associated with a first label for an object. Second sensor data for the vehicle may be identified. The second sensor data may have been captured by a second sensor of the vehicle at a second location at a second point in time outside of the first point in time. The second location is different from the first location. A determination may be made as to whether the object is a static object. Based on the determination that the object is a static object, the first label may be used to automatically generate a second label for the second sensor data.

公开(公告)号：US11556744B1

公开(公告)日：2023-01-17

申请号：US17116255

申请日：2020-12-09

Applicant: Waymo LLC

Inventor： Aditya Joshi ,  Ingrid Fiedler ,  Lo Po Tsui

IPC: G06K9/62 ,  G01S17/89 ,  G06N20/20 ,  G01S17/58 ,  G01S17/04 ,  G06V20/20 ,  G06V20/58

Abstract: Aspects of the disclosure relate to training a labeling model to automatically generate labels for objects detected in a vehicle's environment. In this regard, one or more computing devices may receive sensor data corresponding to a series of frames perceived by the vehicle, each frame being captured at a different time point during a trip of the vehicle. The computing devices may also receive bounding boxes generated by a first labeling model for objects detected in the series of frames. The computing devices may receive user inputs including an adjustment to at least one of the bounding boxes, the adjustment corrects a displacement of the at least one of the bounding boxes caused by a sensing inaccuracy. The computing devices may train a second labeling model using the sensor data, the bounding boxes, and the adjustment to increase accuracy of the second labeling model when automatically generating bounding boxes.

公开(公告)号：US11475263B2

公开(公告)日：2022-10-18

申请号：US16827835

申请日：2020-03-24

Applicant: WAYMO LLC

Inventor： Justin Thorsen ,  Changchang Wu ,  Alper Ayvaci ,  Tiffany Chen ,  Lo Po Tsui ,  Zhinan Xu ,  Chen Wu ,  Sean Rafferty

IPC: G06K19/067 ,  G09F3/00

Abstract: Aspects of the disclosure provide for automatically generating labels for sensor data. For instance, first sensor data for a vehicle may be identified. This first sensor data may have been captured by a first sensor of the vehicle at a first location during a first point in time and may be associated with a first label for an object. Second sensor data for the vehicle may be identified. The second sensor data may have been captured by a second sensor of the vehicle at a second location at a second point in time outside of the first point in time. The second location is different from the first location. A determination may be made as to whether the object is a static object. Based on the determination that the object is a static object, the first label may be used to automatically generate a second label for the second sensor data.