-
1.
公开(公告)号:US12039438B2
公开(公告)日:2024-07-16
申请号:US17112292
申请日:2020-12-04
发明人: Boris Ivanovic , Kuan-Hui Lee , Jie Li , Adrien David Gaidon , Pavel Tokmakov
CPC分类号: G06N3/08 , B60W30/0956 , G06N3/044 , B60W60/0027 , B60W2554/4044 , G05D1/0214
摘要: Systems, methods, and other embodiments described herein relate to improving trajectory forecasting in a device. In one embodiment, a method includes, in response to receiving sensor data about a surrounding environment of the device, identifying an object from the sensor data that is present in the surrounding environment. The method includes determining category probabilities for the object, the category probabilities indicating semantic classes for classifying the object and probabilities that the object belongs to the semantic classes. The method includes forecasting trajectories for the object based, at least in part, on the category probabilities and the sensor data. The method includes controlling the device according to the trajectories.
-
公开(公告)号:US11967141B2
公开(公告)日:2024-04-23
申请号:US18161777
申请日:2023-01-30
发明人: Adrien David Gaidon , Jie Li
IPC分类号: G06N3/04 , G06F17/18 , G06F18/20 , G06F18/2113 , G06F18/25 , G06N3/045 , G06V10/764 , G06V10/771 , G06V10/80 , G06V10/82 , G06V20/56 , G06V40/10
CPC分类号: G06V10/82 , G06F17/18 , G06F18/2113 , G06F18/25 , G06F18/29 , G06N3/045 , G06V10/764 , G06V10/771 , G06V10/80 , G06V20/56 , G06V40/10
摘要: One or more embodiments of the present disclosure include systems and methods that use neural architecture fusion to learn how to combine multiple separate pre-trained networks by fusing their architectures into a single network for better computational efficiency and higher accuracy. For example, a computer implemented method of the disclosure includes obtaining multiple trained networks. Each of the trained networks may be associated with a respective task and has a respective architecture. The method further includes generating a directed acyclic graph that represents at least a partial union of the architectures of the trained networks. The method additionally includes defining a joint objective for the directed acyclic graph that combines a performance term and a distillation term. The method also includes optimizing the joint objective over the directed acyclic graph.
-
公开(公告)号:US20240013409A1
公开(公告)日:2024-01-11
申请号:US18202632
申请日:2023-05-26
发明人: Colton Stearns , Jie Li , Rares A. Ambrus , Vitor Campagnolo Guizilini , Sergey Zakharov , Adrien D. Gaidon , Davis Rempe , Tolga Birdal , Leonidas J. Guibas
IPC分类号: G06T7/246
CPC分类号: G06T7/248 , G06T2207/10028 , G06T2207/20084 , G06T2207/20081
摘要: A method for multiple object tracking includes receiving, with a computing device, a point cloud dataset, detecting one or more objects in the point cloud dataset, each of the detected one or more objects defined by points of the point cloud dataset and a bounding box, querying one or more historical tracklets for historical tracklet states corresponding to each of the one or more detected objects, implementing a 4D encoding backbone comprising two branches: a first branch configured to compute per-point features for each of the one or more objects and the corresponding historical tracklet states, and a second branch configured to obtain 4D point features, concatenating the per-point features and the 4D point features, and predicting, with a decoder receiving the concatenated per-point features, current tracklet states for each of the one or more objects.
-
公开(公告)号:US11868439B2
公开(公告)日:2024-01-09
申请号:US17215646
申请日:2021-03-29
发明人: Vitor Guizilini , Adrien David Gaidon , Jie Li , Rares A. Ambrus
CPC分类号: G06F18/2178 , G06F18/2148 , G06T7/50 , G06T7/74 , G06T9/002 , G06V20/56 , G06V20/64 , G06T2207/10024 , G06T2207/10028 , G06T2207/20081 , G06T2207/20084
摘要: Systems, methods, and other embodiments described herein relate to training a multi-task network using real and virtual data. In one embodiment, a method includes acquiring training data that includes real data and virtual data for training a multi-task network that performs at least depth prediction and semantic segmentation. The method includes generating a first output from the multi-task network using the real data and second output from the multi-task network using the virtual data. The method includes generating a mixed loss by analyzing the first output to produce a real loss and the second output to produce a virtual loss. The method includes updating the multi-task network using the mixed loss.
-
公开(公告)号:US11775615B2
公开(公告)日:2023-10-03
申请号:US17242498
申请日:2021-04-28
CPC分类号: G06F18/22 , G06F18/213 , G06F18/25 , G06N3/08 , G06V20/64
摘要: Systems and methods for tracking objects are disclosed herein. In one embodiment, a system having a processor merges features of detected objects extracted from a point cloud and a corresponding image to generate fused features for the detected objects, generates a learned distance metric for the detected objects using the fused features, determines matched detected objects and unmatched detected objects, applies prior tracking identifiers of the detected objects at the prior time to the matched detected objects, determines a confidence score for the fused features of the unmatched detected objects, and applies new tracking identifiers to the unmatched detected objects based on the confidence score.
-
公开(公告)号:US20220301202A1
公开(公告)日:2022-09-22
申请号:US17333537
申请日:2021-05-28
发明人: Dennis Park , Rares A. Ambrus , Vitor Guizilini , Jie Li , Adrien David Gaidon
摘要: System, methods, and other embodiments described herein relate to performing depth estimation and object detection using a common network architecture. In one embodiment, a method includes generating, using a backbone of a combined network, a feature map at multiple scales from an input image. The method includes decoding, using a top-down pathway of the combined network, the feature map to provide features at the multiple scales. The method includes generating, using a head of the combined network, a depth map from the features for a scene depicted in the input image, and bounding boxes identifying objects in the input image.
-
7.
公开(公告)号:US20220180170A1
公开(公告)日:2022-06-09
申请号:US17112292
申请日:2020-12-04
发明人: Boris Ivanovic , Kuan-Hui Lee , Jie Li , Adrien David Gaidon , Pavel Tokmakov
IPC分类号: G06N3/08 , G06N3/04 , B60W30/095
摘要: System, methods, and other embodiments described herein relate to improving trajectory forecasting in a device. In one embodiment, a method includes, in response to receiving sensor data about a surrounding environment of the device, identifying an object from the sensor data that is present in the surrounding environment. The method includes determining category probabilities for the object, the category probabilities indicating semantic classes for classifying the object and probabilities that the object belongs to the semantic classes. The method includes forecasting trajectories for the object based, at least in part, on the category probabilities and the sensor data. The method includes controlling the device according to the trajectories.
-
公开(公告)号:US20220036126A1
公开(公告)日:2022-02-03
申请号:US16943393
申请日:2020-07-30
发明人: Adrien David Gaidon , Jie Li
摘要: A detector system having a detector model includes one or more processor(s) and a memory. The memory includes an image acquisition module, a training module, and a label propagating module. The modules cause the processor(s) to obtain a first training set, train the detector model using the first training set and a first loss function, label propagate a second training set by the detector model after the detector model is trained with the first training set, and train the detector model using the first training set, the second training set, the first loss function, and a discriminative loss function. The detector model is trained through an intermediate multidimensional feature predicted at each pixel location of the one or more objects of the first training set and the second training set. The intermediate multidimensional feature being an instance identifier expressing the temporal consistency of objects along the temporal axis.
-
公开(公告)号:US12067785B2
公开(公告)日:2024-08-20
申请号:US17358497
申请日:2021-06-25
发明人: Rares A. Ambrus , Dennis Park , Vitor Guizilini , Jie Li , Adrien David Gaidon
IPC分类号: G06K9/62 , G01S17/42 , G01S17/89 , G01S17/931 , G06F18/21 , G06F18/2113 , G06F18/214 , G06F18/25 , G06K9/00 , G06K9/46 , G06N3/04 , G06N3/08 , G06N20/00 , G06T7/10 , G06T7/11 , G06T7/50 , G06V10/46 , G06V10/75 , G06V20/56 , G06V20/58
CPC分类号: G06V20/58 , G01S17/42 , G01S17/89 , G01S17/931 , G06F18/2113 , G06F18/2155 , G06F18/217 , G06F18/251 , G06N3/04 , G06N3/08 , G06N20/00 , G06T7/10 , G06T7/11 , G06T7/50 , G06V10/462 , G06V10/757 , G06V20/56 , G06T2207/10024 , G06T2207/10028 , G06T2207/20016 , G06T2207/20081 , G06T2207/20084 , G06T2207/30248
摘要: Systems, methods, and other embodiments described herein relate to evaluating a perception network in relation to the accuracy of depth estimates and object detections. In one embodiment, a method includes segmenting range data associated with an image according to bounding boxes of objects identified in the image to produce masked data. The method includes comparing the masked data with corresponding depth estimates in the depth map according to an evaluation mask that correlates the depth estimates with the depth map. The method includes providing a metric that quantifies the comparing to assess a network that generated the depth map and the bounding boxes.
-
公开(公告)号:US20230394691A1
公开(公告)日:2023-12-07
申请号:US17834850
申请日:2022-06-07
CPC分类号: G06T7/521 , G06V10/803 , G01S13/867 , G01S13/89 , G06T2207/10028 , G06T2207/20068 , G06T2207/20081 , G06T2207/30248
摘要: Systems and methods are provided for depth estimation from monocular images using a depth model with sparse range sensor data and uncertainty in the range sensor as inputs thereto. According to some embodiments, the methods and systems comprise receiving an image captured by an image sensor, where the image represents a scene of an environment. The method and systems also comprise deriving a point cloud representative of the scene of the environment from range sensor data, and deriving range sensor uncertainty from the range sensor data. Then a depth map can be derived for the image based on the point cloud and the range sensor uncertainty as one or more inputs into a depth model.
-
-
-
-
-
-
-
-
-