-
公开(公告)号:US20180129910A1
公开(公告)日:2018-05-10
申请号:US15709748
申请日:2017-09-20
发明人: Muhammad Zeeshan Zia , Quoc-Huy Tran , Xiang Yu , Manmohan Chandraker , Chi Li
CPC分类号: G06K9/6256 , B60T2201/022 , B60W30/00 , G05D1/0221 , G06F17/5009 , G06K9/00201 , G06K9/00208 , G06K9/00624 , G06K9/00771 , G06K9/00805 , G06K9/4628 , G06K9/6255 , G06N3/02 , G06N3/084 , G06T7/55 , G06T7/74 , G06T11/60 , G06T15/10 , G06T15/40 , G06T2207/20101 , G06T2207/30261 , G06T2210/22 , G08G1/0962 , G08G1/166 , H04N7/00
摘要: A system and method are provided. The system includes an image capture device configured to capture an actual image depicting an object. The system also includes a processor. The processor is configured to render, based on a set of 3D Computer Aided Design (CAD) models, a set of synthetic images with corresponding intermediate shape concept labels. The processor is also configured to form a multi-layer Convolutional Neural Network (CNN) which jointly models multiple intermediate shape concepts, based on the rendered synthetic images. The processor is further configured to perform an intra-class appearance variation-aware and occlusion-aware 3D object parsing on the actual image by applying the CNN to the actual image to output an image pair including a 2D geometric structure and a 3D geometric structure of the object depicted in the actual image.
-
公开(公告)号:US20190065868A1
公开(公告)日:2019-02-28
申请号:US16029167
申请日:2018-07-06
发明人: Quoc-Huy Tran , Mohammed E.F. Salem , Muhammad Zeeshan Zia , Paul Vernaza , Manmohan Chandraker
CPC分类号: G06K9/00798 , G06K9/00785 , G06K9/00825 , G06K9/46 , G06K9/4628 , G06K9/6262 , G06K9/6274 , G06K9/6276 , G06K9/629 , G06K9/66 , G06N3/0454 , G06N3/08 , G06T17/05 , G06T2200/08 , G06T2207/20081
摘要: Systems and methods for detecting traffic scenarios include an image capturing device which captures two or more images of an area of a traffic environment with each image having a different view of vehicles and a road in the traffic environment. A hierarchical feature extractor concurrently extracts features at multiple neural network layers from each of the images, with the features including geometric features and semantic features, and for estimating correspondences between semantic features for each of the images and refining the estimated correspondences with correspondences between the geometric features of each of the images to generate refined correspondence estimates. A traffic localization module uses the refined correspondence estimates to determine locations of vehicles in the environment in three dimensions to automatically determine a traffic scenario according to the locations of vehicles. A notification device generates a notification of the traffic scenario.
-
3.
公开(公告)号:US20180130355A1
公开(公告)日:2018-05-10
申请号:US15709814
申请日:2017-09-20
发明人: Muhammad Zeeshan Zia , Quoc-Huy Tran , Xiang Yu , Manmohan Chandraker , Chi Li
CPC分类号: G06K9/6256 , B60T2201/022 , B60W30/00 , G05D1/0221 , G06F17/5009 , G06K9/00201 , G06K9/00208 , G06K9/00624 , G06K9/00771 , G06K9/00805 , G06K9/4628 , G06K9/6255 , G06N3/02 , G06N3/084 , G06T7/55 , G06T7/74 , G06T11/60 , G06T15/10 , G06T15/40 , G06T2207/20101 , G06T2207/30261 , G06T2210/22 , G08G1/0962 , G08G1/166 , H04N7/00
摘要: A system and method are provided for driving assistance. The system includes an image capture device configured to capture an actual image relative to an outward view from a motor vehicle and depicting an object. The system further includes a processor configured to render, based on a set of 3D CAD models, synthetic images with corresponding intermediate shape concept labels. The processor is further configured to form a multi-layer CNN which jointly models multiple intermediate shape concepts, based on the rendered synthetic images. The processor is also configured to perform an intra-class appearance variation-aware and occlusion-aware 3D object parsing on the actual image by applying the CNN to the actual image to output an image pair including a 2D and 3D geometric structure of the object. The processor is additionally configured to perform an action to mitigate a likelihood of harm involving the motor vehicle, based on the image pair.
-
4.
公开(公告)号:US20180129865A1
公开(公告)日:2018-05-10
申请号:US15709897
申请日:2017-09-20
发明人: Muhammad Zeeshan Zia , Quoc-Huy Tran , Xiang Yu , Manmohan Chandraker , Chi Li
CPC分类号: G06K9/6256 , B60T2201/022 , B60W30/00 , G05D1/0221 , G06F17/5009 , G06K9/00201 , G06K9/00208 , G06K9/00624 , G06K9/00771 , G06K9/00805 , G06K9/4628 , G06K9/6255 , G06N3/02 , G06N3/0454 , G06N3/082 , G06N3/084 , G06T7/55 , G06T7/74 , G06T11/60 , G06T15/10 , G06T15/40 , G06T2207/20101 , G06T2207/30261 , G06T2210/22 , G08G1/0962 , G08G1/166 , H04N7/00
摘要: An action recognition system and method are provided. The action recognition system includes an image capture device configured to capture an actual image depicting an object. The action recognition system includes a processor configured to render, based on a set of 3D CAD models, synthetic images with corresponding intermediate shape concept labels. The processor is configured to form a multi-layer CNN which jointly models multiple intermediate shape concepts, based on the rendered synthetic images. The processor is configured to perform an intra-class appearance variation-aware and occlusion-aware 3D object parsing on the actual image by applying the CNN thereto to generate an image pair including a 2D and 3D geometric structure of the object. The processor is configured to control a device to perform a response action in response to an identification of an action performed by the object, wherein the identification of the action is based on the image pair.
-
公开(公告)号:US10832084B2
公开(公告)日:2020-11-10
申请号:US16526306
申请日:2019-07-30
发明人: Quoc-Huy Tran , Mohammed E. Fathy Salem , Muhammad Zeeshan Zia , Paul Vernaza , Manmohan Chandraker
摘要: A method for estimating dense 3D geometric correspondences between two input point clouds by employing a 3D convolutional neural network (CNN) architecture is presented. The method includes, during a training phase, transforming the two input point clouds into truncated distance function voxel grid representations, feeding the truncated distance function voxel grid representations into individual feature extraction layers with tied weights, extracting low-level features from a first feature extraction layer, extracting high-level features from a second feature extraction layer, normalizing the extracted low-level features and high-level features, and applying deep supervision of multiple contrastive losses and multiple hard negative mining modules at the first and second feature extraction layers. The method further includes, during a testing phase, employing the high-level features capturing high-level semantic information to obtain coarse matching locations, and refining the coarse matching locations with the low-level features to capture low-level geometric information for estimating precise matching locations.
-
公开(公告)号:US10762359B2
公开(公告)日:2020-09-01
申请号:US16029167
申请日:2018-07-06
发明人: Quoc-Huy Tran , Mohammed E. F. Salem , Muhammad Zeeshan Zia , Paul Vernaza , Manmohan Chandraker
摘要: Systems and methods for detecting traffic scenarios include an image capturing device which captures two or more images of an area of a traffic environment with each image having a different view of vehicles and a road in the traffic environment. A hierarchical feature extractor concurrently extracts features at multiple neural network layers from each of the images, with the features including geometric features and semantic features, and for estimating correspondences between semantic features for each of the images and refining the estimated correspondences with correspondences between the geometric features of each of the images to generate refined correspondence estimates. A traffic localization module uses the refined correspondence estimates to determine locations of vehicles in the environment in three dimensions to automatically determine a traffic scenario according to the locations of vehicles. A notification device generates a notification of the traffic scenario.
-
7.
公开(公告)号:US10679075B2
公开(公告)日:2020-06-09
申请号:US16029126
申请日:2018-07-06
发明人: Quoc-Huy Tran , Mohammed E. F. Salem , Muhammad Zeeshan Zia , Paul Vernaza , Manmohan Chandraker
摘要: Systems and methods for correspondence estimation and flexible ground modeling include communicating two-dimensional (2D) images of an environment to a correspondence estimation module, including a first image and a second image captured by an image capturing device. First features, including geometric features and semantic features, are hierarchically extract from the first image with a first convolutional neural network (CNN) according to activation map weights, and second features, including geometric features and semantic features, are hierarchically extracted from the second image with a second CNN according to the activation map weights. Correspondences between the first features and the second features are estimated, including hierarchical fusing of geometric correspondences and semantic correspondences. A 3-dimensional (3D) model of a terrain is estimated using the estimated correspondences belonging to the terrain surface. Relative locations of elements and objects in the environment are determined according to the 3D model of the terrain. A user is notified of the relative locations.
-
公开(公告)号:US10331974B2
公开(公告)日:2019-06-25
申请号:US15709897
申请日:2017-09-20
发明人: Muhammad Zeeshan Zia , Quoc-Huy Tran , Xiang Yu , Manmohan Chandraker , Chi Li
IPC分类号: G06K9/00 , G06K9/62 , G06F17/50 , G06N3/02 , G06T11/60 , G06T15/40 , G05D1/02 , G08G1/16 , G06T7/73 , G06N3/08 , G06T15/10 , B60W30/00 , G08G1/0962 , G06T7/55 , G06K9/46
摘要: An action recognition system and method are provided. The action recognition system includes an image capture device configured to capture an actual image depicting an object. The action recognition system includes a processor configured to render, based on a set of 3D CAD models, synthetic images with corresponding intermediate shape concept labels. The processor is configured to form a multi-layer CNN which jointly models multiple intermediate shape concepts, based on the rendered synthetic images. The processor is configured to perform an intra-class appearance variation-aware and occlusion-aware 3D object parsing on the actual image by applying the CNN thereto to generate an image pair including a 2D and 3D geometric structure of the object. The processor is configured to control a device to perform a response action in response to an identification of an action performed by the object, wherein the identification of the action is based on the image pair.
-
9.
公开(公告)号:US20200058156A1
公开(公告)日:2020-02-20
申请号:US16526306
申请日:2019-07-30
发明人: Quoc-Huy Tran , Mohammed E. Fathy Salem , Muhammad Zeeshan Zia , Paul Vernaza , Manmohan Chandraker
摘要: A method for estimating dense 3D geometric correspondences between two input point clouds by employing a 3D convolutional neural network (CNN) architecture is presented. The method includes, during a training phase, transforming the two input point clouds into truncated distance function voxel grid representations, feeding the truncated distance function voxel grid representations into individual feature extraction layers with tied weights, extracting low-level features from a first feature extraction layer, extracting high-level features from a second feature extraction layer, normalizing the extracted low-level features and high-level features, and applying deep supervision of multiple contrastive losses and multiple hard negative mining modules at the first and second feature extraction layers. The method further includes, during a testing phase, employing the high-level features capturing high-level semantic information to obtain coarse matching locations, and refining the coarse matching locations with the low-level features to capture low-level geometric information for estimating precise matching locations.
-
10.
公开(公告)号:US10289936B2
公开(公告)日:2019-05-14
申请号:US15709849
申请日:2017-09-20
发明人: Muhammad Zeeshan Zia , Quoc-Huy Tran , Xiang Yu , Manmohan Chandraker , Chi Li
IPC分类号: G06K9/00 , G06K9/62 , G06F17/50 , G06N3/02 , G06T11/60 , G06T15/40 , G05D1/02 , G08G1/16 , G06T7/73 , G06N3/08 , G06T15/10 , B60W30/00 , G08G1/0962 , H04N7/00 , G06T7/55 , G06K9/46
摘要: A surveillance system and method are provided. The surveillance system includes an image capture device configured to capture an actual image of a target area depicting an object. The surveillance system further includes a processor. The processor is configured to render, based on a set of 3D Computer Aided Design (CAD) models, synthetic images with intermediate shape corresponding concept labels. The processor is further configured to form a multi-layer Convolutional Neural Network (CNN) which jointly models multiple intermediate shape concepts, based on the rendered synthetic images. The processor is also configured to perform an intra-class appearance variation-aware and occlusion-aware 3D object parsing on the actual image by applying the CNN to the actual image to generate an image pair including a 2D and 3D geometric structure of the object depicted in the actual image. The surveillance system further includes a display device configured to display the image pair.
-
-
-
-
-
-
-
-
-