System for sensor synchronization data analysis in an autonomous driving vehicle

    公开(公告)号:US11136048B2

    公开(公告)日:2021-10-05

    申请号:US16485423

    申请日:2019-07-22

    IPC分类号: B60W60/00 G06K9/00

    摘要: The disclosure describes various embodiments for online system-level validation of sensor synchronization. According to an embodiment, an exemplary method of analyzing sensor synchronization in an autonomous driving vehicle (ADV) include the operations of acquiring raw sensor data from a first sensor and a second sensor mounted on the ADV, the raw sensor data describing a target object in a surrounding environment of the ADV; and generating an accuracy map based on the raw sensor data in view of timestamps extracted from the raw sensor data. The method further includes the operations of generating a first bounding box and a second bounding box around the target object using the raw sensor data; and performing an analysis of the first and second bounding boxes and the accuracy map using a predetermined algorithm in view of one or more pre-configured sensor settings to determine whether the first sensor and the second sensor are synchronized with each other.

    Timestamp and metadata processing for video compression in autonomous driving vehicles

    公开(公告)号:US11245909B2

    公开(公告)日:2022-02-08

    申请号:US16461350

    申请日:2019-04-29

    IPC分类号: H04N7/12 H04N19/172 H04N19/46

    摘要: A method to perform video compression for ADV is disclosed. The method receives multiple frames of image data from multiple cameras. Metadata are appended to each frame of the image data to generate one of multiple frames of uncompressed image data as the image data are received. The frames of uncompressed image data may be stored. To compress the image data later, the method retrieves the frames of uncompressed image data, extracts the metadata from each frame of the uncompressed image data to generate one of multiple frames of processed image data. The method compresses each frame of the processed image data with the metadata extracted to generate one of multiple frames of compressed image data. The method reattaches the metadata to a corresponding frame of the compressed image data to generate one of multiple compressed image frames. The metadata supports time synchronization and error handling of the image data.

    Flexible hardware design for camera calibration and image pre-procesing in autonomous driving vehicles

    公开(公告)号:US11182623B2

    公开(公告)日:2021-11-23

    申请号:US16399457

    申请日:2019-04-30

    申请人: Baidu USA LLC

    摘要: A sensor unit includes a sensor interface, host interface, and pre-processing hardware. The sensor interface is coupled to a plurality of cameras configured to capture images around an autonomous driving vehicle (ADV). The host interface is coupled to a perception and planning system. The pre-processing hardware is coupled to the sensor interface to receive images from the plurality of cameras and to perform one or more pre-processing functions on the images and to transmit pre-processed images to the perception and planning system via the host interface. The perception and planning system is configured to perceive a driving environment surrounding the ADV based on the pre-processed images and to plan a path to control the ADV to navigate through the driving environment. The pre-processing functions can adjust for different calibrations and formats across the plurality of cameras.

    Trigger logic to trigger sensors of an autonomous driving vehicle for capturing data

    公开(公告)号:US10890914B2

    公开(公告)日:2021-01-12

    申请号:US16112349

    申请日:2018-08-24

    申请人: Baidu USA LLC

    摘要: A sensor unit includes a sensor interface coupled to a number of sensors and a host interface coupled to a host system utilized to autonomously drive the vehicle. The sensor unit further includes sensor control modules corresponding to the sensors. Each sensor control module includes delay time control logic, delay adjustment logic, and a trigger signal generator. The delay time control logic is to receive a pulse time adjustment (PTA) value from the host system. The delay adjustment logic is to receive a trigger time adjustment (TTA) value from the host system. The delay adjustment logic is to modify timing of at least a portion of the pulses of a pulse signal based on the PTA value and the TTA value. The trigger signal generator is to generate a trigger signal based on the modified pulse signal and to transmit the trigger signal to a corresponding sensor.

    METHOD AND SYSTEM FOR DESIGNING FPGA BASED ON HARDWARE REQUIREMENTS DEFINED IN SOURCE CODE

    公开(公告)号:US20170323045A1

    公开(公告)日:2017-11-09

    申请号:US15148478

    申请日:2016-05-06

    申请人: Baidu USA LLC

    IPC分类号: G06F17/50

    CPC分类号: G06F17/5054

    摘要: According to one embodiment, a source code is parsed to identify a first routine to perform a first function and a second routine to perform a second function. A control signaling topology is determined between the first routine and the second routine based on one or more statements associated with the first routine and the second routine defined in the source code. A first logic block is allocated describing a first hardware configuration representing the first function of the first routine. A second logic block is allocated describing a second hardware configuration representing the second function of the second routine. A register-transfer level (RTL) netlist is generated based on the first logic block and the second logic block. The second logic block is to perform the second function dependent upon the first function performed by the first logic block based on the control signaling topology.

    Automated factory testflow of processing unit with sensor integration for driving platform

    公开(公告)号:US11198444B2

    公开(公告)日:2021-12-14

    申请号:US16510124

    申请日:2019-07-12

    申请人: Baidu USA LLC

    IPC分类号: B60W50/04 H04N17/00 G07C5/08

    摘要: Diagnosing a sensor processing unit of an autonomous driving vehicle is described. An example computer-implemented method can include transmitting an executable image of a sensor processing application from a host system to the sensor processing unit via at least one of a universal asynchronous receiver-transmitter (UART) or an Ethernet connection. The method also includes causing the sensor processing unit to execute and launch the executable image of the sensor processing application in the DRAM from the eMMC storage device. The method also includes transmitting a sequence of predetermined commands to the executed sensor processing application to perform a plurality of sensor data processing operations on sensor data obtained from a plurality of sensors or sensor simulators associated with an autonomous driving vehicle. The method also includes comparing processing results of the sensor processing operations against expected processing results to determine whether the sensor processing application operates properly.

    Image data acquisition logic of an autonomous driving vehicle for capturing image data using cameras

    公开(公告)号:US10816979B2

    公开(公告)日:2020-10-27

    申请号:US16112370

    申请日:2018-08-24

    申请人: Baidu USA LLC

    摘要: A sensor unit used in an ADV includes a sensor interface that can be coupled to cameras mounted on the ADV. The sensor unit further includes a host interface that can be coupled to a host system. The host system is configured to perceive a driving environment surrounding the ADV based on at least image data obtained from the cameras and to plan a path to autonomously drive the ADV. The sensor unit further includes one or more data acquisition modules corresponding to the cameras. Each data acquisition module includes a pixel alignment module and a frame processing module. The pixel alignment module is configured to reformat pixels of image data from an original format associated with the corresponding camera to a predetermined format. The frame processing module is configured to generate an image frame based on the image data and to transmit the image frame to the host system.