METHOD AND SYSTEM FOR VISIO-LINGUISTIC UNDERSTANDING USING CONTEXTUAL LANGUAGE MODEL REASONERS

    公开(公告)号:EP3926531A1

    公开(公告)日:2021-12-22

    申请号:EP21179781.6

    申请日:2021-06-16

    IPC分类号: G06K9/00 G06K9/32 G06K9/62

    摘要: This disclosure relates generally to visio-linguistic understanding. Conventional methods use contextual visio-linguistic reasoner for visio-linguistic understanding which requires more compute power and large amount of pre-training data. Embodiments of the present disclosure provide a method for visio-linguistic understanding using contextual language model reasoner. The method converts the visual information of an input image into a format that the contextual language model reasoner understands and accepts for a downstream task. The method utilizes the image captions and confidence score associated with the image captions along with a knowledge graph to obtain a combined input in a format compatible with the contextual language model reasoner. Contextual embeddings corresponding to the downstream task is obtained using the combined input. The disclosed method is used to solve several downstream tasks such as scene understanding, visual question answering, visual common-sense reasoning and so on.

    VEHICLE SYSTEM WITH A SAFETY MECHANISM AND METHOD OF OPERATION THEREOF

    公开(公告)号:EP3905121A1

    公开(公告)日:2021-11-03

    申请号:EP21170945.6

    申请日:2021-04-28

    申请人: MOJ.IO Inc.

    发明人: Messer, Alan

    IPC分类号: G06K9/00 G06K9/32

    摘要: A method of operation of a vehicle system includes receiving an image from a visual sensor, identifying an item of interest based on the image, generating a bounding box around the item of interest, categorizing a target object based on the item of interest within the bounding box, calculating a distance based on a width of the bounding box, and communicating the distance for assisting in operation of the vehicle.