Dual-modality relation networks for audio-visual event localization

    公开(公告)号:US11663823B2

    公开(公告)日:2023-05-30

    申请号:US16989387

    申请日:2020-08-10

    摘要: Dual-modality relation networks for audio-visual event localization can be provided. A video feed for audio-visual event localization can be received. Based on a combination of extracted audio features and video features of the video feed, informative features and regions in the video feed can be determined by running a first neural network. Based on the informative features and regions in the video feed determined by the first neural network, relation-aware video features can be determined by running a second neural network. Based on the informative features and regions in the video feed, relation-aware audio features can be determined by running a third neural network. A dual-modality representation can be obtained based on the relation-aware video features and the relation-aware audio features by running a fourth neural network. The dual-modality representation can be input to a classifier to identity an audio-visual event in the video feed.

    THERMAL AND PERFORMANCE MANAGEMENT
    3.
    发明公开

    公开(公告)号:US20240004443A1

    公开(公告)日:2024-01-04

    申请号:US17852699

    申请日:2022-06-29

    IPC分类号: G06F1/26 G06N20/00

    CPC分类号: G06F1/26 G06N20/00

    摘要: Described aspects include a system for optimizing performance of a functional circuit unit, a method of optimizing performance of a functional circuit unit, and a computer program product. In one embodiment, the system may include a functional circuit unit having an associated cooling device and power converter, one or more sensors for the functional circuit unit, the one or more sensors including a power sensor and a temperature sensor, and a first machine learning model. The first machine learning model may be adapted to receive temperature data and power data from the one or more sensors, and to generate control signals for the cooling device and the power converter to optimize performance of the functional circuit unit.

    MULTI-MODAL DEEP LEARNING BASED SURROGATE MODEL FOR HIGH-FIDELITY SIMULATION

    公开(公告)号:US20210279386A1

    公开(公告)日:2021-09-09

    申请号:US16810687

    申请日:2020-03-05

    IPC分类号: G06F30/27 G06F30/28

    摘要: A method of using multiple artificial intelligence models for generating a high fidelity simulation includes generating, by a computing device, multiple artificial intelligence models. Each artificial intelligence model simulating an industry design process. The computing device further fusing the multiple artificial intelligence models to generate a best-fit proposed industry design process. The computing device utilizes a physics constraint model to determine whether the best-fit proposed industry design process is feasible. The best-fit proposed industry design process is displayed in response to determining that the best-fit proposed industry design process is feasible.

    VIDEO RESPONSE GENERATION AND MODIFICATION

    公开(公告)号:US20210133236A1

    公开(公告)日:2021-05-06

    申请号:US16674402

    申请日:2019-11-05

    摘要: A method, system, and program product for generating and modifying a video response is provided. The method includes receiving an audio/video file. Parsed video features of the audio/video file are generated with respect to a first graph. Parsed audio features of the audio/video file are generated with respect to a second graph. The first graph is placed overlaying the second graph and at least one intersection point between the first graph and the second graph is determined. A natural language query is executed with respect to the audio/video file and a parsed query entity is generated from the natural language query. The parsed query entity is analyzed with respect to the intersection point and a node of the intersection point comprising similar features is determined with respect to the parsed query entity. A resulting natural language response with respect to the natural language query is generated.

    Learning neuro-symbolic multi-hop reasoning rules over text

    公开(公告)号:US11645526B2

    公开(公告)日:2023-05-09

    申请号:US16911645

    申请日:2020-06-25

    IPC分类号: G06N3/08 G06N3/044

    CPC分类号: G06N3/08 G06N3/044

    摘要: A method and a system for learning and applying neuro-symbolic multi-hop rules are provided. The method includes inputting training texts into a neural network as well as pre-defined entities. The training texts and the entities relate to a specific domain. The method also includes generating an entity graph made up of nodes and edges. The nodes represent the pre-defined entities, and the edges represent passages in the training texts with co-occurrence of the entities connected together by the edges. The method further includes determining a relation based on the passages for each of the pre-defined entities connected together by the edges, calculating a probability relating to the relation, generating a potential reasoning path between a head entity and a target entity. The method also includes learning a neuro-symbolic rule by converting the edges along the potential reasoning path into symbolic rules and combining those rules into the neuro-symbolic rule.

    Image retrieval using interactive natural language dialog

    公开(公告)号:US10977303B2

    公开(公告)日:2021-04-13

    申请号:US15927309

    申请日:2018-03-21

    摘要: A search engine is modified to perform increasingly precise image searching using iterative Natural Language (NL) interactions. From an NL search input, the modification extracts a set of input features, which includes a set of response features corresponding to an NL statement in the NL search input and a set of image features from a seed image in the NL search input. The modification performs image analysis on an image result in a result set of a query including at least some of the input features. In a next iteration of NL interactions, at least some of the result set is provided. An NL response in the iteration is added to a cumulative NL basis, and a revised result set is provided, which includes a new image result corresponding to a new response feature extracted from the cumulative NL basis.

    IMAGE RETRIEVAL USING INTERACTIVE NATURAL LANGUAGE DIALOG

    公开(公告)号:US20190294702A1

    公开(公告)日:2019-09-26

    申请号:US15927309

    申请日:2018-03-21

    IPC分类号: G06F17/30

    摘要: A search engine is modified to perform increasingly precise image searching using iterative Natural Language (NL) interactions. From an NL search input, the modification extracts a set of input features, which includes a set of response features corresponding to an NL statement in the NL search input and a set of image features from a seed image in the NL search input. The modification performs image analysis on an image result in a result set of a query including at least some of the input features. In a next iteration of NL interactions, at least some of the result set is provided. An NL response in the iteration is added to a cumulative NL basis, and a revised result set is provided, which includes a new image result corresponding to a new response feature extracted from the cumulative NL basis.