公开(公告)号：US12062186B2

公开(公告)日：2024-08-13

申请号：US17496403

申请日：2021-10-07

Applicant: SRI International

Inventor： Han-Pang Chiu ,  Junjiao Tian ,  Zachary Seymour ,  Niluthpol C. Mithun ,  Alex Krasner ,  Mikhail Sizintsev ,  Abhinav Rajvanshi ,  Kevin Kaighn ,  Philip Miller ,  Ryan Villamil ,  Supun Samarasekera

IPC: G06T7/174 ,  G06T3/40 ,  G06T7/38 ,  G06T7/50

CPC classification number: G06T7/174 ,  G06T3/40 ,  G06T7/38 ,  G06T7/50 ,  G06T2207/10016 ,  G06T2207/10024 ,  G06T2207/20112

Abstract: A method, machine readable medium and system for RGBD semantic segmentation of video data includes determining semantic segmentation data and depth segmentation data for less than all classes for images of each frame of a first video, determining semantic segmentation data and depth segmentation data for images of each key frame of a second video including a synchronous combination of respective frames of the RGB video and the depth-aware video in parallel to the determination of the semantic segmentation data and the depth segmentation data for each frame of the first video, temporally and geometrically aligning respective frames of the first video and the second video, and predicting semantic segmentation data and depth segmentation data for images of a subsequent frame of the first video based on the determination of the semantic segmentation data and depth segmentation data for images of a key frame of the second video.

公开(公告)号：US20220092366A1

公开(公告)日：2022-03-24

申请号：US17478177

申请日：2021-09-17

Applicant: SRI International

Inventor： Han-Pang Chiu ,  Junjiao Tian ,  Zachary Seymour ,  Niluthpol C. Mithun

IPC: G06K9/62 ,  G06N20/00

Abstract: Techniques are disclosed for an image understanding system comprising a machine learning system that applies a machine learning model to perform image understanding of each pixel of an image, the pixel labeled with a class, to determine an estimated class to which the pixel belongs. The machine learning system determines, based on the classes with which the pixels are labeled and the estimated classes, a cross entropy loss of each class. The machine learning system determines, based on one or more region metrics, a weight for each class and applies the weight to the cross entropy loss of each class to obtain a weighted cross entropy loss. The machine learning system updates the machine learning model with the weighted cross entropy loss to improve a performance metric of the machine learning model for each class.