公开(公告)号：US20180268265A1

公开(公告)日：2018-09-20

申请号：US15889846

申请日：2018-02-06

Applicant: NEC Laboratories America, Inc.

Inventor： Kihyuk Sohn ,  Xiang Yu ,  Manmohan Chandraker

IPC: G06K9/66 ,  G06T7/70 ,  G06T9/00 ,  G06K9/00 ,  G06K9/62

Abstract: An object recognition system is provided that includes a device configured to capture a video sequence formed from unlabeled testing video frames. The system includes a processor configured to pre-train a recognition engine formed from a reference set of CNNs on a still image domain that includes labeled training still image frames. The processor adapts the recognition engine to a video domain to form an adapted recognition engine, by applying a non-reference set of CNNs to a set of domains that include the still image and video domains and a degraded image domain. The degraded image domain includes labeled synthetically degraded versions of the labeled training still image frames included in the still image domain. The video domain includes random unlabeled training video frames. The processor recognizes, using the adapted engine, a set of objects in the video sequence. A display device displays the set of recognized objects.

公开(公告)号：US20180130324A1

公开(公告)日：2018-05-10

申请号：US15803318

申请日：2017-11-03

Applicant: NEC Laboratories America, Inc.

Inventor： Xiang Yu ,  Kihyuk Sohn ,  Manmohan Chandraker

IPC: G08B13/196 ,  G06K9/00 ,  G06N3/04

Abstract: A computer-implemented method, system, and computer program product is provided for video security. The method includes monitoring an area with a camera. The method also includes capturing, by the camera, live video to provide a live video stream. The method additionally includes detecting and identifying, by a processor using a recognition neural network feeding into a Siamese reconstruction network, a user in the live video stream by employing one or more pose-invariant features. The method further includes controlling, by the processor, an operation of a processor-based machine to change a state of the processor-based machine, responsive to the identified user in the live video stream.

公开(公告)号：US09965610B2

公开(公告)日：2018-05-08

申请号：US15637569

申请日：2017-06-29

Applicant: NEC Laboratories America, Inc. ,  NEC Hong Kong Limited

Inventor： Manmohan Chandraker ,  Xiang Yu ,  Eric Lau ,  Elsa Wong

IPC: G06F21/32 ,  H04L29/06 ,  G07C9/00 ,  G06K9/00 ,  G06N99/00

CPC classification number: G06F21/32 ,  G06F21/6218 ,  G06K9/00221 ,  G06K9/00228 ,  G06K9/00255 ,  G06K9/00281 ,  G06K9/00288 ,  G06K9/00624 ,  G06K9/00791 ,  G06K9/00906 ,  G06K9/4652 ,  G06K9/66 ,  G06N99/005 ,  G07C9/00158 ,  G07C9/00166 ,  H04L63/0861 ,  H04L63/1483

Abstract: A machine access control system and corresponding method are provided. The machine access control system includes a camera configured to capture an input image of a subject purported to be a person associated with operating a particular workplace machine. The machine access control system further includes a memory storing a deep learning model configured to perform multi-task learning for a pair of tasks including a liveness detection task and a face recognition task. The machine access control system also includes a processor configured to apply the deep learning model to the input image to recognize an identity of the subject in the input image regarding being authorized to use the particular workplace machine and a liveness of the subject. The liveness detection task is configured to evaluate a plurality of different distractor modalities corresponding to different physical spoofing materials to prevent face spoofing for the face recognition task.

公开(公告)号：US20240152767A1

公开(公告)日：2024-05-09

申请号：US18497079

申请日：2023-10-30

Applicant: NEC Laboratories America, Inc.

Inventor： Vijay Kumar Baikampady Gopalkrishna ,  Samuel Schulter ,  Xiang Yu ,  Zaid Khan ,  Manmohan Chandraker

IPC: G06N3/096 ,  G06N3/006 ,  G06N3/047

CPC classification number: G06N3/096 ,  G06N3/006 ,  G06N3/047

Abstract: Systems and methods for training a visual question answer model include training a teacher model by performing image conditional visual question generation on a visual language model (VLM) and a targeted visual question answer dataset using images to generate question and answer pairs. Unlabeled images are pseudolabeled using the teacher model to decode synthetic question and answer pairs for the unlabeled images. The synthetic question and answer pairs for the unlabeled images are merged with real data from the targeted visual question answer dataset to generate a self-augmented training set. A student model is trained using the VLM and the self-augmented training set to return visual answers to text queries.

公开(公告)号：US11977602B2

公开(公告)日：2024-05-07

申请号：US17521252

申请日：2021-11-08

Applicant: NEC Laboratories America, Inc.

Inventor： Xiang Yu ,  Yi-Hsuan Tsai ,  Masoud Faraki ,  Ramin Moslemi ,  Manmohan Chandraker ,  Chang Liu

IPC: G06K9/00 ,  G06F18/21 ,  G06F18/214 ,  G06N20/00 ,  G06V40/16

CPC classification number: G06F18/214 ,  G06F18/217 ,  G06N20/00 ,  G06V40/172

Abstract: A method for training a model for face recognition is provided. The method forward trains a training batch of samples to form a face recognition model w(t), and calculates sample weights for the batch. The method obtains a training batch gradient with respect to model weights thereof and updates, using the gradient, the model w(t) to a face recognition model what(t). The method forwards a validation batch of samples to the face recognition model what(t). The method obtains a validation batch gradient, and updates, using the validation batch gradient and what(t), a sample-level importance weight of samples in the training batch to obtain an updated sample-level importance weight. The method obtains a training batch upgraded gradient based on the updated sample-level importance weight of the training batch samples, and updates, using the upgraded gradient, the model w(t) to a trained model w(t+1) corresponding to a next iteration.

公开(公告)号：US11600113B2

公开(公告)日：2023-03-07

申请号：US17091066

申请日：2020-11-06

Applicant: NEC Laboratories America, Inc.

Inventor： Xiang Yu ,  Manmohan Chandraker ,  Kihyuk Sohn ,  Aruni RoyChowdhury

IPC: G06V40/16 ,  G06K9/62 ,  G06N3/04 ,  G06N3/08

Abstract: A computer-implemented method for implementing face recognition includes obtaining a face recognition model trained on labeled face data, separating, using a mixture of probability distributions, a plurality of unlabeled faces corresponding to unlabeled face data into a set of one or more overlapping unlabeled faces that include overlapping identities to those in the labeled face data and a set of one or more disjoint unlabeled faces that include disjoint identities to those in the labeled face data, clustering the one or more disjoint unlabeled faces using a graph convolutional network to generate one or more cluster assignments, generating a clustering uncertainty associated with the one or more cluster assignments, and retraining the face recognition model on the labeled face data and the unlabeled face data to improve face recognition performance by incorporating the clustering uncertainty.

公开(公告)号：US11580780B2

公开(公告)日：2023-02-14

申请号：US17091011

申请日：2020-11-06

Applicant: NEC Laboratories America, Inc.

Inventor： Xiang Yu ,  Manmohan Chandraker ,  Kihyuk Sohn ,  Yichun Shi

IPC: G06N20/00 ,  G06K9/62 ,  G06V40/16

Abstract: A computer-implemented method for implementing face recognition includes receiving training data including a plurality of augmented images each corresponding to a respective one of a plurality of input images augmented by one of a plurality of variations, splitting a feature embedding generated from the training data into a plurality of sub-embeddings each associated with one of the plurality of variations, associating each of the plurality of sub-embeddings with respective ones of a plurality of confidence values, and applying a plurality of losses including a confidence-aware identification loss and a variation-decorrelation loss to the plurality of sub-embeddings and the plurality of confidence values to improve face recognition performance by learning the plurality of sub-embeddings.

公开(公告)号：US20220147767A1

公开(公告)日：2022-05-12

申请号：US17521252

申请日：2021-11-08

Applicant: NEC Laboratories America, Inc.

Inventor： Xiang Yu ,  Yi-Hsuan Tsai ,  Masoud Faraki ,  Ramin Moslemi ,  Manmohan Chandraker ,  Chang Liu

IPC: G06K9/62 ,  G06N20/00 ,  G06K9/00

Abstract: A method for training a model for face recognition is provided. The method forward trains a training batch of samples to form a face recognition model w(t), and calculates sample weights for the batch. The method obtains a training batch gradient with respect to model weights thereof and updates, using the gradient, the model w(t) to a face recognition model what(t). The method forwards a validation batch of samples to the face recognition model what(t). The method obtains a validation batch gradient, and updates, using the validation batch gradient and what(t), a sample-level importance weight of samples in the training batch to obtain an updated sample-level importance weight. The method obtains a training batch upgraded gradient based on the updated sample-level importance weight of the training batch samples, and updates, using the upgraded gradient, the model w(t) to a trained model w(t+1) corresponding to a next iteration.

公开(公告)号：US20220121953A1

公开(公告)日：2022-04-21

申请号：US17496214

申请日：2021-10-07

Applicant: NEC Laboratories America, Inc.

Inventor： Yumin Suh ,  Xiang Yu ,  Masoud Faraki ,  Manmohan Chandraker ,  Weijian Deng

IPC: G06N3/08 ,  G06N3/04

Abstract: A method for multi-task learning via gradient split for rich human analysis is presented. The method includes extracting images from training data having a plurality of datasets, each dataset associated with one task, feeding the training data into a neural network model including a feature extractor and task-specific heads, wherein the feature extractor has a feature extractor shared component and a feature extractor task-specific component, dividing filters of deeper layers of convolutional layers of the feature extractor into N groups, N being a number of tasks, assigning one task to each group of the N groups, and manipulating gradients so that each task loss updates only one subset of filters.

公开(公告)号：US10572777B2

公开(公告)日：2020-02-25

申请号：US15436199

申请日：2017-02-17

Applicant: NEC Laboratories America, Inc.

Inventor： Xiang Yu ,  Feng Zhou ,  Manmohan Chandrakar

IPC: G06K9/62 ,  G06K9/00 ,  G06K9/46 ,  G06T7/73 ,  G06N3/04 ,  G06T7/60 ,  G06N3/08

Abstract: A system and method are provided. The system includes a processor. The processor is configured to generate a response map for an image, using a four stage convolutional structure. The processor is further configured to generate a plurality of landmark points for the image based on the response map, using a shape basis neural network. The processor is additionally configured to generate an optimal shape for the image based on the plurality of landmark points for the image and the response map, using a point deformation neural network. A recognition system configured to identify the image based on the generated optimal shape to generate a recognition result of the image. The processor is also configured to operate a hardware-based machine based on the recognition result.