-
公开(公告)号:US20190138811A1
公开(公告)日:2019-05-09
申请号:US16111298
申请日:2018-08-24
Applicant: NEC Laboratories America, Inc.
Inventor: Kihyuk Sohn , Manmohan Chandraker , Xiang Yu
Abstract: A computer-implemented method, system, and computer program product are provided for activity recognition. The method includes receiving, by a processor, a plurality of videos, the plurality of videos including labeled videos and unlabeled videos. The method also includes extracting, by the processor with a feature extraction convolutional neural network (CNN), frame features for frames from each of the plurality of videos. The method additionally includes estimating, by the processor with a feature aggregation system, a vector representation for one of the plurality of videos responsive to the frame features. The method further includes classifying, by the processor, an activity from the vector representation. The method also includes controlling an operation of a processor-based machine to react in accordance with the activity.
-
62.
公开(公告)号:US20180268203A1
公开(公告)日:2018-09-20
申请号:US15889913
申请日:2018-02-06
Applicant: NEC Laboratories America, Inc.
Inventor: Kihyuk Sohn , Xiang Yu , Manmohan Chandraker
CPC classification number: G06K9/66 , G06K9/00268 , G06K9/00288 , G06K9/00718 , G06K9/00744 , G06K9/00771 , G06K9/4628 , G06K9/6201 , G06K9/6217 , G06K9/6262 , G06K9/6274 , G06K2009/00738 , G06N3/02 , G06N3/0454 , G06N3/08 , G06N3/088 , G06N20/00 , G06T7/70 , G06T9/002 , G06T2207/20081 , G08B13/196 , G08B13/19613
Abstract: A face recognition system is provided that includes a device configured to capture a video sequence formed from a set of unlabeled testing video frames. The system includes a processor configured to pre-train a face recognition engine formed from reference CNNs on a still image domain that includes labeled training still image frames of faces. The processor adapts the face recognition engine to a video domain to form an adapted engine, by applying non-reference CNNs to domains including the still image and video domains and a degraded image domain. The degraded image domain includes labeled synthetically degraded versions of the frames included in the still image domain. The video domain includes random unlabeled training video frames. The processor recognizes, using the adapted engine, identities of persons corresponding to at least one face in the video sequence to obtain a set of identities. A display device displays the set of identities.
-
公开(公告)号:US20180268055A1
公开(公告)日:2018-09-20
申请号:US15888693
申请日:2018-02-05
Applicant: NEC Laboratories America, Inc.
Inventor: Xiang Yu , Kihyuk Sohn , Manmohan Chandraker
CPC classification number: G06K9/00288 , G06F16/71 , G06F16/743 , G06F16/784 , G06K9/00201 , G06K9/00208 , G06K9/00214 , G06K9/00255 , G06K9/00275 , G06K9/00771 , G06K9/00899 , G06K9/4628 , G06K9/6256 , G06T19/20 , G06T2210/44
Abstract: A video retrieval system is provided that includes a server for retrieving video sequences from a remote database responsive to a text specifying a face recognition result as an identity of a subject of an input image. The face recognition result is determined by a processor of the server, which estimates, using a 3DMM conditioned Generative Adversarial Network, 3DMM coefficients for the subject of the input image. The subject varies from an ideal front pose. The processor produces a synthetic frontal face image of the subject of the input image based on the input image and coefficients. An area spanning the frontal face of the subject is made larger in the synthetic than in the input image. The processor provides a decision of whether the synthetic image subject is an actual person and provides the identity of the subject in the input image based on the synthetic and input images.
-
64.
公开(公告)号:US20180130355A1
公开(公告)日:2018-05-10
申请号:US15709814
申请日:2017-09-20
Applicant: NEC Laboratories America, Inc.
Inventor: Muhammad Zeeshan Zia , Quoc-Huy Tran , Xiang Yu , Manmohan Chandraker , Chi Li
CPC classification number: G06K9/6256 , B60T2201/022 , B60W30/00 , G05D1/0221 , G06F17/5009 , G06K9/00201 , G06K9/00208 , G06K9/00624 , G06K9/00771 , G06K9/00805 , G06K9/4628 , G06K9/6255 , G06N3/02 , G06N3/084 , G06T7/55 , G06T7/74 , G06T11/60 , G06T15/10 , G06T15/40 , G06T2207/20101 , G06T2207/30261 , G06T2210/22 , G08G1/0962 , G08G1/166 , H04N7/00
Abstract: A system and method are provided for driving assistance. The system includes an image capture device configured to capture an actual image relative to an outward view from a motor vehicle and depicting an object. The system further includes a processor configured to render, based on a set of 3D CAD models, synthetic images with corresponding intermediate shape concept labels. The processor is further configured to form a multi-layer CNN which jointly models multiple intermediate shape concepts, based on the rendered synthetic images. The processor is also configured to perform an intra-class appearance variation-aware and occlusion-aware 3D object parsing on the actual image by applying the CNN to the actual image to output an image pair including a 2D and 3D geometric structure of the object. The processor is additionally configured to perform an action to mitigate a likelihood of harm involving the motor vehicle, based on the image pair.
-
65.
公开(公告)号:US20180129865A1
公开(公告)日:2018-05-10
申请号:US15709897
申请日:2017-09-20
Applicant: NEC Laboratories America, Inc.
Inventor: Muhammad Zeeshan Zia , Quoc-Huy Tran , Xiang Yu , Manmohan Chandraker , Chi Li
CPC classification number: G06K9/6256 , B60T2201/022 , B60W30/00 , G05D1/0221 , G06F17/5009 , G06K9/00201 , G06K9/00208 , G06K9/00624 , G06K9/00771 , G06K9/00805 , G06K9/4628 , G06K9/6255 , G06N3/02 , G06N3/0454 , G06N3/082 , G06N3/084 , G06T7/55 , G06T7/74 , G06T11/60 , G06T15/10 , G06T15/40 , G06T2207/20101 , G06T2207/30261 , G06T2210/22 , G08G1/0962 , G08G1/166 , H04N7/00
Abstract: An action recognition system and method are provided. The action recognition system includes an image capture device configured to capture an actual image depicting an object. The action recognition system includes a processor configured to render, based on a set of 3D CAD models, synthetic images with corresponding intermediate shape concept labels. The processor is configured to form a multi-layer CNN which jointly models multiple intermediate shape concepts, based on the rendered synthetic images. The processor is configured to perform an intra-class appearance variation-aware and occlusion-aware 3D object parsing on the actual image by applying the CNN thereto to generate an image pair including a 2D and 3D geometric structure of the object. The processor is configured to control a device to perform a response action in response to an identification of an action performed by the object, wherein the identification of the action is based on the image pair.
-
公开(公告)号:US20180025213A1
公开(公告)日:2018-01-25
申请号:US15637368
申请日:2017-06-29
Applicant: NEC Laboratories America, Inc. , NEC Hong Kong Limited
Inventor: Manmohan Chandraker , Xiang Yu , Eric Lau , Elsa Wong
IPC: G06K9/00
CPC classification number: G06F21/32 , G06F21/6218 , G06F2221/2133 , G06K9/00221 , G06K9/00228 , G06K9/00255 , G06K9/00281 , G06K9/00288 , G06K9/00624 , G06K9/00791 , G06K9/00906 , G06K9/4652 , G06K9/66 , G06N99/005 , G07C9/00158 , G07C9/00166 , H04L63/0861 , H04L63/1483
Abstract: A traffic enforcement system and corresponding method are provided. The traffic enforcement system includes a camera configured to capture an input image of one or more subjects in a motor vehicle. The traffic enforcement system further includes a memory storing a deep learning model configured to perform multi-task learning for a pair of tasks including a liveness detection task and a face recognition task on one or more subjects in a motor vehicle depicted in the input image. The traffic enforcement system also includes a processor configured to apply the deep learning model to the input image to recognize an identity the one or more subjects in the motor vehicle and a liveness of the one or more subjects. The liveness detection task is configured to evaluate a plurality of different distractor modalities corresponding to different physical spoofing materials to prevent face spoofing for the face recognition task.
-
公开(公告)号:US20240303365A1
公开(公告)日:2024-09-12
申请号:US18598198
申请日:2024-03-07
Applicant: NEC Laboratories America, Inc.
Inventor: Francesco Pittaluga , Bingbing Zhuang , Xiang Yu
CPC classification number: G06F21/6227 , G06V10/751
Abstract: Systems and methods are provided for privacy-preserving image feature matching in computer vision applications, including receiving a raw image descriptor, and perturbing the raw image descriptor using a subset selection mechanism to generate a perturbed descriptor set that includes the raw image descriptor and additional descriptors. Each descriptor in the perturbed descriptor set is replaced with its nearest neighbor in a predefined descriptor database to reduce the output domain size of the subset selection mechanism. Local differential privacy (LDP) protocols are employed to further perturb the descriptor set, ensuring formal privacy guarantees, and the perturbed descriptor set is matched against a second set of descriptors for image feature matching.
-
公开(公告)号:US20240160938A1
公开(公告)日:2024-05-16
申请号:US18502488
申请日:2023-11-06
Applicant: NEC Laboratories America, Inc.
Inventor: Masoud Faraki , Xiang Yu , Mateusz Michalkiewicz
IPC: G06N3/084
CPC classification number: G06N3/084
Abstract: Methods and systems of training a model include determining a dropout mask based on gradient signal to noise ratio of parameters of a neural network model. The neural network model is trained with parameters zeroed-out according to the dropout mask. The dropout mask is iteratively updated and the training is performed iteratively based on the updated dropout mask.
-
公开(公告)号:US11947626B2
公开(公告)日:2024-04-02
申请号:US17519950
申请日:2021-11-05
Applicant: NEC Laboratories America, Inc.
Inventor: Masoud Faraki , Xiang Yu , Yi-Hsuan Tsai , Yumin Suh , Manmohan Chandraker
IPC: G06F18/214 , G06N3/04 , G06V40/16
CPC classification number: G06F18/214 , G06N3/04 , G06V40/161
Abstract: A method for improving face recognition from unseen domains by learning semantically meaningful representations is presented. The method includes obtaining face images with associated identities from a plurality of datasets, randomly selecting two datasets of the plurality of datasets to train a model, sampling batch face images and their corresponding labels, sampling triplet samples including one anchor face image, a sample face image from a same identity, and a sample face image from a different identity than that of the one anchor face image, performing a forward pass by using the samples of the selected two datasets, finding representations of the face images by using a backbone convolutional neural network (CNN), generating covariances from the representations of the face images and the backbone CNN, the covariances made in different spaces by using positive pairs and negative pairs, and employing the covariances to compute a cross-domain similarity loss function.
-
公开(公告)号:US20230281963A1
公开(公告)日:2023-09-07
申请号:US18175906
申请日:2023-02-28
Applicant: NEC Laboratories America, Inc.
Inventor: Vijay Kumar Baikampady Gopalkrishna , Xiang Yu , Samuel Schulter
IPC: G06V10/77 , G06V10/82 , G06V10/80 , G06V10/774 , G06V10/776 , G06F40/205 , G06F40/284 , G06F40/30
CPC classification number: G06V10/7715 , G06F40/205 , G06F40/284 , G06F40/30 , G06V10/774 , G06V10/776 , G06V10/806 , G06V10/82
Abstract: A method is provided for pretraining vision and language models that includes receiving image-text pairs, each including an image and a text describing the image. The method encodes an image into a set of feature vectors corresponding to input image patches and a CLS token which represents a global image feature. The method parses, by a text tokenizer, the text into a set of feature vectors as tokens for each word in the text. The method encodes the CLS token from the NN based visual encoder and the tokens from the text tokenizer into a set of features by a NN based text and multimodal encoder that shares weights for encoding both the CLS token and the tokens. The method accumulates the weights from multiple iterations as an exponential moving average of the weights during the pretraining until a predetermined error threshold is reduced to be under a threshold amount.
-
-
-
-
-
-
-
-
-