公开(公告)号：US11645497B2

公开(公告)日：2023-05-09

申请号：US16683398

申请日：2019-11-14

Applicant: L'Oreal

Inventor： Eric Elmoznino ,  He Ma ,  Irina Kezele ,  Edmund Phung ,  Alex Levinshtein ,  Parham Aarabi

IPC: G06N3/045 ,  G06T19/00 ,  G06N20/00 ,  G06F3/01 ,  G06N3/08 ,  G06N3/047

CPC classification number: G06N3/045 ,  G06F3/011 ,  G06N3/047 ,  G06N3/08 ,  G06N20/00 ,  G06T19/006 ,  G06T2207/20081

Abstract: Systems and methods relate to a network model to apply an effect to an image such as an augmented reality effect (e.g. makeup, hair, nail, etc.). The network model uses a conditional cycle-consistent generative image-to-image translation model to translate images from a first domain space where the effect is not applied and to a second continuous domain space where the effect is applied. In order to render arbitrary effects (e.g. lipsticks) not seen at training time, the effect's space is represented as a continuous domain (e.g. a conditional variable vector) learned by encoding simple swatch images of the effect, such as are available as product swatches, as well as a null effect. The model is trained end-to-end in an unsupervised fashion. To condition a generator of the model, convolutional conditional batch normalization (CCBN) is used to apply the vector encoding the reference swatch images that represent the makeup properties.

公开(公告)号：US20220284688A1

公开(公告)日：2022-09-08

申请号：US17685691

申请日：2022-03-03

Applicant: L'OREAL

Inventor： Zihao CHEN ,  Zhi Yu ,  Parham Aarabi

IPC: G06V10/24 ,  G06V10/82 ,  G06V10/70 ,  G06V10/776 ,  G06V40/16 ,  G06Q30/06

Abstract: With Convolutional Neural Networks (CNN), facial alignment networks (FAN) have achieved significant accuracy on a wide range of public datasets, which comes along with larger model size and expensive computation costs, making it infeasible to adapt them to real-time applications on edge devices. There is provided a model compression approach on FAN using One-Shot Neural Architecture Search to overcome this problem while preserving performance criteria. Methods and devices provide efficient training and searching (on a single GPU), and resultant models can deploy to run real-time in browser-based applications on edge devices including tablets and smartphones. The compressed models provide comparable cutting-edge accuracy, while having a 30 times smaller model size and can run 40.7 ms per frame in a popular browser on a popular smartphone and OS.

公开(公告)号：US10956009B2

公开(公告)日：2021-03-23

申请号：US16141245

申请日：2018-09-25

Applicant: L'OREAL

Inventor： Parham Aarabi

IPC: G06Q30/00 ,  G06F3/0484

Abstract: Provided is a method and system of providing a cosmetics enhancement interface. The method comprises showing, at the display screen of a computing device having a memory and a processor: a digital photograph including facial features; an interactive dialog portion reflecting a conversational input received and a subsequent response provided thereto from the computing device; and a product display portion; receiving an inquiry, as reflected in the interactive dialog portion, related to a cosmetic product for application onto a selected facial feature; receiving a selection of the cosmetic product based on a matching to the at least one facial feature according to a predefined rule; displaying, at the product display portion, a product representation associated with the selected cosmetic product; receiving an update request; and updating the digital photograph showing a modification to the facial feature on the display screen by simulating application of the selected cosmetic product thereon.

公开(公告)号：US11832958B2

公开(公告)日：2023-12-05

申请号：US18080331

申请日：2022-12-13

Applicant: L'OREAL

Inventor： Ruowei Jiang ,  Junwei Ma ,  He Ma ,  Eric Elmoznino ,  Irina Kezele ,  Alex Levinshtein ,  Julien Despois ,  Matthieu Perrot ,  Frederic Antoinin Raymond Serge Flament ,  Parham Aarabi

IPC: A61B5/00 ,  G06T7/00 ,  G06V40/16 ,  G06N3/08 ,  G06V10/82 ,  G06N3/045 ,  G06V10/44 ,  G06V40/18

CPC classification number: A61B5/441 ,  G06N3/045 ,  G06N3/08 ,  G06T7/0012 ,  G06V10/454 ,  G06V10/82 ,  G06V40/171 ,  G06T2207/30088 ,  G06V40/174 ,  G06V40/18

Abstract: There is shown and described a deep learning based system and method for skin diagnostics as well as testing metrics that show that such a deep learning based system outperforms human experts on the task of apparent skin diagnostics. Also shown and described is a system and method of monitoring a skin treatment regime using a deep learning based system and method for skin diagnostics.

公开(公告)号：US20220198830A1

公开(公告)日：2022-06-23

申请号：US17558955

申请日：2021-12-22

Applicant: L'Oreal

Inventor： Zeqi LI ,  Ruowei Jiang ,  Parham Aarabi

IPC: G06V40/16 ,  G06V10/77 ,  G06T11/00 ,  G06Q30/06

Abstract: There is provided methods, devices and techniques to process an image using a deep learning model to achieve continuous effect simulation by a unified network where a simple (effect class) estimator is embedded into a regular encoder-decoder architecture. The estimator allows learning of model-estimated class embeddings of all effect classes (e.g. progressive degrees of the effect), thus representing the continuous effect information without manual efforts in selecting proper anchor effect groups. In an embodiment, given a target age class, there is derived a personalized age embedding which considers two aspects of face aging: 1) a personalized residual age embedding at a model-estimated age of the subject, preserving the subject's aging information; and 2) exemplar-face aging basis at the target age, encoding the shared aging patterns among the entire population. Training and runtime (inference time) embodiments are described including an AR application that generates recommendations and provides ecommerce services.

公开(公告)号：US10713704B2

公开(公告)日：2020-07-14

申请号：US15798838

申请日：2017-10-31

Applicant: L'OREAL

Inventor： Parham Aarabi

IPC: G06Q30/00 ,  G06Q30/06 ,  G06Q30/02

Abstract: A system and method compute, store and use relativity measures between events in datasets where the measures are stored in a relational memory for querying. From user data and e-commerce shopping session data, relativity measures are computed for a plurality of subsets of data attributes of the user data and session data, each subset comprising two or more data attributes. The relativity measures individually or when combined represent conditional relativities between a set of events within the session data. Only the relativity measures are stored to the relational memory. The measures may be queried for results and applied to the e-commerce service (e.g. to determine which specific product data to present or an order of the presentation of the specific product data). The relativity measures may be computed only for pre-selected relations between particular data attributes which give desired trends and insights into user shopping using the e-commerce service.

公开(公告)号：US12105773B2

公开(公告)日：2024-10-01

申请号：US17361779

申请日：2021-06-29

Applicant: L'Oreal

Inventor： Zeqi Li ,  Ruowei Jiang ,  Parham Aarabi

IPC: G06N5/02 ,  G06F18/22 ,  G06N3/04 ,  G06Q30/0601 ,  G06T19/00

CPC classification number: G06F18/22 ,  G06N3/04 ,  G06N5/02 ,  G06Q30/0631 ,  G06Q30/0643 ,  G06T19/006

Abstract: GANs based generators are useful to perform image to image translations. GANs models have large storage sizes and resource use requirements such that they are too large to be deployed directly on mobile devices. Systems and methods define through conditioning a student GANs model having a student generator that is scaled downwardly from a teacher GANs model (and generator) using knowledge distillation. A semantic relation knowledge distillation loss is used to transfer semantic knowledge from an intermediate layer of the teacher to an intermediate layer of the student. Student generators thus defined are stored and executed by mobile devices such as smartphones and laptops to provide augmented reality experiences. Effects are simulated on images, including makeup, hair, nail and age simulation effects.

公开(公告)号：US11995703B2

公开(公告)日：2024-05-28

申请号：US18102139

申请日：2023-01-27

Applicant: L'OREAL

Inventor： Eric Elmoznino ,  Irina Kezele ,  Parham Aarabi

IPC: G06T5/50 ,  G06F18/214 ,  G06N20/00 ,  G06Q30/0601 ,  G06V10/764 ,  G06V10/774 ,  G06V10/778

CPC classification number: G06Q30/0631 ,  G06F18/214 ,  G06N20/00 ,  G06T5/50 ,  G06V10/764 ,  G06V10/774 ,  G06V10/7788

Abstract: Techniques are provided for computing systems, methods and computer program products to produce efficient image-to-image translation by adapting unpaired datasets for supervised learning. A first model (a powerful model) may be defined and conditioned using unsupervised learning to produce a synthetic paired dataset from the unpaired dataset, translating images from a first domain to a second domain and images from the second domain to the first domain. The synthetic data generated is useful as ground truths in supervised learning. The first model may be conditioned to overfit the unpaired dataset to enhance the quality of the paired dataset (e.g. the synthetic data generated). A run-time model such as for a target device is trained using the synthetic paired dataset and supervised learning. The run-time model is small and fast to meet the processing resources of the target device (e.g. a personal user device such as a smart phone, tablet, etc.)

公开(公告)号：US11861497B2

公开(公告)日：2024-01-02

申请号：US17565581

申请日：2021-12-30

Applicant: L'OREAL

Inventor： Alex Levinshtein ,  Cheng Chang ,  Edmund Phung ,  Irina Kezele ,  Wenzhangzhi Guo ,  Eric Elmoznino ,  Ruowei Jiang ,  Parham Aarabi

IPC: G06N3/08 ,  G06T7/11 ,  G06T7/90 ,  G06T1/20 ,  G06T11/00 ,  G06V10/44 ,  G06V40/16 ,  G06F18/21 ,  G06F18/24 ,  G06V10/82

CPC classification number: G06N3/08 ,  G06F18/21 ,  G06F18/24 ,  G06T1/20 ,  G06T7/11 ,  G06T7/90 ,  G06T11/001 ,  G06V10/454 ,  G06V10/82 ,  G06V40/165 ,  G06V40/171 ,  G06T2200/24 ,  G06T2207/10016 ,  G06T2207/10024 ,  G06T2207/20081 ,  G06T2207/20084

Abstract: A system and method implement deep learning on a mobile device to provide a convolutional neural network (CNN) for real time processing of video, for example, to color hair. Images are processed using the CNN to define a respective hair matte of hair pixels. The respective object mattes may be used to determine which pixels to adjust when adjusting pixel values such as to change color, lighting, texture, etc. The CNN may comprise a (pre-trained) network for image classification adapted to produce the segmentation mask. The CNN may be trained for image segmentation (e.g. using coarse segmentation data) to minimize a mask-image gradient consistency loss. The CNN may further use skip connections between corresponding layers of an encoder stage and a decoder stage where shallower layers in the encoder, which contain high-res but weak features are combined with low resolution but powerful features from deeper decoder layers.

公开(公告)号：US11748888B2

公开(公告)日：2023-09-05

申请号：US17096778

申请日：2020-11-12

Applicant: L'Oreal

Inventor： Abdalla Ahmed ,  Irina Kezele ,  Parham Aarabi ,  Brendan Duke

IPC: G06T7/11 ,  G06T7/174 ,  G06T7/20 ,  G06T9/00 ,  G06N3/08 ,  G06V20/40 ,  G06V40/12 ,  G06V10/771 ,  G06V10/82 ,  G06V10/26 ,  G06V40/10 ,  G06V10/62

CPC classification number: G06T7/11 ,  G06N3/08 ,  G06T7/174 ,  G06T7/20 ,  G06T9/002 ,  G06V10/273 ,  G06V10/771 ,  G06V10/82 ,  G06V20/40 ,  G06V40/107 ,  G06V40/12 ,  G06T2207/10016 ,  G06T2207/20081 ,  G06T2207/20084 ,  G06T2207/20132 ,  G06V10/62

Abstract: There are provided methods and computing devices using semi-supervised learning to perform end-to-end video object segmentation, tracking respective object(s) from a single-frame annotation of a reference frame through a video sequence of frames. A known deep learning model may be used to annotate the reference frame to provide ground truth locations and masks for each respective object. A current frame is processed to determine current frame object locations, defining object scoremaps as a normalized cross-correlation between encoded object features of the current frame and encoded object features of a previous frame. Scoremaps for each of more than one previous frame may be defined. An Intersection over Union (IoU) function, responsive to the scoremaps, ranks candidate object proposals defined from the reference frame annotation to associate the respective objects to respective locations in the current frame. Pixel-wise overlap may be removed using a merge function responsive to the scoremaps.