公开(公告)号：US20240378236A1

公开(公告)日：2024-11-14

申请号：US18314646

申请日：2023-05-09

Applicant: Google LLC

Inventor： Harshit Kharbanda ,  Jessica Lee ,  Christopher James Kelley ,  Belinda Luna Zeng ,  Louis Wang

IPC: G06F16/538 ,  G06F16/583

Abstract: A result image is retrieved based on a similarity between a query image and the result image. A first unit of text is obtained, wherein the first unit of text comprises at least a portion of textual content of a source document that includes the result image. A second unit of text is determined responsive to a prompt associated with the query image, wherein the second unit of text comprises one or more of (a) at least some of the first unit of text, or (b) text derived from the first unit of text. The second unit of text and the result image are provided for display within an interface.

公开(公告)号：US12277635B1

公开(公告)日：2025-04-15

申请号：US18532470

申请日：2023-12-07

Applicant: Google LLC

Inventor： Harshit Kharbanda ,  Louis Wang ,  Christopher James Kelley ,  Jessica Lee

IPC: G06T11/60 ,  G06T13/80

Abstract: A multimodal search system is described. The system can receive image data from a user device. Additionally, the system can receive a prompt associated with the image data. Moreover, the system can determine, using a computer vision model, a first object in the image data that is associated with the prompt. Furthermore, the system can receive, from the user device, a user indication on whether the image data includes the first object. Subsequently, in response to receiving the user indication, the system can generate a response using a large language model.

公开(公告)号：US20250054405A1

公开(公告)日：2025-02-13

申请号：US18446125

申请日：2023-08-08

Applicant: Google LLC

Inventor： Jessica Lee ,  Kimiya Hojjat ,  David Trotter Oleson ,  Daniel Valcarce Silva ,  Andrea D'olimpio ,  Urs Christian Lukas Dönni ,  Christopher Rohrs ,  Kuba Dolecki ,  Balint Miklos ,  Federico Chialvo ,  Lisa Wang ,  Jieru Hu ,  Ryan Muller ,  Chris Heather ,  Sara Wiltberger ,  Saurabh Paliwal ,  Viacheslav Kuznetsov ,  Gleb Makarchuk ,  Philipp Neubeck ,  Ivan Jurin

IPC: G09B7/02 ,  G06F16/9535 ,  G06F16/9538 ,  G06F40/40

Abstract: The present disclosure provides computer-implemented methods, systems, and devices for generating multistep explanations for pedagogical exercises. A computing device receives a query from a user. The computing device determines that the query includes query data describing a pedagogical exercise to be solved. The computing device provides the query data as input to an explanatory machine-learned model. The computing device receives, as output from the explanatory machine-learned model, a pedagogical response, the pedagogical response including a multi-step explanation of a solution to the pedagogical exercise. The computing device provides the pedagogical response for display to a user.

公开(公告)号：US20240403362A1

公开(公告)日：2024-12-05

申请号：US18326496

申请日：2023-05-31

Applicant: Google LLC

Inventor： Harshit Kharbanda ,  Belinda Luna Zeng ,  Viviana Caso Corella ,  Aashi Jain ,  David William Hendon ,  Christopher James Kelley ,  Jessica Lee ,  Dounia Berrada ,  Kai Yu ,  Louis Wang ,  Thomas J. Duerig ,  Radu Soricut ,  Robin Dua

IPC: G06F16/735 ,  G06F16/732 ,  G06F16/783 ,  G06T7/70 ,  G06V10/62 ,  G06V10/774 ,  G06V20/40

Abstract: A multimodal search system using a video query is described. The system can receive video data captured by a camera of a user device. The video data can have a sequence of image frames. Additionally, the system can receive audio data associated with the video data captured by the user device. Moreover, the system can process, using one or more machine-learned models, the sequence of image frames to generate video embeddings related to the sequence of the image frames. The video embeddings can have a plurality of image embeddings associated with the sequence of image frames. Furthermore, the system can determine one or more video results based on the video embeddings and the audio data. Subsequently, the system can transmit, to the user device, the one or more video results.

公开(公告)号：USD1042528S1

公开(公告)日：2024-09-17

申请号：US29869039

申请日：2022-12-20

Applicant: Google LLC

Designer： Jessica Lee ,  Bálint Miklos ,  Harshit Kharbanda ,  Severin Heiniger

Abstract: FIG. 1 is a front view of a display screen or portion thereof with a transitional graphical user interface showing a first image of the claimed design; and,
FIG. 2 is a second image thereof.
The outermost evenly spaced broken lines show an electronic device that forms no part of the claimed design. The dot-dash broken lines show a display screen or portion thereof and form no part of the claimed design. The remaining broken lines and all lined-through text, show portions of the transitional graphical user interface, and form no part of the claimed design.
The appearance of the transitional image sequentially transitions between the views of FIGS. 1-2. The process or period in which an image transitions to another forms no part of the claimed design.

公开(公告)号：USD1042527S1

公开(公告)日：2024-09-17

申请号：US29869038

申请日：2022-12-20

Applicant: Google LLC

Designer： Jessica Lee ,  Alok Aggarwal ,  Ruslan Alfridovich Abdikeev ,  Jessica Katherine Turner ,  Wenjia Yuan ,  Hassan Ali Shojania ,  Viviana Caso Corella ,  Harshit Kharbanda

Abstract: FIG. 1 is a front view of a display screen or portion thereof with a transitional graphical user interface showing a first image of the claimed design;
FIG. 2 is a second image thereof; and,
FIG. 3 is a third image thereof.
The outermost evenly spaced broken lines show an electronic device that forms no part of the claimed design. The dot-dash broken lines show a display screen or portion thereof and form no part of the claimed design. The remaining broken lines and all lined-through text, show portions of the transitional graphical user interface, and form no part of the claimed design.
The appearance of the transitional image sequentially transitions between the views of FIGS. 1-3. The process or period in which an image transitions to another forms no part of the claimed design.

公开(公告)号：US20240233569A9

公开(公告)日：2024-07-11

申请号：US17969303

申请日：2022-10-19

Applicant: Google LLC

Inventor： Jessica Lee ,  David Trotter Oleson ,  Fabian Roth ,  Nils Grimsmo

IPC: G09B7/04 ,  G06F3/04845 ,  G06F40/205 ,  G06T11/60 ,  G06V10/94 ,  G06V20/70 ,  G06V30/12 ,  G06V30/19

CPC classification number: G09B7/04 ,  G06F3/04845 ,  G06F40/205 ,  G06T11/60 ,  G06V10/945 ,  G06V20/70 ,  G06V30/127 ,  G06V30/19133 ,  G06V30/19147

Abstract: Systems and methods for augmented-reality tutoring can utilize optical character recognition, natural language processing, and/or augmented-reality rendering for providing real-time notifications for completing a determined task. The systems and methods can include utilizing one or more machine-learned models trained for quantitative reasoning and can include providing a plurality of different user interface elements at different times.

公开(公告)号：US20240135835A1

公开(公告)日：2024-04-25

申请号：US17969303

申请日：2022-10-18

Applicant: Google LLC

Inventor： Jessica Lee ,  David Trotter Oleson ,  Fabian Roth ,  Nils Grimsmo

IPC: G09B7/04 ,  G06F3/04845 ,  G06F40/205 ,  G06T11/60 ,  G06V10/94 ,  G06V20/70 ,  G06V30/12 ,  G06V30/19

CPC classification number: G09B7/04 ,  G06F3/04845 ,  G06F40/205 ,  G06T11/60 ,  G06V10/945 ,  G06V20/70 ,  G06V30/127 ,  G06V30/19133 ,  G06V30/19147

Abstract: Systems and methods for augmented-reality tutoring can utilize optical character recognition, natural language processing, and/or augmented-reality rendering for providing real-time notifications for completing a determined task. The systems and methods can include utilizing one or more machine-learned models trained for quantitative reasoning and can include providing a plurality of different user interface elements at different times.

公开(公告)号：US12266065B1

公开(公告)日：2025-04-01

申请号：US18409268

申请日：2024-01-10

Applicant: Google LLC

Inventor： Harshit Kharbanda ,  Louis Wang ,  Christopher James Kelley ,  Jessica Lee ,  Igor Bonaci ,  Daniel Valcarce Silva

IPC: G06T19/00 ,  G06V20/20

Abstract: Systems and methods for providing visual indications of generative model responses can include obtaining a user input and processing the user input with a generative model to generate a model-generated-response. The systems and methods can process the model-generated response and an image of an environment to generate an augmented image. The augmented image can include visual indicators of the model-generated response, which can include annotating the image based on detected features within the image. Generation of the augmented image can include object detection and annotation based on the content of the model-generated response.

公开(公告)号：US20250148782A1

公开(公告)日：2025-05-08

申请号：US19015028

申请日：2025-01-09

Applicant: Google LLC

Inventor： Jessica Lee ,  Christopher James Kelley ,  Alok Aggarwal ,  Harshit Kharbanda

IPC: G06V20/20 ,  G06F16/9535 ,  G06T11/00 ,  G06V10/94

Abstract: Systems and methods for providing scene understanding can include obtaining a plurality of images, stitching images associated with the scene, detecting objects in the scene, and providing information associated with the objects in the scene. The systems and methods can include determining filter tags or query tags that can be selected to filter the plurality of objects, which can then be provided as information to the user to provide further insight on the scene. The information may be provided in an augmented-reality experience via text or other user-interface elements anchored to objects in the images.