公开(公告)号：US12112449B2

公开(公告)日：2024-10-08

申请号：US18337769

申请日：2023-06-20

Applicant: Apple Inc.

Inventor： Brett D. Miller ,  Minwoong Kim ,  Ricardo J. Motta

IPC: G06T3/18 ,  G06T7/50 ,  G06T7/521 ,  G06T7/593 ,  G06T7/70 ,  G06V20/10 ,  H04N5/262 ,  H04N9/73 ,  H04N23/63 ,  H04N23/67 ,  H04N23/90

CPC classification number: G06T3/18 ,  G06T7/50 ,  G06T7/70 ,  G06V20/10 ,  H04N5/2621 ,  H04N9/73 ,  H04N23/63 ,  H04N23/90 ,  G06T7/521 ,  G06T7/593 ,  G06T2207/20164 ,  G06T2207/30201 ,  H04N23/67

Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.

公开(公告)号：US11250541B2

公开(公告)日：2022-02-15

申请号：US16125363

申请日：2018-09-07

Applicant: Apple Inc.

Inventor： Brett D. Miller ,  Minwoong Kim ,  Ricardo J. Motta

IPC: G06T3/00 ,  G06T7/50 ,  G06T7/70 ,  G06K9/00 ,  H04N5/232 ,  H04N5/247 ,  H04N5/262 ,  H04N9/73 ,  G06T7/521 ,  G06T7/593

Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.

公开(公告)号：US20180020201A1

公开(公告)日：2018-01-18

申请号：US15588228

申请日：2017-05-05

Applicant: Apple Inc.

Inventor： Ricardo J. Motta ,  Gary L. Vondran, JR. ,  Manohar B. Srikanth ,  Brett D. Miller

IPC: H04N13/00 ,  H04N13/02

CPC classification number: H04N13/156 ,  G06F3/011 ,  G06F3/012 ,  G06F3/013 ,  H04N5/23219 ,  H04N5/247 ,  H04N7/142 ,  H04N7/144 ,  H04N7/147 ,  H04N7/18 ,  H04N13/117 ,  H04N13/122 ,  H04N13/243

Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.

公开(公告)号：US11217021B2

公开(公告)日：2022-01-04

申请号：US16361110

申请日：2019-03-21

Applicant: Apple Inc.

Inventor： Ricardo J. Motta ,  Brett D. Miller ,  Tobias Rick ,  Manohar B. Srikanth

IPC: G06T19/00 ,  H04N13/383 ,  G06F1/16 ,  G06F3/01 ,  G02B27/00 ,  H04N13/344 ,  G02B27/01

Abstract: A mixed reality system that includes a head-mounted display (HMD) that provides 3D virtual views of a user's environment augmented with virtual content. The HMD may include sensors that collect information about the user's environment (e.g., video, depth information, lighting information, etc.), and sensors that collect information about the user (e.g., the user's expressions, eye movement, hand gestures, etc.). The sensors provide the information as inputs to a controller that renders frames including virtual content based at least in part on the inputs from the sensors. The HMD may display the frames generated by the controller to provide a 3D virtual view including the virtual content and a view of the user's environment for viewing by the user.

公开(公告)号：US11187914B2

公开(公告)日：2021-11-30

申请号：US16583193

申请日：2019-09-25

Applicant: Apple Inc.

Inventor： Daniel R. Hennigan ,  Noah D. Bedard ,  Branko Petljanski ,  Jason C. Sauers ,  Edward S. Huo ,  Ricardo J. Motta ,  Brett D. Miller ,  Yury Petrov

IPC: G02B27/14 ,  G02B27/01 ,  G02B5/08 ,  G02B30/35

Abstract: A scene camera system that includes two or more mirrors that reflect light from a respective portion of a field of view (FOV) in front of the system and two or more cameras that each capture the light reflected by a respective one of the two or more mirrors. By using the mirrors to reflect the light, the cameras' entrance pupils are imaged to a location closer to a user's eyes to thus achieve a more accurate representation of the perspective of the user.

公开(公告)号：US20190260979A1

公开(公告)日：2019-08-22

申请号：US16241294

申请日：2019-01-07

Applicant: Apple Inc.

Inventor： Ricardo J. Motta ,  Gary L. Vondran, JR. ,  Manohar B. Srikanth ,  Brett D. Miller

IPC: H04N13/156 ,  H04N7/14 ,  H04N5/232 ,  G06F3/01 ,  H04N13/122 ,  H04N7/18 ,  H04N13/243 ,  H04N13/117 ,  H04N5/247

Abstract: This disclosure pertains to operations, systems, and computer readable media to capture images of a scene using a camera array and process the captured images based on a viewer's point of view (POV) for immersive augmented reality, live display wall, head mounted display, video conferencing, and similar applications. In one implementation, the disclosed subject matter provides a complete view to a viewer by combining images captured by a camera array. In another implementation, the disclosed subject matter tracks the viewer's POV as he moves from one location to another and displays images in accordance with his varying POV. The change of the viewer's POV is inclusive of movements in the X, Y, and Z dimensions.

公开(公告)号：US20190221044A1

公开(公告)日：2019-07-18

申请号：US16361110

申请日：2019-03-21

Applicant: Apple Inc.

Inventor： Ricardo J. Motta ,  Brett D. Miller ,  Tobias Rick ,  Manohar B. Srikanth

IPC: G06T19/00 ,  H04N13/344 ,  G02B27/00 ,  G06F3/01 ,  G02B27/01 ,  G06F1/16 ,  H04N13/383

CPC classification number: G06T19/006 ,  G02B27/0093 ,  G02B27/0172 ,  G02B2027/0136 ,  G02B2027/0138 ,  G02B2027/014 ,  G02B2027/0187 ,  G06F1/163 ,  G06F3/011 ,  G06F3/012 ,  G06F3/013 ,  G06F3/015 ,  G06F3/017 ,  H04N13/344 ,  H04N13/383

Abstract: A mixed reality system that includes a head-mounted display (HMD) that provides 3D virtual views of a user's environment augmented with virtual content. The HMD may include sensors that collect information about the user's environment (e.g., video, depth information, lighting information, etc.), and sensors that collect information about the user (e.g., the user's expressions, eye movement, hand gestures, etc.). The sensors provide the information as inputs to a controller that renders frames including virtual content based at least in part on the inputs from the sensors. The HMD may display the frames generated by the controller to provide a 3D virtual view including the virtual content and a view of the user's environment for viewing by the user.

公开(公告)号：US20190080432A1

公开(公告)日：2019-03-14

申请号：US16125363

申请日：2018-09-07

Applicant: Apple Inc.

Inventor： Brett D. Miller ,  Minwoong Kim ,  Ricardo J. Motta

IPC: G06T3/00 ,  G06T7/50 ,  G06K9/00 ,  H04N5/232 ,  H04N9/73 ,  H04N5/247 ,  G06T7/70 ,  H04N5/262

Abstract: Presenting an image of a scene may include capturing an image of a scene by a camera of an electronic device, wherein the electronic device comprises the camera and a display, and wherein the camera and the display have a first spatial relationship, determining a second spatial relationship between a viewpoint and the display of the electronic device, warping the image to obtain an image of a first portion of the scene based on the first spatial relationship and the second spatial relationship, and presenting the warped image on the display, wherein, from the viewpoint, the image of the first portion of the scene is substantially contiguous with a second portion of the scene visible outside an edge of the electronic device.

公开(公告)号：US20240385454A1

公开(公告)日：2024-11-21

申请号：US18662641

申请日：2024-05-13

Applicant: Apple Inc.

Inventor： Ricardo J. Motta ,  Brett D. Miller ,  Chad A. Bronstein ,  Bennett S. Wilburn ,  Branko Petljanski ,  Mahmut C. Orsan ,  David W. Lum ,  Edward A. Valko ,  Graham B. Myhre ,  Wonjae Choi ,  Manohar B. Srikanth ,  Michael Slootsky ,  Nicolas P. Bonnier ,  Jiaying Wu ,  ZhiBing Ge ,  William W. Sprague ,  Bertrand Nepveu ,  Michael J. Rockwell ,  Edward S. Huo ,  Marinus Meursing ,  Timothy Y. Chang ,  Katharina Buckl

IPC: G02B27/01 ,  G02B7/14 ,  G02B27/00 ,  G06F3/01 ,  H04N13/361

Abstract: A head-mountable display device includes a housing defining a front opening and a rear opening, a display screen disposed in the front opening, a display assembly disposed in the rear opening, a first securement strap coupled to the housing, the first securement strap including a first electronic component, a second securement strap coupled to the housing, the second securement strap including a second electronic component, and a securement band extending between and coupled to the first securement strap and the second securement strap.

公开(公告)号：US20230409278A1

公开(公告)日：2023-12-21

申请号：US18211515

申请日：2023-06-19

Applicant: Apple Inc.

Inventor： Brett D. Miller ,  Daniel K. Boothe ,  Martin E. Johnson

IPC: G06F3/16 ,  H04N7/18

CPC classification number: G06F3/165 ,  H04N7/183

Abstract: In one implementation, a method of playing audio data is performed at a device including a frame configured for insertion into an outer ear, a speaker coupled to the frame, an image sensor coupled to the frame, one or more processors, and non-transitory memory. The method includes capturing, using the image sensor, one or more images of a physical environment. The method includes generating audio data based on the one or more images of the physical environment. The method includes playing, via the speaker, the audio data.