公开(公告)号：US10192528B2

公开(公告)日：2019-01-29

申请号：US15086645

申请日：2016-03-31

Applicant: Sony Computer Entertainment Inc.

Inventor： Andrew Young ,  Jeffrey Roger Stafford

IPC: G06F1/32 ,  G09G5/391 ,  G06F3/01 ,  G02B27/01

Abstract: Foveated rendering based on user gaze tracking may be adjusted to account for the realities of human vision. Gaze tracking error and state parameters may be determined from gaze tracking data representing a user's gaze with respect to one or more images presented to a user. Adjusted foveation data representing an adjusted size and/or shape of one or more regions of interest in one or more images to be subsequently presented to a user may be generated based on the one or more gaze tracking error or state parameters. Foveated image data representing one or more foveated images may be generated with the adjusted foveation data. The foveated images are characterized by level of detail within the one or more regions of interest and lower level of detail outside the one or more regions of interest. The foveated images may then be presented to the user.

公开(公告)号：US20150258432A1

公开(公告)日：2015-09-17

申请号：US14657639

申请日：2015-03-13

Applicant: Sony Computer Entertainment Inc.

Inventor： Jeffrey Roger Stafford ,  Todd Tokubo

IPC: A63F13/213 ,  A63F13/25 ,  A63F13/215

CPC classification number: A63F13/213 ,  A63F13/215 ,  A63F13/23 ,  A63F13/24 ,  A63F13/25 ,  A63F13/26 ,  A63F13/426 ,  A63F13/428 ,  A63F13/5255 ,  A63F13/98 ,  A63F2300/538 ,  G06F3/011 ,  G06F3/017 ,  G06F3/0304 ,  G06F3/04883 ,  G06F2203/04101

Abstract: A method for enhancing detection of a user's hand relative to a head-mounted display (HMD) is described. The method includes sensing a disrupted portion of energy by sensors integrated within a pad device. The disrupted portion of the energy is generated when the hand of the user interferes with the energy. The plurality of sensors that sense the disrupted portion of the energy produce an energy image that mirrors a current position of the hand. The method includes repeating the sensing continuously to produce a stream of energy images. The method includes communicating the stream of energy images to a game console for processing each of the energy images to produce a model of the hand and movement of the model of the hand. The model of the hand is at least partially rendered as a virtual hand in a virtual environment that is displayed in the HMD.

Abstract translation: 描述了一种用于增强用户的手相对于头戴式显示器（HMD）的检测的方法。 该方法包括通过集成在垫装置内的传感器感测中断的能量部分。 当用户的手干扰能量时，产生能量的中断部分。 感测能量的中断部分的多个传感器产生镜像当前手的位置的能量图像。 该方法包括连续重复感测以产生能量图像流。 该方法包括将能量图像流传送到游戏控制台，用于处理每个能量图像以产生手的模型和手的模型的移动。 在HMD中显示的虚拟环境中，手的模型至少部分地被渲染为虚拟手。

公开(公告)号：US20150258431A1

公开(公告)日：2015-09-17

申请号：US14633415

申请日：2015-02-27

Applicant: Sony Computer Entertainment Inc.

Inventor： Jeffrey Roger Stafford ,  Todd Tokubo

IPC: A63F13/213 ,  A63F13/212

CPC classification number: G06F3/017 ,  A63F13/211 ,  A63F13/212 ,  A63F13/213 ,  A63F13/5255 ,  A63F2300/1012 ,  A63F2300/1031 ,  A63F2300/1087 ,  A63F2300/1093 ,  A63F2300/301 ,  A63F2300/303 ,  A63F2300/308 ,  A63F2300/538 ,  G06F3/011 ,  G06F3/014 ,  G06F3/0304 ,  G06K9/00382

Abstract: A method to identify positions of fingers of a hand is described. The method includes capturing images of a first hand using a plurality of cameras that are part of a wearable device. The wearable device is attached to a wrist of a second hand and the plurality of cameras of the wearable device is disposed around the wearable device. The method includes repeating capturing of additional images of the first hand, the images and the additional images captured to produce a stream of captured image data during a session of presenting the virtual environment in a head mounted display (HMD). The method includes sending the stream of captured image data to a computing device that is interfaced with the HMD. The computing device is configured to process the captured image data to identify changes in positions of the fingers of the first hand.

Abstract translation: 描述了用于识别手指的位置的方法。 该方法包括使用作为可穿戴装置的一部分的多个照相机捕获第一只手的图像。 可穿戴装置附接到秒针的手腕，并且穿戴式装置的多个相机设置在可穿戴装置周围。 该方法包括在头戴式显示器（HMD）中呈现虚拟环境的会话期间重复捕获第一只手的附加图像，捕获的图像和附加图像以产生捕获的图像数据流。 该方法包括将捕获的图像数据流发送到与HMD接口的计算设备。 计算设备被配置为处理所捕获的图像数据以识别第一只手的手指的位置的变化。

公开(公告)号：US20150094142A1

公开(公告)日：2015-04-02

申请号：US14333500

申请日：2014-07-16

Applicant: Sony Computer Entertainment Inc.

Inventor： Jeffrey Roger Stafford

IPC: A63F13/25 ,  A63F13/212 ,  G02B27/01

CPC classification number: A63F13/537 ,  A63F13/211 ,  A63F13/212 ,  A63F13/213 ,  A63F13/25 ,  A63F13/26 ,  A63F13/49 ,  A63F13/5255 ,  A63F13/533 ,  A63F2300/306 ,  A63F2300/538 ,  G02B27/017 ,  G06F3/011 ,  G06F3/012 ,  G06F3/016 ,  G06F3/017 ,  G06F3/0304 ,  G06F3/04815 ,  G06F3/16

Abstract: The disclosure provides methods and systems for warning a user of a head mounted display during gameplay of a video game. A game is executed causing interactive scenes of the game to be rendered on a display portion of a head mounted display (HMD) worn by a user. A change in position of the HMD worn by the user, while the user is interacting with the game, is detected. The change in position is evaluated, the evaluation causing a signal to be generated when the change exceeds a pre-defined threshold value. When the signal is generated, content is sent to interrupt the interactive scenes being rendered on the display portion of the HMD. The data sent provides descriptive context for the signal.

Abstract translation: 本公开提供了用于在视频游戏的游戏过程中警告头戴式显示器的用户的方法和系统。 执行游戏，使游戏的交互式场景呈现在用户佩戴的头戴式显示器（HMD）的显示部分上。 检测用户与游戏交互时由用户佩戴的HMD的位置的变化。 评估位置的变化，当变化超过预定义的阈值时，引起产生信号的评估。 当产生信号时，发送内容以中断在HMD的显示部分上呈现的交互式场景。 发送的数据为信号提供了描述性的上下文。

公开(公告)号：US20140361976A1

公开(公告)日：2014-12-11

申请号：US14206219

申请日：2014-03-12

Applicant: Sony Computer Entertainment Inc.

Inventor： Steven Osman ,  Xiadong Mao ,  Jeffrey Roger Stafford

IPC: G06F3/01 ,  G02B27/01 ,  G06T19/00

CPC classification number: G06F3/013 ,  G02B27/0093 ,  G02B27/017 ,  G02B27/0172 ,  G02B27/0179 ,  G02B2027/0187 ,  G06F2203/04804 ,  G06T19/006

Abstract: Methods, systems, and computer programs are presented for managing the display of images on a head mounted device (HMD). One method includes an operation for tracking the gaze of a user wearing the HMD, where the HMD is displaying a scene of a virtual world. In addition, the method includes an operation for detecting that the gaze of the user is fixed on a predetermined area for a predetermined amount of time. In response to the detecting, the method fades out a region of the display in the HMD, while maintaining the scene of the virtual world in an area of the display outside the region. Additionally, the method includes an operation for fading in a view of the real world in the region as if the HMD were transparent to the user while the user is looking through the region. The fading in of the view of the real world includes maintaining the scene of the virtual world outside the region.

Abstract translation: 呈现了用于管理头戴式装置（HMD）上的图像显示的方法，系统和计算机程序。 一种方法包括用于跟踪穿戴HMD的用户的目视的操作，其中HMD正在显示虚拟世界的场景。 此外，该方法包括用于检测用户的凝视在预定区域上固定预定时间量的操作。 响应于该检测，该方法使HMD中的显示区域淡出，同时将虚拟世界的场景保持在区域外的显示区域。 此外，该方法包括在该区域中的现实世界的视图中衰落的操作，如同HMD在用户正在通过该区域查看时对于用户是透明的。 现实世界的褪色包括维护该地区之外的虚拟世界的场景。

公开(公告)号：US20160364881A1

公开(公告)日：2016-12-15

申请号：US14820390

申请日：2015-08-06

Applicant: Sony Computer Entertainment Inc.

Inventor： Dominic Saul Mallinson ,  Eric James Larsen ,  Jeffrey Roger Stafford

IPC: G06T7/20 ,  G06T7/00 ,  G06F3/01 ,  G06K9/00

CPC classification number: G06T7/20 ,  G06F3/013 ,  G06F3/015 ,  G06K9/00604 ,  G06T7/70 ,  G06T2207/30241

Abstract: A method for tracking eye movement is provided. One embodiment of the method includes receiving a first measurement from a first sensor configured to detect a gaze location, determining an initial gaze location based at least on the first measurement, receiving at least one of eye motion amplitude and eye motion direction measurement from a second sensor, and determining an estimated gaze location based at least on the initial gaze location and the at least one of eye motion amplitude and eye motion direction. Systems perform similar steps, and non-transitory computer readable storage mediums each store one or more computer programs.

Abstract translation: 提供了一种跟踪眼动的方法。 该方法的一个实施例包括从配置成检测凝视位置的第一传感器接收第一测量，至少基于第一测量确定初始凝视位置，从第二测量中接收眼睛运动幅度和眼睛运动方向测量中的至少一个 传感器，并且至少基于初始注视位置和眼睛运动幅度和眼睛运动方向中的至少一个来确定估计的注视位置。 系统执行类似的步骤，并且非暂时的计算机可读存储介质每个存储一个或多个计算机程序。

公开(公告)号：US20160091720A1

公开(公告)日：2016-03-31

申请号：US14835558

申请日：2015-08-25

Applicant: Sony Computer Entertainment Inc.

Inventor： Jeffrey Roger Stafford ,  Eric Larsen

IPC: G02B27/01 ,  G02B27/00

CPC classification number: G02B27/0172 ,  G02B27/0068 ,  G02B27/0093 ,  G02B2027/011 ,  G02B2027/0138 ,  G02B2027/014 ,  G02B2027/0187

Abstract: Systems and methods include receiving an image for presenting on a display screen of a head mounted display (HMD). The image is provided by an application. The received image is pre-distorted to enable optics provided in a HMD to render the image. An alignment offset is identified for an eye of a user wearing the HMD by determining a position of the eye relative to an optical axis of at least one lens of the optics of the HMD. The pre-distorted image provided by the application is adjusted to define a corrected pre-distorted image that accounts for the alignment offset. The corrected pre-distorted image is forwarded to the display screen of the HMD for rendering, such that the image presented through the optics of the HMD removes aberrations caused by the alignment offset.

Abstract translation: 系统和方法包括接收用于呈现在头戴式显示器（HMD）的显示屏上的图像。 图像由应用程序提供。 接收到的图像是预失真的，以使HMD中提供的光学元件能够呈现图像。 通过确定眼睛相对于HMD的光学器件的至少一个透镜的光轴的位置来识别佩戴HMD的用户的眼睛的对准偏移。 调整由应用提供的预失真图像以定义校正的预失真图像，其考虑对准偏移。 校正的预失真图像被转发到用于渲染的HMD的显示屏幕，使得通过HMD的光学器件呈现的图像去除由对准偏移引起的像差。

公开(公告)号：US20150092015A1

公开(公告)日：2015-04-02

申请号：US14170470

申请日：2014-01-31

Applicant: Sony Computer Entertainment Inc.

Inventor： Jeffrey Roger Stafford

IPC: H04N13/04 ,  H04N13/02

CPC classification number: H04N13/332 ,  A63F13/213 ,  A63F13/5255 ,  G02B27/017 ,  G02B2027/0178 ,  G02B2027/0187 ,  G06F3/017 ,  G06F2203/012 ,  H04N13/25

Abstract: The disclosure provides methods and systems for warning a user of a head mounted display that the user approaches an edge of field of view of a camera or one or more tangible obstacles. The warning includes presenting audio and/or displayable messages to the user, or moving the display(s) of the head mounted displays away of the user's eyes. The determination that the user approaches the edge of scene or a tangible obstacle is made by dynamically tracking motions of the users through analysis of images and/or depth data obtained from image sensor(s) and/or depth sensor(s) secured to either the head mounted display, arranged outside of the scene and not secured to the head mounted display, or a combination of both.

Abstract translation: 本公开提供了用于警告头戴式显示器的用户的方法和系统，其使用者接近照相机的视野边缘或一个或多个有形障碍物。 警告包括向用户呈现音频和/或可显示的消息，或将头戴式显示器的显示器移动到用户的眼睛之外。 用户接近场景边缘或有形障碍物的确定是通过动态跟踪用户的运动，通过分析从图像传感器和/或深度传感器得到的图像和/或深度数据来确定的， 头戴显示器，布置在场​​景外部并且不固定到头戴式显示器，或两者的组合。

公开(公告)号：US20150015486A1

公开(公告)日：2015-01-15

申请号：US14503218

申请日：2014-09-30

Applicant: Sony Computer Entertainment Inc.

Inventor： Steven Osman ,  Yunpeng Zhu ,  Jeffrey Roger Stafford

IPC: A63F13/211 ,  A63F13/92 ,  A63F13/525 ,  A63F13/213 ,  A63F13/235

CPC classification number: A63F13/5255 ,  A63F13/12 ,  A63F13/211 ,  A63F13/213 ,  A63F13/235 ,  A63F13/323 ,  A63F13/428 ,  A63F13/525 ,  A63F13/655 ,  A63F13/86 ,  A63F13/92 ,  A63F2300/105 ,  A63F2300/1093 ,  A63F2300/403 ,  A63F2300/6676 ,  A63F2300/695

Abstract: A handheld device is provided, comprising: a sensor configured to generate sensor data for determining and tracking a position and orientation of the handheld device during an interactive session of an interactive application presented on a main display, the interactive session being defined for interactivity between a user and the interactive application; a communications module configured to send the sensor data to a computing device, the communications module being further configured to receive from the computing device a spectator video stream of the interactive session that is generated based on a state of the interactive application and the tracked position and orientation of the handheld device, the state of the interactive application being determined based on the interactivity between the user and the interactive application; a display configured to render the spectator video stream.

Abstract translation: 提供了一种手持设备，包括：传感器，被配置为在呈现在主显示器上的交互式应用的交互式会话期间生成用于确定和跟踪手持设备的位置和姿态的传感器数据，所述交互式会话被定义为 用户和交互式应用程序; 通信模块，被配置为将所述传感器数据发送到计算设备，所述通信模块还被配置为从所述计算设备接收基于所述交互式应用的状态和所述跟踪位置生成的所述交互式会话的观众视频流，以及 基于用户和交互式应用之间的交互性来确定交互应用的状态; 被配置为呈现观众视频流的显示器。

公开(公告)号：US20140362110A1

公开(公告)日：2014-12-11

申请号：US14225408

申请日：2014-03-25

Applicant: Sony Computer Entertainment Inc.

Inventor： Jeffrey Roger Stafford

IPC: G06T19/00 ,  G06F3/01

CPC classification number: G06T19/006 ,  G02B27/017 ,  G02B2027/0118 ,  G02B2027/0127 ,  G02B2027/0138 ,  G02B2027/014 ,  G06F3/011 ,  G06F3/013 ,  G06F3/017 ,  G06F3/0346 ,  G09G2340/0407 ,  G09G2340/14

Abstract: Systems and methods for operating a screen of a head mounted display includes executing a program. The execution of the program causes rendering of images on the screen of the HMD. The screen renders the images using a first optical setting. A first image is presented on the screen. The first image has a first size and is presented at a distance. Input is received identifying a clarity level for the first image. A second image is presented on the screen. The second image has a second size and the distance. Input is received identifying the clarity level for the second image. Based on the clarity level received for the first and the second images, the first optical setting for the screen is changed to a second optical setting.

Abstract translation: 用于操作头戴式显示器的屏幕的系统和方法包括执行程序。 程序的执行导致HMD屏幕上的图像渲染。 屏幕使用第一个光学设置渲染图像。 第一个图像显示在屏幕上。 第一个图像具有第一个尺寸，并且呈现在远处。 接收到输入，识别第一个图像的清晰度级别。 第二个图像显示在屏幕上。 第二个图像具有第二个尺寸和距离。 接收到输入，识别第二张图像的清晰度级别。 基于对于第一和第二图像接收的清晰度水平，将屏幕的第一光学设置改变为第二光学设置。