公开(公告)号：US09968847B2

公开(公告)日：2018-05-15

申请号：US15591062

申请日：2017-05-09

Applicant: Sony Interactive Entertainment America LLC

Inventor： Stephen G. Perlman ,  Roger van der Laan ,  Timothy Cotter ,  Scott Furman ,  Robert McCool ,  Ian Buckley

IPC: A63F13/355 ,  H04N21/6583 ,  H04N21/6587 ,  H04N21/658 ,  H04N21/2662 ,  H04N21/2343 ,  H04N19/146 ,  H04L29/06 ,  H04N21/258 ,  A63F13/335 ,  A63F13/71 ,  A63F13/358 ,  A63F13/25 ,  H04N19/169 ,  H04N19/172

CPC classification number: A63F13/355 ,  A63F13/12 ,  A63F13/25 ,  A63F13/335 ,  A63F13/358 ,  A63F13/71 ,  A63F2300/407 ,  A63F2300/534 ,  A63F2300/535 ,  A63F2300/538 ,  A63F2300/552 ,  A63F2300/577 ,  H04L65/403 ,  H04L65/4076 ,  H04L65/605 ,  H04L67/38 ,  H04N7/106 ,  H04N19/107 ,  H04N19/132 ,  H04N19/146 ,  H04N19/169 ,  H04N19/172 ,  H04N19/188 ,  H04N19/436 ,  H04N19/61 ,  H04N21/2343 ,  H04N21/234363 ,  H04N21/23439 ,  H04N21/2381 ,  H04N21/25833 ,  H04N21/266 ,  H04N21/2662 ,  H04N21/6377 ,  H04N21/658 ,  H04N21/6583 ,  H04N21/6587

Abstract: A computer-implemented system and method are described for performing video compression by an online video game service is provided. One example method includes receiving user input by a server and the user input is received from a client device over a network established between the client device and the server. The user input is from an input device associated with the client device and the client device and the input device are used to access the server to play a video game. The method further includes executing the video game on the server while receiving user input from the input device, and the video game produces a video output including a sequence of video images of the video game. Encoding each of the sequence of video images in a video encoding resource of the server to generate a sequence of encoded video frames and transmitting the sequence of encoded video frames by the server over the communication channel to the client device for decoding to produce a corresponding sequence of decoded video frames. The sequence of decoded video frames is configured to be rendered on a display device associated with the client device. The method further includes detecting a change in a communication channel characteristic of the communication channel, and the detected change is based on feedback information received by the server. The feedback information is received from the client device and dynamically adjusting a compression ratio for said encoding of the sequence of encoded video frames based on the detected change in the communication channel characteristic.

公开(公告)号：US09878241B2

公开(公告)日：2018-01-30

申请号：US14792329

申请日：2015-07-06

Applicant: Sony Interactive Entertainment America LLC

Inventor： Steve Perlman ,  Roger van der Laan ,  Timothy Cotter ,  Scott Furman ,  Robert McCool ,  Ian Buckley

IPC: H04N19/65 ,  A63F13/355 ,  H04N19/164 ,  A63F13/358 ,  A63F13/335 ,  H04N21/2343 ,  H04N21/2381 ,  H04N21/266 ,  H04N21/6377 ,  H04N21/658 ,  H04N21/6583 ,  H04N21/6587 ,  H04L29/06 ,  H04N19/172 ,  H04N19/169 ,  H04N19/61 ,  H04N19/107 ,  H04N19/132 ,  H04N19/146 ,  H04N19/436 ,  A63F13/77 ,  H04N7/10

CPC classification number: A63F13/355 ,  A63F13/12 ,  A63F13/335 ,  A63F13/358 ,  A63F13/77 ,  A63F2300/407 ,  A63F2300/535 ,  A63F2300/538 ,  A63F2300/552 ,  A63F2300/577 ,  H04L65/403 ,  H04L65/4076 ,  H04L65/605 ,  H04L67/38 ,  H04N7/106 ,  H04N19/107 ,  H04N19/132 ,  H04N19/146 ,  H04N19/164 ,  H04N19/169 ,  H04N19/172 ,  H04N19/188 ,  H04N19/436 ,  H04N19/61 ,  H04N19/65 ,  H04N21/2343 ,  H04N21/23439 ,  H04N21/2381 ,  H04N21/266 ,  H04N21/6377 ,  H04N21/658 ,  H04N21/6583 ,  H04N21/6587

Abstract: A computer-implemented system and method are provided for performing video compression including encoding a plurality of video frames or portions thereof according to a first encoding format; transmitting the plurality of encoded video frames or portions to a client device; receiving feedback information from the client device, the feedback information usable to detect whether one or more video frames or portions thereof have not been successfully received and/or decoded; determining, in response to detecting that one or more video frames or portions thereof have not been successfully received and/or decoded, whether to encode a new video frame or portion thereof according to a second encoding format or to encode the new video frame to be dependent on the last video frame or portion thereof known to have been successfully received and/or decoded.

公开(公告)号：US09643084B2

公开(公告)日：2017-05-09

申请号：US15097076

申请日：2016-04-12

Applicant: Sony Interactive Entertainment America LLC

Inventor： Stephen G. Perlman ,  Roger van der Laan ,  Timothy Cotter ,  Scott Furman ,  Robert McCool ,  Ian Buckley

IPC: A63F13/355 ,  A63F13/71 ,  A63F13/358 ,  H04N21/2343 ,  H04N21/2381 ,  H04N21/266 ,  H04N21/6377 ,  H04N21/658 ,  H04N21/6583 ,  H04N21/6587 ,  H04L29/06 ,  H04N19/172 ,  H04N19/169 ,  H04N19/61 ,  H04N19/107 ,  H04N19/132 ,  H04N19/146 ,  H04N19/436 ,  A63F13/335 ,  H04N7/10

CPC classification number: A63F13/355 ,  A63F13/12 ,  A63F13/25 ,  A63F13/335 ,  A63F13/358 ,  A63F13/71 ,  A63F2300/407 ,  A63F2300/534 ,  A63F2300/535 ,  A63F2300/538 ,  A63F2300/552 ,  A63F2300/577 ,  H04L65/403 ,  H04L65/4076 ,  H04L65/605 ,  H04L67/38 ,  H04N7/106 ,  H04N19/107 ,  H04N19/132 ,  H04N19/146 ,  H04N19/169 ,  H04N19/172 ,  H04N19/188 ,  H04N19/436 ,  H04N19/61 ,  H04N21/2343 ,  H04N21/234363 ,  H04N21/23439 ,  H04N21/2381 ,  H04N21/25833 ,  H04N21/266 ,  H04N21/2662 ,  H04N21/6377 ,  H04N21/658 ,  H04N21/6583 ,  H04N21/6587

Abstract: A computer-implemented system and method are described for performing video compression. For example, a method according to one embodiment comprises: encoding a plurality of video frames or portions thereof according to a first encoding format; transmitting the plurality of encoded video frames or portions to a client device; receiving feedback information from the client device, the feedback information usable to determine whether data contained in the video frames or portions has not been successfully received and/or decoded; in response to detecting that a video frame or portion thereof has not been successfully received and/or decoded, encoding a video frame or portion thereof according to a second encoding format; and transmitting the video frames or portions thereof to the client device.

公开(公告)号：US11065552B2

公开(公告)日：2021-07-20

申请号：US16596667

申请日：2019-10-08

Applicant: Sony Interactive Entertainment America LLC

Inventor： Stephen G. Perlman ,  Roger van der Laan

IPC: A63F13/87 ,  A63F13/358 ,  A63F13/352 ,  A63F13/30 ,  H04L29/06 ,  A63F13/335 ,  A63F13/33 ,  A63F13/35 ,  H04N21/21 ,  H04N21/214 ,  H04N21/23 ,  H04N21/478 ,  H04N21/61 ,  A63F13/20 ,  A63F13/355 ,  A63F13/533 ,  H04W88/14 ,  A63F13/49 ,  H04N7/10 ,  H04W84/12

Abstract: An apparatus includes one or more servers of a hosting service center operable to execute video game for one or more users remotely located from the hosting service center. The execution of the video game on the one or more servers produces uncompressed video of 3D animation. The one or more servers are operable to integrate live video with the video game such that the live video appears within the uncompressed video of 3D animation. The live video is received at a hosting service center from one or more client devices correspondingly associated with the one or more users. The uncompressed video 3D animation is generated, at least partially, using data streamed from a high-speed storage unit of the hosting service center coupled to the one or more servers. The high-speed storage unit is configured to load geometry for efficient loading and rendering of objects of the 3D animations. A compression unit compresses the uncompressed video 3D animation integrated with the live video, and compressed streaming interactive video being produced therefrom. An outbound routing network device coupled to the compression unit that transmits the compressed streaming interactive video over a packetized network to the one or more users.

公开(公告)号：US11000764B2

公开(公告)日：2021-05-11

申请号：US15645963

申请日：2017-07-10

Applicant: Sony Interactive Entertainment America LLC

Inventor： Stephen G. Perlman ,  Roger van der Laan

IPC: A63F13/355 ,  A63F13/358 ,  H04N21/2343 ,  H04N21/2381 ,  H04N21/478 ,  H04N21/61 ,  H04N21/6587 ,  H04N21/226 ,  H04N19/159 ,  H04N19/172 ,  H04N19/169 ,  H04N19/61 ,  H04N19/107 ,  H04N19/132 ,  H04N19/14 ,  H04N19/137 ,  H04N19/152 ,  H04N19/436 ,  H04N19/85 ,  H04L29/06 ,  H04L12/825 ,  A63F13/335 ,  H04N7/10 ,  H04W4/00 ,  H04W28/06

Abstract: A server center for hosting low-latency streaming interactive audio/video (A/V) includes a plurality of servers that run one or more applications and an inbound routing network that receives packet streams from users via a first network interface and routes the packets to one or more of the servers. The packet streams include user control input. One or more of the servers are operable to compute A/V data responsive to the user control input. A compression unit is coupled to receive the A/V data from the one or more of the servers and output compressed A/V data therefrom. An output routing network that routes the compressed A/V data to each of the users over a corresponding communication channel via a second interface, the compression unit is operable to modify a compression rate responsive to current characteristics of the corresponding communication channel for each user so as to optimize performance of the one or more applications.

公开(公告)号：US10722791B2

公开(公告)日：2020-07-28

申请号：US16108079

申请日：2018-08-21

Applicant: Sony Interactive Entertainment America LLC

Inventor： Stephen G. Perlman ,  Roger van der Laan

IPC: A63F13/86 ,  A63F13/358 ,  A63F13/335 ,  H04L12/841 ,  H04N7/173 ,  H04N21/222 ,  H04N21/2343 ,  H04N21/238 ,  H04N21/24 ,  H04N21/478 ,  H04N21/61 ,  H04N21/6587 ,  H04N21/8545 ,  H04L29/06 ,  A63F13/30 ,  A63F13/35 ,  H04N21/234 ,  H04N21/6405 ,  H04N21/643 ,  H04N21/81 ,  H04N7/10 ,  H04W4/06 ,  H04W28/16 ,  H04W84/12

Abstract: Systems and methods for creating and delivering an interactive display of thumbnail windows of visual content of different video game sessions are presented. In one embodiment, a method provides for sending, by a server of hosting service, an interactive interface for display on display of a client device. The interactive interface has an array of thumbnail windows and each thumbnail window depicts visual content of different video game sessions being played. Responsive to a selection of a thumbnail window by a user of the client device, the method provides for sending a video stream associated with the selected thumbnail window from the array of thumbnail windows. The video stream enables the user of the client device to view interactive game play of a video game associated with the video stream.

公开(公告)号：US20200230505A1

公开(公告)日：2020-07-23

申请号：US16812191

申请日：2020-03-06

Applicant: Sony Interactive Entertainment America LLC

Inventor： Roger van der Laan ,  Stephen G. Perlman

IPC: A63F13/87 ,  A63F13/355 ,  A63F13/358 ,  A63F13/352 ,  A63F13/30 ,  H04L29/06 ,  A63F13/335 ,  A63F13/33 ,  A63F13/35 ,  H04N21/21 ,  H04N21/214 ,  H04N21/23 ,  H04N21/478 ,  H04N21/61 ,  A63F13/20

Abstract: A system and method are described below for encoding interactive video used in video games streamed from a server to a client. For example, one embodiment of a computer-implemented method for performing video compression, and detecting a maximum data rate of a communication channel between a server and a client. The method includes transmitting a video stream comprising a series of compressed frames from the server to the client. The method includes detecting that the maximum data rate will be exceeded if a particular frame or frames of the sequence of frames are transmitted from the server to the client over the communication channel. The method includes reducing the frame rate of the video stream by adjusting compression of the video frames being sent from the server to the client to avoid exceeding the maximum data rate.

公开(公告)号：US10434425B2

公开(公告)日：2019-10-08

申请号：US15653463

申请日：2017-07-18

Applicant: Sony Interactive Entertainment America LLC

Inventor： Stephen G. Perlman ,  Roger van der Laan

IPC: A63F13/87 ,  A63F13/355 ,  A63F13/358 ,  A63F13/352 ,  A63F13/30 ,  H04L29/06 ,  A63F13/335 ,  A63F13/33 ,  A63F13/35 ,  H04N21/21 ,  H04N21/214 ,  H04N21/23 ,  H04N21/478 ,  H04N21/61 ,  A63F13/20 ,  A63F13/49 ,  H04N7/10 ,  H04W84/12 ,  H04W88/14

Abstract: An apparatus includes one or more servers of a hosting service center operable to execute a twitch video game or real-time application for one or more users remotely located from the hosting service center. The execution of the twitch video game or real-time application on the one or more servers produces uncompressed video of 3D animation. The one or more servers are operable to integrate live video with the twitch video game or real-time application such that the live video appears within the uncompressed video of 3D animation. The live video is received at a hosting service center from one or more client devices correspondingly associated with the one or more users. The uncompressed video 3D animation is generated, at least partially, using data streamed from a high-speed storage unit of the hosting service center coupled to the one or more servers. The high-speed storage unit is configured to load geometry for efficient loading and rendering of objects of the 3D animations. A compression unit compresses the uncompressed video 3D animation integrated with the live video, and compressed streaming interactive video being produced therefrom. An outbound routing network device coupled to the compression unit that transmits the compressed streaming interactive video over a packetized network to the one or more users.

公开(公告)号：US20190151756A1

公开(公告)日：2019-05-23

申请号：US16216988

申请日：2018-12-11

Applicant: Sony Interactive Entertainment America LLC

Inventor： Stephen G. Perlman ,  Roger van der Laan

IPC: A63F13/358 ,  A63F13/335 ,  H04L29/06 ,  H04N21/262 ,  H04L29/08 ,  A63F13/52 ,  A63F9/24 ,  A63F13/355 ,  H04N21/214 ,  H04N21/233 ,  H04N21/2343 ,  H04N21/2381 ,  H04N21/43 ,  H04N21/439 ,  H04N21/478 ,  H04N21/61 ,  H04N19/169 ,  H04N19/436 ,  H04N19/146 ,  H04N19/132 ,  H04N19/107 ,  H04N19/61 ,  H04N19/172 ,  H04N21/6587 ,  H04N21/658 ,  H04N21/6405 ,  H04N21/6377

Abstract: Methods for hosting and operating a service and systems are provided. One method includes executing one or more video games on one or more servers of a hosting service. The one or more video games are played or used by one or more users on one or more client devices remote to the hosting service via compressed streaming interactive video. The compressed streaming interactive video is sent to the one or more client devices where decompression of the compressed streaming interactive video is preformed to display video of each of said one or more video games and said one or more servers are configured to receive user control input to driver interactivity with said one or more video games. The method includes broadcasting the one or more video games as one or more streams of real-time compressed digital video to enable one or more viewing client devices to view play of said one or more video games over the Internet. Each of the one or more viewing client devices receives one or more of the streams of real-time compressed digital video. The method includes overlaying with the streams of real-time compressed digital video, audio from one or more of the viewing client devices, such that the audio is included in the broadcasting of the one or more video games. And, adjusting a compression rate for said broadcasting to each of said viewing client devices based on a respective connection performance each of said viewing client devices has with respect to said one or more servers of the hosting service.

公开(公告)号：US20190060747A1

公开(公告)日：2019-02-28

申请号：US16115503

申请日：2018-08-28

Applicant: Sony Interactive Entertainment America LLC

Inventor： Stephen G. Perlman ,  Roger van der Laan

IPC: A63F13/30 ,  A63F13/77 ,  A63F13/49 ,  A63F13/358 ,  A63F13/25 ,  H04N21/61 ,  H04N21/478 ,  H04N21/214

Abstract: Methods for hosting low-latency streaming interactive audio/video (A/V) include executing one or more video games or applications on a server communicatively coupled to a data network. Packet streams are received from a plurality of users and routed to the one or more video games. The packet streams include user control input that are used to compute A/V data in response. The A/V data are received from the video games or applications. Portions of the A/V data are compressed in parallel using processing units resulting in low-latency streaming compressed A/V data. The low-latency streaming compressed A/V data are routed to each of the users over a corresponding data network communication channel. The executing of video games, receiving of packet streams and A/V data, compressing portions of the A/V data and routing the compressed A/V data are performed with a latency such that at least one user has the perception that the controlled video game is responding instantly.