公开(公告)号：US20240307783A1

公开(公告)日：2024-09-19

申请号：US18121268

申请日：2023-03-14

Applicant: Snap Inc.

Inventor： Willi Menapace ,  Aliaksandr Siarohin ,  Sergey Tulyakov

IPC: A63F13/67 ,  A63F13/57

CPC classification number: A63F13/67 ,  A63F13/57

Abstract: A framework trains game-engine-like neural models from annotated videos to generate a Learnable Game Engine (LGE) that maintains states of the scene, objects and agents in it, and enables rendering the environment from a controllable viewpoint. The LGE models the logic of the game and the rules of physics, making it possible for the user to play the game by specifying both high- and low-level action sequences. The LGE also unlocks a director's mode where the game is played by plotting behind the scenes, specifying high-level actions and goals for the agents using text-based instructions. To implement the director's mode, a trained diffusion-based animation model navigates the scene using high-level constraints, to enable play against an adversary, and to devise the strategy to win a point. To render the resulting state of the environment and its agents, a compositional neural radiance field (NeRF) representation is used in a synthesis model.