System and method for creating personalized game experiences

A method for optimizing a computer-implemented game for a target metric is disclosed. Based on a detecting that an optimization point has been reached during a runtime of the computer game, user data, game state data, and personalized gaming experience (PGE) question data is communicated to a PGE server. The PGE question data is linked to the optimization point. An answer corresponding to the PGE question data is received from the PGE server. The answer is selected from a plurality of answers linked to the optimization point based on an application of a machine-learned model to the user data, game state data, and PGE question data. The received answer is implemented within the computer-implemented game.

Method and system for merging distant spaces

A method of merging distant virtual spaces is disclosed. Data describing an environment surrounding a MR merging device is received. A first slice plane is generated, positioned, and displayed within the environment. A second MR merging device is connective with in a second environment. Data describing inbound content from the second MR merging device is received. Content data is sent from the MR merging device to the second MR merging device. The inbound content data is processed and displayed on the first slice plane.

Method and system for aligning a digital model of a structure with a video stream

A method of aligning a digital model of a structure with a displayed portion of the structure within a video stream is disclosed. An approximate position of the camera device in the digital model is determined. A position and an orientation are determined for a plurality of digital surfaces within the digital model visible from the approximate position of the camera device. A position and an orientation of a plurality of object surfaces visible in a video stream are determined. A 3D translation, a 3D scale, and a 3D rotation that maximize an alignment of the position and orientation of the plurality of digital surfaces with the position and orientation of the plurality of object surfaces are determined. The 3D translation, the 3D scale, and the 3D rotation are applied to the digital model and the digital model is displayed contemporaneously with a display of the video stream.

Method and system for populating a digital environment using a semantic map

A method of populating a digital environment with digital content is disclosed. Environment data describing the digital environment is accessed. Populator data describing a populator digital object is accessed. The populator data includes semantic data describing the populator digital object. The populator digital object is placed within the digital environment. A semantic map representation of the populator digital object is generated. The semantic map representation is divided into a plurality of cells. A target cell of the plurality of cells is selected as a placeholder in the digital environment for a digital object that is optionally subsequently instantiated. The selecting of the target cell is based on an analysis of the environment data, the populator data, and the semantic map representation. Placeholder data is recorded in the semantic map representation. The placeholder data includes properties corresponding to the digital object that is optionally subsequently instantiated.

Systems and methods for creating, broadcasting, and viewing 3D content

A method is disclosed for allowing a custom version of video session to be created for presentation on at least one viewer device. Game metadata generated from a live or recorded video feed is received. The game metadata includes three-dimensional modeling data associated with the live or recorded video feed. Viewer metadata collected from a plurality of viewer devices is received. The viewer metadata includes information pertaining to a plurality of responses of a plurality of viewers to a presentation of the video session on a plurality of viewer devices. Additional game metadata is created based on the game metadata and the viewer metadata. The additional game metadata includes camera data based on the three-dimensional modeling data. The additional game metadata is integrated into the game metadata for at least near-real-time presentation of the custom version of the video session.

System and method for temporal manipulation in virtual environments

A system includes a first and a second hand tracking device, the first and second hand tracking devices being configured to provide position information associated with hands of a user; hardware processors; a memory; and a temporal manipulation module. The temporal manipulation module is configured to perform operations including recording changes made in a virtual reality (VR) environment involving the user, thereby creating a recorded content, detecting a first revolution gesture performed by the user with the first and second hand tracking devices, the first revolution gesture including the first and second hand tracking devices revolving in a circular motion about a common axis, stopping the recording based on detecting the first revolution gesture, rewinding through the recorded content based on the first revolution gesture and, during the rewinding, displaying the recorded content in the VR environment, the displaying including altering the VR environment based on the recorded content.

Method and system for creating a neural net based lossy renderer

A method of generating a high-resolution image frame for a state of a video game within a 2D or 3D environment is disclosed. A low-resolution data map of a virtual camera frustum view of the 2D or 3D environment for the state is determined. The data map is of a data type. A high-resolution output data map of the data type is generated from the low-resolution data map. The generating of the high-resolution output data map includes training a neural network. The training includes associating a low-resolution data map of the data type with a high-resolution data map of the data type within the 2D or 3D environment. A high-resolution image of the frustum view is generated from the high-resolution output data map. The generated high-resolution image is displayed on a display device.

Method and system to generate authoring conditions for digital content in a mixed reality environment

Systems and methods for spawning a digital object in an environment are disclosed. Data describing the environment is received. The data includes data describing properties of the environment, a state of the environment, and properties of a plurality of objects within the environment. The data is analyzed to detect and categorize one or more of the plurality of objects, and to detect one or more surfaces related to the plurality of objects. Data is received that describes a placement of the digital object on one of the detected surfaces or detected objects and determines properties of the placement. Conditions are associated with the placed digital object, the conditions including data describing properties of the placement, data describing properties of the detected object, and data describing a state of the detected object. The spawning of the digital object is performed in the environment based on the conditions.

Method and system for transitioning between a 2D video and 3D environment

A system includes a display device, a memory storing 2-dimensional (2D) content and 3-dimensional (3D) game content associated with a content item, and a playback engine. The playback engine is configured to receive the 2D content, the 2D content including one or more reference identifiers (IDs), display the 2D content on the display device for a user and, during display of the 2D content, receive an indication of a transition event. The playback engine is also configured to determine a transition reference identifier (ID) from the one or more reference IDs, identify the 3D game content based on the transition reference ID, render a 3D game environment based on the 3D content, and transition the display device from the displaying the 2D content to providing the 3D game environment.

Method and system for populating a digital environment using a semantic map

A method of populating a digital environment with digital content is disclosed. Environment data describing the digital environment is accessed. Populator data describing a populator digital object is accessed. The populator data includes semantic data describing the populator digital object. The populator digital object is placed within the digital environment. A semantic map representation of the populator digital object is generated. The semantic map representation is divided into a plurality of cells. A target cell of the plurality of cells is selected as a placeholder in the digital environment for a digital object that is optionally subsequently instantiated. The selecting of the target cell is based on an analysis of the environment data, the populator data, and the semantic map representation. Placeholder data is recorded in the semantic map representation. The placeholder data includes properties corresponding to the digital object that is optionally subsequently instantiated.

System and method for generation of 3D virtual objects

A system includes one or more hardware processors, a memory storing vosset data for a virtual object, the vosset data including a first displacement threshold and a first component, and a vosset module executable by the one or more hardware processors. The vosset module is configured to display the first component of the virtual object to a user in a virtual environment, display a manipulator object adjacent to the virtual object in the virtual environment, the manipulator identifies a pre-determined direction of displacement, detect a displacement event associated with the manipulator object, the displacement event being initiated by the user performing a grab event on the manipulator object and extending the manipulator object a displacement distance in the direction of displacement, determine that the displacement distance exceeds a displacement threshold, and alter the virtual object based on detecting the determining that the displacement distance exceeds the displacement threshold ...

Systems and methods for processing a 2D video

A system includes one or more hardware processors, a head mounted display (HMD), a digital camera device, and a path extraction engine. The path extraction engine is configured to receive two-dimensional (2D) video from the digital camera device, the 2D video presenting a 2D view of a three-dimensional (3D) object, the digital camera device moving along a camera path as the digital camera device captures the 2D video, identify a plurality of 2D features associated with the 3D object within the 2D video, track the plurality of 2D features across multiple frames of the 2D video, estimate motion in pixels associated with each 2D feature of the plurality of 2D features, using the estimated motion in the pixels, identify 2D motion caused by the digital camera device moving along the camera path, and compute a first camera position of the digital camera device based on the identified 2D motion.

Method and system for a compiler and decompiler

A method, system, and computer-readable medium including operations for optimizing computer code is disclosed. A block of mixed intermediate representation (MIR) code is received. A partially-decompiled block of computer code is generated from the MIR code. For each instruction in the block of MIR code, in reverse order, a native expression vector for the instruction is computed. A set of pattern-matching operations is repeated until no transformations occur. A fully-decompiled block of computer code is generated from the partially-decompiled block of computer code, the fully-decompiled block of computer code having a semantic level that is raised. The fully-decompiled block of computer code is provided for deployment on an architecture, the deployment including lowering the semantic of the computer code to a level that corresponds to a CPU or GPU supported by the architecture.

Method and system for story development with a dynamic grid

A system includes a display device, one or more hardware processors, and a story development engine executable by the one or more hardware processors. The story development engine is configured to perform operations comprising generating a story grid, the story grid including one or more columns, each column of the one or more columns representing one part in a digital story, displaying the story grid to a user via the display device, receiving a first content element, creating a first cell within a first column, associating the first cell with the first content element, and marking the first cell as a selected cell from a plurality of cells within the first column, the selected cell being identified for inclusion as the one part of the digital story and excluding the other cells of the plurality of cells from the digital story.

Method and system for filtering shadow maps with sub-frame accumulation

A method of rendering an image of an environment is disclosed. Environment data for the environment is accessed. The environment data corresponds to a frame of a video. A plurality of subframes associated with the frame is determined. An angle for each of the plurality of subframes is determined. One or more lights corresponding to the environment are selected. For each light of the one or more lights, a shadow map is generated. The shadow map corresponds to a subframe of the plurality of subframes based on a frustum view oriented at the angle determined for the subframe. The image of the environment is rendered. The rendering includes using the generated shadow map for each light of the one or more lights.

Method and system for automated camera collision and composition preservation

There is described herein systems and methods for camera colliders and shot composition preservation within a 3D virtual environment that prevent a virtual procedural camera from getting stuck behind an object, or penetrating into an object, when filming a subject, while at the same time also maintaining the screen composition of the subject in the camera shot.

Methods and systems for displaying UI elements in mixed reality environments

A method for improving a display of a user interface element in a mixed reality environment is disclosed. A request to display the user interface element is received. The request includes display instructions, angle threshold data, distance threshold data, and velocity threshold data. Display operations are continuously performed while sensor data is continuously received from a mixed reality user interface device. The display operations include displaying the user interface element according to the display instructions, and, based on the sensor data indicating a distance between the user interface element and the mixed reality user interface device in the mixed reality environment has exceeded a distance threshold or based on the sensor data indicating an angle of view of the mixed reality user interface device has exceeded an angle threshold with respect to the user interface element in the mixed reality environment, hiding the user interface element.

Method and system for merging distant spaces

A method of merging distant virtual spaces is disclosed. Data describing an environment surrounding a MR merging device is received. A first slice plane is generated, positioned, and displayed within the environment. A second MR merging device is connective with in a second environment. Data describing inbound content from the second MR merging device is received. Content data is sent from the MR merging device to the second MR merging device. The inbound content data is processed and displayed on the first slice plane.

System and method for providing user interface tools

A system includes one or more hardware processors, a head mounted display configured to display a virtual environment to a user, an input device, and a virtual mini-board module. The mini-board module is configured to render the virtual environment for presentation to the user via the HMD, the virtual environment is rendered from a first perspective providing a field of view of the virtual environment to the user, provide a virtual mini-board to the user within the field of view, the virtual mini-board displaying a region of the virtual environment, detect an interaction event performed by the user on the virtual mini-board, identify the first object based on the interaction event performed on the virtual mini-board, and perform the interaction event on the first object within the virtual environment based on the interaction event performed on the virtual mini-board.

Systems and methods for a virtual reality editor

A system includes processors, a head mounted display, a hand-tracking input device, and an editor engine. The editor engine performs operations including identifying a set of virtual assets, each virtual asset includes data associated with a 3D object, creating a card tray within a virtual environment, creating one or more virtual cards within the card tray, the one or more virtual cards including a first virtual card, the first virtual card is configured with a first interaction mechanic including a triggering gesture and an associated gesture response, the triggering gesture allows the user to interact with the first virtual card within the virtual environment by performing the triggering gesture, detecting performance of the triggering gesture by the user on the first virtual card using the first hand-tracking input device, and based on detecting performance of the triggering gesture, performing the gesture response associated with the first interaction mechanic.

System and method for digital token exchange and delivery

A system includes hardware processors and a token exchange module configured to create a uniquely identified first digital token including an owner ID field identifying the current possessor of the digital token, associate the first digital token with digital content presented to the first user in a mixed reality environment, present the digital within the MR environment, make the first digital token available for acquisition, receive a request to acquire the first digital token, assign possession of the first digital token, via the owner ID field, to the first unique user ID of the first user based on the request to acquire the first digital token, receive a request to transfer the first digital token from the first user to the second user, the second user having a second unique user ID, and changing the owner ID field to the second unique user ID based on the request to transfer.

System and method for creating personalized game experiences

A method for optimizing a computer-implemented game for a target metric is disclosed. Based on a detecting that an optimization point has been reached during a runtime of the computer game, user data, game state data, and personalized gaming experience (PGE) question data is communicated to a PGE server. The PGE question data is linked to the optimization point. An answer corresponding to the PGE question data is received from the PGE server. The answer is selected from a plurality of answers linked to the optimization point based on an application of a machine-learned model to the user data, game state data, and PGE question data. The received answer is implemented within the computer-implemented game.

Method and system for viewing and manipulating interiors of continuous meshes

A method of rendering a view is disclosed. Three occlusion planes associated with an interior cavity of a three-dimensional object included in the view are created. The three occlusion planes are positioned based on a camera position and orientation. Any objects or parts of objects that are in a line of sight between the camera and any one of the three occlusion planes are culled. The view is rendered from the perspective of the camera.

Method and system for intelligent importing and transformation of digital assets

A method of intelligently transforming a digital asset for a target environment is disclosed. Asset data describing the digital asset is received. The received asset data is analyzed to determine a classification type for the digital asset. A database is communicated with to request additional data associated with the determined classification type. The additional data includes semantic data associated with the classification type. The additional data is compared to the received asset data. The comparing includes determining missing data and conflicting data. The missing data includes data from the additional data which is absent from the asset data. The missing data is added to the asset data.

Methods and systems for game system creation

A method for automatically connecting game items for sharing by a plurality of game systems is disclosed. A first game system is added to a computer-implemented game. The first game system is associated with a first game system definition. The first game system definition corresponding to a first game object. The first game system is automatically connected to a second game system. The automatic connecting includes, at least: identifying a correspondence between the first game system definition and a second game system definition, the second game system definition being associated with the second game system, the second game system definition corresponding to a second game object; and preventing a change to the first game object during runtime of the computer-implemented game based on a detection that the change to the first game object would violate a rule associated with the second game system definition.

Determining a job group status based on a relationship between a generation counter value and a ticket value for scheduling the job group for execution

A job scheduler system includes one or more hardware processors, a memory including a job group queue stored in the memory, and a job scheduler engine configured to create a first job group in the job group queue, the first job group includes a generation counter having an initial value, receive a first request to steal the first job group, determine a state of the first job group based at least in part on the generation counter, the state indicating that the first job group is available to steal, based on the determining the state of the first job group, atomically increment the generation counter, thereby making the first job group unavailable for stealing, and alter an execution order of the first job group ahead of at least one other job group in the job group queue.

System and method for texturing in virtual reality and mixed reality environments

A system includes hardware processors and a texturing module configured to display a texturing tray within a virtual reality environment or a mixed reality environment, the texturing tray including a first plurality of texturing mode objects, detect that a user has grabbed a first texturing mode object of the first plurality of texturing mode objects from the texturing tray with a virtual hand of the user, based on motion of the virtual hand after the grabbing, moving the first texturing mode object into a position adjacent to a surface of a target object, detect that the user has applied a texturing agent toward the first texturing mode object, and apply a texture effect to the surface of the target object based on a virtual interaction between the texturing agent and the first texturing mode object, the texture effect altering the appearance of the surface of the target object.

System and method for generating character poses using deep learning

A method of generating or modifying poses in an animation of a character are disclosed. Variable numbers and types of supplied inputs are combined into a single input. The variable numbers and types of supplied inputs correspond to one or more effector constraints for one or more joints of the character. The single input is transformed into a pose embedding. The pose embedding includes a machine-learned representation of the single input. The pose embedding is expanded into a pose representation output. The pose representation output includes local rotation data and global position data for the one or more joints of the character.

Methods and systems for optimal transport of non-linear transformations

Herein is presented a light-weight, high-quality texture synthesis algorithm that generalizes to other applications. We utilize an optimal transport optimization process within a bottleneck layer of an auto-encoder, achieving quality and flexibility on par with expensive back-propagation based neural texture synthesis methods, but at interactive rates. In addition to superior synthesis quality, our statistically motivated approach generalizes better to other special case texture synthesis problems such as Style Transfer, Inverse-Texture Synthesis, Texture Mixing, Multi-Scale Texture Synthesis, Structured Image Hybrids and Texture Painting. We treat the texture synthesis problem as the optimal transport between Probably Density Function of the deep neural activation vectors of the image being synthesized and the exemplar texture. We present a fast algorithm that matches random sliced 1-Dimensional histograms projected from the full N-Dimensional distribution and we propose an extension of ...

Contextual applications in a mixed reality environment

A method of implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user is disclosed. At least one real-world object is identified in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device. A first contextual application of the plurality of contextual applications is used to determine an association between a first set of contextual triggers and a second contextual application of the plurality of contextual applications. A second contextual application is initiated based on a satisfying of the at least one contextual trigger. A function is invoked within the second contextual application based on an interaction of the user with at least one virtual object satisfying a second set of contextual triggers associated with the second contextual application.

System and method for digital token exchange and delivery

A system includes hardware processors and a token exchange module configured to create a uniquely identified first digital token including an owner ID field identifying the current possessor of the digital token, associate the first digital token with digital content presented to the first user in a mixed reality environment, present the digital within the MR environment, make the first digital token available for acquisition, receive a request to acquire the first digital token, assign possession of the first digital token, via the owner ID field, to the first unique user ID of the first user based on the request to acquire the first digital token, receive a request to transfer the first digital token from the first user to the second user, the second user having a second unique user ID, and changing the owner ID field to the second unique user ID based on the request to transfer.

Method and system for improved performance of a video game engine

Methods and apparatuses to improve the performance of a video game engine using an Entity Component System (ECS) are described herein. In accordance with an embodiment, the ECS creates and uses entities, to represent game objects, which are constructed entirely using value data types. The ECS constructs the entities within a memory in a densely packed linear way, and whereby the ECS constantly monitors (e.g., during game play) objects within a game and adjusts the entity distribution within the memory so that a maximum density of memory usage is maintained in real time as the game is being played.

Method and system for matching conditions for digital objects in augmented reality

A method of spawning a digital island in a three-dimensional environment is disclosed. Data describing a three-dimensional environment is accessed. The data is partitioned into a plurality of contexts based on properties identified in the data, the properties corresponding to surfaces or objects in the three-dimensional environment. One or more values of one or more traits corresponding to a context of the plurality of context are identified. A digital island is matched to the context. The matching includes analyzing one or more conditions associated with the digital island with respect to the one or more values of the one or more traits corresponding to the context. Based on the matching, the spawning of the digital island is performed in the three-dimensional environment for the context.

System and method for providing user interface tools

A system includes one or more hardware processors, a head mounted display (HMD) configured to display a virtual environment to a user wearing the HMD, an input device configured to allow the user to interact with virtual objects presented in the virtual environment, and a virtual mini-board module executable by the one or more hardware processors. The virtual mini-board module is configured to perform operations including providing a virtual mini-board to the user within the virtual environment, the virtual mini-board including a representation of a region of the virtual environment, detecting a scroll operation performed by the user, modifying the region of the virtual environment based on the scroll operation, and updating one or more of (1) the virtual environment and (2) the representation of the region of the virtual environment on the virtual mini-board, based on the modifying.

System and method for assymetric rendering to eyes in augmented reality and virtual reality devices

A head-mounted display device including first and second display surfaces associated with first and second eyes of the user, a graphics processing unit, one or more hardware processors, and an adaptive rendering module. The adaptive rendering module is configured to identify a threshold frame time, the threshold frame time representing an upper threshold of time to render frame data by the GPU, receiving a first frame time associated with rendering a first frame to the first eye and second eye of the user, the first frame being rendered at a target resolution, determining that the first frame time exceeds the threshold frame time, and lowering the resolution below the target resolution for parts of a second frame associated with the first eye of the user while maintaining the resolution for parts of the second frame at the target resolution for images associated with the second eye of the user.

Contextual applications in a mixed reality environment

A method of implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user is disclosed. At least one real-world object is identified in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device. A first contextual application of the plurality of contextual applications is used to determine an association between a first set of contextual triggers and a second contextual application of the plurality of contextual applications. A second contextual application is initiated based on a satisfying of the at least one contextual trigger. A function is invoked within the second contextual application based on an interaction of the user with at least one virtual object satisfying a second set of contextual triggers associated with the second contextual application.

Method and system for designing game play and economies

A method of improving game development is disclosed. A game model graph of a video game is created or modified using visual scripting nodes. The game model graph represents one or more game systems and/or one or more economies. The nodes are linked to game resources from the video game. Player profiles describing a plurality of different player types are accessed to be used during a simulation of the game model graph. One or more additional simulations are performed. Each of the one or more additional simulations includes executing the game model graph using the player profiles and the game resources. Data is extracted from the one or more additional simulations to determine behavior of the one or more game systems, the one or more game economies, and/or the resources within the video game over time and across the player types.

Onion skin animation platform and time bar tool

A system includes hardware processor(s), an HMD, an input device, and an onion skin animation module. The animation modules is configured to receive a character rig of a 3D character, receive a first 3D animation of the 3D character, the first 3D animation defines a motion sequence of the 3D character based on the character rig, create a virtual time bar within the virtual environment, the virtual time bar displaying a timeline associated with the first 3D animation, identify a first animation time within the first 3D animation, the first animation time is a point in time during the motion sequence, create a first pose object of the 3D character in the virtual environment, pose the first pose object based on the first 3D animation at the animation time, and positioning the first pose object within the virtual environment proximate the first animation time on the virtual time bar.

Addressable assets in software development

A method for tracking game asset locations is disclosed. Content is created for an application using a first asset. The first asset includes asset data that describes at least a part of the content associated with the asset. The asset data is placed within one or more files within a location within one or more memories. A catalog associated with the application is created on the server. A catalog entry is created for the first asset. The catalog entry includes an address that uniquely identifies the first asset. A request is received from the application for asset data associated with the first asset, the request including the address. The address and the catalog are used to determine location data associated with the address. The determined location data is used to retrieve the asset data at the determined location. The retrieved asset data is returned to the application.

System and method for device synchronization in augmented reality

A method of registering multiple devices to a single coordinate system is disclosed. A digital video is received from a digital camera device. The digital video presents a 2D view of a three-dimensional environment surrounding the mobile device. A set of simulated images is generated based on each video frame in the digital video. The simulated images and video frame are analyzed to determine features and create scale and orientation invariant descriptors that include depth information for the features. The descriptors and other data are sent to the server over a network. The descriptors are matched to feature descriptors from a second mobile device. A registration matrix is determined that represents a transformation from a coordinate system in the mobile device to a coordinate system in the second mobile device. The registration matrix is stored on the server to perform the registering of the multiple devices.

Method and system for transitioning between a 2D video and 3D environment

A system includes one or more hardware processors, a display device, a memory storing 2-dimensional (2D) content and 3-dimensional (3D) content associated with a content item, and a playback engine. The playback engine is configured to receive the 2D content, the 2D content including one or more reference identifiers (IDs), display the 2D content on the display device for a user and, during display of the 2D content, receive an indication of a transition event. The playback engine is also configured to determine a transition reference identifier (ID) from the one or more reference IDs, identify the 3D content based on the transition reference ID, render a 3D environment based on the 3D content, and transition the display device from the displaying the 2D content to the 3D environment.

Method and system for creating character poses in a virtual reality environment

A system includes hardware processor(s), an HMD, an input device, and an onion skin animation module. The animation modules is configured to receive a character rig of a 3D character, receive a first 3D animation of the 3D character, the first 3D animation defines a motion sequence of the 3D character based on the character rig, create a virtual time bar within the virtual environment, the virtual time bar displaying a timeline associated with the first 3D animation, identify a first animation time within the first 3D animation, the first animation time is a point in time during the motion sequence, create a first pose object of the 3D character in the virtual environment, pose the first pose object based on the first 3D animation at the animation time, and positioning the first pose object within the virtual environment proximate the first animation time on the virtual time bar.

Contextual applications in a mixed reality environment

A method of implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user is disclosed. At least one real-world object is identified in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device. A first contextual application of the plurality of contextual applications is used to determine an association between a first set of contextual triggers and a second contextual application of the plurality of contextual applications. A second contextual application is initiated based on a satisfying of the at least one contextual trigger. A function is invoked within the second contextual application based on an interaction of the user with at least one virtual object satisfying a second set of contextual triggers associated with the second contextual application.

Method and system for improved performance of a video game engine

Methods and apparatuses to improve the performance of a video game engine using an Entity Component System (ECS) are described herein. In accordance with an embodiment, the ECS creates and uses entities, to represent game objects, which are constructed entirely using value data types. The ECS constructs the entities within a memory in a densely packed linear way, and whereby the ECS constantly monitors (e.g., during game play) objects within a game and adjusts the entity distribution within the memory so that a maximum density of memory usage is maintained in real time as the game is being played.

Determining a job group status based on a relationship between a generation counter value and a ticket value for scheduling the job group for execution

A job scheduler system includes one or more hardware processors, a memory including a job group queue stored in the memory, and a job scheduler engine configured to create a first job group in the job group queue, the first job group includes a generation counter having an initial value, receive a first request to steal the first job group, determine a state of the first job group based at least in part on the generation counter, the state indicating that the first job group is available to steal, based on the determining the state of the first job group, atomically increment the generation counter, thereby making the first job group unavailable for stealing, and alter an execution order of the first job group ahead of at least one other job group in the job group queue.

Method and system for dynamic notation in a game system

Systems and methods for generating a dynamic note and associating the dynamic note to a game object are disclosed. The game object is selected in a digital environment. The dynamic note is associated to the game object. Data is added to the dynamic note. The added data can include data describing a state of the environment and a state of the game object and include a link to the game object. Data associated with the dynamic note is sent over a network to a data storage device on a server. An aspect of the dynamic note is displayed within the digital environment.

Determining a job group status based on a relationship between a generation counter value and a ticket value for scheduling the job group for execution

A job scheduler system includes one or more hardware processors, a memory including a job group queue stored in the memory, and a job scheduler engine configured to create a first job group in the job group queue, the first job group includes a generation counter having an initial value, receive a first request to steal the first job group, determine a state of the first job group based at least in part on the generation counter, the state indicating that the first job group is available to steal, based on the determining the state of the first job group, atomically increment the generation counter, thereby making the first job group unavailable for stealing, and alter an execution order of the first job group ahead of at least one other job group in the job group queue.

Method and system for real-time animation generation using machine learning

A method of generating an image frame for depicting a posing of a character in a game is disclosed. A dominant atom from a set of active atoms associated with the character is determined. A motion controlling ability from a plurality of abilities of the character is determined. A motion fragment for the dominant atom is determined based on pose data and future trajectory data in a motion library associated with the dominant atom; A predicted future trajectory of the character is calculated. Based on a detection of a future collision or problem between the predicted future trajectory and an environment within the game, a combined future trajectory is calculated based on the predicted future trajectory and an additional future trajectory. The character is posed in the image frame based on a weighted combination of poses associated with the set of active atoms.

Method and system for real-time animation generation using machine learning

A method of generating an image frame for depicting a posing of a character in a game is disclosed. A dominant atom from a set of active atoms associated with the character is determined. A motion controlling ability from a plurality of abilities of the character is determined. A motion fragment for the dominant atom is determined based on pose data and future trajectory data in a motion library associated with the dominant atom; A predicted future trajectory of the character is calculated. Based on a detection of a future collision or problem between the predicted future trajectory and an environment within the game, a combined future trajectory is calculated based on the predicted future trajectory and an additional future trajectory. The character is posed in the image frame based on a weighted combination of poses associated with the set of active atoms.

Method and system for retargeting a human component of a camera motion

A method of retargeting camera motion is disclosed. Data is received that describes a position of a target along a target path. Data is received that describes a position and an orientation of a first camera along a first camera path wherein the first camera is filming the target. A second camera path is computed, the second path corresponding to a second camera. The second camera path corresponds to the first camera path with one or more exceptions. The one or more exceptions include one or more of modification of orientation data corresponding to the second camera path to place the target at the center of a camera screen associated with the second camera and removing of a camera rotation roll for the second camera. A difference is computed between the first camera path and the second camera path. The computed difference is applied to a third camera path.

Systems and methods for processing a 2D video

A system includes one or more hardware processors, a head mounted display (HMD), a digital camera device, and a path extraction engine. The path extraction engine is configured to receive two-dimensional (2D) video from the digital camera device, the 2D video presenting a 2D view of a three-dimensional (3D) object, the digital camera device moving along a camera path as the digital camera device captures the 2D video, identify a plurality of 2D features associated with the 3D object within the 2D video, track the plurality of 2D features across multiple frames of the 2D video, estimate motion in pixels associated with each 2D feature of the plurality of 2D features, using the estimated motion in the pixels, identify 2D motion caused by the digital camera device moving along the camera path, and compute a first camera position of the digital camera device based on the identified 2D motion.

Method and system for automated camera collision and composition preservation

There is described herein systems and methods for camera colliders and shot composition preservation within a 3D virtual environment that prevent a virtual procedural camera from getting stuck behind an object, or penetrating into an object, when filming a subject, while at the same time also maintaining the screen composition of the subject in the camera shot.

Method and system for incremental topological update within a data flow graph in gaming

A method of rendering a virtual environment is disclosed. First application data is received. The first application data includes first graph data. The first graph data corresponds to a first state of an application. Second application data is received after the first application data. The second application data corresponds to a second state of the application. The first application data and the second application data are analyzed to determine a change in the first graph data associated with the second application data. An island subgraph within the first graph data that includes the change is determined. Second graph data is generated for the second state. The generating of the second graph data includes partially reconstructing the first graph data. The partial reconstructing includes rebuilding the determined island subgraph. The generated second graph data is communicated for rendering of the virtual environment in the second state.

Method and system for synchronizing a plurality of augmented reality devices to a virtual reality device

A method of synchronizing digital content between a first mobile device and a second mobile device is disclosed. Device position and orientation data is received at a first application executing in an operating system of the first mobile device. Sensor data is used to determine at least two 3D points associated with a physical location of the second mobile device. At least two 3D points are used to determine an offset for the position, orientation and scale of the first mobile device relative to the second mobile device. The offset and digital content data captured by the first mobile device are shared over a network with a second application executing in an operating system of the second mobile device. The second mobile device uses the offset to display the digital content captured by the first mobile device together with digital content data captured by the second mobile device.

Dynamic assets for creating game experiences

A system includes a memory, processors, and an asset player module configured to identify a playable asset configuration for a first playable asset, including a first graph configuration identifying processing nodes and edges, each node in the graph configuration represents a media processing component configured to modify media inputs to generate a media output, construct a graph in the memory based on the first graph configuration, receive a first set of media inputs, execute the media processing components in an order based on the graph configuration and using the first set of media inputs as the one or more input media components, based on said executing, generate a media output configured to be played by a conventional media player, alter the graph at runtime, thereby changing the media processing components identified within the graph, and execute the media processing components of the graph after the altering.

System and method for frustum context aware digital asset suggestions

A method of determining a suggested digital object to place into a 3D environment is disclosed. Scene data within a frustum volume of a camera within a 3D environment is determined. The scene data includes a set of digital objects that are located within the frustum volume. A set of traits is determined based on the scene data. At least one suggested digital object is suggested for placing into the 3D environment based on a correlation between the suggested digital object and the set of traits.

Method and system for interactive imitation learning in video games

In example embodiments, a method of interactive imitation learning method is disclosed. An input is received from an input device. The input includes data describing a first set of example actions defining a behavior for a virtual character. Inverse reinforcement learning is used to estimate a reward function for the set of example actions. The reward function and the set of example actions is used as input to a reinforcement learning model to train a machine learning agent to mimic the behavior in a training environment. Based on a measure of failure of the training of the machine learning agent reaching a threshold, the training of the machine learning agent is paused to request a second set of example actions from the input device. The second set of example actions is used in addition to the first set of example actions to estimate a new reward function.

System and method for digital token exchange and delivery

A system includes hardware processors and a token exchange module configured to create a uniquely identified first digital token including an owner ID field identifying the current possessor of the digital token, associate the first digital token with digital content presented to the first user in a mixed reality environment, present the digital within the MR environment, make the first digital token available for acquisition, receive a request to acquire the first digital token, assign possession of the first digital token, via the owner ID field, to the first unique user ID of the first user based on the request to acquire the first digital token, receive a request to transfer the first digital token from the first user to the second user, the second user having a second unique user ID, and changing the owner ID field to the second unique user ID based on the request to transfer.

Contextual applications in a mixed reality environment

A method of implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user is disclosed. At least one real-world object is identified in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real-world object that are captured by sensors of the MR-capable device. A first contextual application of the plurality of contextual applications is used to determine an association between a first set of contextual triggers and a second contextual application of the plurality of contextual applications. A second contextual application is initiated based on a satisfying of the at least one contextual trigger. A function is invoked within the second contextual application based on an interaction of the user with at least one virtual object satisfying a second set of contextual triggers associated with the second contextual application.

Method and system for interactive imitation learning in video games

In example embodiments, a method of interactive imitation learning method is disclosed. An input is received from an input device. The input includes data describing a first set of example actions defining a behavior for a virtual character. Inverse reinforcement learning is used to estimate a reward function for the set of example actions. The reward function and the set of example actions is used as input to a reinforcement learning model to train a machine learning agent to mimic the behavior in a training environment. Based on a measure of failure of the training of the machine learning agent reaching a threshold, the training of the machine learning agent is paused to request a second set of example actions from the input device. The second set of example actions is used in addition to the first set of example actions to estimate a new reward function.

Systems and methods for creating, broadcasting, and viewing 3D content

A method is disclosed for allowing a custom version of video session to be created for presentation on at least one viewer device. Game metadata generated from a live or recorded video feed is received. The game metadata includes three-dimensional modeling data associated with the live or recorded video feed. Viewer metadata collected from a plurality of viewer devices is received. The viewer metadata includes information pertaining to a plurality of responses of a plurality of viewers to a presentation of the video session on a plurality of viewer devices. Additional game metadata is created based on the game metadata and the viewer metadata. The additional game metadata includes camera data based on the three-dimensional modeling data. The additional game metadata is integrated into the game metadata for at least near-real-time presentation of the custom version of the video session.

Systems and methods for creating, broadcasting, and viewing 3D content

A method is disclosed for allowing a custom version of video session to be created for presentation on at least one viewer device. Game metadata generated from a live or recorded video feed is received. The game metadata includes three-dimensional modeling data associated with the live or recorded video feed. Viewer metadata collected from a plurality of viewer devices is received. The viewer metadata includes information pertaining to a plurality of responses of a plurality of viewers to a presentation of the video session on a plurality of viewer devices. Additional game metadata is created based on the game metadata and the viewer metadata. The additional game metadata includes camera data based on the three-dimensional modeling data. The additional game metadata is integrated into the game metadata for at least near-real-time presentation of the custom version of the video session.

Determining a job group status based on a relationship between a generation counter value and a ticket value for scheduling the job group for execution

A job scheduler system includes one or more hardware processors, a memory including a job group queue stored in the memory, and a job scheduler engine configured to create a first job group in the job group queue, the first job group includes a generation counter having an initial value, receive a first request to steal the first job group, determine a state of the first job group based at least in part on the generation counter, the state indicating that the first job group is available to steal, based on the determining the state of the first job group, atomically increment the generation counter, thereby making the first job group unavailable for stealing, and alter an execution order of the first job group ahead of at least one other job group in the job group queue.

Addressable assets in software development

A method for tracking game asset locations is disclosed. Content is created for an application using a first asset. The first asset includes asset data that describes at least a part of the content associated with the asset. The asset data is placed within one or more files within a location within one or more memories. A catalog associated with the application is created on the server. A catalog entry is created for the first asset. The catalog entry includes an address that uniquely identifies the first asset. A request is received from the application for asset data associated with the first asset, the request including the address. The address and the catalog are used to determine location data associated with the address. The determined location data is used to retrieve the asset data at the determined location. The retrieved asset data is returned to the application.

Methods and systems for displaying UI elements in mixed reality environments

A method for improving a display of a user interface element in a mixed reality environment is disclosed. A request to display the user interface element is received. The request includes display instructions, angle threshold data, distance threshold data, and velocity threshold data. Display operations are continuously performed while sensor data is continuously received from a mixed reality user interface device. The display operations include displaying the user interface element according to the display instructions, and, based on the sensor data indicating a distance between the user interface element and the mixed reality user interface device in the mixed reality environment has exceeded a distance threshold or based on the sensor data indicating an angle of view of the mixed reality user interface device has exceeded an angle threshold with respect to the user interface element in the mixed reality environment, hiding the user interface element.

Method and system for addressing and segmenting portions of the real world for visual digital authoring in a mixed reality environment

A method of generating a digital island is disclosed. A base shape of a digital island virtual object is created with respect to a local coordinate system of a digital island. Conditions are associated with the digital island. The conditions include criteria for properties of objects within an environment. Additional virtual objects are associated with the digital island. Data describing the environment is received. The data is analyzed to determine one or more parts of the environment that match the conditions. Based on the matching of conditions, a display of the digital island is positioned, scaled, and oriented in a mixed reality display device contemporaneously with a view of the environment. The digital island with is aligned with at least one part of the one or more parts based on the conditions and a movement of one or more sensors and the mixed reality display device.

System and method for creating personalized game experiences

A method for optimizing a computer-implemented game for a target metric is disclosed. Based on a detecting that an optimization point has been reached during a runtime of the computer game, user data, game state data, and personalized gaming experience (PGE) question data is communicated to a PGE server. The PGE question data is linked to the optimization point. An answer corresponding to the PGE question data is received from the PGE server. The answer is selected from a plurality of answers linked to the optimization point based on an application of a machine-learned model to the user data, game state data, and PGE question data. The received answer is implemented within the computer-implemented game.

Method and system for a behavior generator using deep learning and an auto planner

A method of behavior generation is disclosed. Planning state data in a planning domain language format is received and a state description and an associated action description based on the planning state data are generated. The state description and the associated action description are parsed into a series of tokens for a machine learning encoded state and associated ML encoded action. The series of tokens describe the state and the action. The ML encoded state and ML encoded action is processed with a recurrent neural network to generate an estimate of a value of the state description and the action description. Output of the RNN is taken as input into a neural network to generate a value estimate for a state-action pair. A plan that includes a plurality of sequential actions for an agent is generated. The plurality of sequential actions is chosen based on at least the value estimate.

Method and system for synchronizing a plurality of augmented reality devices to a virtual reality device

A method of synchronizing digital content between a first mobile device and a second mobile device is disclosed. Device position and orientation data is received at a first application executing in an operating system of the first mobile device. Sensor data is used to determine at least two 3D points associated with a physical location of the second mobile device. At least two 3D points are used to determine an offset for the position, orientation and scale of the first mobile device relative to the second mobile device. The offset and digital content data captured by the first mobile device are shared over a network with a second application executing in an operating system of the second mobile device. The second mobile device uses the offset to display the digital content captured by the first mobile device together with digital content data captured by the second mobile device.

Systems and methods for creating, broadcasting, and viewing 3d content

A method is disclosed for allowing a custom version of video session to be created for presentation on at least one viewer device. Game metadata generated from a live or recorded video feed is received. The game metadata includes three-dimensional modeling data associated with the live or recorded video feed. Viewer metadata collected from a plurality of viewer devices is received. The viewer metadata includes information pertaining to a plurality of responses of a plurality of viewers to a presentation of the video session on a plurality of viewer devices. Additional game metadata is created based on the game metadata and the viewer metadata. The additional game metadata includes camera data based on the three-dimensional modeling data. The additional game metadata is integrated into the game metadata for at least near-real-time presentation of the custom version of the video session.

Method and system for real-time animation generation using machine learning

A method of generating an image frame for depicting a posing of a character in a game is disclosed. A dominant atom from a set of active atoms associated with the character is determined. A motion controlling ability from a plurality of abilities of the character is determined. A motion fragment for the dominant atom is determined based on pose data and future trajectory data in a motion library associated with the dominant atom; A predicted future trajectory of the character is calculated. Based on a detection of a future collision or problem between the predicted future trajectory and an environment within the game, a combined future trajectory is calculated based on the predicted future trajectory and an additional future trajectory. The character is posed in the image frame based on a weighted combination of poses associated with the set of active atoms.

System and method for creating personalized game experiences

A method for optimizing a computer-implemented game for a target metric is disclosed. Based on a detecting that an optimization point has been reached during a runtime of the computer game, user data, game state data, and personalized gaming experience (PGE) question data is communicated to a PGE server. The PGE question data is linked to the optimization point. An answer corresponding to the PGE question data is received from the PGE server. The answer is selected from a plurality of answers linked to the optimization point based on an application of a machine-learned model to the user data, game state data, and PGE question data. The received answer is implemented within the computer-implemented game.

Method and system for automated camera collision and composition preservation

There is described herein systems and methods for camera colliders and shot composition preservation within a 3D virtual environment that prevent a virtual procedural camera from getting stuck behind an object, or penetrating into an object, when filming a subject, while at the same time also maintaining the screen composition of the subject in the camera shot.

Method and system for automated camera collision and composition preservation

There is described herein systems and methods for camera colliders and shot composition preservation within a 3D virtual environment that prevent a virtual procedural camera from getting stuck behind an object, or penetrating into an object, when filming a subject, while at the same time also maintaining the screen composition of the subject in the camera shot.

Method and system for automated camera collision and composition preservation

There is described herein systems and methods for camera colliders and shot composition preservation within a 3D virtual environment that prevent a virtual procedural camera from getting stuck behind an object, or penetrating into an object, when filming a subject, while at the same time also maintaining the screen composition of the subject in the camera shot.

Contextual applications in a mixed reality environment

A method of implementing a plurality of contextual applications within a mixed reality (MR) environment on an MR-capable device of a user is disclosed. At least one real-world object is identified in the MR environment by applying an object recognition algorithm to one or more attributes of the at least one real- world object that are captured by sensors of the MR-capable device. A first contextual application of the plurality of contextual applications is used to determine an association between a first set of contextual triggers and a second contextual application of the plurality of contextual applications. A second contextual application is initiated based on a satisfying of the at least one contextual trigger. A function is invoked within the second contextual application based on an interaction of the user with at least one virtual object satisfying a second set of contextual triggers associated with the second contextual application.

Contextual applications in mixed reality environments

公开了一种在用户的具有MR功能的装置上在混合现实(MR)环境内实现多个情境应用的方法。通过向由具有MR功能的装置的传感器捕获的至少一个现实世界对象的一个或更多个属性应用对象识别算法，来在MR环境中识别至少一个现实世界对象。使用多个情境应用中的第一情境应用来确定第一情境触发集与多个情境应用中的第二情境应用之间的关联。基于对至少一个情境触发的满足来启动第二情境应用。基于用户与至少一个虚拟对象的交互满足与第二情境应用相关联的第二情境触发集来调用第二情境应用内的功能。

Real-time animation generation using machine learning

A method of generating an image frame for depicting a posing of a character in a game is disclosed. A dominant atom from a set of active atoms associated with the character is determined. A motion controlling ability from a plurality of abilities of the character is determined. A motion fragment for the dominant atom is determined based on pose data and future trajectory data in a motion library associated with the dominant atom; A predicted future trajectory of the character is calculated. Based on a detection of a future collision or problem between the predicted future trajectory and an environment within the game, a combined future trajectory is calculated based on the predicted future trajectory and an additional future trajectory. The character is posed in the image frame based on a weighted combination of poses associated with the set of active atoms.

Method and system for filtering shadow maps with sub-frame accumulation

A method of rendering an image of an environment is disclosed. Environment data for the environment is accessed. The environment data corresponds to a frame of a video. A plurality of subframes associated with the frame is determined. An angle for each of the plurality of subframes is determined. One or more lights corresponding to the environment are selected. For each light of the one or more lights, a shadow map is generated. The shadow map corresponds to a subframe of the plurality of subframes based on a frustum view oriented at the angle determined for the subframe. The image of the environment is rendered. The rendering includes using the generated shadow map for each light of the one or more lights.

Real-time animation generation using machine learning

A method of generating an image frame for depicting a posing of a character in a game is disclosed. A dominant atom from a set of active atoms associated with the character is determined. A motion controlling ability from a plurality of abilities of the character is determined. A motion fragment for the dominant atom is determined based on pose data and future trajectory data in a motion library associated with the dominant atom; A predicted future trajectory of the character is calculated. Based on a detection of a future collision or problem between the predicted future trajectory and an environment within the game, a combined future trajectory is calculated based on the predicted future trajectory and an additional future trajectory. The character is posed in the image frame based on a weighted combination of poses associated with the set of active atoms.

Method and system for intelligent importing and transformation of digital assets

A method of intelligently transforming a digital asset for a target environment is disclosed. Asset data describing the digital asset is received. The received asset data is analyzed to determine a classification type for the digital asset. A database is communicated with to request additional data associated with the determined classification type. The additional data includes semantic data associated with the classification type. The additional data is compared to the received asset data. The comparing includes determining missing data and conflicting data. The missing data includes data from the additional data which is absent from the asset data. The missing data is added to the asset data.

Optimizing lifetime value of computer game players

A method for optimizing LTV related to cumulative future rewards for a player of a plurality of computer-implemented games is disclosed. Data is collected from a game of the plurality of games. The data includes game event data associated with the player, a playing environment within the game, and engine actions performable by the LTV module. The data is analyzed with a first machine-learning (ML) system to create a time-dependent state representation of the game, the player, and the playing environment. The state representation is provided as input to a second ML system to create and optimize an ML policy over time. The ML policy includes a functional relationship proposing a selection of one or more of the engine actions to maximize the LTV. The ML policy and state representation is provided to an LTV optimization module to choose and implement one or more of the engine actions.

System and method for generating character poses using deep learning

A method of generating or modifying poses in an animation of a character are disclosed. Variable numbers and types of supplied inputs are combined into a single input. The variable numbers and types of supplied inputs correspond to one or more effector constraints for one or more joints of the character. The single input is transformed into a pose embedding. The pose embedding includes a machine-learned representation of the single input. The pose embedding is expanded into a pose representation output. The pose representation output includes local rotation data and global position data for the one or more joints of the character.

System and method for texturing in virtual reality and mixed reality environments

A system includes hardware processors and a texturing module configured to display a texturing tray within a virtual reality environment or a mixed reality environment, the texturing tray including a first plurality of texturing mode objects, detect that a user has grabbed a first texturing mode object of the first plurality of texturing mode objects from the texturing tray with a virtual hand of the user, based on motion of the virtual hand after the grabbing, moving the first texturing mode object into a position adjacent to a surface of a target object, detect that the user has applied a texturing agent toward the first texturing mode object, and apply a texture effect to the surface of the target object based on a virtual interaction between the texturing agent and the first texturing mode object, the texture effect altering the appearance of the surface of the target object.

Method and system for aligning a digital model of a structure with a video stream

Methods and systems for game system creation

A method for automatically connecting game items for sharing by a plurality of game systems is disclosed. A first game system is added to a computer-implemented game. The first game system is associated with a first game system definition. The first game system definition corresponding to a first game object. The first game system is automatically connected to a second game system. The automatic connecting includes, at least: identifying a correspondence between the first game system definition and a second game system definition, the second game system definition being associated with the second game system, the second game system definition corresponding to a second game object; and preventing a change to the first game object during runtime of the computer-implemented game based on a detection that the change to the first game object would violate a rule associated with the second game system definition.

软件开发中的可寻址资产

公开了一种用于跟踪游戏资产位置的方法。使用第一资产创建针对应用的内容。第一资产包括描述与该资产相关联的内容的至少一部分的资产数据。将资产数据放置在一个或更多个存储器内的位置内的一个或更多个文件内。在服务器上创建与应用相关联的目录。创建针对第一资产的目录条目。目录条目包括唯一地标识第一资产的地址。接收来自应用的对与第一资产相关联的资产数据的请求，该请求包括地址。使用地址和目录来确定与地址相关联的位置数据。使用所确定的位置数据来检索在所确定的位置处的资产数据。将所检索到的资产数据返回至应用。

System and method for digital token exchange and delivery

A system includes hardware processors and a token exchange module configured to create a uniquely identified first digital token including an owner ID field identifying the current possessor of the digital token, associate the first digital token with digital content presented to the first user in a mixed reality environment, present the digital within the MR environment, make the first digital token available for acquisition, receive a request to acquire the first digital token, assign possession of the first digital token, via the owner ID field, to the first unique user ID of the first user based on the request to acquire the first digital token, receive a request to transfer the first digital token from the first user to the second user, the second user having a second unique user ID, and changing the owner ID field to the second unique user ID based on the request to transfer.

Method and system for incremental topological update within a data flow graph in gaming

A method of rendering a virtual environment is disclosed. First application data is received. The first application data includes first graph data. The first graph data corresponds to a first state of an application. Second application data is received after the first application data. The second application data corresponds to a second state of the application. The first application data and the second application data are analyzed to determine a change in the first graph data associated with the second application data. An island subgraph within the first graph data that includes the change is determined. Second graph data is generated for the second state. The generating of the second graph data includes partially reconstructing the first graph data. The partial reconstructing includes rebuilding the determined island subgraph. The generated second graph data is communicated for rendering of the virtual environment in the second state.

Method and system for dynamic notation in a game system

Systems and methods for generating a dynamic note and associating the dynamic note to a game object are disclosed. The game object is selected in a digital environment. The dynamic note is associated to the game object. Data is added to the dynamic note. The added data can include data describing a state of the environment and a state of the game object and include a link to the game object. Data associated with the dynamic note is sent over a network to a data storage device on a server. An aspect of the dynamic note is displayed within the digital environment.

System and method for digital token exchange and delivery

A system includes hardware processors and a token exchange module configured to create a uniquely identified first digital token including an owner ID field identifying the current possessor of the digital token, associate the first digital token with digital content presented to the first user in a mixed reality environment, present the digital within the MR environment, make the first digital token available for acquisition, receive a request to acquire the first digital token, assign possession of the first digital token, via the owner ID field, to the first unique user ID of the first user based on the request to acquire the first digital token, receive a request to transfer the first digital token from the first user to the second user, the second user having a second unique user ID, and changing the owner ID field to the second unique user ID based on the request to transfer.

用于在混合现实环境中针对视觉数字创作处理和划分真实世界的部分的方法和系统

公开了一种生成数字岛的方法。关于数字岛的局部坐标系，创建数字岛虚拟对象的基本形状。将条件与数字岛相关联。条件包括用于环境内对象的属性的标准。将附加虚拟对象与数字岛相关联。接收描述环境的数据。分析数据以确定与条件匹配的环境的一个或更多个部分。基于条件的匹配，对数字岛同时与环境的视图一起在混合现实显示装置中的显示进行定位、缩放和定向。基于条件以及一个或更多个传感器和混合现实显示装置的运动，将数字岛与一个或更多个部分中的至少一个部分对准。

Method and system for merging distant spaces

A method of merging distant virtual spaces is disclosed. Data describing an environment surrounding a MR merging device is received. A first slice plane is generated, positioned, and displayed within the environment. A second MR merging device is connective with in a second environment. Data describing inbound content from the second MR merging device is received. Content data is sent from the MR merging device to the second MR merging device. The inbound content data is processed and displayed on the first slice plane.

System and method for generating character poses using deep learning

A method of generating or modifying poses in an animation of a character are disclosed. Variable numbers and types of supplied inputs are combined into a single input. The variable numbers and types of supplied inputs correspond to one or more effector constraints for one or more joints of the character. The single input is transformed into a pose embedding. The pose embedding includes a machine-learned representation of the single input. The pose embedding is expanded into a pose representation output. The pose representation output includes local rotation data and global position data for the one or more joints of the character.

Addressable assets in software development

Method and system for aligning a digital model of a structure with a video stream

A method of aligning a digital model of a structure with a displayed portion of the structure within a video stream is disclosed. An approximate position of the camera device in the digital model is determined. A position and an orientation are determined for a plurality of digital surfaces within the digital model visible from the approximate position of the camera device. A position and an orientation of a plurality of object surfaces visible in a video stream are determined. A 3D translation, a 3D scale, and a 3D rotation that maximize an alignment of the position and orientation of the plurality of digital surfaces with the position and orientation of the plurality of object surfaces are determined. The 3D translation, the 3D scale, and the 3D rotation are applied to the digital model and the digital model is displayed contemporaneously with a display of the video stream.

Addressable assets in software development

A method for tracking game asset locations is disclosed. Content is created for an application using a first asset. The first asset includes asset data that describes at least a part of the content associated with the asset. The asset data is placed within one or more files within a location within one or more memories. A catalog associated with the application is created on the server. A catalog entry is created for the first asset. The catalog entry includes an address that uniquely identifies the first asset. A request is received from the application for asset data associated with the first asset, the request including the address. The address and the catalog are used to determine location data associated with the address. The determined location data is used to retrieve the asset data at the determined location. The retrieved asset data is returned to the application.

Method and system for an improved job scheduler

A job scheduler system includes one or more hardware processors, a memory including a job group queue stored in the memory, and a job scheduler engine configured to create a first job group in the job group queue, the first job group includes a generation counter having an initial value, receive a first request to steal the first job group, determine a state of the first job group based at least in part on the generation counter, the state indicating that the first job group is available to steal, based on the determining the state of the first job group, atomically increment the generation counter, thereby making the first job group unavailable for stealing, and alter an execution order of the first job group ahead of at least one other job group in the job group queue.

Method and system for a virtual reality animation tool

A system includes hardware processor(s), an HMD, an input device, and an onion skin animation module. The animation modules is configured to receive a character rig of a 3D character, receive a first 3D animation of the 3D character, the first 3D animation defines a motion sequence of the 3D character based on the character rig, create a virtual time bar within the virtual environment, the virtual time bar displaying a timeline associated with the first 3D animation, identify a first animation time within the first 3D animation, the first animation time is a point in time during the motion sequence, create a first pose object of the 3D character in the virtual environment, pose the first pose object based on the first 3D animation at the animation time, and positioning the first pose object within the virtual environment proximate the first animation time on the virtual time bar.

Addressable assets in software development

A method for tracking game asset locations is disclosed. Content is created for an application using a first asset. The first asset includes asset data that describes at least a part of the content associated with the asset. The asset data is placed within one or more files within a location within one or more memories. A catalog associated with the application is created on the server. A catalog entry is created for the first asset. The catalog entry includes an address that uniquely identifies the first asset. A request is received from the application for asset data associated with the first asset, the request including the address. The address and the catalog are used to determine location data associated with the address. The determined location data is used to retrieve the asset data at the determined location. The retrieved asset data is returned to the application.

ソフトウェア開発におけるアドレス可能なアセット

ゲームアセットの場所を追跡するための方法が開示される。コンテンツは、第１のアセットを使用してアプリケーション用に作成される。第１のアセットは、アセットに関連付けられたコンテンツの少なくとも一部分を記述するアセットデータを含む。アセットデータは、１つ以上のメモリ内の場所において１つ以上のファイル内に配置される。アプリケーションに関連付けられたカタログは、サーバー上に作成される。カタログエントリーは、第１のアセットに対して作成される。カタログエントリーは、第１のアセットを一意に識別するアドレスを含む。第１のアセットに関連付けられたアセットデータの要求がアプリケーションから受信され、要求はアドレスを含む。アドレスおよびカタログを用いてアドレスに関連付けられたロケーションデータを決定する。決定されたロケーションデータを用いて決定された場所のアセットデータを検索する。検索したアセットデータはアプリケーションに返される。

Methods and systems for optimal transport of non-linear transformations

Herein is presented a light-weight, high-quality texture synthesis algorithm that generalizes to other applications. We utilize an optimal transport optimization process within a bottleneck layer of an auto-encoder, achieving quality and flexibility on par with expensive back-propagation based neural texture synthesis methods, but at interactive rates. In addition to superior synthesis quality, our statistically motivated approach generalizes better to other special case texture synthesis problems such as Style Transfer, Inverse-Texture Synthesis, Texture Mixing, Multi-Scale Texture Synthesis, Structured Image Hybrids and Texture Painting. We treat the texture synthesis problem as the optimal transport between Probably Density Function of the deep neural activation vectors of the image being synthesized and the exemplar texture. We present a fast algorithm that matches random sliced 1-Dimensional histograms projected from the full N-Dimensional distribution and we propose an extension of this algorithm that reduces dimensionality of neural feature space.

Methods and systems for displaying ui elements in mixed reality environments

A method for improving a display of a user interface element in a mixed reality environment is disclosed. A request to display the user interface element is received. The request includes display instructions, angle threshold data, distance threshold data, and velocity threshold data. Display operations are continuously performed while sensor data is continuously received from a mixed reality user interface device. The display operations include displaying the user interface element according to the display instructions, and, based on the sensor data indicating a distance between the user interface element and the mixed reality user interface device in the mixed reality environment has exceeded a distance threshold or based on the sensor data indicating an angle of view of the mixed reality user interface device has exceeded an angle threshold with respect to the user interface element in the mixed reality environment, hiding the user interface element.

Method and system for behavior generation with a trait based planning domain language

A method for generating behavior with a trait-based planning domain language is disclosed. A world model of a dynamic environment is created. The world model includes data defining a state for the world model. The data defining the state includes data describing objects within the environment. Input to update the state for the world model is received. The input includes data to change the state and data defining a goal for a future state. A machine-learning model is used to generate a planning state from the state for the world model. The planning state includes a plurality of planning domain objects and associated traits. Based on instructions associated with an action, one or more of modifying values within a trait associated with the planning domain object, adding a trait to the planning domain object, or removing a trait from the planning domain object are performed.

Determining a job group status based on a relationship between a generation counter value and a ticket value for scheduling the job group for execution

A job scheduler system includes one or more hardware processors, a memory including a job group queue stored in the memory, and a job scheduler engine configured to create a first job group in the job group queue, the first job group includes a generation counter having an initial value, receive a first request to steal the first job group, determine a state of the first job group based at least in part on the generation counter, the state indicating that the first job group is available to steal, based on the determining the state of the first job group, atomically increment the generation counter, thereby making the first job group unavailable for stealing, and alter an execution order of the first job group ahead of at least one other job group in the job group queue.

Method and system for displaying a large 3d model on a remote device

A method of displaying a digital object on a device is disclosed. A set of unique identifiers and associated is downloaded for a set of digital objects within a 3D model. A position and an orientation of a virtual camera is determined relative to the positions of the set of digital objects. A visibility score for each digital object of the set of digital objects is computed based on a position of the digital object relative to the position and the orientation of the virtual camera. A polygon mesh associated with a digital object with a highest visibility score is downloaded based on the polygon mesh being absent. The downloading uses a unique identifier of the set of unique identifiers associated with the polygon mesh. The polygon mesh of the digital object with the highest score is rendered and displayed on a display device.

Method and system for merging distant spaces

A method of merging distant virtual spaces is disclosed. Data describing an environment surrounding a MR merging device is received. A first slice plane is generated, positioned, and displayed within the environment. A second MR merging device is connective with in a second environment. Data describing inbound content from the second MR merging device is received. Content data is sent from the MR merging device to the second MR merging device. The inbound content data is processed and displayed on the first slice plane.

Determining a job group status based on a relationship between a generation counter value and a ticket value for scheduling the job group for execution

A job scheduler system includes one or more hardware processors, a memory including a job group queue stored in the memory, and a job scheduler engine configured to create a first job group in the job group queue, the first job group includes a generation counter having an initial value, receive a first request to steal the first job group, determine a state of the first job group based at least in part on the generation counter, the state indicating that the first job group is available to steal, based on the determining the state of the first job group, atomically increment the generation counter, thereby making the first job group unavailable for stealing, and alter an execution order of the first job group ahead of at least one other job group in the job group queue.

Determining a job group status based on a relationship between a generation counter value and a ticket value for scheduling the job group for execution

A job scheduler system includes one or more hardware processors, a memory including a job group queue stored in the memory, and a job scheduler engine configured to create a first job group in the job group queue, the first job group includes a generation counter having an initial value, receive a first request to steal the first job group, determine a state of the first job group based at least in part on the generation counter, the state indicating that the first job group is available to steal, based on the determining the state of the first job group, atomically increment the generation counter, thereby making the first job group unavailable for stealing, and alter an execution order of the first job group ahead of at least one other job group in the job group queue.

Method and system for designing game play and economies

A method of improving game development is disclosed. A game model graph of a video game is created or modified using visual scripting nodes. The game model graph represents one or more game systems and/or one or more economies. The nodes are linked to game resources from the video game. Player profiles describing a plurality of different player types are accessed to be used during a simulation of the game model graph. One or more additional simulations are performed. Each of the one or more additional simulations includes executing the game model graph using the player profiles and the game resources. Data is extracted from the one or more additional simulations to determine behavior of the one or more game systems, the one or more game economies, and/or the resources within the video game over time and across the player types.

Method and system for addressing and segmenting portions of the real world for visual digital authoring in a mixed reality environment

A method of generating a digital island is disclosed. A base shape of a digital island virtual object is created with respect to a local coordinate system of a digital island. Conditions are associated with the digital island. The conditions include criteria for properties of objects within an environment. Additional virtual objects are associated with the digital island. Data describing the environment is received. The data is analyzed to determine one or more parts of the environment that match the conditions. Based on the matching of conditions, a display of the digital island is positioned, scaled, and oriented in a mixed reality display device contemporaneously with a view of the environment. The digital island with is aligned with at least one part of the one or more parts based on the conditions and a movement of one or more sensors and the mixed reality display device.

Methods and systems for optimal transport of non-linear transformations

Herein is presented a light-weight, high-quality texture synthesis algorithm that generalizes to other applications. We utilize an optimal transport optimization process within a bottleneck layer of an auto-encoder, achieving quality and flexibility on par with expensive back-propagation based neural texture synthesis methods, but at interactive rates. In addition to superior synthesis quality, our statistically motivated approach generalizes better to other special case texture synthesis problems such as Style Transfer, Inverse-Texture Synthesis, Texture Mixing, Multi-Scale Texture Synthesis, Structured Image Hybrids and Texture Painting. We treat the texture synthesis problem as the optimal transport between Probably Density Function of the deep neural activation vectors of the image being synthesized and the exemplar texture. We present a fast algorithm that matches random sliced 1-Dimensional histograms projected from the full N-Dimensional distribution and we propose an extension of this algorithm that reduces dimensionality of neural feature space.

Method and system for behavior generation with a trait based planning domain language

A method for generating behavior with a trait-based planning domain language is disclosed. A world model of a dynamic environment is created. The world model includes data defining a state for the world model. The data defining the state includes data describing objects within the environment. Input to update the state for the world model is received. The input includes data to change the state and data defining a goal for a future state. A machine-learning model is used to generate a planning state from the state for the world model. The planning state includes a plurality of planning domain objects and associated traits. Based on instructions associated with an action, one or more of modifying values within a trait associated with the planning domain object, adding a trait to the planning domain object, or removing a trait from the planning domain object are performed.

Determining a job group status based on a relationship between a generation counter value and a ticket value for scheduling the job group for execution

A job scheduler system includes one or more hardware processors, a memory including a job group queue stored in the memory, and a job scheduler engine configured to create a first job group in the job group queue, the first job group includes a generation counter having an initial value, receive a first request to steal the first job group, determine a state of the first job group based at least in part on the generation counter, the state indicating that the first job group is available to steal, based on the determining the state of the first job group, atomically increment the generation counter, thereby making the first job group unavailable for stealing, and alter an execution order of the first job group ahead of at least one other job group in the job group queue.

Method and system for transitioning between a 2d video and 3d environment

A system includes a display device, a memory storing 2-dimensional (2D) content and 3-dimensional (3D) game content associated with a content item, and a playback engine. The playback engine is configured to receive the 2D content, the 2D content including one or more reference identifiers (IDs), display the 2D content on the display device for a user and, during display of the 2D content, receive an indication of a transition event. The playback engine is also configured to determine a transition reference identifier (ID) from the one or more reference IDs, identify the 3D game content based on the transition reference ID, render a 3D game environment based on the 3D content, and transition the display device from the displaying the 2D content to providing the 3D game environment.

Contextual applications in a mixed reality environment

Method and system for interactive imitation learning in video games

In example embodiments, a method of interactive imitation learning method is disclosed. An input is received from an input device. The input includes data describing a first set of example actions defining a behavior for a virtual character. Inverse reinforcement learning is used to estimate a reward function for the set of example actions. The reward function and the set of example actions is used as input to a reinforcement learning model to train a machine learning agent to mimic the behavior in a training environment. Based on a measure of failure of the training of the machine learning agent reaching a threshold, the training of the machine learning agent is paused to request a second set of example actions from the input device. The second set of example actions is used in addition to the first set of example actions to estimate a new reward function.

Method and system for real-time animation generation using machine learning

用于管理故事中的对象的情绪相关性的方法和系统

公开了一种用于确定与视频游戏相关联的视频的一帧内的感兴趣对象的电影镜头质量的方法。从来自视频游戏内的对象集合中确定多个感兴趣对象。从游戏接收游戏状态数据。游戏状态数据描述在游戏内发生的至少事件集合。确定多个感兴趣水平值。针对多个感兴趣对象确定多个饥饿值。针对多个感兴趣对象确定多个紧急性值。生成用于管理与该帧相关联的电影摄影的数据结构。该数据结构便利于查找多个感兴趣水平值和紧急性值。

用于在混合现实环境中针对视觉数字创作处理和划分真实世界的部分的方法和系统

公开了一种生成数字岛的方法。关于数字岛的局部坐标系，创建数字岛虚拟对象的基本形状。将条件与数字岛相关联。条件包括用于环境内对象的属性的标准。将附加虚拟对象与数字岛相关联。接收描述环境的数据。分析数据以确定与条件匹配的环境的一个或更多个部分。基于条件的匹配，对数字岛同时与环境的视图一起在混合现实显示装置中的显示进行定位、缩放和定向。基于条件以及一个或更多个传感器和混合现实显示装置的运动，将数字岛与一个或更多个部分中的至少一个部分对准。

软件开发中的可寻址资产

公开了一种用于跟踪游戏资产位置的方法。使用第一资产创建针对应用的内容。第一资产包括描述与该资产相关联的内容的至少一部分的资产数据。将资产数据放置在一个或更多个存储器内的位置内的一个或更多个文件内。在服务器上创建与应用相关联的目录。创建针对第一资产的目录条目。目录条目包括唯一地标识第一资产的地址。接收来自应用的对与第一资产相关联的资产数据的请求，该请求包括地址。使用地址和目录来确定与地址相关联的位置数据。使用所确定的位置数据来检索在所确定的位置处的资产数据。将所检索到的资产数据返回至应用。

混合现实环境中的情境应用

公开了一种在用户的具有MR功能的装置上在混合现实(MR)环境内实现多个情境应用的方法。通过向由具有MR功能的装置的传感器捕获的至少一个现实世界对象的一个或更多个属性应用对象识别算法，来在MR环境中识别至少一个现实世界对象。使用多个情境应用中的第一情境应用来确定第一情境触发集与多个情境应用中的第二情境应用之间的关联。基于对至少一个情境触发的满足来启动第二情境应用。基于用户与至少一个虚拟对象的交互满足与第二情境应用相关联的第二情境触发集来调用第二情境应用内的功能。