Augmented reality enhanced situational awareness

A method for enhancing the situational awareness of a user to a problem within an area. The method includes one or more computer receiving visual information corresponding to an area from a device associated with a user. The method further includes receiving data from a group of one or more sensors within the area, where the area includes a plurality of physical elements. The method further includes determining that a first problem is present within the area and a first physical element in the area that corresponds to the first problem, based on analysing the data received from the group of sensors. The method further includes generating augmented reality (AR) content related to the first problem present within the area. The method further includes displaying, via the device associated with the user, the generated AR content related to the problem within the visual information corresponding to the area.

Augmented reality enabled autonomous vehicle command centre

Data processing apparatus and method

A method of data processing comprises receiving an image 400 of a user in an environment, determining a region 402 of the environment in the image in which motion of the user is expected, and identifying one or more attributes of a detected object 300-303 in the image, a portion of said object being within the determined region. An appropriate predetermined process is then selected and performed. The motion of the user is associated with interactive content experienced by the user, such as virtual reality (VR) content, and the determined region may be a play area for the VR system. The attribute(s) may comprise a classification of the detected object and/or one or more physical characteristics of the detected object such as hardness, weight, fragility, etc. (see figure 5). The predetermined process may comprise causing a visual indicator 603A-C to be output to indicate to the user the physical characteristics of the detected object(s) and/or to remove the detected object(s) from the play ...

컨텍스트 기반 증강 현실 통신

... 본 개시내용의 양태들은 네트워크를 통해 다수의 사용자들 사이의 텍스트 기반 증강 현실 통신을 용이하게 하는 적어도 하나의 프로그램을 저장한 컴퓨터 판독가능 저장 매체를 포함하는 시스템, 방법, 및 사용자 인터페이스를 수반한다. 제2 디바이스를 통해 제2 사용자에게 보이는 실세계 환경에 적용하기 위한 가상 콘텐츠 아이템의 제1 사용자에 의한 선택을 나타내는 가상 콘텐츠 아이템 구성 데이터가 제1 디바이스로부터 수신된다. 가상 콘텐츠 아이템 구성 데이터는 또한 실세계 환경에 대한 가상 콘텐츠 아이템의 적용을 트리거하기 위한 하나 이상의 기준을 포함한다. 제2 디바이스에서 생성된 컨텍스트 데이터로부터 결정된 하나 이상의 기준의 충족에 기초하여 트리거링 이벤트가 검출된다. 제2 디바이스는 트리거링 이벤트에 기초하여 제2 사용자에게 보이는 실세계 환경에 오버레이된 가상 콘텐츠 아이템을 제시한다.

AUGMENTED REALITY ENABLED DYNAMIC PRODUCT PRESENTATION

Methods and systems for providing a dynamic product presentation are disclosed. In one example, a method comprises providing, by a processor, a three-dimensional representation of a product in a virtual environment for display on a customer device; and responsive to the processor identifying a surface in a camera feed of the customer device having a dimensionality suitable for the product, generating, by the processor, an augmented media containing an augmented reality representation of a three-dimensional model for the product on the surface.

Augmented reality based image protection in enterprise settings

Disclosed are various examples for augmented reality based image protection in enterprise settings. In one example, a managed camera application can generate an artificial reality based camera user interface using image data from a field of view of a camera. An indoor position can be identified using global positioning system (GPS) and indoor positioning data. A sector of the field of view can be identified as a protected image area that depicts a protected or confidential object, and the user interface can be updated to include an AR user interface element that is generated relative to the protected image area that is identified.

Systems and methods for generating graphical user interfaces

A system for generating graphical user interfaces. The system may include processors and storage devices storing instructions. The instructions may configure the one or more processors to perform operations including identifying a plurality of attributes from an image captured with a client device, identifying a plurality of first results based on the attributes, generating a first graphical user interface for display in the client device. The first graphical user interface may include a plurality of result icons corresponding to a subset of the first results having confidence scores above a threshold, a plurality of filter icons displaying options, and a search button. The operations may also include receiving a selection of at least one of the result icons or at least one of the filter icons, performing a search, based on the selection, and generating a second graphical user interface.

Generating and rendering motion graphics effects based on recognized content in camera view finder

Systems and methods are described for providing co-presence in an augmented reality environment. The method may include receiving a visual scene within a viewing window depicting a multi-frame real-time visual scene captured by a camera onboard an electronic device associated with the augmented reality environment, identifying a plurality of elements of the visual scene, detecting at least one graphic indicator associated with at least one of the plurality of elements, detecting at least one boundary associated with the at least one element, and generating, in the viewing window and based on the detection of the at least one graphic indicator, Augmented Reality (AR) motion graphics within the detected boundary. In response to determining that content related to the at least one element is available, the method may include retrieving the content and visually indicating an AR tracked control on the at least one element within the viewing window.

Methods and systems for training an object detection algorithm using synthetic images

A method includes: (A) receiving a selection of a 3D model stored in one or more memories, the 3D model corresponding to an object and (B) setting a camera parameter set for a camera for use in detecting a pose of the object in a real scene. The method also includes (C) generating at least one 2D synthetic image based at least on the camera parameter set by rendering the 3D model in a view range for generating training data.

SYSTEMS AND METHODS FOR OBSCURING RESTRICTED TEXT AND/OR IMAGES IN A VIDEO CONFERENCING SESSION

Systems and methods for obscuring images and/or text during a screen sharing operation in a video conferencing session are described herein. In some embodiments, a client device detects a screen sharing operation. As part of the screen sharing operation, the client device captures an image of a display. The client device recognizes images and/or text in the image of the display and determines whether any of the images and/or text are restricted. If the images and/or text are determined to be restricted, the client device obscures the images and/or text prior to encoding of the image of the display for transmission.

System for physical-virtual environment fusion

A sensing system includes a sensor with a computing system and a memory in communication with the computing system, the memory storing a plurality of endpoints. The computing system is configured to receive activity preferences from a device at an endpoint and further determines the likeability of the activities at the endpoint. Further, it receives semantic identification preferences from the device in communication with the computing system and the system blurs the corresponding semantic identities based on the received preferences.

Eye glint imaging in see-through computer display systems

Disclosure herein concerns a method that includes illuminating a user's eye with an illumination source in a head-worn display, capturing an image of the user's eye with an eye camera in the head-worn display, wherein the image includes an eye glint produced by light from the illumination source that is reflected from a surface of the user's eye, determining a size of an eye glint in the captured image, and identifying a change in focus distance for the user's eye in correspondence with a change in the size of the eye glint.

Eyepieces for augmented reality display system

An eyepiece waveguide for an augmented reality display system may include an optically transmissive substrate, an input coupling grating (ICG) region, a multi-directional pupil expander (MPE) region, and an exit pupil expander (EPE) region. The ICG region may receive an input beam of light and couple the input beam into the substrate as a guided beam. The MPE region may include a plurality of diffractive features which exhibit periodicity along at least a first axis of periodicity and a second axis of periodicity. The MPE region may be positioned to receive the guided beam from the ICG region and to diffract it in a plurality of directions to create a plurality of diffracted beams. The EPE region may overlap the MPE region and may out couple one or more of the diffracted beams from the optically transmissive substrate as output beams.

USER SCANNING AND ONE-WAY AUGMENTED REALITY VIEWING SYSTEM

A method for providing augmented reality content includes identifying a customer at a customer interaction location, retrieving customer-specific interaction data, determining locations of a customer service representative in an area adjacent to a first side of a transparent panel and the customer in an area adjacent to a second side of the transparent panel, and projecting an augmented reality element to the first side of a transparent panel, the augmented reality element displaying the customer-specific interaction data on the first side of the transparent panel.

Generating assistive indications based on detected characters

Methods, systems, devices, and tangible non-transitory computer readable media for generating assistive indications are provided. The disclosed technology can include accessing image data that includes at least one image. Character data can be generated based at least in part on the image data and one or more optical character recognition operations. Further, the character data can include one or more characters associated with the at least one image. One or more characters that are associated with one or more recognized words and the one or more characters that are associated with one or more unrecognized words can be determined based on the character data. One or more auditory indications including a synthetic voice reciting the one or more recognized words and the one or more unrecognized words can be generated. Furthermore, the synthetic voice can recite each of the one or more unrecognized words one character at a time.

Information providing system, information providing method and management apparatus for information providing system for providing information obtained through a robot

An information providing system includes: a memory unit configured to store an image data acquired by an imaging unit a robot in association with a position data and a time data acquired by an information acquisition portion; a data extraction portion configured to extract an image data corresponding to a time and a position instructed by the user from among the stored image data; and a display unit configured to display an image based on the extracted image data. The memory unit stores the image data acquired by the imaging unit of a first robot moving with a first user and the image data acquired by the imaging unit of a second robot moving with a second user. The data extraction portion extracts an image data corresponding to a time and a position instructed by the first user from among the image data acquired by the second robot stored in the memory unit when the image data corresponding to the time and the position instructed by the first user is not present among the image data acquired ...

Item collection guidance system

A method includes: obtaining a task definition including (i) an item identifier, (ii) an original item quantity, and (iii) a normalized item quantity; presenting the normalized item quantity; obtaining an image of a collected item set; deriving collected item data from the image; determining whether the collected item data matches the item identifier and the original item quantity; and generating first output data or second output data according to whether the collected item data matches the item identifier and the original item quantity, or does not match the item identifier and the original item quantity.

Systems and methods for using augmented reality to locate objects, identify persons, and interact with inanimate objects

A system is provided which utilizes multiple combinations of object location technology to locate objects and direct users to them, and which provides reliable owner recognition and ownership verification with the use of displayed augmented reality with a predefined image of the object and/or the user. Further, the system utilizes augmented reality fingerprint markers. When the augmented reality fingerprint marker is positioned on an object and scanned with a smart device, information relating to the object is superimposed on the object displayed on the smart device.

Object replacement system

Aspects of the present disclosure involve a system comprising a computer-readable storage medium storing a program and a method for performing operations comprising: receiving an image that includes a depiction of a real-world environment; processing the image to obtain data indicating presence of a real-world object in the real-world environment; receiving input that selects an AR experience comprising an AR object; determining that the real-world object detected in the real-world environment depicted in the image indicated in the obtained data corresponds to the AR object; applying a machine learning technique to the image to generate a new image that depicts the real-world environment without the real-world object; and applying the AR object to the new image to generate a modified new image that depicts the real-world environment including the AR object in place of the real-world object.

SYNCHRONIZING PHYSICAL AND VIRTUAL ENVIRONMENTS USING QUANTUM ENTANGLEMENT

An embodiment includes accessing captured video of a first physical object in a physical environment. The embodiment also includes detecting a feature of the first physical object in a first frame of the video. The embodiment encodes a first qubit with a first quantum state based on a first value of the feature of the first physical object, and then entangles the first qubit with a second qubit forming an entangled qubit pair. The embodiment detects a second quantum state of the second qubit of the entangled qubit pair and renders computer-generated content of a virtual environment that includes a computer-generated first virtual object that corresponds to the first physical object, where the rendering comprises determining a property of the first virtual object based on the detected second quantum state.

Roomfinder platform

A method may include receiving, from a client device, a reservation time and image data relating to a desired room; receiving, from a network storage device, facility data for a plurality of rooms; identifying, using an image recognition model, the desired room based on the image data and the facility data; determining an availability of the desired room based on the reservation time and the facility data; generating a first reservation option to reserve the desired room and/or a second reservation option to reserve an alternate room; transmitting, to the client device, the first reservation option and/or the second reservation option; receiving, from the client device, a user selection of the first reservation option and/or the second reservation option; and transmitting, to the network storage device, an instruction to reserve the desired room and/or the alternate room.

SYSTEMS/METHODS FOR IDENTIFYING PRODUCTS WITHIN AUDIO-VISUAL CONTENT AND ENABLING SEAMLESS PURCHASING OF SUCH IDENTIFIED PRODUCTS BY VIEWERS/USERS OF THE AUDIO-VISUAL CONTENT

An automated system/method for identifying and enabling viewer selection/purchase of products or services associated with digital content presented on a display device. Products within the digital content are identified and existing product placement data is ascertained. For products that do not include such data, other methodologies, with the assistance of third-party servers, are employed to assess identity and purchase availability. Viewer input designate products to assess or products can be automatically assessed. Viewers initiate purchase of identified products via the display device or other electronic devices controlled by viewers, such as via viewers' smart phones. Various processes for identifying products include use of AI processing, access to data on third-party servers, crowd sourcing and other methodologies. Various techniques for selecting products for purchases are employed including employing 3D codes (e.g., QR codes) alongside presented products to enable other portable ...

ELECTRONIC DEVICE FOR DISPLAYING VISUAL OBJECT BASED ON LOCATION OF EXTERNAL ELECTRONIC DEVICE AND METHOD THEREOF

According to an embodiment, a processor of a wearable device is configured to identify, based on an image output from a camera, an external electronic device. The processor, based on the identification, is configured to request outputting of an optical signal to the external electronic device through a communication circuit. The processor is configured to identify, in a portion of the image where the external electronic device is displayed, a position of the optical signal output from the external electronic device. The processor is configured to control a display to display a visual object having an extended shape based on the identified position in the display.

VISUAL TEACH AND REPEAT MOBILE MANIPULATION SYSTEM

MEDIA ANNOTATION WITH PRODUCT SOURCE LINKING

COMPUTER VISION CAMERAS FOR INFRARED LIGHT DETECTION

OCCLUSION DETECTION SYSTEM

SYSTEMS, METHODS, AND GRAPHICAL USER INTERFACES FOR ANNOTATING, MEASURING, AND MODELING ENVIRONMENTS

A computer system displays an annotation placement user interface that includes a representation of a field of view of one or more cameras that is updated over time based on changes in the field of view, and a placement user interface element indicating a virtual annotation placement location. If the placement user interface element is over a representation of a first type of feature in the physical environment, the appearance of the placement user interface element changes to indicate an anchor point corresponding to the first type of feature, and the system displays a first set of guides. If the placement user interface element is over a representation of a second, different type of feature in the physical environment, the appearance of the placement user interface element changes to indicate an anchor point corresponding to the second type of feature, and the system displays a second, different set of guides.

Model training method and apparatus, and prediction method and apparatus

Mobile device based VR control

... the present disclosure relates to controlling virtual reality (VR) content displayed on a VR head mounted display (HMD). Communication can be established between a computer system, a VR HMD, and a mobile device. A user input configured to control VR content displayed on a display of the VR HMD can be received on the mobile device. The VR content displayed on the VR HMD can then be controlled based on the user input received on the mobile device.

Interface translation using one or more neural networks

Using one or more neural networks 304, 306, 308, 310 to generate second graphical user interfaces (GUI) 316 based, at least in part, on a first graphical user interface 302. The neural networks may further be used to generate the second graphical user interfaces based on visual features of the first graphical user interfaces. The visual and functional features may be translated to a set of feature vectors adhering to a common schema. The feature vectors may be encoded into a latent space which define constraints of the second graphical user interface by variational autoencoders which may be included in the neural networks. The neural networks may include generative adversarial networks (GAN). This allows for graphical user interfaces to be translated onto alternate devices while preserving the functionality of the original interface.

Optical authentication structure with augmented reality feature

Disclosed is an optical authentication structure 1, with machine-executable instructions and method for authenticating it. The instructions are executed by a processor of an authentication apparatus 10, e.g. a smartphone or tablet computer. The instructions enable the apparatus to read a machine-readable security feature 3 on the optical authentication structure, and cause a display 4 of the apparatus to show the optical authentication structure and augmented reality imagery 5. The augmented reality imagery allows the optical authentication structure to be authenticated. Data captured from the machine-readable security feature may be compared to determine whether the optical authenticating structure is genuine/authentic. The augmented reality imagery may provide a likeness to a visible reference image arranged on the optical authentication structure as an authenticating reference. The optical authentication structure is arranged to be read by a camera 6 of the apparatus to generate the ...

An image generator and method for an augmented reality system

Displaying an Augmented Reality (AR) image of a real-world environment having one of more objects located therein and one or more AR objects each corresponding to a respective object. The system comprises a display on which the image of the real-world environment and the AR objects are displayed. Data relating to the real-world environment and objects therein is received via sensos. This data is used to create the AR objects from the real-world objects by determining additional information relating to said real-world object. The AR object is represented by at least one of symbols, annotations and fonts to be attached to the AR object to act as a displayed component. The AR objects, once created, are stored for future retrieval based on a unique identifier for the AR object which remains active as long as the object is of interest. In an embodiment, when a stored AR object enters the Field of View (FOV) of the display, the stored AR object is retrieved for display. A further embodiment relates ...

Recognition apparatus and method

METHOD, SYSTEM AND RECORDING MEDIUM FOR ACCESSORY PAIRING

A method, a system and a recording medium for accessory pairing are provided. The method comprises: determining a type of the accessory based on the identifying information of an accessory; recognizing a type of a user body part according to an image; in response to the accessory is applicable to the user body part according to the type of accessory and the type of the user body part, obtaining a first position of the accessory within an environment and obtaining a second position of the user body part within the environment; and configuring the accessory to pair the user body part according to the first position of the accessory and the second position of the user body part.

Comparing planogram compliance to checkout data

The present disclosure relates to systems and methods for identifying products and monitoring planogram compliance using analysis of image data. In one implementation, the method may include accessing at least one planogram describing a desired placement of a plurality of product types on shelves of a plurality of retail stores; receiving image data from the plurality of retail stores; analyzing the image data to determine an actual placement of the plurality of product types on the shelves of the plurality of retail stores; determining at least one characteristic of planogram compliance based on detected differences between the at least one planogram and the actual placement of the plurality of product types on the shelves of the plurality of retail stores; and receiving checkout data from the plurality of retail stores reflecting sales of at least one product type from the plurality of product types.

LIGHT-EMITTING USER INPUT DEVICE FOR CALIBRATION OR PAIRING

A light emitting user input device can include a touch sensitive portion configured to accept user input (e.g., from a user's thumb) and a light emitting portion configured to output a light pattern. The light pattern can be used to assist the user in interacting with the user input device. Examples include emulating a multi-degree-of-freedom controller, indicating scrolling or swiping actions, indicating presence of objects nearby the device, indicating receipt of notifications, assisting pairing the user input device with another device, or assisting calibrating the user input device. The light emitting user input device can be used to provide user input to a wearable device, such as, e.g., a head mounted display device.

SCENE ANNOTATION AND ACTION DESCRIPTION USING MACHINE LEARNING

A system enhances existing audio-visual content with an action a scene annotation module, an action description module, both of which are coupled to a controller. The scene annotation module classifies scene elements from an image frame received from a host system and generates a caption describing the scene elements. The scene annotation module includes a first neural network configured to generate a feature vector from the image frame and a second neural network configured to generate a caption describing elements within the image frame from the feature vector. The action description module recognizes action happening within one or more image frames received from the host system and generates a description of the action happening within one or more image frames.

SHADOW-BASED ESTIMATION OF 3D LIGHTING PARAMETERS FROM REFERENCE OBJECT AND REFERENCE VIRTUAL VIEWPOINT

A method and device are provided for processing images. In one embodiment, the method comprises receiving an image is received of a scene having a plurality of real objects and determining if at least one of these real objects is a reference object stored in a database. Subsequently, candidate shadow map of the reference object identified is retrieved when available and when a reference object cannot be identified, an object in the scene with adequate parameters is selected and the candidate shadow maps of this new reference object using said lighting parameters is computed.

PROVIDING VISUAL GUIDANCE FOR PRESENTING VISUAL CONTENT IN A VENUE

Systems, methods, and computer program products can provide visual guidance on presenting content on a media surface of a venue. These systems, methods, and computer program products can operate by mapping visual content onto a media surface of the venue and a key feature unique to the visual content and/or the media surface being extracted. Thereafter, these systems, methods, and computer program products can retrieve an experiment metric corresponding to the visual content and/or the media surface and can determine a viewer location metric and/or a media surface metric based on the experiment metric and the key feature. These systems, methods, and computer program products can utilize the viewer location metric and/or the media surface metric to provide a hint of the attribute to a user.

COOKING APPLIANCE WITH AN IMAGING DEVICE

A cooking appliance can include a cooking chamber and an imaging device for capturing an image of a food item inside the chamber. The appliance can include a transparent, touchscreen display for displaying the interior contents of the cooking appliance to a user without requiring the user to open the cooking appliance, as well as providing for the user controlling, operating, and interacting with the cooking appliance.

Attention guidance for ground control labeling in street view imagery

The disclosure provides methods, apparatus, and products for attention guidance and labeling. In one aspect, a method comprises receiving metadata associated with a perspective image captured by an image capture device located at the image position and having the image pose, wherein the metadata comprises the image position and the image pose. The method comprises defining a field-of-view indicator having a first indicator position located at the image position and having the image pose; receiving ground control point (GCP) information identifying a GCP, wherein the GCP information comprises a GCP position; defining a GCP indicator having a second indicator position located at the GCP position; and causing display of an indicator layer comprising the field-of-view indicator and the GCP indicator in a second portion of an interactive user interface (IUI) of a labeling tool, wherein at least the perspective image is displayed in a first portion of the IUI.

Augmented-reality-based facility semantic mapping

Aspects of augmented reality (AR)-based facility mapping are described. In some examples, an AR device detects optical data using an optical device, and position data using a position detection device. An AR-based modification user interface is displayed using the AR device. The AR-based modification user interface includes an AR map and an AR object. The AR-based modification user interface is generated using the facility data and the sensor data. The AR-based modification user interface identifies a user action that modifies the AR object.

User scanning and one-way augmented reality viewing system

An augmented reality customer interaction system includes a transparent panel having a first side and a second side that is opposite to the first side, and a camera device configured to capture visual data from an area adjacent to the second side of the transparent panel. The visual data includes identifying features of a customer located in the area with respect to the second side of the transparent panel. The system further includes a projection system configured to project information on the first side of the transparent panel. The information projected on the first side of the transparent panel may include customer interaction data retrieved from a data store based on the identifying features of the customer.

Methods and systems for developing a personalized non-profit venue experience and presenting personalized multimedia to a mobile computing device

A method for developing content for a personalized non-profit venue experience using a mobile computing device, comprising: providing an interface for management of a set of multimedia assets; receiving a mapping of multimedia assets to a set of locations on a site plan for the non-profit venue; receiving metadata for the set of multimedia assets; receiving personalized interest data indicating personal philanthropic interests of a visitor to the non-profit venue; and applying a machine learning algorithm to analyze the metadata and the personalized interest data to provide, to a mobile computing device, a selection of a multimedia asset from the set of multimedia assets for consumption by the visitor.

Reconfigurable headset that transitions between virtual reality, augmented reality, and actual reality

A method for switching a configuration of an enhanced reality headset is provided. The method includes identifying a scenario to switch a configuration of an enhanced reality headset between a virtual reality configuration, an augmented reality configuration, and a direct reality configuration, wherein the virtual reality configuration and the augmented reality configuration include a computer generated image, and the direct reality configuration and the augmented reality configuration include a real-time image of an environment of a user of the enhanced reality headset. The method includes providing a notification to a user of the enhanced reality headset including an intent to switch the configuration of the enhanced reality headset, and switching the configuration of the enhanced reality headset according to the scenario.

Output of virtual content

This specification describes an apparatus, method and computer program relating to virtual reality, particularly augmented reality (AR) or mixed reality (MR). The method may comprise providing, based on a position associated with a display means, different first and second sets of virtual content for overlaid display at the display means at a first time and determining that the first set of virtual content is prioritized over the second set of virtual content. Based on the determination, the method may comprise prioritizing display of the first set of virtual content over the second set of virtual content at the display means, and enabling display of the second set of virtual content at a second, subsequent, time.

Optical character detection using a low-power sensor

Various aspects of the present disclosure generally relate to optical character detection. In some aspects, a user device may receive, from a vision sensor, a first image that is associated with a first optical character image. The user device may determine, using an image processing model, that the first image depicts the first optical character image. The user device may cause, based at least in part on determining that the first image depicts the first optical character image, a camera to capture a second image that is associated with a second optical character image. The user device may perform an action associated with the second image. Numerous other aspects are provided.

AUTOMATIC NETWORK DEVICE IDENTIFICATION AND PRESENTATION OF NETWORK INFORMATION VIA AUGMENTED REALITY

Automatic network device identification and presentation of network information via augmented reality is disclosed. Real time imagery of a scene within a field of view of a camera is received. It is determined that the imagery depicts a first object. It is determined that the first object comprises a first network-attached device attached to a network. Information that corresponds to the first network-attached device is obtained, and the information is presented in conjunction with a real time view of the scene.

MIXED REALITY (MR) PROVIDING DEVICE FOR PROVIDING IMMERSIVE MR, AND CONTROL METHOD THEREOF

A mixed reality (MR) providing device is disclosed. The MR providing device includes: a camera, a communication unit comprising circuitry configured to communicate with an electronic device providing video, an optical display unit comprising a display configured to simultaneously display real space within a preset range of viewing angle and a virtual image, and a processor. The processor is configured to: capture the preset range of viewing angle through the camera to acquire an image, identify at least one semantic anchor spot of the acquired image in which an object may be positioned, transmit characteristic information of the semantic anchor spot related to the object that may be positioned to the electronic device through the communication unit, receive an object region including the object corresponding to the characteristic information and included in an image frame of the video from the electronic device through the communication unit, and control the optical display unit to display ...

Finding and filtering elements of a visual scene

In a general aspect, a method can include receiving, by an electronic device, a visual scene; identifying, by the electronic device, a plurality of elements of the visual scene; and determining, based on the plurality of elements identified in the visual scene, a context of the visual scene. The method can further include applying, based on the determined context of the visual scene, at least one filter to identify at least one element of the plurality of elements corresponding with the at least one filter; and visually indicate, in the visual scene on a display of the electronic device, the at least one element identified using the at least one filter.

Suggested actions for images

Implementations relate to causing a command to be executed based on an image. In some implementations, a computer-implemented method includes obtaining and programmatically analyzing an image to determine suggested actions. The method causes a user interface to be displayed that includes user interface elements corresponding to default actions, and to suggested actions that are determined based on analyzing the image. The method receives user input indicative of selection of a particular action from the default actions and the suggested actions. The method causes a command to be executed by a computing device for the particular action that was selected.

Registration of local content between first and second augmented reality viewers

A method of viewing image data of local content is disclosed. An augmented reality view is created by storing a first device coordinate frame (DCF), moving a first registration marker to select a first feature point (FP1) and a second feature point (FP2) on at least one real-word object viewable by the user through a display. A uniform coordinate system (UCS) alignment module stores locations of the registration marker when selecting the FP1 and the FP2, determines a user coordinate frame (UCF) based on the locations of the first registration marker when selecting the FP1 and the FP2, transforms the DCF to the UCF and displays image data of local content received on a first data source with a projector through the display to the user based on the transformation from the first DCF to the first UCF.

XR system and method for controlling the same

The present invention relates to an XR system and its control method, and more specifically, may be applied to 5G communication technology field, robot technology field, autonomous driving technology field and AI technology field. The XR system includes an XR device and a storage case that stores the XR device. The control method of the XR system includes recording and learning user experiences and providing a mate service to the user based on an image captured by a camera unit of the storage case and user information including the learned user experience.

Use of a convolutional neural network to auto-determine a floor height and floor height elevation of a building

A system, apparatus, computer program product, and method use a convolutional neural network to auto-determine a first floor height (FFH) and a FFH elevation (FFE) of a building. The FFH, and FFE of the building are determined with respect to the terrain or surface of the parcel of land on which the building is located. In turn, by knowing the FFH and/or FFE of the building on the parcel, it is possible to use that information while performing a flood risk assessment to a property without requiring a personal inspection of the parcel by a human.

OPTIMAL ASSISTANCE FOR OBJECT-REARRANGEMENT TASKS IN AUGMENTED REALITY

The disclosed computer-implemented method may include identifying, via an artificial reality system, a current state of a physical object in a real-world environment of a user and determining a desired end state for the physical object. The method may also include determining a sequence of action steps for manipulating the object to place the object in the desired end state, and presenting, via the artificial reality system, a notification to the user indicative of the sequence of action steps. Various other methods, systems, and computer-readable media are also disclosed.

Virtual content positioned based on detected object

Various implementations disclosed herein include devices, systems, and methods that use an object as a background for virtual content. Some implementations involve obtaining an image of a physical environment. A location of a surface of an object is detected based on the image. A virtual content location to display virtual content is determined, where the virtual content location corresponds to the location of the surface of the object. Then, a view of the physical environment and virtual content displayed at the virtual content location is provided.

Systems and methods for using augmented reality for antenna adjustment

An antenna position determination system determines a position for an antenna to communicate with a broadcast tower and present the antenna to a user via augmented reality. The antenna position determination system receives information indicating one or more broadcast towers including a location for each broadcast tower. The antenna position determination system selects a broadcast tower of the one or more broadcast towers and receives an image of an antenna. The antenna position determination system determines a position for the antenna to communicate with the selected broadcast tower based on the image of the antenna and the selected broadcast tower. The antenna position determination system presents the determined position of the antenna to a user via an augmented reality user interface.

Enhanced pose determination for display device

To determine the head pose of a user, a head-mounted display system having an imaging device can obtain a current image of a real-world environment, with points corresponding to salient points which will be used to determine the head pose. The salient points are patch-based and include: a first salient point being projected onto the current image from a previous image, and with a second salient point included in the current image being extracted from the current image. Each salient point is subsequently matched with real-world points based on descriptor-based map information indicating locations of salient points in the real-world environment. The orientation of the imaging devices is determined based on the matching and based on the relative positions of the salient points in the view captured in the current image. The orientation may be used to extrapolate the head pose of the wearer of the head-mounted display system.

DATA GENERATION METHOD AND APPARATUS, AND ELECTRONIC DEVICE

Disclosed in the present disclosure are a data generation method and apparatus, and an electronic device. The method includes: obtaining first image data, the first image data representing a real environment in which a user is located; obtaining category information and plane information of a target object, the target object being an object in the first image data, and the plane information including information of an outer surface of the target object; obtaining second image data, the second image data containing a virtual object; and generating target image data by mixing the first image data with the second image data based on the category information and the plane information, the target image data containing the target object and the virtual object.

METHOD AND SYSTEM FOR REDUCING MANUAL REVIEW OF LICENSE PLATE IMAGES FOR ASSESSING TOLL CHARGES

A tolling system is operable to reduce the number of manual reviews of a toll point images needed to process toll fee charges by separately reporting from both toll points and mobile device in vehicles running a tolling application program the lane and crossing time when traversing a toll point. A tolling service can match records produced by the toll points with records providing by the mobile device when the toll point cannot immediately determine the identity of the toll customer passing through the toll point.

Systems, methods, and apparatus for enhanced cameras

In accordance with some embodiments, systems, apparatus, interfaces, methods, and articles of manufacture are provided for providing information about objects, such as background information and task information, and for providing alerts related to objects. In various embodiments, data is captured about an object and about a user via a camera. Based on the data, information about the object may be provided to the user.

Systems and methods for virtual and augmented reality

Disclosed herein are systems and methods for distributed computing and/or networking for mixed reality systems. A method may include capturing an image via a camera of a head-wearable device. Inertial data may be captured via an inertial measurement unit of the head-wearable device. A position of the head-wearable device can be estimated based on the image and the inertial data via one or more processors of the head-wearable device. The image can be transmitted to a remote server. A neural network can be trained based on the image via the remote server. A trained neural network can be transmitted to the head-wearable device.

Event argument extraction method, event argument extraction apparatus and electronic device

An event argument extraction (EAE) method, an EAE apparatus and an electronic device, relates to the technical field of knowledge graphs. A specific implementation scheme includes acquiring a to-be-extracted event content; and performing argument extraction on the to-be-extracted event content based on a trained EAE model, to obtain a target argument of the to-be-extracted event content; where the trained EAE model is obtained by training a pre-trained model with event news annotation data and a weight of each argument annotated in the event news annotation data.

Virtual planogram management systems and methods

Systems and methods of constructing and managing virtual planograms are presented. Contemplated systems allow for construction of a virtual planogram, which can be used to present consumers virtual inventory items as being available for purchase via a display device. The display device can include an electronic billboard within an establishment, a mall for example, or include a user's smart device, a cell phone or tablet for example.

Augmented reality methods and systems

Methods and systems employing augmented reality techniques via real-world objects for various purposes. Computer implemented methods are provided for animated augmentation of real-time video of static real-world objects. A computing device receives first data defining features and movements of a human subject, captures a video image of an object having features corresponding to human features, maps features defined by first data to corresponding features of the object, and uses the first data to animate the corresponding features of the object to correspond with the movements of the human subject defined by the first data. The object may be a physical 3D object. Embodiments may include positioning and orienting a 3D mesh representation of the object in registration with the object video image, prior to mapping the features defined by the first data to corresponding features on the 3D mesh. Recorded audio messages may be played in synchronism with the object.

Automatic appending of subtitles based on media context

A processor may automatically generate one or more transcripts based on a media context. The processor may append at least one of the one or more transcripts to the media. The processor may modify the at least one of the one or more transcripts based on an adjustment to a weight factor.

Augmented, virtual and mixed-reality content selection and display

Systems, methods and techniques for automatically recognizing two or 3-dimensional real world objects with an augmented reality display device (smartphone or glasses etc.), and augmenting or enhancing the display of such real world objects by superimposing virtual images such as a still or video advertisement, an opportunity to buy, a story or other virtual image presentation. In non-limiting embodiments, the real world object includes visible features including visible security features and a recognition process takes the visible security features into account when recognizing the object and/or displaying superimposed virtual images.

Varying extended reality content based on risk level of a driving environment

Extended reality content in a video can be varied based on a risk level of an external environment of a vehicle. The video can be of an external environment of a vehicle can be presented in real-time on a display located within the vehicle. The display can be a video pass through display. The display can be an in-vehicle display, or it can be a part of a video pass-through extended reality headset. The video can present a view of an external environment of the vehicle as well as extended reality content. A risk level of the external environment of the vehicle can be determined. An amount of the extended reality content presented in the video can be varied based on the risk level.

VIRTUAL TRY-ON SYSTEMS FOR SPECTACLES USING REFERENCE FRAMES

OBFUSCATED CONTROL INTERFACES FOR EXTENDED REALITY

LIDAR SIMULTANEOUS LOCALIZATION AND MAPPING

SYSTEM AND METHOD FOR TRACKING A PORTION OF THE USER AS A PROXY FOR NON-MONITORED INSTRUMENT

METHOD OF ALIGNING VIRTUAL AND REAL OBJECTS

PORTABLE SCANNING DEVICE FOR ASCERTAINING ATTRIBUTES OF SAMPLE MATERIALS

REGISTRATION FOR AUGMENTED REALITY SYSTEM FOR VIEWING AN EVENT

CACHING AND UPDATING OF DENSE 3D RECONSTRUCTION DATA

Augmented reality multi-plane model animation interaction method and device, apparatus, and storage medium

An augmented reality multi-plane model animation interaction method and device, an apparatus, and a storage medium. The method comprises: acquiring a video image of a real environment (S1); performing computational processing on the video image to identify multiple real planes in the real environment (S2); placing a virtual object, corresponding to a model, on one of the multiple real planes (S3); and generating, according to the identified multiple real planes, an animation track of the virtual object between the multiple real planes (S4). In the method, real planes of a real environment are identified to generate an animation track of a virtual object, thereby associating an animation effect of the virtual object with a real scenario, and enhancing a realistic sensory experience of the user.

Positioning apparatus and method

A positioning apparatus/method for positioning an operator, e.g. maintenance worker, at an object, such as an aircraft. The apparatus comprises a personal sensor 12, whereby the operator’s environment is assessed to generate sensor data. Personal feedback devices 14, 18 issue at least one alert signal to the operator and a processing unit 20 is connected to the sensor 12 and the feedback devices 14, 18. Processing unit 20 comprises an exclusion zone module 1021 defining a region the operator should remain outside of, and a positioning module 24 for generating positioning data for determining the position of the sensor 12 relative to the object. A task module 1100 assigns a task to the operator and generates an associated desired position for the operator. A path planning module 1200 generates a path from the operator position to the desired position, which avoids the exclusion zone. An alert module 26 issues a set of routing alert signals to the operator in dependence on the position of ...

WEARABLE TERMINAL DEVICE, CONTROL METHOD, AND SYSTEM

A wearable terminal device capable of wirelessly communicating with a mobile terminal device having a display unit, the wearable terminal device comprising receiving status information of the mobile terminal device from the mobile terminal device when status of the mobile terminal device changes, determining whether or not to select display of the wearable terminal device based on the status information, and controlling the display of the wearable terminal device based on a determination result by the determining.

Systems and methods for improving the classification of objects

Systems, methods, and other embodiments described herein relate to improving the classification of objects depicted in a scene. In one embodiment, a method includes generating, using an ontological detector, a type classification of a detected object according to a detector ontology of known classes. The detected object is represented as segmented data from sensor data about a surrounding environment. The method includes, in response to determining that the type classification specifies an unknown class that is not defined in the detector ontology, annotating the segmented data as unknown. The method includes providing the segmented data to specify that the type classification for the detected object is unknown.

SYSTEMS AND METHODS FOR ASSESSING BALANCE AND FORM DURING BODY MOVEMENT

The disclosure relates to a system for evaluating movement of a body of a user. The system may include a video display, one or more digital cameras, and a processor. The processor may control the one or more cameras to generate images of at least the part of the body over a period of time. The processor may estimate a position of a plurality of joints of the body. The processor may receive a selection of a tracked pose, and determine, from the plurality of joints, a set of joints associated with the tracked pose. The processor may generate at least one joint vector connecting joints in the set of joints, and assign, based on changes in the joint vector over the period of time, a form score to a performance of the tracked pose. The processor may then generate a user interface that depicts the form score.

METHOD AND SYSTEM TO ANALYZE SPACES FOR CLEANING BASED ON THE PRESENCE OF AN INDIVIDUAL

A method for monitoring and controlling a cleanliness of a space including: monitoring a distance of one or more individuals relative to an object within the space using a space monitoring system; determining when the distance is less than a selected distance; determining a number of the one or more individuals within the selected distance of object; determining a duration that the one or more individuals are within the selected distance of the object; and determining that the object requires a cleaning when at least one of: the number of the one or more individuals within the selected distance of object exceeds a threshold number, or the duration that the one or more individuals are within the selected distance of the object exceeds a threshold time.

SYSTEMS AND METHODS FOR PRESENTING SUPPLEMENTAL CONTENT IN AUGMENTED REALITY

Systems and methods provided for presenting supplemental content in an augmented reality environment where an object within a field of view of an augmented reality device of a user is identified and processed to detect a reference related to a participant in an event. A user profile or user social network is searched to identify a message from the user about the participant. The message may be combined with the object in the augmented reality field of view.

Virtual reality training application for surgical scrubbing-in procedure

Disclosed herein are immersive virtual and/or augmented reality education/training tools useable to teach operating room personnel any or all of scrubbing-in procedures, gowning/gloving procedures, and proper operating room etiquette. The training tools provide individuals with active practice in the operating room setting prior to the real-world applications, and thus allows the user to become more adept with the necessary procedures prior to entering the operating room. The disclosed virtual and/or augmented reality training will make for a better, less stressful, risk-reducing (e.e., infection of patient), operating room environment that is more conducive for both learning by the students and surgical performance by the experienced operating room personnel.

Mobile terminal

Disclosed is a mobile terminal comprising: a communication unit for communicating with a plurality of target devices in a Bluetooth Low Energy (BLE) manner; a camera for capturing an image; and a controller configured for: receiving a plurality of advertisement information respectively including a plurality of shape information of the plurality of target devices from the plurality of target devices; and when first and second target devices among the plurality of target devices have the same shape information and when a distance of the first target device is equal to a distance of the second target device, activating a connection between the mobile terminal and the first target device based on an operation of at least one of the first target device or the second target device.

Image processing device, image processing method, and image processing system

In a current image processing environment, providing feedback for a user operation is slow, and a user cannot confirm the result of the operation in real time. Thus, the user interrupts the operation every time the user confirms the feedback, and a time period for performing image processing is long. An image processing device includes a guidance information generator. The guidance information generator acquires trajectory information to be used to segment an image object on an image input from a user, generates guidance information indicating a segmentation region desired by the user based on the trajectory information, and presents the segmentation region based on a user's intention in real-time. The guidance information generator provides a function of smoothly continuously segmenting an image object.

Augmented reality device for visualizing luminaire fixtures

The invention provides an augmented reality device and method for assisting a user in choosing appropriate luminaire fixtures to install within their home. A user may point the camera of a mobile device toward the region or location in a room where a new luminaire is desired, and based upon data generated by an orientation determination means included within the device, an appropriate luminaire or luminaire category is selected automatically for the user from a stored catalogue or database. Once an appropriate luminaire has been chosen, it is inserted within an image captured by the camera to generate an augmented reality image depicting the luminaire fixture in place within the user's room.

Augmented reality pharmacy system and method

An augmented reality pharmacy system and method for assisting pharmacy personnel with health care-related tasks is presented. The augmented reality pharmacy system can reduce or eliminate common pharmacy fulfillment errors attributable to human error. The present disclosure improves pharmacy fulfilment error rates by providing image processing systems adapted to characterize prescription and drug related information to verify prescription fulfillment; incorporates augmented reality systems with pharmacy systems to provide enhanced fulfillment systems where real-world objects can be over-laid with virtual reality elements to provide additional pharmaceutical information to a user; and shortens the time spent by a user looking for a specific drug bottle in a maze of similar-looking drug bottles by incorporating wayfinding technology coupled with location based components.

SYSTEMS AND METHODS FOR IMPLEMENTED MIXED REALITY IN LABORATORY AUTOMATION

Systems and methods for providing instructing mixed reality overlays for configuring automation protocols in laboratory processes using computing systems include receiving an image from an image feed of an environment. Component markers in the image of the environment are detected and matched with an associated environment component, an environment component action or both. The environment component or environment component action, or both are selected and an animation of an instruction is rendered. An overlay of the animation is caused to display in an augmented reality device associated with a user to appear in a location in the environment of the environment component or environment component action, or both.

Training a neural network with representations of user interface devices

A wearable display system can be capable of determining a user interface (UI) event with respect to a virtual UI device (e.g., a button) and a pointer (e.g., a finger or a stylus) using a neural network. The wearable display system can render a representation of the UI device onto an image of the pointer captured when the virtual UI device is shown to the user and the user uses the pointer to interact with the virtual UI device. The representation of the UI device can include concentric shapes (or shapes with similar or the same centers of gravity) of high contrast. The neural network can be trained using training images with representations of virtual UI devices and pointers.

Apparatuses, computer program products, and methods for generation of augmented reality interfaces

Methods, apparatuses, and computer program products are disclosed for generating an augmented reality interface, for example for provision of promotions. Some example embodiments receive camera data and location data associated with a consumer device; receive map data based on the location data; identify at least a provider location in a field of view based on the map data; determine a number of consumer devices within a proximity radius of the provider location; determine the number of consumer devices satisfies a predetermined number of consumer devices; determine, based on the determining that the number of consumer devices within the proximity radius satisfies the predetermined number of consumer devices, a promotion for the provider location; and cause display of an impression of the promotion in an augmented reality displayed via an interface of the consumer device.

Traffic signal management for autonomous vehicle operation

Systems and methods to perform traffic signal management for operation of autonomous vehicles involve obtaining vehicle data from two or more vehicles at an intersection with one or more traffic lights. The vehicle data includes vehicle location, vehicle speed, and image information or images. A method includes determining at least one of three types of information about the one or more traffic lights based on the vehicle data. The three types of information include a location of the one or more traffic lights, a signal phase and timing (SPaT) of the one or more traffic lights, and a lane correspondence of the one or more traffic lights. The method also includes providing the at least one of the three types of information about the one or more traffic lights for the operation of autonomous vehicles.

SYSTEM AND METHOD FOR PRODUCT SELECTION IN AN AUGMENTED REALITY ENVIRONMENT

A system and method includes a computing system that captures images of multiple objects and/or entities in the physical environment, and that identifies the objects/entities in the captured images. The computing system retrieves information related to the identified objects/entities, and outputs the information to a user of the computing system. The computing system may access information locally and/or from external resources in the identification of the multiple objects/entities and in the retrieval of the related information. The information to be output by the computing system may be selected in response to a user input, or based on set user preferences or known user data. The output of information related to the multiple objects/entities may allow a user of the computing system to compare the multiple objects/entities and make an informed decision related to the selection of one or more of the objects/entities without the need for multiple individual searches.

Computer System and Method for Documenting Items Identified While at a Construction Site

In order to improve a user's task of documenting items that are identified by the user while at a construction site, a computing system may be configured to (i) receive, from a first client station associated with a first user, media content that is descriptive of a given on-site item of a construction site, (ii) apply a speech recognition technique to transcribe audio data included in the media content into text that is descriptive of the given on-site item, (iii) generate a data record for the given on-site item, (iv) pre-populate at least a portion of the transcribed text into one or more fields of the data record for the given on-site item, and (v) cause a second client station to display the data record to a second user for review.

COMPUTED SYSTEMS LAYOUT LEVERAGING AUGMENTED REALITY IN THE DATA CENTER

Systems and methods for creating an instance of a 3D model of an IT component within an AR world coordinate space are provided. According to one embodiment, a layout of a system of IT components may be specified in terms of respective rack unit locations within a rack. Based on a relative position with respect to an anchor component and the location of the anchor component within the coordinate space, a high-accuracy calibrated model of an IT component with reference to which AR entities have been defined for various back panel elements may be used to create an instance of a 3D model of the IT component within the coordinate space. The instantiated 3D model may then be used to precisely overlay the AR entities on corresponding back panel elements within a through-the-lens view of the IT component, for example, to facilitate a guided wire-up of the system.

METHODS AND DEVICES FOR DECOMMISSIONING MICRO-CLIMATE SENSORS

A repository device for decommissioning a plurality of micro-climate sensors for generating product flow data for each of a plurality of items of sale flowing through a supply chain is provided. The repository device includes an opening, at least one space, a plurality of sensors, and a sensor decommission circuit. The opening is for receiving at least two micro-climate sensors of the plurality. The at least one space is for storing the at least two micro-climate sensors, a plurality of sensors operative to detect a presence of the at least two micro-climate sensors; and a sensor decommission circuit structured to generate a decommission command value in response to detection of the at least two micro-climate sensors via the plurality of sensors.

ENHANCED POSE DETERMINATION FOR DISPLAY DEVICE

To determine the head pose of a user, a head-mounted display system having an imaging device can obtain a current image of a real-world environment, with points corresponding to salient points which will be used to determine the head pose. The salient points are patch-based and include: a first salient point being projected onto the current image from a previous image, and with a second salient point included in the current image being extracted from the current image. Each salient point is subsequently matched with real-world points based on descriptor-based map information indicating locations of salient points in the real-world environment. The orientation of the imaging devices is determined based on the matching and based on the relative positions of the salient points in the view captured in the current image. The orientation may be used to extrapolate the head pose of the wearer of the head-mounted display system.

Pharmaceutical Manufacturing Process Control, Support and Analysis

A computer apparatus and computer-implemented process for generating a batch record (BR) from a master batch record (MBR) during manufacture of a batch of pharmaceutical product. In the control software, a graphical analysis interface is provided for processing graphs of spectroscopic or chromatographic analysis. A group of fields in the MBR is assigned to the analysis. The operator wears an augmented reality headset and uses this to capture an image of a graph from an instrument display. The graph image is then processed to extract the graph and its metadata, and then further processed to find peaks and assign attribute labels to the peaks, and thus populate the analysis fields. Overlay images are then transmitted to AR headset to present to the operator the populated analysis fields. The operator then accepts or rejects the populated analysis fields by issuing user interface commands.

APPARATUS AND METHOD FOR RECOGNIZING HAND GESTURES IN A VIRTUAL REALITY HEADSET

Method and apparatus for producing and reproducing augmented reality contents in mobile terminal

A method and an apparatus produce and reproduce Augmented Reality (AR) contents in a mobile terminal. In the method, contents are produced. An image including an object corresponding to the contents is recognized. Recognition information for the object corresponding to the contents is obtained based on a recognition result. AR contents including the contents and the recognition information are generated. Therefore, AR contents for an input image may be easily produced and reproduced, and the AR contents may be used as independent multimedia contents, not an auxiliary means of other contents.

Augmented reality system and method for remotely sharing augmented reality service

An augmented reality (AR) system and method for remotely sharing an AR service is provided. The AR system includes a plurality of client devices and a host device. The AR system allows information related to a marker and information related an AR object to be shared between client devices participating in an AR session, which may be separated by a reference distance, through a host device. Accordingly, an AR service may be shared between the client devices.

Automated annotation of a view

In a view, e.g. of scenery, of a shopping or museum display, or of a meeting or conference, automated processing can be used to annotate objects which are visible from a viewer position. Annotation can be of objects selected by the viewer, and can be displayed visually, for example, with or without an image of the view.

Method for Implementing a High-Level Image Representation for Image Analysis

Robust low-level image features have been proven to be effective representations for a variety of visual recognition tasks such as object recognition and scene classification; but pixels, or even local image patches, carry little semantic meanings. For high-level visual tasks, such low-level image representations are potentially not enough. The present invention provides a high-level image representation where an image is represented as a scale-invariant response map of a large number of pre-trained generic object detectors, blind to the testing dataset or visual task. Leveraging on this representation, superior performances on high-level visual recognition tasks are achieved with relatively classifiers such as logistic regression and linear SVM classifiers.

Enhanced information for viewer-selected video object

The user of an Internet-enabled CE device such as a TV can select an object in a video presented on the display of the CE device by either touch sensitive screen or input device. The processor of the CE device can then determine the boundaries of the object, identify it using face/image recognition software, search for information/websites related to the object, and present those information/websites to the viewer on the display screen.

Adaptable Framework for Cloud Assisted Augmented Reality

A mobile platform efficiently processes sensor data, including image data, using distributed processing in which latency sensitive operations are performed on the mobile platform, while latency insensitive, but computationally intensive operations are performed on a remote server. The mobile platform acquires sensor data, such as image data, and determines whether there is a trigger event to transmit the sensor data to the server. The trigger event may be a change in the sensor data relative to previously acquired sensor data, e.g., a scene change in an image. When a change is present, the sensor data may be transmitted to the server for processing. The server processes the sensor data and returns information related to the sensor data, such as identification of an object in an image or a reference image or model. The mobile platform may then perform reference based tracking using the identified object or reference image or model.

Method for displaying status of power consumption and portable device thereof

Provided is a method for displaying electricity use status and a portable device thereof by identifying at least one electronic appliance in an image on a screen captured through a camera, receiving electricity use information of the at least one electronic appliance, determining the electricity use status of the at least one electronic appliance using the received electricity use information, and displaying one of a plurality of Graphical User Interfaces (GUI), wherein each GUI corresponds to a different stage of the electricity use status of the at least one electronic appliance

Camera distance measurement device

A camera distance measurement device displays an image in which a plurality of graduation lines which are arranged in the form of a grid with respect to a vehicle is superimposed on a camera image which is captured by a camera mounted to the vehicle on a display unit, and estimates a distance in a direction of the width of the vehicle, and a distance in a direction of the capturing by the camera from a unit distance defined for each grid side of the graduation lines.

Apparatus and Method for Using Augmented Reality Vision System in Surgical Procedures

A system and method for improving a surgeon's vision by overlaying augmented reality information onto a video image of the surgical site. A high definition video camera sends a video image in real time. Prior to the surgery, a pre-operative image is created from MRI, x-ray, ultrasound, or other method of diagnosis using imaging technology. The pre-operative image is stored within the computer. The computer processes the pre-operative image to decipher organs, anatomical geometries, vessels, tissue planes, orientation, and other structures. As the surgeon performs the surgery, the AR controller augments the real time video image with the processed pre-operative image and displays the augmented image on an interface to provide further guidance to the surgeon during the surgical procedure.

Information processing apparatus, information processing method, program, recording medium, and information processing system

Provided is an information processing apparatus including an image acquisition unit for acquiring an image that includes a target a user is paying attention to, an access information acquisition unit for acquiring access information assigned with a time-limited access right to public information that is associated with the target in the image, a public information acquisition unit for acquiring the public information using the access information, and a display control unit for controlling display of the public information.

Augmented reality for table games

A method includes acquiring media content of a wagering game table at a wagering game establishment with a camera of a mobile device. A location of the mobile device is determined when the media content is acquired. A direction that a lens of the camera is facing when the media content is acquired is determined. The wagering game table is identified based on the location and the direction. Overlay imagery derived from wagering game activity of the wagering game table is downloaded into the mobile device from a server. The overlay imagery is composited onto the media content to create a composited media content. The composited media content is displayed on a display of the mobile device.

Adaptive Projector

An apparatus for processing data includes a projector, which is configured to project content onto at least a part of a scene. A processor is configured to detect a location of an eye of a person in the scene and to control the projector so as to reduce an intensity of the projected content in an area of the eye.

Feeling-expressing-word processing device, feeling-expressing-word processing method, and feeling-expressing-word processing program

The present approach enables an impression of the atmosphere of a scene or an object present in the scene at the time of photography to be pictured in a person's mind as though the person were actually at the photographed scene. A feeling-expressing-word processing device has: a feeling information calculating unit 11 for analyzing a photographed image, and calculating feeling information which indicates a temporal change in a scene shown in the photographed image or a movement of an object present in the scene; and a feeling-expressing-word extracting unit 12 for extracting, from among feeling-expressing words which express feelings and are stored in a feeling-expressing-word database 21 in association with the feeling information, a feeling-expressing word which corresponds to the feeling information calculated by the feeling information calculating unit 11.

Preventing classification of object contextual information

Technology is disclosed for preventing classification of objects, e.g., in an augmented reality system. The technology can identify a set of objects to be classified, determine whether context information for one or more objects in the identified set of objects to be classified is identified as not to be employed during classifiation, and during classification of two different objects, include context information for one object but not the other.

Virtual environment generating system

A system and related methods for visually augmenting an appearance of a physical environment as seen by a user through a head-mounted display device are provided. In one embodiment, a virtual environment generating program receives eye-tracking information, lighting information, and depth information from the head-mounted display. The program generates a virtual environment that models the physical environment and is based on the lighting information and the distance of a real-world object from the head-mounted display. The program visually augments a virtual object representation in the virtual environment based on the eye-tracking information, and renders the virtual object representation on a transparent display of the head-mounted display device.

Locking Plate with Screw Fixation from Opposite Cortex

An apparatus for supporting a bone includes a plate configured for positioning on a tension side of the bone. Two or more fasteners are configured to extend from an opposite side of the bone to the plate, and engage the plate after extending through the bone. The plate has an attachment mechanism operable to engage with the fasteners. In other aspects, a method of performing a surgical procedure includes positioning a plate on a tension side of a bone, and forming a hole from an opposite side of the bone to the plate. A fastener is inserted through the hole to the plate, and the plate is adjusted into position on the bone surface on the tension side of the bone. One or more additional holes are formed from the opposite side of the bone to the plate, and one or more additional fasteners are inserted.

Multifunction Portable Electronic Device and Mobile Phone with Touch Screen, Internet Connectivity, and Intelligent Voice Recognition Assistant

A multifunction portable electronic device, mobile device, or smart phone may have a touch screen, cellular radio, microphone, speakers, video, camera, voicemail, website, and Internet connectivity. The device records spoken audio, processes the audio, transcribes the audio into text, and displays the text on the screen. The spoken audio may be used for composition of emails or documents. The mobile device may accept voice commands for actions including calling, dialing a number, or accessing an Internet site. The device may use local or server-based voice recognition software. Voice patterns may be stored in a database specific to an individual. Voice may be used to control functions on the device, a server, Internet site, or an intelligent television. The mobile device may communicate using an inside line path such as a Bluetooth, Ethernet, USB, WiFi or 802.11x or a cellular, GSM, CDMA, LTE, GPS, or other outside line path.

Systems and methods for landmark detection

A computer-implemented method for detecting a landmark. An image is received. A feature in the received image is detected. The detected feature is compared to a plurality of images of landmarks stored in a database. Upon determining the detected feature matches an image of a landmark, information associated with the landmark is retrieved from the database. The retrieved information is displayed on a computing device.

Image Capture and Identification System and Process

A digital image of the object is captured and the object is recognized from plurality of objects in a database. An information address corresponding to the object is then used to access information and initiate communication pertinent to the object.

Image Capture and Identification System and Process

A digital image of the object is captured and the object is recognized from plurality of objects in a database. An information address corresponding to the object is then used to access information and initiate communication pertinent to the object.

Electronic system with augmented reality mechanism and method of operation thereof

A method of operation of an electronic system includes: scanning an image for detecting a subject; detecting a potential adjustment for moving the subject within the image; and selecting an augmentation for recommending the potential adjustment and for displaying the augmentation on a device.

Image networks for mobile communication

A mobile communication system based on images, enabling communication between a plurality of mobile devices and servers, wherein the images have associated additional properties in the nature of voice, audio, data and other information. The system further enabling the formation of one or more image networks wherein the images are stored, organized, connected and linked to each other by one or more methods inclusive of one to one connection between images, a hierarchical connection between images and or other methods of connection between images to facilitate efficient image based communication between mobile devices, stationary devices and servers based on the mobile device identification, mobile device specific profiles and user specific profiles. The imaged base network system further having the ability to learn and form intelligent association between objects, people and other entities; between images and the associated data relating to both animate-and inanimate entities for intelligent image based communication in a network.

Monitoring through a transparent display

Traffic obstacles or other objectives in front of a vehicle or a person can be monitored by using a monitoring system including a display device, a camera unit, and a control unit. The display device includes a transparent display which allows a user to view a scene through the transparent display. The camera unit produces images of the scene. The control unit determines objective(s) according to recognition of the scene images, and transmits objective data corresponding to the objective(s) to the display device. The transparent display of the display device displays objective information indicating virtual image(s) of the objective(s) projected on the transparent display according to the objective data.

Electronic Personal Companion

Personal companions crowd-source and/or crowd-share characterizations, and optionally raw data, from real-world, virtual and/or mixed-reality experiences. Characterizations can advantageously be stored in one or more self-evolving, structured databases, and can be organized according to objects, actions, events and thoughts. Characterizations can be weighted differently for different users, and “forgotten” over time, especially in favor of maintaining higher level characterizations. Personal companions can be used to obtain additional information, and conduct interpersonal, commercial, or other interactions or transactions.

Object Information Derived from Object Images

Search terms are derived automatically from images captured by a camera equipped cell phone, PDA, or other image capturing device, submitted to a search engine to obtain information of interest, and at least a portion of the resulting information is transmitted back locally to, or nearby, the device that captured the image.

Device and method for scanning of books and other items for order and inventory control

A method and device for creating digital optical images of bindings of books or other items in order to determine whether the books are in the correct order, such as on a bookshelf. The digital images are analyzed by a computer to determine whether the unique identifier of the book indicates the books is out of order relative to one or more next adjacent books bearing unique identifiers. If a book is out of order, an augmented reality image is produced with the digital image on the display of the device to indicate which books is out of order, and which books are in order. The user of the system can correct the disorder and re-scan to confirm order. The system also retains information about which books were scanned, which books were adjacent each scanned book, and whether the books were in or out of order.

Image Capture and Identification System and Process

A digital image of the object is captured and the object is recognized from plurality of objects in a database. An information address corresponding to the object is then used to access information and initiate communication pertinent to the object.

Method of constructing street guidance information database, and street guidance apparatus and method using street guidance information database

A walking guidance apparatus using a walking guidance information database includes a feature point extracting unit configured to extract a feature point from an acquired image, a corresponding point search unit configured to search for a corresponding point based on a correspondence relationship between feature points extracted from consecutive images, a current position and walking direction calculation unit configured to calculate a current position and a walking direction of a pedestrian by calculating a 3D position and pose of a camera between the images by using camera internal parameters and a relationship between corresponding points, a guidance information generating unit configured to generate guidance information according to the current position and the walking direction of the pedestrian, and a guidance sound source reproducing unit configured to reproduce a guidance sound source corresponding to the guidance information in 3D based on the current position and the walking direction of the pedestrian.

Gaze-based user interactions

In an exemplary process for interacting with user interface objects using an eye gaze, an affordance associated with a first object is displayed. A gaze direction or a gaze depth is determined. While the gaze direction or the gaze depth is determined to correspond to a gaze at the affordance, a first input representing user instruction to take action on the affordance is received, and the affordance is selected responsive to receiving the first input.

Systems and Methods for Remote Measurement using Artificial Intelligence

A system and method for remotely measuring a site or object with artificial intelligence. The system comprising: a computing device for remote measuring of at least one object within a captured image, the computing device comprising: a processor, a frontend service having a user interface, a database comprising one or more detected objects, an estimator controller, and a detector controller. The method comprising a step for obtaining a captured image with an identified detected object, a step for retrieving a reference image matched with the detected object and a set of dimensions associated with the reference image, a step for determining that at least one threshold has been met by the detected object, and a step for generating a depth map based off the captured image.

Multiscale feature representations for object recognition and detection

Embodiments of the present invention are directed to a computer-implemented method for multiscale representation of input data. A non-limiting example of the computer-implemented method includes a processor receiving an original input. The processor downsamples the original input into a downscaled input. The processor runs a first convolutional neural network (“CNN”) on the downscaled input. The processor runs a second CNN on the original input, where the second CNN has fewer layers than the first CNN. The processor merges the output of the first CNN with the output of the second CNN and provides a result following the merging of the outputs.

Information processing device, terminal device, information processing method, information output method, customer service assistance method, and recording medium

An information processing device ( 210 ) comprises the following: an acquisition unit ( 211 ) for acquiring first information indicating a position; an identification unit ( 212 ) for identifying a target present within a prescribed range from the position, using the first information acquired by the acquisition unit ( 211 ); a generation unit ( 213 ) for generating second information relating to the target identified by the identification unit ( 212 ), using third information indicating the movement history of the target; and an output unit ( 214 ) for outputting the second information generated by the generation unit ( 213 ).

Computer aided rebar measurement and inspection system

Embodiments of the present invention generally relate to computer aided rebar measurement and inspection systems. In some embodiments, the system may include a data acquisition system configured to obtain fine-level rebar measurements, images or videos of rebar structures, a 3D point cloud model generation system configured to generate a 3D point cloud model representation of the rebar structure from information acquired by the data acquisition system, a rebar detection system configured to detect rebar within the 3D point cloud model generated or the rebar images or videos of the rebar structures, a rebar measurement system to measure features of the rebar and rebar structures detected by the rebar detection system, and a discrepancy detection system configured to compare the measured features of the rebar structures detected by the rebar detection system with a 3D Building Information Model (BIM) of the rebar structures, and determine any discrepancies between them.

Augmented reality toy

A toy, for use with a viewing device, may include a plurality of component parts at least two of which include at least one augmented reality marker of a plurality of augmented reality markers enabling digital content associated with the at least one augmented reality marker to be unlocked and influencing at least one of an appearance and function of the toy on the viewing device. The plurality of augmented reality markers, each of which may be a physically independent augmented reality image marker, may be combined and converted into a composite augmented reality marker to produce a single, unique physical item and a composite digital representation.

Recognition assistant

A method provides for assistance in recognition of an entity. A set of data and associated information corresponding to a plurality of entities known to an assisted user is received, such that an instance of the set of data and associated information includes identification of a respective entity of the plurality of entities known to an assisted user. Real-time data corresponding to a first entity is received from one or more devices capturing the real-time data. The real-time data is compared to the set of data and associated information corresponding to the plurality of entities known to the user to determine whether the first entity has a known relevance to the user, and in response to determining the first entity does have a known relevance to the user, the processor provides the identity and relevance of the first entity to the user.

Device and Method for Imaging Structures Positioned Behind or Within a Surface

The present invention relates to a surface scanning and imaging device. The surface scanning and imaging device is designed for viewing through walls, ceilings, and other surfaces in a building to view structural components by positioning the device in proximity to the desired surface. The device includes one or more sensors for transmitting surface penetrating signals, a transceiver for receiving signals reflected from one or more structures positioned behind the surface, and an image processor to process the received signals to create a 2D or 3D image of the structural component that reflected the transmitted signals. The device displays the image of the structural component located behind the surface, allowing a user to determine where a repair or modification can be made. The structural components can be one or more of wires, circuits, studs, pipes, taps and other construction components.

Shopping Facility Assistance System and Method to Retrieve In-Store Abandoned Mobile Item Containers

A central computer system identifies a mobile item container in a retail shopping facility as being abandoned. The central computer system then directs a motorized transport unit through the retail shopping facility to the abandoned mobile item container and causes that motorized transport unit to physically attach to the abandoned mobile item container. The central computer system then directs that motorized transport unit through the retail shopping facility with the attached abandoned mobile item container to a specified destination within the retail shopping facility. Abandonment can be determined as a function, at least in part, of determining that the mobile item container is both stationary and unattended for at least a predetermined amount of time. By one approach the central computer system can use different predetermined amounts of time when assessing abandonment depending upon where in the retail shopping facility the mobile item containers are located.

Computer Vision Cameras for IR Light Detection

A head-mounted device (HMD) is structured to include at least one computer vision camera that omits an IR light filter. Consequently, this computer vision's sensor is able to detect IR light, including IR laser light, in the environment. The HMD is configured to generate an image of the environment using the computer vision camera. This image is then fed as input into a machine learning (ML) algorithm that identifies IR laser light, which is detected by the sensor and which is recorded in the image. The HMD then visually displays a notification comprising information corresponding to the detected IR laser light.

Object Information Derived from Object Images

An object is recognized from image data as a target object and linked to a user based on an interaction by the user, information about the target object is obtained and a purchase of the target object is initiated.

Augmented Reality Methods and Devices

Augmented reality methods and systems are described. According to one aspect, an augmented reality computer system includes processing circuitry configured to access an image of the real world, wherein the image includes a real world object, and evaluate the image using a neural network to determine a plurality of augmented reality estimands which are indicative of a pose of the real world object and which are useable to generate augmented content regarding the real world object. Other methods and systems are disclosed including additional aspects directed towards training and using neural networks.

Embedded eye tracker with dichroic mirror

An eyewear device has an optical element, a source, a dichroic mirror, and a camera. The optical element has a front surface, a back surface, a rim, and an angled portion of the rim. The source emits light in a first band of light and is configured to illuminate a portion of an eye of a user of the eyewear device. The dichroic mirror is arranged proximate to the angled portion of the rim, is reflective in the first band of light, is transmissive in a second band of light, and is configured to direct light in the first band reflected from the portion of the eye toward a first position. The camera is located in the first position that is located in a plane of the optical element, and the camera is configured to capture images of the light in the first band reflected by the dichroic mirror.

Congestion-state-monitoring system

The present invention provides a system for visualizing the congestion degree of an entire facility using a monitoring camera. The system detects a crowd from a monitoring camera video and derives the magnitude (number of people) of the crowd from the edge amount thereof, and thereby estimates a congestion state in a range of camera video. The system also calculates a human flow (amount of movement of people) on the basis of a number of passing people counter and the magnitude and movement speed of the crowd, etc., and estimates the number of people in an area other than the range of camera video on the basis of a modeled crowd behavior pattern. The model is constructed by selecting a distribution function and a parameter thereof by a statistical model selection technique such as a Bayesian Information Criteria (BIC). The congestion state is plotted on a map, allowing the congestion state of an area of interest to be grasped at once.

Image segmentation for object modeling

Systems, devices, and methods are presented for segmenting an image of a video stream with a client device by accessing a set of images within a video stream, identifying an object of interest within one or more images of the set of images, and detecting a region of interest within the one or more images. The systems, devices, and method identify a first set of median pixels in a first portion of the object of interest and a second set of median pixels in a second portion of the object of interest. The systems, devices, and methods determine a polyline approximating the first and second sets of median pixels and generate a model for the polyline.

Methods and systems for fire detection

In accordance with various aspects of the present disclosure, methods and systems for fire detection are disclosed. In some embodiments, a method for fire detection includes: acquiring data related to a monitored area, wherein the data comprises image data related to the monitored area; determining, based on the acquired data, whether a first mode or a second mode is to be executed for fire detection, wherein the first mode comprises a first smoke detection, and the second mode comprises a first flame detection; and executing, by a hardware processor, at least one of the first mode or the second mode based on a result of the determination.

Intraoperative surgical event summary

Systems and methods for generating decision support data for surgical videos are disclosed, the system may include at least on processor configured to implement a method. The method may include receiving video footage of a surgical procedure performed by a surgeon on a patient in an operating room and implementing a model on the received video footage to determine an existence of a surgical decision-making junction. The model may be trained to detect decision-making junctions using training data based on historical video footage depicting surgical situations. The method may include accessing, in a data structure, a correlation between an outcome and a specific action taken at the decision-making junction, and, based on the existence of the decision-making junction and the correlation, outputting a recommendation to at least one of a user device or a user interface related to the specific action.

Method, device, and computer program product for model updating

The present disclosure relates to a method, a device, and a computer program product for model updating. The method includes: acquiring a first image set and first annotation information, wherein the first annotation information indicates whether a corresponding image in the first image set includes a target object; updating a first version of an object verification model using the first image set and the first annotation information to obtain a second version, wherein the first version of the object verification model has been deployed to determine whether an input image includes the target object; determining the accuracy of the second version of the object verification model; and updating, if it is determined that the accuracy is lower than a preset accuracy threshold, the second version of the object verification model using a second image set and second annotation information to obtain a third version of the object verification model.

Methods, devices and computer program products for tracking of objects in a transportation system

A method for tracking an object in a transportation system is described. The method includes determining, at a first time, a first location of the object based on a positioning system. A first area includes the first location. The method includes determining, based on a movement sensor, that the object is moving. The movement sensor determines that the object is moving at a second time that is after the first time. The method includes determining, at a third time that is after the second time, based on the positioning system, a second location of the object, and setting the moving time of the object to be the second time, responsive to determining that the object has moved to the second location that is in a second area that is different from the first area. Related systems, devices and computer program products are also described.

Information processing apparatus, information processing method, and program

There is provided an information processing apparatus including an image acquiring unit configured to acquire at least one captured image; and an associating unit configured to associate a first information corresponding to a first object and a second information corresponding to a second object, wherein the acquired at least one captured image comprises at least one selectable object depicted therewithin, and at least one of the first object and the second object corresponds to a respective one or ones of the at least one selectable object.

System and method for drive through order processing

A system and a method for automating drive-thru orders are provided. In particular, a bridge board is provided that can integrate existing drive-thru hardware with computer devices executing machine learning models to detect and analyze speech from a drive-thru. The system and method employ vehicle tracking to account for vehicle behavior during its time at the drive-thru as well as enhanced vehicle analytics. Additionally, the system and method employ tools for assessing customer speed-of-service. Cameras and vehicle image analysis are used to link drive-thru and/or on-line food/beverage orders with vehicles entering the eatery property to accelerate food/beverage delivery to vehicle occupants.

Methods, devices, and systems for illuminating spatial light modulators

An optical device comprising may include a light turning element. The optical device can include a first surface that is parallel to a horizontal axis and a second surface opposite to the first surface. The optical device may include a light module that includes a plurality of light emitters. The light module can be configured to combine light for the plurality of emitters. The optical device can further include a light input surface that is between the first and the second surfaces and is disposed with respect to the light module to receive light emitted from the plurality of emitters. The optical device may include an end reflector that is disposed on a side opposite the light input surface. The light coupled into the light turning element may be reflected by the end reflector and/or reflected from the second surface towards the first surface.

Updating online store inventory based on physical store inventory

A system for identifying products and tracking inventory in a retail store based on analysis of image data is provided. The system may comprise at least one processor configured to receive image data from a plurality of image sensors mounted in a retail store; analyze the image data to estimate a current inventory of at least one product type in the retail store; receive product supply information associated with the at least one product type in the retail store; determine that an online order from a virtual store made during a first time period will be fulfilled by an employee of the retail store during a second time period; determine a predicted inventory of the at least one product type during the second time period; and provide information to the virtual store regarding the predicted inventory of the at least one product type during the second time period.

Information display method

The present invention relates to an information display method and system, and a terminal. The method includes: collecting a visual signal in a target area of a display device by using an augmented reality device; judging a user reading scene according to the visual signal; generating a first signal according to the user reading scene; generating a second signal after processing the first signal; collecting a target parameter of the display device; generating a display signal after fusing the second signal with the target parameter; and performing information display on a display interface of the augmented reality device according to the display signal. By adopting the information display method and system, and the terminal of the present invention, the reading efficiency of the user may be improved.

Visual-Inertial Positional Awareness for Autonomous and Non-Autonomous Tracking

The described positional awareness techniques employing visual-inertial sensory data gathering and analysis hardware with reference to specific example implementations implement improvements in the use of sensors, techniques and hardware design that can enable specific embodiments to provide positional awareness to machines with improved speed and accuracy.

Interactive Generation and Publication of an Augmented-Reality Application

An electronic device that specifies or determines information associated with an application is described. During operation, the electronic device may identify one or more objects of interest in an acquired image. Then, the electronic device may determine or specify classifications (such as names) for the one or more objects of interest. Moreover, the electronic device may analyze a context of the one or more objects of interest in order to determine one or more inspection criteria. Once the one or more inspection criteria are determined, the electronic device may receive publishing information (such as designated recipients) and privacy settings (such as is the application private or public). Next, the electronic device may generate the application using the specified or determined information, and may publish or provide the application to one or more other electronic devices associated with the designated recipients using the publishing information and the privacy settings.

Virtual Barrier Objects

A computer system presents a virtual object on a display of the computer system which represents an action at a location in space. If a user of the computer system approaches the location, the computer system presents one or more stimuli to encourage movement away from that location. The appearance of the object may vary as the user of the computer system moves. The appearance of the object may include information and may vary with a change of state. Multiple such virtual objects may be used to indicate a path.

Method and system for teledentistry images

A method ( 300 ) for obtaining one or more oral images ( 90 ) using an imaging device or system ( 100, 200 ) includes: (i) obtaining ( 320 ), using an imager, an oral image; (ii) extracting ( 330 ), by a controller, one or more features from the oral image, wherein the extracted one or more features comprises at least an image quality feature and an object recognition feature; (iii) determining ( 340 ), by a decision module of the controller, whether the one or more extracted features satisfy a predetermined feature threshold; (iv) providing ( 350 ) feedback to a user regarding the obtained oral image if the one or more extracted features does not satisfy the predetermined feature threshold, or authorizing ( 370 ) transmission of the oral image to a dental professional if the one or more extracted features satisfy the predetermined feature threshold; and (v) transmitting ( 380 ), if transmission is authorized, the obtained oral image to a dental professional.

Storing metadata related to captured images

The present disclosure relates to user-selected metadata related to images captured by a camera of a client device. User-selected metadata may include contextual information and/or information provided by a user when the images are captured. In various implementations, a free form input may be received at a first client device of one or more client devices operated by a user. A task request may be recognized from the free form input, and it may be determined that the task request includes a request to store metadata related to one or more images captured by a camera of the first client device. The metadata may be selected based on content of the task request. The metadata may then be stored, e.g., in association with one or more images captured by the camera, in computer-readable media. The computer-readable media may be searchable by the metadata.

Information processing method, information processing apparatus, and non-transitory recording medium storing instructions for executing an information processing method

An information processing method, apparatus, and non-transitory recording medium storing instructions for executing an information processing method. The information processing method, performed by a computer connectable to a terminal via a network, includes determining a status of a device based on a first image of the device transmitted from the terminal, specifying a solution for a problem occurring in the device based on the status of the device, displaying a first message on the terminal to indicate the specified solution, displaying a second message on the terminal to request a second image obtained by photographing a specified part of the device or the device in a specified state, updating the status of the device based on the second image, specifying the solution for the problem in accordance with the updated status of the device, and displaying a third message on the terminal to indicate the specified solution for the problem.

Smartphone dock and diagnostic-test reader plus related methods

Disclosed is are a diagnostic test reader and related methods or systems. In one embodiment, a smartphone, including diagnostic software and a camera, may be docked onto the reader so that the camera of the phone is coupled to a diffuser for imaging test strips or other test cartridges provided to the reader. Preferably, the diagnostic software is configured to interpret an image of the strip and provide diagnostic feedback via the interface of the phone. Light for producing the image may be from the flash of the smartphone camera or a light source inside the reader. The more specific details of the disclosed reader are described in connection with the figures.

Vision system for training an assembly system through virtual assembly of objects

This invention provides a system and method for guiding the workpieces to optimal positions to train an assembly system that is generally free of the use of a CMM or similar metrology device. The system and method expresses the image features of the workpieces, when they are in their respective stations, in a common coordinate system. This ability allows a user to visualize the result of assembling the workpieces without actually assembling them, in a “virtual assembly”. The virtual assembly assists guiding placement of workpieces in respective stations into a desired relative alignment. The system and method illustratively generates a composite image using the images from cameras used in guiding the workpieces that helps the user visualize how the part would appear following assembly. The user can reposition the images of workpieces in their respective stations until the composite image has a desired appearance.

Dynamic merchandising communication system

Provided herein are display systems and units, including those configured for dynamic communication in a physical location, such as in retail settings. Also included herein are methods for dynamically displaying product information in a physical location, such as a retail setting.

Methods and systems for providing an augmented reality interface for saving information for recognized objects

A method includes displaying and capturing image data containing an object and accessing a plurality of records related to objects, selecting a record related to the captured object, obtaining an identifier of a vendor of the object of the selected data record, combining the selected data record and the vendor identifier to form a search record, displaying, based on the search record, an augmented reality interface to receive a first interactive action for saving the search record, receiving the first interactive action, saving, in response to receiving the first interactive action, the search record into a searchable data structure, receiving a second interactive action, retrieving, in response to receiving the second interactive action, the search record from the searchable data structure, updating the vendor identifier based on the retrieved search record, and displaying information related to the search record.

H.v.e.

The present invention illustrates a story book virtual reality application that includes, but is not limited to scanning capability, 3D graphics presentation and video presentation that immerses participants into experiencing the story line of a story book though mobile virtual reality viewing devices or cast onto to television via app. The book cover or a page in the book itself can be scanned with a smart device (smart phone, tablet, Kindle, Nuke, MAC, PC, laptop), which activates the virtual reality book application to get the appropriate content of the VR-Book. The participant can then experience themselves being fully immersed into experiencing the story line through their virtual reality viewing device, when displaying the video VR content using the VR application, through their mobile phone or cast to television via app. Therefore, the invention allows participants to be immersed into the story book as real time spectating participant in the story book using pre filmed real life actors and participants who represent the selected character or characters in the story book.

Methods and arrangements employing sensor-equipped smart phones

The present technology concerns improvements to smart phones and related sensor-equipped systems. Some embodiments involve spoken clues, e.g., by which a user can assist a smart phone in identifying what portion of imagery captured by a smart phone camera should be processed, or identifying what type of image processing should be conducted. Some arrangements include the degradation of captured content information in accordance with privacy rules, which may be location-dependent, or based on the unusualness of the captured content, or responsive to later consultation of the stored content information by the user. A great variety of other features and arrangements are also detailed.

Information processing apparatus, transmission method, and storage medium

An information processing apparatus includes a first acquisition unit, a recognition unit, a specification unit, and a transmission unit. The first acquisition unit acquires an image. The recognition unit recognizes an object included in the image. The specification unit causes a user to specify data to be transmitted to the object recognized by the recognition unit. The transmission unit transmits the specified data to an apparatus identified by destination information corresponding to the object, according to the specification by the specification unit.

Systems, methods, and computer programs, for analyzing images of a portion of a person to detect a severity of a medical condition

Methods, systems, and computer programs for monitoring skin condition of a person. In one aspect, a method can include obtaining data representing a first image, the first image depicting skin from at least a portion of a body of a person, generating a severity score that indicates a likelihood that the person is trending towards an increased severity of an auto-immune condition or trending towards a decreased severity of an auto-immune condition, comparing, the severity score to a historical severity score, wherein the historical severity score is indicative of a likelihood that a historical image of the user depicts skin of a person having the auto-immune condition, and determining based on the comparison, whether the person is trending towards an increased severity of the auto-immune condition or trending towards a decreased severity of the auto-immune condition.

Textured mesh building

Systems and methods are provided for receiving a two-dimensional (2D) image comprising a 2D object; identifying a contour of the 2D object; generating a three-dimensional (3D) mesh based on the contour of the 2D object; and applying a texture of the 2D object to the 3D mesh to output a 3D object representing the 2D object.

Visual search refinement for computer generated rendering environments

Various implementations disclosed herein include devices, systems, and methods that enable faster and more efficient real-time physical object recognition, information retrieval, and updating of a CGR environment. In some implementations, the CGR environment is provided at a first device based on a classification of the physical object, image or video data including the physical object is transmitted by the first device to a second device, and the CGR environment is updated by the first device based on a response associated with the physical object received from the second device.

Markerless augmented reality (ar) system

A markerless augmented reality (AR) can track 2D feature points among video frames, generate 2D point clouds and 3D point clouds based thereon, and match a 3D model against 3D point cloud to obtain proper positional information of the model with respect to a frame. The AR system can use the 3D model with the obtained positional information to render and project AR content to a user's view. Additionally, the AR system can maintain associations between frames and 3D model positional information for search and retrieval.

Notification system and notification control apparatus used in the same

A notification system includes a detector, an identifier, a notifier, a determination module, and a controller. The detector detects that a commodity is placed at a display location. The identifier identifies the commodity. The notifier executes a notification operation with a plurality of different notification methods. The determination module determines a degree of conformity between the commodity identified by the identifier and the display location at which the commodity is placed. The controller controls the notifier such that the notifier executes the notification operation with one notification method associated to the degree of conformity determined by the determination module from the plurality of notification methods.

Augmented reality techniques

A visual characteristic on a physical surface in the physical space is determined from an image of a physical space, with the physical surface including a first area inside the visual characteristic and a second area outside of the visual characteristic. A first region of virtual space that corresponds to the first area and a second region of virtual space that corresponds to the second area is determined based at least in part on the visual characteristic. Information that includes a position and orientation of a digital object relative to the first region and the second region in the virtual space is determined based at least in part on the visual characteristic. The digital object is rendered at a position and orientation in the virtual space in accordance with the information.

Room-scale virtual reality mobile unit

A virtual reality system, including one or more modular walls configured to define an interior area, at least one modular wall having a monochromatic interior surface, a camera positioned within the interior area, the camera configured to capture an image of a player with the monochromatic interior surface as a background, one or more processors configured to generate a virtual scene, to receive the image of the player, and to generate a composite image including at least a portion of the image of the player composited into the virtual scene, and a display configured to receive and display the composite image. The one or more modular walls may define a number of interior areas for multiple players.

Alerts of mixed reality devices

In some examples, a system for alerts of mixed reality (MR) devices includes a receiver engine to receive an input from a sensor of a MR device in response to the sensor detecting a person in an environment in which the MR device is located, and a generator engine to cause the MR device to generate an alert in response to the detected person based on an identity of the detected person.

Point cloud data transmission apparatus, point cloud data transmission method, point cloud data reception apparatus and point cloud data reception method

According to embodiments, point cloud data transmission method may include encoding point cloud data, encapsulating a bitstream that includes the encoded point cloud data and signaling data into a file, and transmitting the file, the bitstream is stored either in a single track or in multiple tracks of the file, the signaling data include at least one parameter set, and the encoded point cloud data include a geometry bitstream containing geometry data and an attribute bitstream containing attribute data.

System And Method For Accessing Electronic Data Via An Image Search Engine

The present invention provides a system and method for accessing electronic data through entry of images as queries in search engine. The system uses various image capturing devices and communication devices to capture images and enter them into image database. Image recognition techniques encode images in a computer readable format. The processed image is then entered for comparison into at least one database populated with images and associated information. Once the newly captured image is matched with an image in the database, the information linked with that image is returned to the user.

Augmented reality systems and methods incorporating wearable pin badges

Systems and methods disclosed in this application are directed to augmented reality for used with pin badges. Pin badges can be worn, held, or set within view of an AR device having a camera. The AR device sends images or video from its camera to a platform server that determines whether a pin badge exists in view of the camera. If a pin badge exists, it is identified and augmented reality imagery related to the pin badge is transmitted back to the AR device so that the AR device can incorporate that augmented reality imagery into a video stream from its camera as shown on its display.

Image processing method and image processing device

Provided are an image processing method and an image processing device. The image processing method includes generating an image based on viewpoint information of a user; rendering the image based on information about what is in front of the user; and outputting the rendered image using an optical element.

Training a neural network with representations of user interface devices

Disclosed herein are examples of a wearable display system capable of determining a user interface (UI) event with respect to a virtual UI device (e.g., a button) and a pointer (e.g., a finger or a stylus) using a neural network. The wearable display system can render a representation of the UI device onto an image of the pointer captured when the virtual UI device is shown to the user and the user uses the pointer to interact with the virtual UI device. The representation of the UI device can include concentric shapes (or shapes with similar or the same centers of gravity) of high contrast. The neural network can be trained using training images with representations of virtual UI devices and pointers.

Commerce facilitation system

A commerce facilitation system enables the seamless purchase of available products as displayed on the screen of a digital device. A portal uses an image capture of a posted product to access an identifier of the product so that an invoice for the product is transmitted directly to the user upon creating of the image capture.

Providing an augmented reality overlay for display over a view of a user

The present disclosure is directed toward systems and methods for generating and providing an augmented reality overlay for display in connection with an augmented reality display device. For example, systems and methods described herein identify a user being viewed through an augmented reality display device, and builds an augmented reality overlay for the user that is displayed on a view of the user through the augmented reality display device. Systems and methods described herein build the augmented reality overlay based on the location of the augmented reality display device, and on other networking system information including a networking system relationship between the user wearing the augmented reality display device and the user who is being looked at through the augmented reality display device.

Systems and methods for computer assisted operation

An augmented reality method includes selecting an image of an object to be installed; converting the image into a 3D model; capturing a view of an environment and selecting a place to insert the 3D model; overlaying the 3D model with the view of the environment in a mashed-up and aligned manner wherein the 3D model view is displayed contemporaneously and contiguously to form an augmented reality view; and enabling user interaction with the displayed 3D model view to update the 3D model of the environment in the 3D modeling system.

Refuse contamination analysis

A computer-implemented method for analyzing refuse includes operations of receiving sensor data indicating an operational state of a vehicle body component of a refuse collection vehicle (RCV); analyzing the sensor data to detect a presence of a triggering condition based at least partly on a particular operational state of the vehicle body component, as indicated by the sensor data; in response to detecting the triggering condition, accessing image data indicating a physical state of refuse collected by the RCV; providing the image data as input to at least one contaminant detection model trained, using at least one machine learning (ML) algorithm, to output a classification of the image data, the classification indicating a degree of contamination of the refuse; and storing, in a machine-readable medium, the classification of the image data.

Augmented reality system capable of manipulating an augmented reality object

An augmented reality system according to the present invention comprises a mobile terminal which, in displaying a 3D virtual image on a display, displays a dotted guide along the boundary of characters displayed on the display and when handwriting is detected along the dotted guide, recognizes the characters and displays a virtual object corresponding to the content of the characters, wherein, if the virtual object is touched, a pre-configured motion of the virtual object corresponding to the touched area is reproduced.

Information processing method, information processing device, and computer-readable non-transitory storage medium storing program

A clothing item being captured is appropriately measured without requiring a dedicated sheet of paper or the like. An information processing method is performed by one or more processors included in an information processing device, the method including acquiring, from a memory, image data including a clothing item being captured; detecting a reference plane including the clothing item; transmitting the image data to an other information processing device; acquiring, from the other information processing device, each piece of coordinate data, in an image coordinate system, indicating each measured point regarding a size of the clothing item included in the image data, the coordinate data being obtained as a result of image processing on the image data; projecting and plotting, on the reference plane, each measured point indicated by each piece of coordinate data in the image coordinate system; calculating, in relation to each measured point in a three-dimensional coordinate system plotted on the reference plane, a distance between predetermined two points, the distance indicating the size of the clothing item; and performing control such that the distance between the predetermined two points that is calculated is displayed on the image data being captured.

Vehicle display system

The purpose of the present invention is to contribute to safe vehicle operation by suppressing the movement of a user's line-of-sight and causing the degree of a potential risk to be intuitively presented and recognized. A vehicle display system comprises: a periphery recognition unit that detects the position, type, and the like of a specified object on a road surface; a display unit that displays a wall surface image which is superimposed onto the actual scenery outside of a user's vehicle so as to be visible to a user; and a control unit that controls the display unit. The control unit displays, with the specified object as the point of origin, a lateral wall image having a wall surface section that protrudes upward.

Business discovery from imagery

Aspects of the present disclosure relate to a method includes training a deep neural network using training images and data identifying one or more business storefront locations in the training images. The deep neural network outputs tight bounding boxes on each image. At the deep neural network, a first image may be received. The first image may be evaluated using the deep neural network. Bounding boxes may then be generated identifying business storefront locations in the first image.

Automatic annotation of video quality impairment training data for generating machine learning models of video quality prediction

Exemplary embodiments are directed to methods and systems for annotating generating training data for video quality impairment detection. A video stream recorder extracts one or more image frames from a reference video stream as it is played. A video image labeler embeds a unique label into each of the one or more extracted image frames. The video stream recorder records the one or more labeled image frames as a labeled video stream. The video stream player then plays the labeled video stream through an impaired communication channel to generate a degraded video stream. A video image comparator compares one or more corresponding frames of the labeled video stream and the degraded video stream to generate one or more difference frames. An impaired image recorder annotates at least one of the one or more difference frames according to a corrupted region of the at least one difference frame.

Mirror type display device and method for controlling same

The present invention relates to a mirror type display device and a control method thereof, and a mirror type display device of the present invention includes a mirror type display; a camera provided on a front surface of the display; an interface for recognizing a user's dress(wear) located in front of the display and a mobile terminal carried by the user through the camera and recommending an operation mode of the recognized mobile terminal based on the recognized dress(wear), and a controller. Thus, the user projecting the image on the mirror-type display device can be photographed to recognize the dress, and the mobile terminal recognized by the user can be recognized to recommend the mode of the mobile terminal suitable for the current dress mode.

Attention guidance for ground control labeling in street view imagery

The disclosure provides methods, apparatus, and products for attention guidance and labeling. In one aspect, a method comprises receiving metadata associated with a perspective image captured by an image capture device located at the image position and having the image pose, wherein the metadata comprises the image position and the image pose. The method comprises defining a field-of-view indicator having a first indicator position located at the image position and having the image pose; receiving ground control point (GCP) information identifying a GCP, wherein the GCP information comprises a GCP position; defining a GCP indicator having a second indicator position located at the GCP position; and causing display of an indicator layer comprising the field-of-view indicator and the GCP indicator in a second portion of an interactive user interface (IUI) of a labeling tool, wherein at least the perspective image is displayed in a first portion of the IUI.

Guide Cane And Method For Guiding Used For Guide Cane

Embodiments of the present disclosure provide a guide cane and a method for guiding used for a guide cane. The guide cane comprises a movement component and an object acquisition component. The object acquisition component is provided on the movement component, and the movement component is electrically coupled to the object acquisition component. The object acquisition component acquires a positional information of the target object in a target space, and determines a navigation route according to the target position indicated by the positional information and an initial position where the guide cane is currently located. The movement component guides, according to the navigation route, a user to arrive at the target position.

Method and Apparatus for Configuring Privacy Settings for Publishing Electronic Images

A method and apparatus configure privacy settings for publishing electronic images. An image including first image content and second image content can be received. A first image content data file can be created for the first image content and a second image content data file can be created for the second image content. A publication privilege can be assigned to the first image content data file. The publication privilege can be based on a relationship between a consumer of the image and a subject of the first image content. The first image content data file with the publication privilege can be stored separate from the second image content data file. Image reconstruction data can be stored. The image reconstruction data can provide information on how to reconstruct the image from the first image content data file and the second image content data file

Role-based perception filter

Systems and methods for filtering objects of interest associated with an investigation. One method includes receiving a role identifier. The method includes retrieving metadata corresponding to the plurality of objects of interest. The method includes determining, with a classifier, a plurality of relationships between at least two of the objects of interest based on the metadata. The method includes identifying, based on the metadata, a subset of the plurality of relationships that are associated with the role identifier, the subset of the plurality of relationships including a subset of the plurality of objects of interest. The method includes generating a graphical representation including a first indication of the subset of the objects of interest and a second indication of the subset of the plurality of relationships. The method includes presenting the graphical representation on a display communicatively coupled to the electronic processor.

Augmented reality head gesture recognition systems

A method, system, apparatus, and/or device for moving or scrolling a virtual object in a virtual or augmented reality environment. The method, system, apparatus, and/or device may include: detecting, by a first sensor, a first gesture associated with selecting a first virtual object in an augmented reality environment displayed by a head-mounted display; displaying, by the head-mounted display, a first indicator indicating a selection of the first virtual object by a user; detecting, using the first sensor or a second sensor, a first movement of the head-mounted display associated with a first movement command; and in response to detecting the first movement of the head-mounted display, executing the first movement command, where the first movement command is a scrolling function to scroll text or a graphical object of the first virtual object or a movement function to move the text or the graphical object of the first virtual object.

Visual, audible, and/or haptic feedback for optical see-through head mounted display with user interaction tracking

A method, an apparatus, and a computer program product provide feedback to a user of an augmented reality (AR) device having an optical see-through head mounted display (HMD). The apparatus obtains a location on the HMD corresponding to a user interaction with an object displayed on the HMD. The object may be an icon on the HMD and the user interaction may be an attempt by the user to select the icon through an eye gaze or gesture. The apparatus determines whether a spatial relationship between the location of user interaction and the object satisfies a criterion, and outputs a sensory indication, e.g., visual display, sound, vibration, when the criterion is satisfied. The apparatus may be configured to output a sensory indication when user interaction is successful, e.g., the icon was selected. Alternatively, the apparatus may be configured to output a sensory indication when the user interaction fails.

Map generation for an environment based on captured images

Systems and methods for map generation for an environment based on captured images are disclosed. According to an aspect, a method includes capturing a first image of an environment. The method also includes identifying a reference in the first image. Further, the method includes generating, based on the identified reference, a map of the environment to use for physically orienting a computing device within the environment based on a second image including the reference.

Medical device for improving environmental perception for blind or visually-impaired users

A device for improving environmental perception for blind or visually impaired users, including a set of mechanical actuators intended to be in contact with the skin of a user, at least one digital camera designed to acquire a current digital image of an environment facing the user, a processing circuit connected to the camera for receiving pixel signals from the acquired digital image and converting at least one portion of the pixel signals into control signals, each of which powers a mechanical actuator of the set of actuators, an eye-tracking module for tracking each eye of the user to identify a gaze direction of the user. The processing circuit then selects, in the environment filmed by the camera, an area of acquired current image which is a function of the gaze direction and converts the pixel signals of said area into control signals, each of which powers an actuator of the set to stimulate the user's skin.

Method and apparatus for providing augmented reality-based dynamic service

The present invention includes the steps of: collecting data associated with a preset authoring item based on a predetermined viewing point within content to be guided, when an authoring mode; rendering authoring item-specific data collected based on a preset augmented reality content service policy, matching the rendered data to metadata corresponding to the viewing point, and storing the matched data; checking the metadata of the viewing point; and as a result of the checking, matching the metadata to the content displayed in the viewing region and augmenting the content.

Word flow annotation

An augmented reality (AR) device can be configured to monitor ambient audio data. The AR device can detect speech in the ambient audio data, convert the detected speech into text, or detect keywords such as rare words in the speech. When a rare word is detected, the AR device can retrieve auxiliary information (e.g., a definition) related to the rare word from a public or private source. The AR device can display the auxiliary information for a user to help the user better understand the speech. The AR device may perform translation of foreign speech, may display text (or the translation) of a speaker's speech to the user, or display statistical or other information associated with the speech.

Commodity recognition apparatus and commodity recognition method

A candidate output element configured to output recognition target commodities as candidates of a recognized commodity in a descending order of the similarity degrees calculated by the similarity degree calculation element, a distance measurement element configured to measure the distance from the image capturing section to a commodity photographed by the image capturing section, and a changing element configured to change the number of candidates of a recognized commodity output by the candidate output element according to the distance measured by the distance measurement element.

Interactive touch screen device for wine

Interactive display and rendering device, system and method for wine bottles. are disclosed. An interactive wine management and display device and method using a scanning instrument capable of scanning information regarding a bottle of wine; an imaging instrument capable of collecting an image of at least a portion of a bottle of wine; a connection to a wine database containing information regarding wines; a connection to a database having images of wine bottles. The device is able to render the images as well as information on the wine to a display for user interaction.