СЪЕМКА ВИДОВ И ДВИЖЕНИЙ АКТЕРОВ, ДЕЙСТВУЮЩИХ В СГЕНЕРИРОВАННЫХ ДЕКОРАЦИЯХ

Изобретение относится к моделированию работы виртуальной камеры, действующей в декорациях, сгенерированных для кинофильмов и видеоигр. Технический результат - обеспечение генерирования декораций для виртуальной среды визуальной развлекательной программы. Способ генерирования декорации для виртуальной среды визуальной развлекательной программы содержит этапы, на которых: выполняют первую сессию съемки движения для записи движений множества меток для съемки движений, надетых на актера, внутри физического объема пространства; первая сессия съемки выполнена с возможностью генерирования сцены декораций для виртуальной среды визуальной развлекательной программы; выполняют вторую сессию съемки движения для отслеживания и записи движений нательной камеры, которая является камерой, такой как объект, надеваемый на актера, выполненный с возможностью съемки и преобразования движений и видов указанного актера, выполняемых в указанных сценах декораций; преобразуют движения и виды актера, выполняемые ...

ОБЛЕГЧЕНИЕ ВЗАИМОДЕЙСТВИЯ МЕЖДУ ПОЛЬЗОВАТЕЛЯМИ И ИХ ОКРУЖАЮЩИМИ СРЕДАМИ С ПОМОЩЬЮ ГАРНИТУРЫ, ИМЕЮЩЕЙ МЕХАНИЗМЫ ВВОДА

Изобретение относится к вычислительной технике. Технический результат заключается в обеспечении безопасности перемещения в пространстве. Система содержит гарнитуру, выполненную с возможностью представления аудиоинформации пользователю по мере того, как пользователь взаимодействует с пространством, причем гарнитура содержит набор механизмов ввода, выполненных с возможностью приема команд от пользователя, вызывающих выполнение модулем взаимодействия с пространством соответствующих относящихся к взаимодействию с пространством функций, причем гарнитура дополнительно содержит механизм определения ориентации гарнитуры для определения ориентации гарнитуры, чтобы предоставлять информацию об ориентации гарнитуры. Система дополнительно содержит модуль взаимодействия с пространством, выполненный с возможностью запускать режим исследования, находясь в режиме исследования определять направление внимания пользователя, идентифицировать набор представляющих интерес элементов, которые связаны с подпространством ...

ИСПОЛЬЗОВАНИЕ ВТОРОГО ЭКРАНА В КАЧЕСТВЕ ПЕРСОНАЛЬНОГО ПРОЕКЦИОННОГО ДИСПЛЕЯ НА ОЧКАХ

Изобретение относится к способу расширения области обзора дисплейного устройства для отображения контента. Технический результат заключается в расширении ограниченной экранной области, имеющейся в вычислительных устройствах, в целях обеспечения возможности работы в многозадачном режиме для пользователей. Сущность изобретения заключается в том, что способ включает этапы, на которых определяют наличие окулярного вычислительного устройства вблизи первичного вычислительного устройства. Первичное вычислительное устройство имеет первичный дисплей, который выполнен с возможностью отображения контента. Окулярное вычислительное устройство включает вторичный дисплей. Окулярное вычислительное устройство соединяют с первичным вычислительным устройством. Определяют направление обзора для окулярного вычислительного устройства относительно первичного дисплея. Отображают на вторичном дисплее дополнительный контент, который является дополнением к контенту, отображаемому на первичном дисплее. Вторичный дисплей ...

СПОСОБ И УСТРОЙСТВО УПРАВЛЕНИЯ ДИСПЛЕЕМ

Изобретение относится к способу управления экранным дисплеем. Технический результат заключается в расширении арсенала средств. Способ управления экранным дисплеем (550) электронного устройства (500) включает в себя этапы, на которых определяют (110) посредством электронного устройства разрешение камеры (720) путем выбора одного из высокого разрешения и низкого разрешения, распознают лицо пользователя с использованием камеры, работающей с определенным разрешением (120), и управляют экранным дисплеем посредством распознавания лица. При этом, когда камера работает с низким разрешением, распознают позицию лица пользователя, а когда камера работает с высоким разрешением, получают позицию взгляда пользователя, и при этом определение разрешения содержит этапы, на которых определяют состояние питания электронного устройства; и выбирают одно из высокого разрешения или низкого разрешения в зависимости от состояния питания электронного устройства. 2 н. и 16 з.п. ф-лы, 12 ил.

СПОСОБ И СИСТЕМА ДЛЯ ОБРАБОТКИ АУДИОКОНТЕНТА, СПОСОБ И СИСТЕМА ДЛЯ ОБРАБОТКИ МУЛЬТИМЕДИЙНОГО КОНТЕНТА ДЛЯ ВОСПРОИЗВЕДЕНИЯ

Изобретение относится к области информационных технологий, а именно к средствам обработки мультимедийного контента. Технический результат направлен на повышение точности обработки и снижения времени задержки. Способ обработки аудиоконтента, содержащий этапы, на которых: получают, используя один или более процессоров, первую информацию положения, указывающую первую позицию или ориентацию пользователя, получают, используя один или более процессоров, прогнозированную информацию положения на основе первой информации положения и информации предыдущего положения, подготавливают посредством рендеринга аудиоконтент и передают, используя один или более процессоров, подготовленный посредством рендеринга аудиоконтент и прогнозированную информацию положения в первое оборудование для воспроизведения, при этом прогнозированная информация положения и вторая информация положения используются для обновления подготовленного посредством рендеринга аудиоконтента перед воспроизведением обновленного подготовленного ...

УСТРОЙСТВО, СПОСОБ И СИСТЕМА ОБЕСПЕЧЕНИЯ УВЕЛИЧЕННОГО ДИСПЛЕЯ С ИСПОЛЬЗОВАНИЕМ ШЛЕМА-ДИСПЛЕЯ

Шлем-дисплей содержит логику формирования, прозрачный дисплей и модуль презентации. Прозрачный дисплей выполнен с возможностью предоставления пользователю шлема-дисплея просматривать по меньшей мере один виртуальный дисплей и физический дисплей в сцене реального мира. Указанный по меньшей мере один виртуальный дисплей представлен на прозрачном дисплее в качестве дополнительного объекта в сцене реального мира, и указанный по меньшей мере один виртуальный дисплей используется в соединении с физическим дисплеем для представления сформированной компьютером информации на расширенном рабочем столе с визуальной точки зрения пользователя. Технический результат заключается в обеспечении мобильности развертывания одного или более дисплеев для портативных устройств для вывода на экран сформированной компьютером информации в соединении с физическим дисплеем вычислительной системы. 3 н. и 15 з.п. ф-лы, 5 ил.

СПОСОБ, УСТРОЙСТВО И ПОТОК ФОРМАТИРОВАНИЯ ИММЕРСИВНОГО ВИДЕО ДЛЯ УСТРОЙСТВ УНАСЛЕДОВАННОГО И ИММЕРСИВНОГО РЕНДЕРИНГА

Изобретение относится к области кодирования и декодирования видео. Технический результат заключается в улучшении кодирования обратно совместимого иммерсивного видео. Поток переносит данные, представляющие иммерсивное видео, состоящее из кадра, организованного согласно схеме, содержащей первую область, кодированную согласно прямоугольному преобразованию, вторую область, кодированную согласно преобразованию, переходящему от прямоугольного преобразования к иммерсивному преобразованию, и третью область, кодированную согласно иммерсивному преобразованию. Для обратной совместимости поток дополнительно содержит первую информацию, представляющую размер и местоположение первой области в видеокадре, и вторую информацию, содержащую, по меньшей мере, тип выбранной схемы, поле зрения первой части, размер упомянутой второй области в видеокадре и опорное направление. 6 н. и 9 з.п. ф-лы, 11 ил.

Способ, устройство и электронное оборудование для управления виртуальной реальностью

Изобретение относится к способам управления виртуальной реальностью (VR). Технический результат заключается в повышении точности ввода данных. Такой результат достигается за счет получения данных датчика, указывающих на передвижение носимого VR-устройства в трехмерном пространстве в течение заданного периода времени, определяемого первым моментом времени, в который принимается первая инструкция управления, и вторым моментом времени, в который принимается вторая инструкция управления; определение траектории передвижения носимого VR-устройства в течение заданного периода времени на основе данных датчика, полученных в течение заданного периода времени; управление устройством выполнения функций VR для выполнения операций, соответствующих траектории передвижения, и отображения на дисплее VR-устройства результатов операций, соответствующих траектории передвижения; получение данных датчика, указывающих на передвижение носимого VR-устройства в трехмерном пространстве, от первого гироскопического ...

УСТРОЙСТВО ОЦЕНКИ ПОЛОЖЕНИЯ ДВИЖУЩЕГОСЯ ТЕЛА И СПОСОБ ОЦЕНКИ ПОЛОЖЕНИЯ ДВИЖУЩЕГОСЯ ТЕЛА

Устройство оценки положения движущегося тела – транспортного средства - основано на том, что сопоставляют данные карты, хранящейся в блоке хранения, и изображений, полученных камерой, с использованием фильтра частиц. При этом блок вычисления значения оценки сравнивает изображение края объекта и виртуальное изображение для каждой частицы, присваивает более высокое значение оценки, если имеется большее количество перекрывающихся краев между изображениями, а также присваивает более высокое значение оценки, если имеется большее количество краев, которые не являются перекрывающимися краями и для которых расстояние от края до края, которое представляет собой расстояние между краем в изображении края и краем в виртуальном изображении, меньше или равно заданному значению. Технический результат заключается в повышении стабильности оценки положения движущегося тела. 4 н. и 2 з.п. ф-лы, 9 ил.

ОРИЕНТАЦИЯ И ВИЗУАЛИЗАЦИЯ ВИРТУАЛЬНОГО ОБЪЕКТА

Изобретение относится к ориентации и визуализации виртуального объекта. Техническим результатом является обеспечение создания виртуального объекта с привязкой к ракурсу. Способ содержит этапы, на которых: принимают данные местоположения объекта и данные потребления для совместно используемого виртуального объекта; отображают виртуальный маркер, ассоциированный с совместно используемым виртуальным объектом, в положении маркера, соответствующем географическому местоположению объекта; определяют географическое местоположение потребляющего устройства относительно положения маркера; определяют положение потребляющего устройства и ориентацию потребляющего устройства в системе локальных координат и выводят совместно используемый виртуальный объект на потребляющем устройстве в качестве реакции на совпадение положения потребляющего устройства и ориентации потребляющего устройства соответственно с единственно возможным положением потребления и единственно возможной ориентацией потребления и в качестве ...

УСТРОЙСТВО ОБРАБОТКИ ИНФОРМАЦИИ, СПОСОБ ОБРАБОТКИ ИНФОРМАЦИИ И ПРОГРАММА

УСТРОЙСТВО КОНТРОЛЯ СИСТЕМЫ ДВИЖЕНИЙ

Изобретение относится к вычислительной технике, в частности к устройствам ввода, и может быть использовано в компьютерных играх, для управления объектами в моделируемых на ЭВМ пространствах, в области машиностроения, самолетостроения, космонавтики, направлено на повышение надежности и точности управления системой движений, это достигается за счет того, что устройство включает в себя кнопки 1, корпус 2 с шарниром 3, хвостовиком 4, жестко соединенным с сиденьем оператора 13, кронштейном 14 с зажимами 15. Корпус 2 жестко соединен с управляющим элементом 5, выполненном в виде Н-образной рамы с зажимами 11, с криволинейными опорными поверхностями 6 для расположения предплечий оператора с возможностью движения за счет шарнира 3, жесткость хода которого может регулироваться с помощью расположенного в корпусе 2 регулятора 12, на передних частях 7 управляющего элемента 5 по периферии напротив друг друга с боковых сторон расположены кнопки 1, кнопки шарнирные 8, кнопки многоконтактные 9, ролики 10 ...

УСТРОЙСТВО ДЛЯ ОБРАБОТКИ ИНФОРМАЦИИ, СПОСОБ УПРАВЛЕНИЯ ДИСПЛЕЕМ И ПРОГРАММА ДЛЯ УПРАВЛЕНИЯ ДИСПЛЕЕМ

... 1. Способ для управления содержимым, представленным на дисплее, содержащий: ! - прием сигналов, представляющих первые и вторые характеристики пользователя; ! - представление первого и второго изображения на дисплее, при этом первое и второе изображения, соответственно, связаны с первой и второй характеристиками; и ! - представление на дисплее ассоциативной информации, при этом ассоциативная информация может обозначать связь между первым и вторым изображениями. ! 2. Способ по п.1, в котором первая характеристика соответствует лицу пользователя, а вторая характеристика соответствует руке пользователя. ! 3. Способ по п.1, дополнительно содержащий: ! - запись данных изображения, в которых принятые сигналы основываются на данных изображения; ! - обработка данных изображения, чтобы распознать первую и вторую характеристики пользователя, при этом распознавание первой характеристики включает в себя обнаружение лица пользователя, а распознавание второй характеристики включает в себя обнаружение ...

ОСНОВАННОЕ НА ВЗГЛЯДЕ РАЗМЕЩЕНИЕ ОБЪЕКТОВ В СРЕДЕ ВИРТУАЛЬНОЙ РЕАЛЬНОСТИ

СИСТЕМА ОТОБРАЖЕНИЯ ВИРТУАЛЬНОЙ РЕАЛЬНОСТИ

... 1. Устройство, содержащее:процессор (216) изображений, сконфигурированный для приема информации (226) о положении первого глаза (234) на голове (236) пользователя (206) и информации (228) о положении второго глаза (240) на голове (236) пользователя (206);создания первого изображения (252) виртуальной реальности (204) для первого глаза (234) на основе информации (226) о положении первого глаза (234);создания второго изображения (254) виртуальной реальности (204) для второго глаза (240) на основе информации (228) о положении второго глаза (240); ипередачу первого изображения (252) и второго изображения (254) для отображения для пользователя (206).2. Устройство по п.1, дополнительно содержащее:контроллер (214), сконфигурированный для создания виртуальной реальности (204);средство (405) слежения за головой, сконфигурированное для отслеживания положения (264) головы (236) и получения информации (230) о положении головы (236); и,причем, процессор (216) изображений дополнительно сконфигурирован ...

СПОСОБ И УСТРОЙСТВО ОТОБРАЖЕНИЯ С ОПТИМИЗАЦИЕЙ ПЕРЕРАСПРЕДЕЛЕНИЯ ПИКСЕЛЕЙ

СИСТЕМА И СПОСОБ УПРАВЛЕНИЯ ДОПОЛНЕННЫМИ АТТРАКЦИОНАМИ ДЛЯ КАТАНИЯ

СПОСОБ (ВАРИАНТЫ) И УСТРОЙСТВО ДЛЯ УПРАВЛЕНИЯ ИСТОЧНИКОМ ТОКА, А ТАКЖЕ СВАРОЧНЫЙ ШЛЕМ ДЛЯ ПОЛЬЗОВАТЕЛЯ ИСТОЧНИКА ТОКА

... 1. Способ управления со стороны пользователя соединенным с ручным рабочим прибором (4) источником тока (1), причем посредством элементов управления (3) блока (2) управления и индикации на источнике тока (1) настраивают параметры источника тока (1), а посредством расположенного в источнике тока (1) блока обработки (9) регистрируют движение рабочего прибора (4) в пространстве (6), отличающийся тем, что на источнике тока (1) до, после или между проводимыми рабочим прибором (4) работами активируют функцию управления, после чего регистрируют движение рабочего прибора (4) в пространстве (6) и результирующее из этого движения положение рабочего прибора (4) назначают элементу управления (3) блока (2) и назначенный элемент управления (3) подтверждают, после подтверждения элемента управления (3) настраивают или изменяют, по меньшей мере, один соответствующий элементу управления (3) параметр источника тока (1), после чего снова дезактивируют функцию управления на источнике тока (2).2. Способ по п.1 ...

СИСТЕМА УСТРОЙСТВА ПРОСМОТРА СМЕШАННОЙ РЕАЛЬНОСТИ И СПОСОБ ДЛЯ НЕГО

СПОСОБ И СИСТЕМА ОБЕСПЕЧЕНИЯ ВИРТУАЛЬНОГО ОТОБРАЖЕНИЯ ФИЗИЧЕСКОЙ СРЕДЫ

СПОСОБ И СИСТЕМА РЕАЛИЗАЦИИ МНОГОПОЛЬЗОВАТЕЛЬСКОЙ ВИРТУАЛЬНОЙ СРЕДЫ

PORTABLE VIRTUAL REALITY

SYSTEM AND PROCEDURE FOR OPERATING IN a VIRTUAL ONE 3D-RAUM

CAMERA MOVEMENT CONTROLLER

Kamerabewegungssteuerung (1) für eine Kamera (3) eines Endoskops (10) mit einer Kamerasteuereinheit (21) zur Steuerung der Kamerabewegungen durch eine Bedienperson (15), wobei eine Aktuatoreinheit (22) zum Antrieb der Kamerasteuereinheit (21) vorgesehen ist, und dass eine Kopfbewegungserfassungseinheit (23) vorgesehen ist, die konfiguriert ist, Kopfbewegungen der Bedienperson zu erfassen und in Ansteuersignale für die Aktuatoreinheit (22) umzuwandeln, sodass die Steuerung der Bewegung der Endoskopkamera (3) durch die Kopfbewegungen der Bedienperson (15) erfolgt.

SYSTEM FOR THE PURSUIT OF HEAD MOVEMENTS

COMPACT HEAD PUR-FOLLOWING SYSTEMS FOR CHEAP VIRTUAL REALITY SYSTEM

Head operated computer pointer

Display system and method

Disclosed is a method of operation in a virtual image presentation system, the method comprising: detecting a head movement of an end user; determining respective resolutions for each of at least two portions of a frame in response to the detected head movement of the end user; modifying a first resolution of a first portion of the frame such that it is different from a second resolution of a second portion of the frame by adjusting a first slope of a first drive signal corresponding to the first portion of the frame such that it is different from a second slope of a second drive signal corresponding to the second portion of the frame; and displaying a virtual object to the end user based on the first and second resolutions of the respective first and second portions of the frame.

DIRECTIONAL BACKLIGHT

A directional display may include a waveguide. The waveguide may include light extraction features arranged to direct light from an array of light sources by total internal reflection to an array of viewing windows and a reflector arranged to direct light from the waveguide by transmission through extraction features of the waveguide to the same array of viewing windows. The brightness of the directional display can be increased. An efficient and bright autostereoscopic display system can be achieved. WO 2014/130860 PCT/US2014/017779 z A '12 { - 310 311 315 312 z 300 FIG. 27 1 -- 226 x 314 310 312 ...

Headphones with interactive display

A novel headphone system includes a first speaker assembly, a second speaker assembly, and an interactive visual display system. The interactive visual display system includes a display operative to output visual contents according to interaction with a user. In one embodiment, one of the displays can be a transparent display placed in front of the user's eyes. Live images passing through the transparent display can be merged with the digital image generated by the transparent display to give the user an augmented reality experience. In another embodiment, the headphone system includes an external device interface that enables a user to interact with the contents displayed from the screen via an external device or via the Internet. In another embodiment, the interactive visual display system is removable from the rest of the headphone system and can optionally be installed in another compatible non-headphone device.

Microcurrent-stimulation-therapy apparatus and method

A system and method for applying stimulation therapy to a patient, the system including a first stimulation strip that includes a first elongated portion configured to be placed on the upper eyelid of the first eye of the patient and a second elongated portion configured to be placed on the lower eyelid of the first eye of the patient, wherein the first stimulation strip includes: a first plurality of individually controlled electrodes configured to deliver a microcurrent stimulation therapy to the patient, a first plurality of individually controlled light emitters configured to deliver light stimulation therapy to the patient, and a first plurality of individually controlled heat sources configured to deliver heat therapy to the patient; and a controller operatively coupled to the first stimulation strip and configured to control delivery of the microcurrent stimulation therapy, the light stimulation therapy, and the heat therapy.

Three-dimensional environment authoring and generation

Aspects of the present disclosure relate to three-dimensional (3D) environment authoring and generation. In an example, a 3D environment may be authored using one or more models, wherein two-dimensional (2D) representations of the models may be manipulated using an authoring application. Models may comprise anchor points, which may be used to stitch the models together when rendering the 3D environment. In another example, a model may comprise one or more content points, which may be used to position content within the 3D environment. An environment data file may be generated based on the one or more models and content associated with content points, thereby creating a file that may be distributed to other computing devices. A viewer application may be used to generate the 3D environment based on the environment data file. Accordingly, the viewer application may stitch the models and populate the 3D environment with content.

METHOD AND DEVICE FOR HUMAN-MACHINE INTERACTION IN A STORAGE UNIT, STORAGE UNIT, AND STORAGE MEDIUM

The present disclosure relates to a method and a device for human-machine interaction in a storage unit, a storage unit, and a storage medium. The method includes: detecting a user behavior in a preset area, the preset area including a sensing area of the storage unit; extracting voice 5 information and/or facial information of a user according to the user behavior; determining a target position of an item of interest to the user according to the voice information and/or facial information; obtaining an item identification corresponding to the target position according to a preset corresponding relationship between a position of a stored item and an identification of a stored item in the storage unit; and pushing item information corresponding to the item 0 identification. Based on the disclosure, the user can know the item information that he/she is interested in, and the problems of the traditional mode of shopping consumption can be solved. DETECTING A USER BEHAVIOR IN A PRESETAREA, ...

Improved systems and methods for diagnosis and therapy of vision stability dysfunction

Abstract Described herein are improved systems and methods for diagnosis and therapy of vision stability dysfunctions in a patient. One embodiment provides a device for obtaining data from a patient. Device includes a frame configured for mounting to the patient, a target module for providing a drive signal to define a selectively movable target object, an eye tracking module mounted to the frame, the module being responsive to the drive signal for obtaining first data indicative of one or more characteristics of the patient's eyes; and a head tracking module mounted to the frame, the head tracking module being responsive to the drive signal for obtaining second data indicative of one or more characteristics of the motion of the patient's head.

Augmented reality-based system and method to show the location of personnel and sensors inside occluded structures and provide increased situation awareness

VIRTUAL REALITY IMMERSION SYSTEM

Display system and method

One embodiment is directed to a user display device comprising a housing frame mountable on the head of the user, a lens mountable on the housing frame and a projection sub system coupled to the housing frame to determine a location of appearance of a display object in a field of view of the user based at least in part on at least one of a detection of a head movement of the user and a prediction of a head movement of the user, and to project the display object to the user based on the determined location of appearance of the display object.

A surgical mechanism control system

Systems and methods for biomechanically-based eye signals for interacting with real and virtual objects

Systems and methods are provided for discerning the intent of a device wearer primarily based on movements of the eyes. The system can be included within unobtrusive headwear that performs eye tracking and controls screen display. The system can also utilize remote eye tracking camera(s), remote displays and/or other ancillary inputs. Screen layout is optimized to facilitate the formation and reliable detection of rapid eye signals. The detection of eye signals is based on tracking physiological movements of the eye that are under voluntary control by the device wearer. The detection of eye signals results in actions that are compatible with wearable computing and a wide range of display devices.

Gaze-based object placement within a virtual reality environment

A head mounted display (HMD) device operating in a real world physical environment is configured with a sensor package that enables determination of an intersection of a device user's projected gaze with a location in a virtual reality environment so that virtual objects can be placed into the environment with high precision. Surface reconstruction of the physical environment can be applied using data from the sensor package to determine the user's view position in the virtual world. A gaze ray originating from the view position is projected outward and a cursor or similar indicator is rendered on the HMD display at the ray's closest intersection with the virtual world such as a virtual object, floor/ground, etc. In response to user input, such as a gesture, voice interaction, or control manipulation, a virtual object is placed at the point of intersection between the projected gaze ray and the virtual reality environment.

Head-mounted display for virtual and mixed reality with inside-out positional, user body and environment tracking

A Head-Mounted Display system together with associated techniques for performing accurate and automatic inside-out positional, user body and environment tracking for virtual or mixed reality are disclosed. The system uses computer vision methods and data fusion from multiple sensors to achieve real-time tracking. High frame rate and low latency is achieved by performing part of the processing on the HMD itself.

Systems and methods for improved data integration in augmented reality architectures

Systems and methods are provided for improved data integration in augmented reality devices. The systems and methods include obtaining contextual information associated with an individual and device information associated with an augmented reality device associated with the individual from the augmented reality device, obtaining a plurality of data sets associated with the individual or augmented reality device from a plurality of data sources, determining a subset of information from the data sets that is relevant to the individual or the augmented reality device wherein the relevancy of the subset of information is based on an analysis of the contextual information and the device information, generating display data based on the determined subset of information, providing the display data to the augmented reality device for display on the augmented reality device wherein a graphical overlay representative of the display data is displayed in front of the individual's field of view. - 0 ...

3D navigation system using complementary head-mounted and stationary infrared beam detection units

Video processing methods and apparatus for gaze point tracking

Orientation angle detector

SYSTEM AND METHOD FOR THE INTERACTIVE DISPLAY OF DATA IN A MOTION CAPTURE ENVIRONMENT

A system includes an analysis system for performing an analysis and a mot ion capture environment interfaced with the analysis system. The motion capt ure system includes at least one sensor-tracker for tracking a location of a tracked object within the motion capture environment and one or more comput ers collectively operable to generate a virtual reality environment correspo nding to the analysis.

HEAD-MOUNTED DISPLAY AND METHOD TO REDUCE VISUALLY INDUCED MOTION SICKNESS IN A CONNECTED REMOTE DISPLAY

A Head-Mounted Display (HMD) (100) is provided which comprises a camera (101) configured to capture a video of a real-world scene with a first field-of-view (FoV) (111), a network interface circuit (102) configured to stream video to a receiving display device (140), and processing means (103) which is operative to generate a 3D model of the real-world scene, and to generate a video from the 3D model using a second FoV which is wider than the first FoV (111). The processing means (103) is further operative to estimate a motion of the camera (101), and to stream the generated video to the receiving display device (140) if the estimated motion of the camera (101) satisfies one or more conditions indicative of rapid motion, else stream the captured video to the receiving display device (140). Further provided are a method performed by an HMD, a corresponding computer program, and a corresponding computer-readable storage medium.

SYNCHRONISATION DEVICE HAVING A BASE STATION FOR SYNCHRONIZING HEAD-MOUNTED DISPLAYS WITH A VIRTUAL WORLD IN AN AMUSEMENT RIDE, AMUSEMENT RIDE HAVING A SYNCHRONIZATION DEVICE OF THIS TYPE, AND METHOD FOR OPERATING AN AMUSEMENT RIDE OF THIS TYPE

The invention relates to a synchronisation device for synchronising head-mounted displays (22) with a virtual world in an amusement ride (10), comprising at least one head-mounted display (22) with which a virtual reality can be represented, a base station (24) for placing down the head-mounted display (22), wherein the base station (24) has receiving means with which the head-mounted display (22) can be placed down in a defined position and orientation in the base station (24), and a display position detection device (28) for detecting the position and orientation of the head-mounted display (22), where in the synchronisation device (12) is designed in such a way that the virtual reality is synchronised with the head-mounted display (22), taking into consideration the position and orientation of the head-mounted display (22) detected by the display position detection device (28) in relation to the position and orientation of the head-mounted display (22) in the base station (24), as soon ...

METHODS, DEVICES, AND SYSTEMS FOR ILLUMINATING SPATIAL LIGHT MODULATORS

An optical device can include a wedge-shaped light turning element having a first surface that is parallel to a horizontal axis and a second surface opposite to the first surface, which is inclined with respect to the horizontal axis by a wedge angle. A light module, which includes a number of light emitters, can be configured to combine light from the emitters. A light input surface is between the first and the second surfaces and receives light emitted from the emitters. An end reflector is disposed on a side opposite the light input surface. The second surface may be inclined such that a height of the light input surface is less than a height of the side opposite the light input surface. The light coupled into the turning element may be reflected by the end reflector and/or reflected from the second surface towards the first surface.

MULTIPLE OBSERVER IMPROVEMENT FOR A VIRTUAL ENVIRONMENT

A virtual reality imaging system for two or more users including: at least one screen for presenting images of a virtual environment to two or more users; an imaging assembly, including two or more sets of projectors corresponding to the two or more users for producing images on the at least one screen; a tracking system that is configured to track each of the two or more users; a computer that is programmed to respond to the tracking system and thereby control the imaging assembly to produce stereo 3D images of the virtual environment corresponding to tracking data from the tracking system for each of the two or more users for each user to perceive different views of the same virtual environment appropriate to their position; stereo 3D glasses worn by each of the two or more users in order for the users to perceive the stereo 3D images; and an image separation arrangement to present images from the two or more sets of projectors to corresponding ones of the two or more users whereby each ...

TRACKING SYSTEM

A tracking system for use with a head mounted display on a watercraft is disclosed. The tracking system comprises: optical tracking circuitry to determine first tracking information of the head mounted display using optical tracking means; non-optical circuitry to determine second tracking information of the head mounted display using non-optical tracking means; and correction circuitry to determine an updated tracking information of the head mounted display based on the first tracking information and the second tracking information. The selection of at least the optical tracking means or non-optical tracking means to use to determine the first tracking information and the second tracking information is based on a location of the head mounted display.

METHOD AND SYSTEM FOR GENERATING A THERAPEUTIC SESSION

The present invention is in the field of clinical therapy. In particular, the present invention provides systems and methods for generating at least a part of a therapeutic session for controlling a subject via sensory stimulations, such as graphical and/or auditory stimulations.

SELECTIVE APPLICATION OF REPROJECTION PROCESSING ON LAYER SUB-REGIONS FOR OPTIMIZING LATE STAGE REPROJECTION POWER

Optimizations are provided for late stage reprojection processing for a multi-layered scene. A scene is generated, which is based on a predicted pose of a portion of a computer system. A sub-region is identified within one of the layers and is isolated from the other regions in the scene. Thereafter, late stage reprojection processing is applied to that sub-region selectively/differently than other regions in the scene that do not undergo the same late state reprojection processing.

THREE-DIMENSIONAL ENVIRONMENT AUTHORING AND GENERATION

Aspects of the present disclosure relate to three-dimensional (3D) environment authoring and generation. In an example, a 3D environment may be authored using one or more models, wherein two-dimensional (2D) representations of the models may be manipulated using an authoring application. Models may comprise anchor points, which may be used to stitch the models together when rendering the 3D environment. In another example, a model may comprise one or more content points, which may be used to position content within the 3D environment. An environment data file may be generated based on the one or more models and content associated with content points, thereby creating a file that may be distributed to other computing devices. A viewer application may be used to generate the 3D environment based on the environment data file. Accordingly, the viewer application may stitch the models and populate the 3D environment with content.

SYSTEMS AND METHODS FOR VESTIBULAR REHABILITATION

Systems and methods for controlling, conducting, and/or performing vestibular rehabilitation exercises. The individual performing a vestibular rehabilitation exercise progresses through the exercise by selecting target word or picture representations that are sequentially presented on a display. When read sequentially, the target word or picture representations form at least a portion of a phrase, a sentence, or a visual message. In use, a given exercise can correspond to the selection of a series of correct word or picture representations that correspond to the displayed target word or picture representations and form a book or story.

METHODS AND SYSTEMS FOR DISPLAYING ADDITIONAL CONTENT ON A HEADS UP DISPLAY DISPLAYING A VIRTUAL REALITY ENVIRONMENT

Methods and systems for presenting additional content in virtual reality environments on heads up displays showing main content without interfering with a user's viewing of the main content. Specifically, a movement of a center of gaze of the user is detected. A visual field of the user based on the movement of the center of gaze is determined. If it is determined that the portion of the virtual reality environment in which the additional content is generated for display corresponds to a foreground area of the user's visual field, the additional content is generated for display in a portion of the virtual reality environment corresponding to a peripheral area of the user's visual field.

METHOD AND SYSTEM FOR SYNCHRONIZING VIBRO-KINETIC EFFECTS TO A VIRTUAL REALITY SESSION

A method for synchronizing an output of a motion signal with a virtual reality program being played comprises mixing a multi-channel audio output of the virtual reality program into at least one mixed audio track. A time position of the virtual reality program being played is obtained using the at least one mixed audio track. A motion signal corresponding to virtual reality program being played and synchronized with the identified time position of the virtual reality program being played is obtained, the motion signal comprising motion platform actuator-driving instructions. The motion signal is output to an actuation system of a motion platform for producing vibro-kinetic effects in the motion platform corresponding to and synchronized with the virtual reality program being played. A motion controller for synchronizing an output of a motion signal with a virtual reality program being played is also provided.

HYBRID RENDERING

One disclosed example provides a computing device comprising a processing device and a storage device storing instructions executable by the processing device to execute in a first local process an application that outputs digital content for rendering and display. During execution of the application, the instructions are executable to provide, to a second local or remote process, object information regarding an object to be rendered by the second local or remote process, receive, from the second local or remote process, a rendering of the object, output the rendering of the object to display the object, receive a manipulation made to the object, provide, to the second local or remote process, updated object information based on the manipulation made to the object, receive, from the second local or remote process, an updated rendering of the object, and output the updated rendering of the object to display the object.

ASSOCIATING A USER VOICE QUERY WITH HEAD DIRECTION

A method and apparatus, e.g. a digital assistant, for associating a query to a direction is provided herein. During operation, a virtual partner will attempt to determine a direction to which the query was originated to. When answering the query, the virtual partner will adjust multiple speaker outputs so that the reply is perceived to come from the same direction where the query was originated to. Additionally, if another follow-on query is made into a particular direction, the follow-on query will be associated with the prior query originating from the same particular direction.

METHODS, DEVICES, AND SYSTEMS FOR ILLUMINATING SPATIAL LIGHT MODULATORS

An optical device comprising may include a light turning element. The optical device can include a first surface that is parallel to a horizontal axis and a second surface opposite to the first surface. The optical device may include a light module that includes a plurality of light emitters. The light module can be configured to combine light for the plurality of emitters. The optical device can further include a light input surface that is between the first and the second surfaces and is disposed with respect to the light module to receive light emitted from the plurality of emitters. The optical device may include an end reflector that is disposed on a side opposite the light input surface. The light coupled into the light turning element may be reflected by the end reflector and/or reflected from the second surface towards the first surface.

EYE TRACKING CALIBRATION TECHNIQUES

Systems and methods for eye tracking calibration in a wearable system are described. The wearable system can present three-dimensional (3D) virtual content and allow a user to interact with the 3D virtual content using eye gaze. During an eye tracking calibration, the wearable system can validate that a user is indeed looking at a calibration target while the eye tracking data is acquired. The validation may be performed based on data associated with the user's head pose and vestibulo-ocular reflex.

EYE GAZE FOR SPOKEN LANGUAGE UNDERSTANDING IN MULTI-MODAL CONVERSATIONAL INTERACTIONS

Improving accuracy in understanding and/or resolving references to visual elements in a visual context associated with a computerized conversational system is described. Techniques described herein leverage gaze input with gestures and/or speech input to improve spoken language understanding in computerized conversational systems. Leveraging gaze input and speech input improves spoken language understanding in conversational systems by improving the accuracy by which the system can resolve references-or interpret a user's intent-with respect to visual elements in a visual context. In at least one example, the techniques herein describe tracking gaze to generate gaze input, recognizing speech input, and extracting gaze features and lexical features from the user input. Based at least in part on the gaze features and lexical features, user utterances directed to visual elements in a visual context can be resolved.

METHOD AND SYSTEM FOR 360 DEGREE HEAD-MOUNTED DISPLAY MONITORING BETWEEN SOFTWARE PROGRAM MODULES USING VIDEO OR IMAGE TEXTURE SHARING

Presented is a method and system for displaying spherical 180 or 360 degree video or still images while editing in a host editing or visual effects software program as equirectangular projection, or other spherical projection, to the input of a simultaneously running software program on the same device that can continuously acquire the orientation and position data from a wired or wirelessly connected head-mounted display's orientation sensors, and simultaneously render a representative monoscopic or stereoscopic view of that orientation to the head mounted display, in real time. The benefits of this system and/or method in accordance with one or more embodiments can include a dramatic decrease in the amount of time required to align, manipulate, or edit spherical 360 degree video or images.

ANTI-TRIP WHEN IMMERSED IN A VIRTUAL REALITY ENVIRONMENT

An HMD device with a see-through display and depth sensing capability is configured to selectively dim or fade out a display of a virtual reality environment to enable a user to see the real world without obstruction by the virtual world when a distance between the user and a real world object is determined to be less than a threshold distance. The current height of the user's head (i.e., the distance from head to ground) may be utilized when performing the dimming/fading so that different threshold distances can be used depending on whether the user is standing or seated.

DIRECTIONAL BACKLIGHT

A directional display may include a waveguide. The waveguide may include light extraction features arranged to direct light from an array of light sources by total internal reflection to an array of viewing windows and a reflector arranged to direct light from the waveguide by transmission through extraction features of the waveguide to the same array of viewing windows. The brightness of the directional display can be increased. An efficient and bright autostereoscopic display system can be achieved.

AUTOMATIC VOICE RECOGNITION WITH DETECTION OF AT LEAST ONE CONTEXTUAL ELEMENT, APPLICATION TO STEERING AND MAINTENANCE OF AN AIRCRAFT

Ce dispositif de reconnaissance vocale automatique (30) comprend une unité (32) d'acquisition d'un signal audio, un dispositif de détection (36) pour détecter l'état d'au moins un élément contextuel, et un décodeur linguistique (38) pour la détermination d'une instruction orale correspondant au signal audio. Le décodeur linguistique (38) comprend au moins un modèle acoustique (42) définissant une loi de probabilité acoustique et au moins deux modèles syntaxiques (44) définissant chacun une loi de probabilité syntaxique. Le décodeur linguistique (38) comprend également un algorithme de construction d'instruction orale (46) mettant en oeuvre le modèle acoustique (42) et une pluralité de modèles syntaxiques actifs pris parmi les modèles syntaxiques (44), un processeur de contextualisation (48) pour sélectionner, en fonction de l'état du ou de chaque élément contextuel détecté par le dispositif de détection (36), au moins un modèle syntaxique sélectionné parmi la pluralité de modèles syntaxiques ...

MENU NAVIGATION IN A HEAD-MOUNTED DISPLAY

A wearable computing device includes a head-mounted display (HMD) that generates a virtual reality environment. Through the generation and tracking of positional data, a focal point may be tracked with respect to one or menu navigation elements. Following the fixated positioning of the focal point over the menu navigation element for a predetermined amount of time, a process corresponding to the menu navigation element is executed.

DISPLAY VISIBILITY BASED ON EYE CONVERGENCE

Gaze information of a user can be determined by a computing device that analyzes images of the user. Gaze information of a user includes information such as the user's line of sight, point of regard information, the direction of the user's gaze, the depth of convergence of the user's gaze, and the like. The computing device is able to estimate the distance from the user at which the user is focusing (for example, at a screen near the user or at an object farther away). The visibility and display characteristics of objects displayed on the HUD may be based on the gaze information. For example, content on a heads-up display (HUD) on a windshield may be more transparent while the user is looking through the windshield and more opaque (or otherwise enhanced) while the user is focusing on the HUD.

SYSTEMS AND METHODS FOR IMPROVED DATA INTEGRATION IN VIRTUAL REALITY ARCHITECTURES

The embodiments described herein provide technologies and techniques for using available data (from a variety of data sources) to provide an integrated and virtual reality experience. Embodiments described herein include systems and methods for obtaining a selection of a travel destination from a virtual reality device, obtaining destination information associated with the travel destination, including travel services for the destination, obtaining contextual information associated with the individual including a device identifier or an individual profile identifier, obtaining profile information associated with the individual and device, and filtering the services based on the profile and destination, providing a virtual environment of the destination including controls, providing the travel services, using the controls and the environment; receiving, the individual's movements in the virtual environment, determining interaction with the virtual environment and updates to the controls ...

METHOD AND SYSTEM FOR IMPLEMENTING A MULTI-USER VIRTUAL ENVIRONMENT

The present disclosure provides computer systems, apparatuses, computer- executable methods and one or more non-transitory computer-readable media for implementing a multi-user virtual reality environment. An example of a computer- implemented method for implementing a multi-user virtual reality environments includes receiving sensor data from a sensor at a local device, providing the sensor data to a network interface for electronic transmission to a remote device, providing the sensor data to a virtual environment engine executing on a processor of the local device, determining, by the virtual environment engine and based at least in part on the sensor data, virtual environment data related to a virtual reality environment, providing the virtual environment data to the network interface for electronic transmission to the remote device, and electronically transmitting, using the network interface, the sensor data and the virtual environment data to the remote device for use in generating ...

ORIENTATION DETERMINATION FOR A MOBILE DEVICE

A system determines an orientation of a user of a mobile device has a head-mounted accessory device including a first digital compass for providing a first orientation signal and a short-range wireless transmitter for transmitting the first orientation signal to the mobile device. The mobile device includes a short-range wireless receiver for receiving the first orientation signal from the head-mounted accessory device. A processor in the mobile device determines the orientation based on the first orientation signal. The mobile device may include a second digital compass. The system with two compasses provides greater accuracy. By comparing the two orientation signals, the device can infer whether the user is viewing the mobile device. Tilt sensors in the accessory device and mobile device can also be used with the orientation signals to infer whether the user is viewing the mobile device and thus to control content delivered by the mobile device.

SYSTEM AND METHOD FOR AUGMENTED AND VIRTUAL REALITY

One embodiment is directed to a system for enabling two or more users to interact within a virtual world comprising virtual world data, comprising a computer network comprising one or more computing devices, the one or more computing devices comprising memory, processing circuitry, and software stored at least in part in the memory and executable by the processing circuitry to process at least a portion of the virtual world data; wherein at least a first portion of the virtual world data originates from a first user virtual world local to a first user, and wherein the computer network is operable to transmit the first portion to a user device for presentation to a second user, such that the second user may experience the first portion from the location of the second user, such that aspects of the first user virtual world are effectively passed to the second user.

IMAGE PROCESSING APPARATUS AND METHOD, AND PROVIDING MEDIUM

A whole image (51) of a low resolution is captured from an inputted image so as to accurately track a predetermined portion, an image (52) of an intermediate resolution mainly containing the head is extracted from the whole image (51), and further, after the extraction an image (53) of a high resolution mainly containing the eyes is extracted. A predetermined image different from the inputted image is changed and displayed according to the extracted image (53).

Device with the function of a computer mouse.

Virtual reality goggles.

Die Erfindung betrifft eine Virtual-Reality-Brille, bestehend aus einem gefalteten, im Rohzustand flachen Zuschnitt aus leichtem Material, vorzugsweise aus Karton, mit einer Mehrzahl von miteinander verbundenen und zusammengefalteten Abschnitten sowie mit zwei Sehöffnungen, gegebenenfalls mit eingesetzten Linsen, hinter welche eine Anzeigeeinrichtung, vorzugsweise ein Smartphone, platziert werden kann, sowie deren Verwendung als Werbemittel. Um eine leicht und einfach aus einem flachen Zuschnitt zusammenzustellende Brille zu erhalten, ist ein erster, zentraler und bei Gebrauch parallel zur Anzeigeeinrichtung ausgerichteter Abschnitt vorgesehen, mit einem ersten, von einer Seite des Ausschnitts ausgehenden Nasenausschnitt (2) sowie zwei Sehöffnungen (3) zu beiden Seiten des Nasenausschnittes (2).

VIRTUAL REALITY SYSTEM BASED ON SMARTPHONE AND INCLINED MIRROR

Techniques for compensating for variable display device latency in image display of virtual reality

Facilitating dynamic eye torsion-based eye tracking on computing devices

Motion tracking system using one or more magnetic fields

HAPTIC DIMENSIONS IN A VARIABLE GAZE ORIENTATION VIRTUAL ENVIRONMENT

Eyeglasses type operation device, eyeglasses type operating system and electronic devices

The invention provides an eyeglasses type operation device configured on an eyeglasses framework to carrying out controlling and processing of external electronic equipment, an eyeglasses type operating system and electronic devices. The eyeglasses type operation device comprises a frame 10 configured to be worn on a head of a user, a sensor 20 that outputs a detection signal by sensing, the sensor being disposed inside the frame 10 and a connector 31 operatively connected to the sensor to receive the detection signal, the connector being configured to detachably connect to an external electronic device 40, and the connector being configured to output the detection signal to the external electronic device when the external electronic device 40 is connected to the connector.

Direction based electronic device for displaying object and method thereof

Systems and methods for generating augmented and virtual reality images

Attributing user action based on biometric identity

Closestool for parturient after childbirth

ELECTRONIC DEVICE AND SCREEN DISPLAY METHOD THEREOF

HEAD-MOUNTED DISPLAY DEVICE AND METHOD OF CONTROLLING HEAD-MOUNTED DISPLAY DEVICE

A transmissive head-mounted display device includes an image display unit that generates image light representing an image, allows a user to visually recognize the image light, and transmits outside scenery when worn on a head of a user, a detection unit that detects at least one of a line-of-sight direction of the user and a shielding object that is located at a predetermined distance from an eye of the user and shields a line of sight of the user, and a control unit that performs control of the image light based on the detected detection result.

Face tracking for additional modalities in spatial interaction

REALITY MIXER FOR MIXED REALITY

HEAD TRACKING WITH ADAPTIVE REFERENCE

Information processing system and information processing method

For measuring digital signboards audience-scale method and device

Image forming apparatus and Image forming method

Method and device for determining points of interest in immersive content

SYSTEM FOR PRESENTING VISUAL INFORMATION FOR THE SAFETY FOR DRIVER HAS CONTROL D'ACTIVATION-TO-DEACTIVATING PROJECTION MODULE, FOR THE SAFETY DEVICE AND METHOD THEREFOR

Apparatus and method for controlling interface

In an apparatus and method for controlling an interface, a user interface (UI) may be controlled using information on a hand motion and a gaze of a user without separate tools such as a mouse and a keyboard. That is, the UI control method provides more intuitive, immersive, and united control of the UI. Since a region of interest (ROI) sensing the hand motion of the user is calculated using a UI object that is controlled based on the hand motion within the ROI, the user may control the UI object in the same method and feel regardless of a distance from the user to a sensor. In addition, since positions and directions of view points are adjusted based on a position and direction of the gaze, a binocular 2D/3D image based on motion parallax may be provided.

Hands-Free Control of Mobile Communication Device Based on Head Movement

A user controls a function of a mobile terminal with the aid of a motion sensor integrated in a headset. A motion sensor in the headset detects movement of the user's head and transmits head movement signals to the mobile. The mobile terminal uses the head movement signals received from the headset to control a function of the mobile terminal. For example, the mobile terminal may lock or unlock the mobile terminal based on the user's head movement. The mobile terminal may also accept, decline, initiate, or terminate a call based on the user's head movement. Head movement may also be used to navigate desktop icons and menus to perform tasks such as starting programs and selecting music for play back.

Method and computing device in a system for motion detection

A computing device in a system for motion detection comprises an image processing device to determine a motion of an object of interest, and a graphical user interface (GUI) module to drive a virtual role based on the motion determined by the image processing device. The image processing device comprises a foreground extracting module to extract a foreground image from each of a first image of the object of interest taken by a first camera and a second image of the object of interest taken by a second camera, a feature point detecting module to detect feature points in the foreground image, a depth calculating module to calculate the depth of each of the feature points based on disparity images associated with the each feature point, the depth calculating module and the feature point detecting module identifying a three-dimensional (3D) position of each of the feature points, and a motion matching module to identify vectors associated with the 3D positions of the feature points and determine a motion of the object of interest based on the vectors.

Contextual help based on facial recognition

A computer program product includes a computer readable storage medium to store a computer readable program, wherein the computer readable program, when executed on a computer, causes the computer to perform operations for providing contextual help based on a user facial expression. The operations include: capturing a user facial expression using a camera device connected to a computing device; categorizing the user facial expression into a facial expression category; collecting an application context from the computing device in conjunction with an application, wherein the application context includes a recently performed task and a current application state, wherein the current application state comprises information on a current performance of an application in which the user is operating; determining a set of available tasks relating to the application context; and automatically executing one of the set of available tasks based on the facial expression category and the application context.

Mobile device and method for proximity detection verification

A mobile device includes a touch screen display and touch screen circuit for receiving touch inputs. A camera configured to acquire one or more images in front of the display. A proximity sensor configured to provide an indication that a user is in close proximity to the touch screen display. A processor being effective to determine, in response to the indication of the proximity sensor, whether the user is in a position to view the display based on the one or more images of the camera. If the proximity sensor provides an indication that the user is in close proximity to the display and the processor determines, based on the one or more images, that the user is in a position to view the display, then at least one of the display and the touch screen circuit are not disabled.

Methods, systems, and computer program products for managing attention of a user of a portable electronic device

Methods and systems are described for managing attention of a user of a portable electronic device. A detection is made that a portable electronic device is in motion relative to a first object separate from the portable electronic device An interaction is detected between a user and the portable electronic device while the portable electronic device is in the motion, wherein in the interaction the portable electronic device is a first source of sensory input for the user. A determination is made, based on detecting the motion, that a specified attention criterion is met in response to detection of the interaction. In response to detecting that the attention criterion is met, attention information is sent for presenting an attention output, via an output device, for directing the user to an object, not included in the portable electronic device, as a second source of sensory input for the user.

Control systems and methods for head-mounted information systems

A head-mounted information system is provided, the head-mounted information system comprising a frame configured to be mounted on a head of a user, a display unit coupled to the frame, a sensor unit coupled to the frame comprising one or more motion sensors, and, a processor unit coupled to the frame and connected to receive signals from the motion sensors. The processor unit comprises a processor and a memory accessible by the processor. The processor unit is configured to monitor the received signals and enter a gesture control mode upon detection of a gesture control enable signal. In the gesture control mode the processor is configured to convert signals received from the motion sensors into menu navigation commands.

Advanced remote control of host application using motion and voice commands

A head mounted microdisplay (HMD) device that uses hand, body, head movement or gesture and voice commands to control interaction with a local processor or host computer. The local processor overlays information, such as onto an operating system desktop or application program menu, indicating the motion, gesture, and/or voice input needed to activate an associated action.

Natural Human to Robot Remote Control

The subject disclosure is directed towards controlling a robot based upon sensing a user's natural and intuitive movements and expressions. User movements and/or facial expressions are captured by an image and depth camera, resulting in skeletal data and/or image data that is used to control a robot's operation, e.g., in a real time, remote (e.g., over the Internet) telepresence session. Robot components that may be controlled include robot “expressions” (e.g., audiovisual data output by the robot), robot head movements, robot mobility drive operations (e.g., to propel and/or turn the robot), and robot manipulator operations, e.g., an arm-like mechanism and/or hand-like mechanism.

Virtual Touchpad Using a Depth Camera

The subject disclosure is directed towards a virtual touchpad comprising a region in space positioned relative to a detected user with which a user interacts by hand movements as determined from frames of depth data obtained via a depth camera. The user's hand position or positions in the virtual touchpad region may be converted to coordinates, such as for posting to a message queue for use by an application. The computing device and depth camera may be incorporated into a robot that moves on a floor, with the depth camera angled upwardly and the virtual touchpad region tilted to facilitate user interaction.

System and Method for Operating a Helmet Mounted Display

A display system comprises a transparent display device mountable on the head of a user. A scene of an environment is visible to the user through the transparent display device. The system further comprises a user interface image adapted for display on the display device. The user interface image remains fixed relative to the scene as the user and the display device move within the environment. The display system also comprises an indicator image adapted for display on the display device. The indicator is movable relative to the user interface image in response to movement of the user's head.

Scaling of Visual Content Based Upon User Proximity

A mechanism is disclosed for automatically scaling the size of a set of visual content based upon how close a user's face is to a display. In one implementation, the mechanism initially causes a set of visual content on a display to be sized according to a first scaling factor when the user's face is at a first distance from the display. The mechanism then determines that the user's face has moved relative to the display such that the user's face is no longer at the first distance from the display. In response, the mechanism causes the set of visual content on the display to be sized according to a second and different scaling factor. By doing so, the mechanism effectively causes the display size of the visual content to automatically change as the distance between the user's face and the display changes.

Information source for mobile communicators

An information source device adapted to be in communication with a plurality of users includes a receiver adapted to determine a time-of-visual-gaze duration of a user viewing the information source device, a transmitter adapted to wirelessly transmit information to the user when the time-of-visual-gaze duration exceeds a pre-determined value.

method of aligning a helmet mounted display

A method and system of aligning a helmet mounted display mounted to a helmet is disclosed. The method comprises the steps of;—aligning a first helmet mounted display guide symbol with a reference direction;—selecting the alignment of the first helmet mounted display guide symbol with the reference direction;—aligning a first stabilised guide symbol, starting at the position of the aligned first symbol, with a reference direction, the first stabilised symbol being stabilised to reduce the movement of the stabilised symbol with respect to the reference direction in response to movements of the head of the user;—selecting the alignment of the first stabilised guide symbol with the reference direction;—aligning a second helmet mounted display guide symbol with the reference direction;—selecting the alignment of the second helmet mounted display guide symbol with the reference direction;—aligning a second stabilised guide symbol, starting at the position of the aligned second symbol, with a reference direction, the second stabilised symbol being stabilised to reduce the movement of the stabilised symbol with respect to the reference direction in response to movements of the head of the user; —selecting the alignment of the second stabilised display guide symbol with the reference direction;—determining a relative position and orientation between the helmet and the display mounted to the helmet according to the positions and orientations of the helmet at said alignments.

Method And Apparatus For Display Zoom Control Using Object Detection

A zoom method and apparatus utilizing object detection. For example, some embodiments allow a user to zoom in or out from the digital content being displayed by moving their head towards or away from the display screen.

Natural user interfaces for mobile image viewing

The mobile image viewing technique described herein provides a hands-free interface for viewing large imagery (e.g., 360 degree panoramas, parallax image sequences, and long multi-perspective panoramas) on mobile devices. The technique controls the imagery displayed on a display of a mobile device by movement of the mobile device. The technique uses sensors to track the mobile device's orientation and position, and front facing camera to track the user's viewing distance and viewing angle. The technique adjusts the view of a rendered imagery on the mobile device's display according to the tracked data. In one embodiment the technique can employ a sensor fusion methodology that combines viewer tracking using a front facing camera with gyroscope data from the mobile device to produce a robust signal that defines the viewer's 3D position relative to the display.

Methods and systems for enabling depth and direction detection when interfacing with a computer program

One or more images can be captured with a depth camera having a capture location in a coordinate space. First and second objects in the one or more images can be identified and assigned corresponding first and second object locations in the coordinate space. A relative position can be identified in the coordinate space between the first object location and the second object location when viewed from the capture location by computing an azimuth angle and an altitude angle between the first object location and the object location in relation to the capture location. The relative position includes a dimension of depth with respect to the coordinate space. The dimension of depth is determined from analysis of the one or more images. A state of a computer program is changed based on the relative position.

Region-Specific User Input

In general, the subject matter described in this specification can be embodied in methods, systems, and program products for receiving user input. A computing system recognizes a touch input gesture that includes a first touch input in a first zone of a touchscreen that occurs simultaneous with a second touch input in a second zone of the touchscreen. The first and second zones of the touchscreen abut a periphery of the touchscreen and have been established by the computing system as being zones for causing the computing system to react to touch input in a particular manner, in contrast to at least another zone of the touchscreen for which the computing system does not react to touch input in the particular manner. The computing system modifies, in response to recognizing the touch input gesture, a display of the touchscreen to react in the particular manner.

Sound localization for user in motion

Methods, apparatus, and computer programs for simulating the source of sound are provided. One method includes operations for determining a location in space of the head of a user utilizing face recognition of images of the user. Further, the method includes an operation for determining a sound for two speakers, and an operation for determining an emanating location in space for the sound, each speaker being associated with one ear of the user. The acoustic signals for each speaker are established based on the location in space of the head, the sound, the emanating location in space, and the auditory characteristics of the user. In addition, the acoustic signals are transmitted to the two speakers. When the acoustic signals are played by the two speakers, the acoustic signals simulate that the sound originated at the emanating location in space.

Locking Mechanism Based on Unnatural Movement of Head-Mounted Display

A wearable computer determines unnatural movements of a head-mounted display (HMD) and triggers a locking mechanism. In one embodiment, the wearable computer receives movement data from one or more sensors and determines that the movement of the HMD is unnatural. In one embodiment, the wearable computer receives movement data from one or more sensors and determines that the HMD is being worn by an unauthorized user. In response to determining an unnatural movement and/or an unauthorized user wearing the HMD, the wearable computer triggers a locking mechanism, which can beneficially provide security measures for the wearable computer.

MULTIMODAL COMMUNICATION SYSTEM

The present invention, in various embodiments, comprises systems and methods for providing a communication system. In one embodiment, the system is an assistive technology (AT) in a single, highly integrated, multimodal, multifunctional, multipurpose, minimally invasive, unobtrusive, wireless, wearable, easy to use, low cost, and reliable AT that can potentially provide people with severe disabilities with flexible and effective computer access and environmental control in various conditions. In one embodiment, a multimodal Tongue Drive System (mTDS) is disclosed that uses tongue motion as its primary input modality. Secondary input modalities including speech, head motion, and diaphragm control are added to the tongue motion as additional input channels to enhance the system speed, accuracy, robustness, and flexibility, which are expected to address many of the aforementioned issues with traditional ATs that have limited number of input channels/modalities and can only be used in certain conditions by a certain group of users. 1. A multi-modal communication system for use by a subject , the system comprising: a tracer unit for use on a tongue of the subject;', 'a sensing unit comprising a primary sensor configured for placement in proximity to tongue carrying the tracer unit, wherein the primary sensor detects a position of the tracer unit to output a first type of communication; and, 'a tongue tracking unit comprising, 'a primary modality comprisinga plurality of secondary modalities comprising one or more secondary sensors to output a second type of communication.2. The system of claim 1 , wherein the first type of communication is proportional and the one or second types of communications are discrete.3. The system of claim 1 , wherein the first type of communication is discrete and the second type of communication is proportional.4. The system of claim 1 , wherein the tracer unit comprises a magnet.5. The system of claim 4 , wherein the magnet is coated with a ...

Eye Gaze Detection to Determine Speed of Image Movement

A wearable computing device or a head-mounted display (HMD) may be configured to track the gaze axis of an eye of the wearer. In particular, the device may be configured to observe movement of a wearer's pupil and, based on the movement, determine inputs to a user interface. For example, using eye gaze detection, the HMD may change a tracking rate of a displayed virtual image based on where the user is looking. Gazing at the center of the HMD field of view may, for instance, allow for fine movements of the virtual display. Gazing near an edge of the HMD field of view may provide coarser movements.

System and Method for Augmented and Virtual Reality

One embodiment is directed to a system for enabling two or more users to interact within a virtual world comprising virtual world data, comprising a computer network comprising one or more computing devices, the one or more computing devices comprising memory, processing circuitry, and software stored at least in part in the memory and executable by the processing circuitry to process at least a portion of the virtual world data; wherein at least a first portion of the virtual world data originates from a first user virtual world local to a first user, and wherein the computer network is operable to transmit the first portion to a user device for presentation to a second user, such that the second user may experience the first portion from the location of the second user, such that aspects of the first user virtual world are effectively passed to the second user.

Head-Angle-Trigger-Based Action

Methods and devices for applying at least one manipulative action to a selected content object are disclosed. In one aspect, a head-mounted-device (HMD) system includes at least one processor and data storage with user-interface logic executable by the at least one processor to apply at least one manipulative action to a displayed content object based on received data that indicates a first direction in which the HMD is tilted and an extent to which the HMD is tilted in the first direction. The at least one manipulative action is applied to a degree corresponding to the indicated extent to which the HMD is tilted in the first direction.

System and method for providing an interactive data-bearing mirror interface

An interactive interface of an embodiment of the present invention comprises a mirror surface; a sensor configured to receive an input from a user; a processor communicatively coupled to the sensor; the processor configured to identify a user identification based on the input, retrieve user specific content associated with the user identification; and identify one or more interactions with the user, wherein the processor comprises a speech processor and a video processor; and an output configured to display content associated with the user identification and responsive to the interactions on the mirror surface.

Systems and methods for a gaze and gesture interface

A system and methods for activating and interacting by a user with at least a 3D object displayed on a 3D computer display by at least the user's gestures which may be combined with a user's gaze at the 3D computer display. In a first instance the 3D object is a 3D CAD object. In a second instance the 3D object is a radial menu. A user's gaze is captured by ahead frame containing at least an endo camera and an exo camera worn by a user. A user's gesture is captured by a camera and is recognized from a plurality of gestures. User's gestures are captured by a sensor and are calibrated to the 3D computer display.

Image Adjusting

An apparatus including a display configured to display an image; and a system for adjusting the image on the display based upon location of a user of the apparatus relative to the apparatus. The system for adjusting includes a camera and an orientation sensor. The system for adjusting is configured to use signals from both the camera and the sensor to determine the location of the user relative to the display.

User Interface Apparatus and Method

The apparatus and method of the invention relate to data entry and menu selection. Applications include: (a) data entry for ideographic languages, including Chinese, Japanese and Korean; (b) fast food ordering; (c) correction of documents generated by optical character recognition; and (d) computer access and speech synthesis by persons temporarily or permanently lacking normal motor capabilities. In a preferred embodiment, each option of a menu is associated respectively with a selectable region displayed adjacent an edge of a display, forming a perimeter menu and leaving a region in the center of the perimeter menu for the output of an application program. Selectable regions may be on the display, outside the display, or both. A menu option may be selected by clicking on the associated selectable region, by dwelling on it for a selection threshold period or by a cursor path toward the selectable region, or by a combination thereof. Remaining dwell time required to select a selectable region is preferably indicated by the brightness of the selectable region. Submenus of a perimeter menu may also be perimeter menus and the location of a submenu option may be foretold by the appearance of its parent menu option. Menu options may be ideographs sharing a sound, a structure or another characteristic. Ideographs, which may be homophones of one another, may be associated with colored indicating regions and selection of an ideograph may be made by speaking the name of the associated color.

Control apparatus, electronic device, control method, and program

There is provided a control apparatus including a control unit configured to control information to be displayed on a display screen in a manner that a different type of information is visually recognized according to a positional relationship between the display screen and a predetermined portion of a user viewing the display screen. The information to be displayed on the display screen under control of the control unit is information extracted from information capable of being displayed on the display screen as information having an attribute satisfying a predetermined condition set for each positional relationship.

INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND COMPUTER PROGRAM

There is provided an information processing apparatus including an operation detection unit configured to detect an orientation of a user's face and operations performed by the user, and an area selection unit configured to, when the operation detection unit detects that the user has performed a first operation, select an area on a screen based on the orientation of the user's face during the first operation. 1. An information processing apparatus comprising:an operation detection unit configured to detect an orientation of a user's face and operations performed by the user; andan area selection unit configured to, when the operation detection unit detects that the user has performed a first operation, select an area on a screen based on the orientation of the user's face during the first operation.2. The information processing apparatus according to claim 1 , wherein when the operation detection unit detects that the user has switched from the first operation to a second operation claim 1 , the area selection unit is configured to change the selected area based on the second operation.3. The information processing apparatus according to claim 1 , further comprisingan information display unit configured to display information on the screen,wherein the information display unit is configured to display information on an area on the screen selected by the area selection unit.4. The information processing apparatus according to claim 3 , wherein the information display unit is configured to display information based on a position of the user's face detected by the operation detection unit.5. The information processing apparatus according to claim 3 , wherein the information display unit is configured to change the displayed information based on a content of the first operation detected by the operation detection unit.6. The information processing apparatus according to claim 3 , wherein the information display unit is configured to change a display when the user has ...

Virtual image display apparatus

An attitude information detection unit detects the attitude of a wearer, and an arithmetic processing unit and a device position drive unit, which form an image area position adjustment mechanism, translate a liquid crystal display device that is a light modulating device relative to a light guide section in a plane parallel to a light incident surface of the light guide section based on a detection result from the attitude information detection unit, whereby the position of an image area can be so changed that an image moves in the direction opposite the direction in which the wearer moves. As a result, when the wearer moves, the image does not follow the motion of the wearer as if the image remained still, whereby the wearer who moves will have a reduced amount of unpleasant sensation resulting from the motion.

Managing controller pairing in a multiplayer game

A method for managing multi-player interactivity with an interactive application is provided, including the following method operations: determining a location of a first controller; determining a location of a second controller; determining a location of a biometric identifier of a first user; determining a location of a biometric identifier of a second user; pairing the first controller to the first user based on the location of the first controller and the location of the biometric identifier of the first user; pairing the second controller to the second user based on the location of the second controller and the location of the biometric identifier of the second user.

Source conditioning for imaging directional backlights

Disclosed is an imaging directional backlight apparatus for providing large area uniform directed illumination from localized light sources. Within an exemplary optical valve system, a waveguide comprises a stepped structure, where the steps comprise extraction features hidden to guided light, propagating in a first forward direction. Returning light propagating in a second backward direction may be refracted, diffracted, or reflected by the features to provide discrete illumination beams exiting from the top surface of the waveguide. Such controlled illumination may provide for efficient, multi-user autostereoscopic displays as well as improved 2D display functionality. Illumination uniformity is provided by the positioning, packaging, and optically modifying of individual input sources. The latter employs non-imaging and refractive optics.

INFORMATION PROCESSING APPARATUS, INFORMATION PROCESSING METHOD, AND PROGRAM PRODUCT

According to one embodiment, an information processing apparatus includes: a detector configured to set a detection area curved in an arc shape to a frame image included in a video that is based on input video data, with reference to a position of a face image included in the video to detect a movement of an operator giving an operation instruction in the detection area; and an output module configured to output operation data indicating the operation instruction given by the movement detected by the detector. 1. An information processing apparatus comprising:a detector configured to set a detection area curved in an arc shape to a frame image included in a video that is based on input video data, with reference to a position of a face image included in the video to detect a movement of an operator giving an operation instruction in the detection area; andan output module configured to output operation data indicating the operation instruction given by the movement detected by the detector.2. The information processing apparatus of claim 1 , wherein the detection area is configured claim 1 , when the operator rotates an arm about an elbow claim 1 , to be set in a manner following an arc-like trajectory of the arm.3. The information processing apparatus of claim 1 , whereinthe detector is configured to detect the movement represented in a polar coordinate system from the detection area to convert the movement detected to the movement in a Cartesian coordinate system, andthe output module is configured to output the operation data indicating the operation instruction given by the movement represented in the Cartesian coordinate system, and to output the operation data indicating the operation instruction given by the movement converted.4. The information processing apparatus of claim 1 , wherein the detection area is configured to be set in line symmetry with respect to a line extending in a vertical direction of the face image.5. An information processing method ...

Control of displayed content in virtual environments

A system and method are disclosed for controlling content displayed to a user in a virtual environment. The virtual environment may include virtual controls with which a user may interact using predefined gestures. Interacting with a virtual control may adjust an aspect of the displayed content, including for example one or more of fast forwarding of the content, rewinding of the content, pausing of the content, stopping the content, changing a volume of content, recording the content, changing a brightness of the content, changing a contrast of the content and changing the content from a first still image to a second still image.

Enabling and Disabling Features of a Headset Computer Based on Real-Time Image Analysis

Operating conditions for a headset computer are determined using input from a speed sensor or accelerometer together with the results of scene analysis performed on images captured by a camera embedded in the headset computer. If the headset is travelling above a predetermined speed, and if the scene analysis returns a decision that the wearer is sitting in a driver's seat of the vehicle, then one or more features of the headset computer are disabled or restricted. The headset computer may disable display operation, mobile phone operation, or change audio interface options, or take other actions. 1. A method of controlling operation of a headset computer comprising:determining whether an acceleration or a velocity of a headset computer is greater than a predetermined threshold;through a camera of the headset computer, capturing an image from a perspective of a user of the headset computer;comparing the captured image against one or more template images representative of elements of a vehicle as seen by occupants of the vehicle; anddisabling one or more features of the headset computer based on the comparison of the captured image and the template images indicating that the user of the headset computer is operating a vehicle.2. The method of controlling operation of a headset computer of claim 1 , wherein the one or more features disabled includes operation of a micro-display.3. The method of controlling operation of a headset computer of claim 1 , wherein the one or more features disabled includes operation of a 3G/4G cellular radio.4. The method of controlling operation of a headset computer of claim 1 , further including enabling one or more features of the headset computer based on the comparison of the captured image and the template images indicating that the user of the headset computer is not operating a vehicle.5. The method of controlling operation of a headset computer of claim 4 , wherein the one or more features enabled includes operation of the headset ...

Saving augmented realities

Saving augmented realities includes collecting, with an augmented reality device, observation information of a physical space including an object, and obtaining, with the augmented reality device, an augmentation associated with the object. An augmented view of the physical space including a visual representation of the augmentation is visually presented with the augmented reality device, and the augmented view is saved for subsequent playback.

Portable proprioceptive peripatetic polylinear video player

Departing from one-way linear cinema played on a single rectangular screen, this multi-channel virtual environment involves a cinematic paradigm that undoes habitual ways of framing things, employing architectural concepts in a polylinear video/sound construction to create a type of experience that allows the world to reveal itself and permits discovery on the part of participants. Techniques are disclosed for peripatetic navigation through virtual space with a handheld computing device, leveraging human spatial memory to form a proprioceptive sense of location, allowing a participant to easily navigate amongst a plurality of simultaneously playing videos and to center in front of individual video panes in said space, making it comfortable for a participant to rest in a fixed posture and orientation while selectively viewing any one of the video streams, and providing spatialized 3D audio cues that invite awareness of other content unfolding simultaneously in the virtual environment.

Controlling light sources of a directional backlight

Disclosed is an imaging directional backlight including an array of light sources, and a control system arranged to provide variable distribution of luminous fluxes, scaled inversely by the width associated with the respective light sources in the lateral direction, across the array of light sources. The luminous intensity distribution of output optical windows may be controlled to provide desirable luminance distributions in the window plane of an autostereoscopic display, a directional display operating in wide angle 2D mode, privacy mode and low power consumption mode. Image quality may be improved and power consumption reduced.

Lip activity detection

Provided is a method of detecting lip activity. The method determines magnitude of optical flow in lip region and at least one non-lip region of a detected face. The ratio of magnitude of optical flow in lip region and at least one non-lip region is compared against a threshold. If the ratio is found to be greater than the threshold, lip activity of the detected face is recognized.

Display apparatus control system and method and apparatus for controlling a plurality of displays

A display apparatus control system and a method and apparatus for controlling a plurality of display apparatuses are provided. The method of controlling a plurality of display apparatuses includes capturing a region at which a user is located, generating a user-captured image, determining which display apparatus the user is looking at from among the plurality of display apparatuses, based on the user-captured image, and determining the determined display apparatus to be a control target display apparatus, and transmitting a control signal which corresponds to a user's command to be the control target display apparatus when the user's command is input.

ELECTRONIC DEVICE AND METHOD FOR REGULATING DISPLAY SCREEN

In a method for regulating the display on a screen of an electronic device, an initial state of a headpiece on a user is detected when the electronic device is powered on, and any deviation direction or deviation angle of the headpiece is determined by comparing a current orientation of the headpiece and an initial state thereof. A regulating direction and a regulating angle of the display screen or a displayed image on the display screen are determined according to any deviation direction and deviation angle such that when a deviation angle is greater than a threshold value, the physical display screen itself or the displayed image is adjusted according to the determined regulating direction and the determined regulating angle. 1. A computer-implemented method for regulating a display screen of an electronic device , the method comprising:detecting a current state of a head equipment on a user's head when the electronic device is powered on;determining a deviation direction and a deviation angle of the head equipment by comparing the current state of the head equipment with an initial state of the head equipment;determining a regulating direction and a regulating angle of the display screen or a displayed image on the display screen according to the deviation direction and the deviation angle when the deviation angle is greater than a threshold value; andadjusting the display screen or the displayed image according to the determined regulating direction, the determined regulating angle, and a preset adjustment mode.2. The method according to claim 1 , further comprising:setting the initial state of the head equipment and the adjustment mode of the display screen, and storing information of the initial state and the adjustment mode in a storage device of the electronic device.3. The method according to claim 1 , wherein the initial state of the head equipment corresponds to an initial angle and an initial direction of the head equipment claim 1 , and the current ...

INFORMATION DISPLAY DEVICE, CONTROL METHOD, AND PROGRAM

An information display device includes: an orientation change degree computation unit that computes a degree of change of a state of orientation detected by an orientation detection unit; an orientation change degree determination unit that determines whether the degree of change of the state of orientation of the information display device computed by the orientation change degree computation unit is equal to or more than a threshold; an image capture unit controller that activates the imaging unit upon the orientation change degree determination unit determining that the degree of change of the state of orientation of the information display device is equal to or more than the threshold; a face detection unit that detects a person's face in an image obtained by imaging of the imaging unit; a face direction specification unit that specifies a vertical orientation of the person's face detected by the face detection unit; a display direction determination unit that determines whether a vertical orientation of a screen display that is displayed on a display unit is the same orientation as the vertical orientation of the person's face specified by the face direction specification unit; and a display direction switch unit that switches the vertical orientation of the screen display that is displayed on the display unit to the same orientation as the vertical orientation of the person's face upon the display direction determination unit determining that the vertical orientation of the screen display that is displayed on the display unit and the vertical orientation of the person's face are not the same orientation. 1. An information display device comprising:an orientation change degree computation unit that computes a degree of change of a state of orientation of the information display device;an orientation change degree determination unit that determines whether the degree of change of the state of orientation of the information display device computed by the ...

Mobile communicator device including user attentiveness detector

Disclosed herein is a mobile communicator that includes a speed detection system for determining whether the mobile communicator is moving faster than a threshold speed. The mobile communicator further includes a user attentiveness detector configured to determine whether a user is inattentive and a sensor element configured to determine orientation of the mobile communicator. The mobile communicator further includes a circuit configured to ensure that a functionality of the mobile communicator is in a turned off state when the speed detection system determines that the mobile communicator is moving faster than the threshold speed and at least one of: (a) the user attentiveness detector determines that the user is inattentive; and (b) the sensor element determines that the mobile communicator is in a substantially vertical position.

DISPLAY DEVICE

An information processing apparatus including a communication interface configured to be connected to an external posture detecting device to be worn by a user; a display configured to rotatably display a display image; and circuitry configured to control a rotation angle of the image displayed by the display based on posture information received from the external posture detecting device. 1. An information processing apparatus comprising:a communication interface configured to be connected to an external posture detecting device to be worn by a user;a display configured to display a display image; andcircuitry configured to control a rotation angle of the display image based on posture information received from the external posture detecting device.2. The information processing apparatus of claim 1 , further comprising:an attitude detecting device configured to detect an attitude of the information processing apparatus.3. The information processing apparatus of claim 2 , whereinthe circuitry is configured to control the rotation angle of the image displayed by the display based on the posture information received from the external posture detecting device and an attitude of the information processing apparatus detected by the attitude detecting device.4. The information processing apparatus of claim 1 , whereinthe external posture detecting device includes a triaxial geomagnetic sensor and a tiraxial acceleration sensor.5. The information processing apparatus of claim 4 , whereinthe external posture detecting device is configured to detect a gravity direction using the acceleration sensor, and detect a bearing of the external posture detecting device in relation to a horizontal plane using the geomagnetic sensor6. The information processing apparatus of claim 1 , whereinthe circuitry is configured to control the rotation angle of the image displayed by the display so that the image displayed by the display becomes an upright state for the user.7. The information ...

Gesture recognition apparatus, control method thereof, display instrument, and computer readable medium

A gesture recognition apparatus for recognizing a gesture of a user from a moving image in which the user is photographed is provided, the gesture recognition apparatus comprising: a sight line direction estimation part configured to estimate a sight line direction of the user; a determination part configured to determine that the user intends to start the gesture when an angle formed by a first predetermined direction and the sight line direction is less than a predetermined value in a predetermined period; and a notification part configured to notify the user that the determination is made, when the determination part determines that the user intends to start the gesture.

APPARATUS AND METHOD OF PROVIDING USER INTERFACE ON HEAD MOUNTED DISPLAY AND HEAD MOUNTED DISPLAY THEREOF

An apparatus and method of providing a user interface (UI) on head mounted display and the head mounted display (HMD) thereof are discussed. The apparatus comprises a sensor unit detecting whether an object exists in the proximity of the HMD and if the object is detected, the sensor unit senses a distance between the object and the HMD. The apparatus further comprises a processor controlling a User Interface (UI) of the HMD based on a result of the sensor unit. A physical User Interface (UI) mode is applied if the detected object is within a predetermined distance from the HMD and a non-physical User Interface (UI) mode is applied if the object is not detected or is not within the predetermined distance from the HMD. 1. A method providing a User Interface (UI) for a Head Mounted Display (HMD) , the method comprising:detecting whether an object exists in the proximity of the HMD;determining one of a physical UI mode and a non-physical UI mode as an HMD UI mode based on the detecting step, wherein if the object is within a predetermined distance from the HMD, the physical UI mode is applied, and if the object is not detected or is not within the predetermined distance from the HMD, the non-physical UI mode is applied;detecting whether a state of the object is changed; andchanging the HMD UI mode from the physical UI mode to the non-physical UI mode if the state of the detected object is varied beyond the predetermined distance from the HMD, and changing the HMD UI mode from the non-physical UI mode to the physical UI mode if the state of the detected object is varied within the predetermined distance from the HMD.2. The method of claim 1 , further comprising:receiving a user command in the physical UI mode by contacting the object and in the non-physical UI mode by not contacting the object.3. The method of claim 1 , wherein the physical UI mode displays an imaginary UI function on a surface of the detected object.4. The method of claim 3 , wherein the physical UI ...

VIDEO IMAGE DISPLAY SYSTEM AND HEAD MOUNTED DISPLAY

A video image display system including an information apparatus and a transmissive head mounted display that allows a user to visually recognize video images distributed from the information apparatus as virtual images is provided. The information apparatus includes a video image distributor that distributes video images corresponding to a specific geographic region to the head mounted display. The head mounted display includes a motion detector that detects motion of the user's head and allows the user to visually recognize, as virtual images, video images selected based on motion information representing the motion. 1. A video image display system comprising:an information apparatus; anda transmissive head mounted display that allows a user to visually recognize video images distributed from the information apparatus as virtual images,wherein the information apparatus includes a video image distributor that distributes video images corresponding to a specific geographic region to the head mounted display, andthe head mounted display includes a motion detector that detects motion of the user's head and allows the user to visually recognize, as virtual images, the video images selected based on motion information representing the motion.2. The video image display system according to claim 1 ,wherein the head mounted display includes an information transmitter that transmits the motion information to the information apparatus,the information apparatus includes an information receiver that receives the motion information from the head mounted display located in the specific geographic region, andthe video image distributor selects at least one of the multiple sets of video images corresponding to the specific geographic region based on the motion information and distributes the selected video images to the head mounted display from which the motion information has been transmitted.3. The video image display system according to claim 2 ,wherein the multiple sets of ...

Control method of mobile terminal apparatus

A mobile device system that includes left and right earphones, and a mobile device connectable to the left and right earphones. Each of the left and right earphones includes at least one of an acceleration sensor and a geomagnetic sensor. The mobile device includes a controller that monitors a wearing state of the individual left and right earphones by a user on a basis of output from the least one of the acceleration sensors and the geomagnetic sensors, and controls a behavior of the mobile device connected to the left and right earphones according to the monitored wearing state of the left and right earphones by the user.

Eye/Head Controls for Camera Pointing

A setting of a video camera is remotely controlled. Video from a video camera is displayed to a user using a video display. At least one eye of the user is imaged as the user is observing the video display, a change in an image of at least one eye of the user is measured over time, and an eye/head activity variable is calculated from the measured change in the image using an eyetracker. The eye/head activity variable is translated into a camera control setting, and an actuator connected to the video camera is instructed to apply the camera control setting to the video camera using a processor.

Apparatus and method

In an aspect of the invention there is provided apparatus comprising: a plurality of image processing modules, each image processing module being one selected from amongst an image capture module and an image projection module, the apparatus comprising at least one image capture module and at least one image projection module; and a reflector member, the image projection module being arranged to project an image onto the reflector member for viewing by a user, the image capture module being arranged to capture an image of the user via the reflector member, at least one of the image processing modules being arranged to receive operational data from a source external to the module.

DETECTION OF HUMAN-MACHINE INTERACTION ERRORS

Disclosed are a system and method of detection of an interaction-error. The interaction-error is derived from an incorrect decision and is directed to interacting with a machine. During human-machine interaction, command related data values are obtained. Command related data values characterize any one of an interacting-command and an interacting-action. The command related data values are compared with command related reference data values, and an interaction-error is identified if a difference between the command related data values and the command related reference data values complies with a predefined criterion. 140-. (canceled)41. A human-machine interface system configured and operable to control the machine operation via monitoring cognitive brain commands of a user while operating the machine , wherein the human-machine interface system comprises:at least one sensor configured and operable to generate sensing data indicative of user's cognitive brain commands, anda computer processor configured for data communication with said at least one sensor, the computer processor being adapted to process said sensing data to identify, in said data indicative of user's cognitive brain commands, motor-command related data indicative of motor commands controlling the user interaction with the machine to perform one or more actions, wherein the computer processor comprises an error detector configured and operable to analyze the data indicative of motor-command related data and determine a relation of said motor command related data to command related reference data indicative of a condition of user's predicted error in his cognitive brain command, and upon identifying, based on the determined relation, that the motor command related data corresponds to said condition of user's predicted error in his cognitive brain command, generate and store data indicative of said predicted error condition, thereby enabling to carry out at least one of the following: (i) selectively ...

CAPACITIVE SENSING ASSEMBLY FOR DETECTING PROXIMITY OF USER TO A CONTROLLER DEVICE

A capacitive sensing assembly comprises a nonconductive interface material, a conductive plastic, an electrode, and a controller. The capacitive sensing assembly is configured to detect a proximity of skin to the nonconductive interface material. The nonconductive interface material comprises a first surface and a second surface that are opposite one another. The first surface receives contact with the skin. The conductive plastic is coupled to the second surface of the interface material. The electrode is electrically coupled to the conductive plastic and configured to provide an electrical signal. The electrical signal corresponds to a surface capacitance of the nonconductive interface material that is based in part on the conductive plastic, the nonconductive interface material, and the proximity of the skin to the interface material. The controller determines the proximity of the skin based on the electrical signal that corresponds to the surface capacitance. 1. A capacitive sensing assembly comprising:a nonconductive interface material comprising a first surface and a second surface that is opposite the first surface, the first surface configured to receive contact with skin of a user;a conductive plastic coupled to the second surface of the interface material;an electrode electrically coupled to the conductive plastic and configured to provide an electrical signal that corresponds to a surface capacitance that is based in part on the conductive plastic, the nonconductive interface material, and proximity of the skin of the user to the first surface of the interface material; anda controller electrically coupled to the electrode and configured to determine a proximity of the skin of the user based on the electrical signal that corresponds to the surface capacitance.2. The capacitive sensing assembly of claim 1 , wherein the nonconductive interface material forms a button on a controller device.3. The capacitive sensing assembly of claim 1 , wherein the ...

NATURAL USER INTERFACES FOR MOBILE IMAGE VIEWING

The mobile image viewing technique described herein provides a hands-free interface for viewing large imagery (e.g., 360 degree panoramas, parallax image sequences, and long multi-perspective panoramas) on mobile devices. The technique controls the imagery displayed on a display of a mobile device by movement of the mobile device. The technique uses sensors to track the mobile device's orientation and position, and front facing camera to track the user's viewing distance and viewing angle. The technique adjusts the view of a rendered imagery on the mobile device's display according to the tracked data. In one embodiment the technique can employ a sensor fusion methodology that combines viewer tracking using a front facing camera with gyroscope data from the mobile device to produce a robust signal that defines the viewer's 3D position relative to the display. 1. A computer-implemented process for viewing imagery on a mobile device , comprising:tracking a mobile device's orientation and position;using a camera and viewer tracker on the mobile device to track a user's face looking at a screen on the mobile device;computing image transformations of imagery displayed on the screen of the mobile device using the tracked mobile device's orientation and position and the tracked user's face to allow dynamic change of the perspective of the displayed imagery by moving the mobile device relative to the user's face.2. The computer-implemented process of claim 1 , wherein the perspective of the imagery displayed can be changed even when the user's face is outside of the camera's field of view.3. The computer-implemented process of claim 1 , further comprising:computing the viewing angle and a viewing distance between the user and the screen on the mobile device by using the tracked orientation and position of the mobile device, and the tracked position of the user's face relative to screen of the mobile device in order to compute the image transformations.4. The computer- ...

DISPLAY CONTROL PROGRAM, DISLAY CONTROL APPARATUS AND DISPLAY CONTROL METHOD

Methods and apparatus provide for carrying out a method for rendering a virtual-three dimensional space on a display, including: acquiring information indicative of a position of a head mounted display worn on a head of a user; acquiring information indicative of a position of an input apparatus for use by the user; acquiring information indicative of an instruction entered in the input apparatus; and generating an image of the virtual three-dimensional space to be displayed on the head mounted display at a viewpoint position and a sightline direction, where the generating includes generating an image of the input apparatus at the position in the image determined on a basis of a relative position with the head mounted display of the input apparatus, and where the generating includes generating an image of a hand of the user when the user controls the input apparatus. 1. A display control apparatus comprising:a head mounted display information acquisition block configured to acquire information indicative of a position of a head mounted display worn on a head of a user;an input apparatus information acquisition block configured to acquire information indicative of a position of an input apparatus for use by the user;an instruction input acquisition block configured to acquire information indicative of an instruction entered in the input apparatus; andan image generation block configured to generate an image of the virtual three-dimensional space to be displayed on the head mounted display at a viewpoint position and a sightline direction,wherein the image generation block displays an image of the input apparatus at the position in the image determined on a basis of a relative position with the head mounted display of the input apparatus, andwherein the image generation block displays an image of a hand of the user when the user controls the input apparatus.2. The display control apparatus according to claim 1 , wherein the image generation block changes the image of ...

ALIGNMENT SENSOR SYSTEM AND ALIGNMENT SENSOR SYSTEM METHOD

A system and a method detect alignment of a movement machine in a cost-effective manner. An alignment sensor system includes a movement instrument () for carrying out movements by displacing the center of gravity or actuating the muscles of a human body, with an alignment element, and a mass produced sensor, which is fixed relative to the alignment element. 1. An alignment sensor system comprising:a movement instrument for carrying out movements by displacing, with an alignment element a center of gravity or actuating muscles of a user, anda first mass-produced sensor the first mass-produced sensor being fixed relative to the alignment element.2. The alignment sensor system of claim 1 , wherein the first mass produced sensor is a portable electronic device claim 1 , the device comprising either a position sensor or an alignment sensor and a data transmitter claim 1 , the device being produced by a mass production method and being directed to an end-consumer as buyer.3. The alignment sensor system of claim 1 , wherein the first mass-produced sensor is a mobile phone claim 1 , a media player claim 1 , an activity band claim 1 , a smartwatch claim 1 , or a handheld controller for a games console claim 1 ,wherein the mobile phone, the media player, the activity band, the smartwatch, and the handheld controller for the games console comprising alignment sensors.4. The alignment sensor system of claim 1 ,wherein the alignment element comprises a support device, the support device for supporting at least in part the user, andwherein the first mass-produced sensor is fixed relative to the support device.5. The alignment sensor system of claim 1 , further comprising a computer claim 1 , the computer for receiving and processing alignment information from the first mass-produced claim 1 , sensor.6. The alignment sensor system of further comprising a display unit claim 5 , the display unit being fixable to a head of the user claim 5 ,wherein the display unit is connected to ...

INFORMATION PROCESSING METHOD, INFORMATION PROCESSING APPARATUS AND USER EQUIPMENT

An information processing method, an information processing apparatus and a user equipment are provided. A method comprises: acquiring eye movement information related to that at least one eye of a user gazes a user equipment; and determining relative movement information of the user equipment relative to the head of the user at least according to a non-saccade eye movement part in the eye movement information. Relative movement between a user equipment and the head of a user can be determined according to a non-saccade eye movement part(s) of eye movement information related to the user gazes the user equipment, providing a basis for increasing the accuracy of compensation corresponding to the relative movement. 1. A method , comprising:acquiring, by a system comprising a processor, eye movement information related to determining that an eye of a user gazes at a user equipment; anddetermining relative movement information of the user equipment relative to a head of the user at least according to a non-saccade eye movement part in the eye movement information.2. The method of claim 1 , further comprising:processing the eye movement information to obtain the non-saccade eye movement part.3. The method of claim 1 , wherein the non-saccade eye movement part comprises:a smooth pursuit eye movement part.4. The method of claim 3 , wherein the non-saccade eye movement part further comprises:a fixation eye movement part.5. The method of claim 1 , wherein the determining the relative movement information at least according to the non-saccade eye movement part comprises:determining the relative movement information at least according to the non-saccade eye movement part and a reference non-saccade eye movement information.6. The method of claim 1 , wherein the determining the relative movement information at least according to the non-saccade eye movement part comprises:determining the relative movement information at least according to the non-saccade eye movement part and ...

DISPLAY CONTROL METHOD, DISPLAY CONTROL APPARATUS AND USER EQUIPMENT

A display control method, a display control apparatus and a user equipment are provided. A method comprises acquiring eye movement information related to that at least one eye of a user gazes a user equipment; and adjusting display of a display content of the user equipment at least according to a non-saccade eye movement part in the eye movement information. A display content of a user equipment can be adjusted according to a non-saccade eye movement part of the user equipment gazed by a user, to enable relative movement between the head of the user and the adjusted display content to disappear or decrease, thus improving the user's reading experience in a scenario of having relative movement with the user equipment. 1. A method , comprising:acquiring, by a system comprising a processor, eye movement information related to determining that an eye of a user gazes at a user equipment; andadjusting a display of a display content of the user equipment at least according to a non-saccade eye movement part in the eye movement information.2. The method of claim 1 , further comprising:processing the eye movement information to obtain the non-saccade eye movement part.3. The method of claim 1 , wherein the non-saccade eye movement part comprises:a smooth pursuit eye movement part.4. The method of claim 3 , wherein the non-saccade eye movement part further comprises:a fixation eye movement part.5. The method of claim 1 , wherein the adjusting the display of a display content of the user equipment at least according to the non-saccade eye movement part comprises:determining a display shifting parameter of the display content of the user equipment at least according to the non-saccade eye movement part; anddisplaying the display content according to the display shifting parameter,wherein the display shifting parameter is configured to perform display stabilizing compensation on a relative movement between a head of the user and the user equipment corresponding to the non- ...

VIRTUAL REALITY SYSTEM WITH CONTROL COMMAND GESTURES

A virtual reality system that uses gestures to obtain commands from a user. Embodiments may use sensors mounted on a virtual reality headset to detect head movements, and may recognize selected head motions as gestures associated with commands. Commands associated with gestures may modify the user's virtual reality experience, for example by selecting or modifying a virtual world or by altering the user's viewpoint within the virtual world. Embodiments may define specific gestures to place the system into command mode or user input mode, for example to temporarily disable normal head tracking within the virtual environment. Embodiments may also recognize gestures of other body parts, such as wrist movements measured by a smart watch. 1. A virtual reality system with control command gestures , comprising:at least one display viewable by a user;at least one sensor that generates sensor data that measures one or more aspects of a pose of one or more body parts of said user;a pose analyzer coupled to said at least one sensor, that calculates pose data of said pose of one or more body parts of said user, based on said sensor data generated by said at least one sensor;a control state;one or more control commands, each configured to modify said control state when executed, each associated with one or more gestures of one or more of said one or more body parts of said user; receives said pose data from said pose analyzer;', 'determines whether said user has performed a gesture associated with a control command; and,', 'executes said control command to modify said control state when said user has performed said gesture associated with said control command;, 'a gesture recognizer coupled to said pose analyzer and to said one or more control commands, wherein said gesture recognizer'}a 3D model of a scene; and, optionally modifies or selects said 3D model of a scene based on said control state;', 'receives said pose data from said pose analyzer;', 'calculates one or more ...

Efficient orientation estimation system using magnetic, angular rate, and gravity sensors

A system that efficiently estimates an object's orientation using magnetic, angular rate, and gravity sensors. The object may be for example a virtual reality headset or a user in a virtual reality environment. Magnetic and gravity data are used to correct errors that accumulate from integrating angular velocity. Unlike systems that use Kalman filter approaches, embodiments of the system apply a simple, highly efficient technique to generate magnetic and gravity error vectors; these error vectors are added directly to the angular velocity prior to integration. Error calculations are performed in the sensor reference frame rather than in the Earth reference frame. Magnetic error correction uses only the horizontal component of the magnetic field, which is efficiently calculated by subtracting off the projection of the magnetic field onto the measured gravity vector. Sensors and processors for calculating orientation may be integrated into a low-latency virtual reality display system.

CONTROL DEVICE, HEAD-MOUNT DISPLAY DEVICE, PROGRAM, AND CONTROL METHOD FOR DETECTING HEAD MOTION OF A USER

It is possible to provide a technique for accurately performing an operation desired by a user. A user's head operation is identified according to information detected by a head motion detection unit. A process desired by the user is executed according to an angular velocity of the head motion. Moreover, the technique uses a control unit which can accurately execute an operation by the user's head operation without reflecting the return motion of the user's head in the process. The control unit executes a process for a start and an end of each process corresponding to the detected angular velocity according to a predetermined threshold value. 1. A head-mount display to be worn on a user's head , comprising:a housing;a display unit which is placed in the housing, and configured to display an image for the user who wears the head-mount display;a sensor unit which is placed within the housing, and configured to detect an information with respect to a velocity of the user's head motion;a control unit which is placed within the housing, and configured to perform either one of a first control to control the head-mount display or a second control differs from the first control, based on the information detected by the sensor unit.2. The head-mount display of claim 1 , wherein:the control unit is configured to perform the second control when the information is larger than a first predetermined value and the first control when the information is smaller than the first predetermined value.3. The head-mount display of claim 1 , whereinthe sensor unit is configured to detect the information in a first direction and the control unit is configured to perform the first control or the second control based on the information in the first direction.4. The head-mount display of claim 3 , wherein the second control when the information in the first direction is larger than the first predetermined value; and', 'the first control when the information in the first direction is smaller ...

System, Method, Device and Computer Readable Medium for Use with Virtual Environments

According to the invention, there is disclosed a system, method, device and computer readable medium for a user to interact with objects in a virtual environment. The invention includes a gesture controller, associated with an aspect of the user, and operative to generate spatial data corresponding to the position of the aspect of the user. A mobile device processor is operative to receive the spatial data of the gesture controller and to automatically process the spatial data to generate a spatial representation in the virtual environment corresponding to the position of the aspect of the user. Thus, the invention is operative to facilitate the user interacting with the objects in the virtual environment using the spatial representation of the gesture controller based on the position of the aspect of the user.

Method and Apparatus for Construction Machine Visualization

A system for displaying information to an operator of a machine comprises a head tracking system and a projection system. One system for displaying information uses a projection system and a see-through display to present two-dimensional images to an operator. One system uses a projection system and a see-through display to present three-dimensional images to an operator. One system uses a pair of smart glasses to display information to a user based a direction a user is looking. 1. A method comprising:determining a direction of a user's gaze;determining information to be displayed to the user based on the direction of the user's gaze, a current state of a construction site, and a current state of a machine; anddisplaying the information to the user.2. The method of claim 1 , wherein the direction of the user's gaze is determined based on a head position of the user.3. The method of claim 1 , wherein the direction of the user's gaze is determined based on a position of the user's eyes and a direction the user's eyes are looking.4. The method of claim 1 , the determining information to be displayed to a user comprising:determining a difference between the current state of the construction site and a desired site plan;determining an operation to be performed based on the difference, wherein the information comprises the operation to be performed.5. The method of claim 1 , the determining information to be displayed to a user comprising:determining a difference between the current state of the construction site and a desired site plan;generating a value pertaining to a percent of completion of the construction site based on the determining the difference between the current state of the construction site and the desired site plan,wherein information comprises the percent of completion of the construction site.6. The method of claim 1 , further comprising:determining that a user's gaze has changed;determining new information to be displayed to the user based on the ...

Display apparatus

In a display apparatus provided with an electrophoresis layer, such a display apparatus capable of operating a driving electrode for use in displaying an image as an electrode for use in detecting an input position is provided. A display apparatus includes: a substrate; another substrate that is disposed so as to face the substrate; an electrophoresis layer sandwiched between the substrate and another substrate; a plurality of pixel electrodes formed on the substrate; and a plurality of driving electrodes formed on another substrate. An electric field is formed between each of the plurality of pixel electrodes and each of the plurality of driving electrodes, so that an image is displayed, and an input position is detected based on an electrostatic capacitance of each of the plurality of driving electrodes.

GROUND PLANE ADJUSTMENT IN A VIRTUAL REALITY ENVIRONMENT

An HMD device is configured to vertically adjust the ground plane of a rendered virtual reality environment that has varying elevations to match the flat real world floor so that the device user can move around to navigate and explore the environment and always be properly located on the virtual ground and not be above it or underneath it. Rather than continuously adjust the virtual reality ground plane, which can introduce cognitive dissonance discomfort to the user, when the user is not engaged in some form of locomotion (e.g., walking), the HMD device establishes a threshold radius around the user within which virtual ground plane adjustment is not performed. The user can make movements within the threshold radius without the HMD device shifting the virtual terrain. When the user moves past the threshold radius, the device will perform an adjustment as needed to match the ground plane of the virtual reality environment to the real world floor. 1. A method for rendering a virtual reality environment having a virtual ground plane with variable elevations on a head mounted display (HMD) device located in a physical space having a floor , comprising:rendering, to a user of the HMD device, one or more virtual images in a stream of frames in which the virtual images are rendered with respect to the virtual ground plane;using a depth sensor to generate surface reconstruction data of the physical space;determining a height of the user's head from the floor of the physical space using the surface reconstruction data; andbased on the determined height, dynamically adjusting the virtual ground plane to align with the floor of the physical space on a frame-by-frame basis, or group of frames basis.2. The method of including filtering or smoothing the adjustment of the virtual ground plane to align with the floor.3. The method of including using the surface reconstruction data to determine a location of the user's head in the physical space claim 1 , establishing a threshold ...

IMAGE DISPLAY DEVICE, IMAGE DISPLAY METHOD, AND RECORDING MEDIUM

Provided is an image display device used by being mounted to a head or a face of a user, including a first display unit that displays an internal image seen from a side of the user, a second display unit that displays an external image seen from outside the image display device, and a control unit that controls display of the internal image and the external image. 1. An image control device , comprising:an image generating unit configured to generate a user image based on information related to a status of a user, wherein the user image is related to the user who mounts a display device to one of a head or a face of the user; anda control unit configured to control a display of the user image generated by the image generating unit.2. The image control device according to claim 1 , wherein the user image is different from an internal image that is displayed on a user side of the display device.3. The image control device according to claim 1 , wherein the control unit is further configured to control the display of the user image for a person different from the user.4. The image control device according to claim 1 , wherein the image generating unit is further configured to generate the user image based on at least one of an eyelid status or a gaze direction of the user.5. The image control device according to claim 1 , wherein the image generating unit is further configured to generate the user image based on sensor information received from the display device.6. The image control device according to claim 2 , wherein the control unit is further configured to control a display of at least one of a first image that is same as the internal image claim 2 , a second image that is different from the internal image claim 2 , information related to a surrounding environment claim 2 , or the status of the user to be displayed as the user image.7. The image control device according to claim 2 , wherein the control unit is further configured to control the display of the user ...

Apparatus and Method for Improving, Augmenting or Enhancing Vision

The present invention provides various apparatus and methods for improving, augmenting or enhancing a vision of a person that include a first camera, one or more sensors, a microdisplay, and one or more processors communicably coupled to the first camera, the one or more sensors and the microdisplay. The first camera is configured to acquire a first image of a scene facing away from an eye of the person. The microdisplay is configured to display a modified first image to the eye. In one operational mode, the one or more processors are configured to acquire the first image of the scene using the first camera, modify the first image based on one or more vision improvement parameters, and display the modified first image on the microdisplay to improve, augment or enhance the vision of the person.

Mixed-Reality Golf Tracking and Simulation

A mixed-reality golf simulation system includes a ball-tracking sub-system to generate ball-tracking data when a golf ball is hit by a user, a near-eye display, a storage device to store images of a hole of a golf course associated with location coordinates of a plurality of locations along the hole, and a controller. The controller may direct the near-eye display to display a mixed-reality environment from the perspective of a location on the hole based on images associated with the location, receive ball-tracking data including a landing location of a ball hit by the user, alert the user if the landing location is a target pin, and direct the near-eye display to display a mixed-reality environment from the perspective of the landing location based on images associated with the landing location if the landing location is not the target pin. 1. A mixed-reality golf simulation system comprising:a ball-tracking sub-system configured to generate ball-tracking data when a golf ball is hit by a user;a near-eye display with a partially-transparent display element configured to display mixed-reality virtual objects displayed over physical objects within a user field of view, wherein the near-eye display includes a user input device;a storage device to store images of a hole of a golf course from a plurality of locations along the hole, wherein the images are associated with location coordinates of the plurality of locations along the hole; and directing the near-eye display to display a mixed-reality environment including a virtual reality scene from the perspective of a first location on the hole over at least a portion of the user field of view, wherein the virtual reality scene from the perspective of the first location includes images from the storage device associated with location coordinates of the first location;', 'receiving ball-tracking data of a ball hit by the user in real-time from the ball-tracking sub-system, wherein the ball-tracking data includes a ...

TRAILER MOUNTED AIRCRAFT FUSELAGES WITH VIRTUAL ENVIRONMENT SYSTEMS

A system for virtually experiencing an aircraft includes a a trailer, an aircraft fuselage, a cabin module, a head mounted display, and a controller. The aircraft fuselage is disposed in the trailer and defines an interior with a first portion and a second portion. The first portion is substantially unfurnished. The cabin module is disposed in the second portion and includes furnishings. The head mounted display is configured for movement within the interior. The controller is configured for generating a virtual aircraft interior based on a position and an orientation of the head mounted display relative to the interior of the aircraft fuselage and for controlling the head mounted display to display the virtual aircraft interior. 1. A system for virtually experiencing an aircraft , the system comprising:a trailer;a portion of an aircraft fuselage disposed in the trailer and defining an interior with a first portion and a second portion, wherein the first portion is substantially unfurnished;a cabin module disposed in the second portion and including furnishings;a head mounted display configured for movement within the interior; and generating a virtual aircraft interior based on a position and an orientation of the head mounted display relative to the interior, and', 'controlling the head mounted display to display the virtual aircraft interior., 'a controller communicatively coupled with the head mounted display and configured for2. The system of claim 1 , wherein the furnishings include an aircraft cabin chair claim 1 , an aircraft cabin table claim 1 , and an aircraft cabin side ledge.3. The system of claim 1 , wherein the cabin module is configured to relocate within the interior and the controller is configured for generating the virtual aircraft interior further based on cabin layout selections.4. The system of claim 1 , wherein the controller is further configured for generating a finished cabin interior as the virtual interior further based on cabin ...

Focus-Based Debugging And Inspection For A Display System

A method for debugging includes determining an eye focus depth for a user, determining a virtual focus point relative to a virtual view location in a virtual environment based on the eye focus depth for the user, transitioning a first object from the virtual environment from a first rendering mode to a second rendering mode based on a location of the virtual focus point relative to the first object, wherein visibility of a second object from the virtual view location is occluded by the first object in the first rendering mode and visibility of the second object from the virtual view location is not occluded by the first object in the second rendering mode, and activating a function of a development interface relative to the second object while the first object is in the second rendering mode.

GAZE BASED INTERFACE FOR AUGMENTED REALITY ENVIRONMENT

In an embodiment, a processing system provides an augmented reality object for display by a head-mounted device (HMD) worn by a user. The processing system provides an augmented reality graphic for display by the HMD on a plane and overlaid on the augmented reality object. The processing system determines a gaze direction of the user using sensor data captured by a sensor of the HMD. Responsive to determining that the gaze direction intersects with the augmented reality graphic on the plane and remains intersecting for at least a period of time, the processing system determines a position of intersection between the gaze direction and the augmented reality graphic on the plane. The processing system provides a modified version of the augmented reality object for display by the HMD according to the position of intersection during the period of time. 1. A system comprising:a head-mounted device (HMD) worn by a user, the HMD including a sensor, and provide an augmented reality object for display by the HMD;', 'provide an augmented reality graphic for display by the HMD on a plane and overlaid on the augmented reality object;', 'determine a gaze direction of the user using sensor data captured by the sensor; and', determine a position of intersection between the gaze direction and the augmented reality graphic on the plane, and', 'provide a modified version of the augmented reality object for display by the HMD according to the position of intersection during the period of time., 'responsive to determining that the gaze direction intersects with the augmented reality graphic on the plane and remains intersecting for at least a period of time], 'a non-transitory computer-readable storage medium storing instructions, the instructions when executed by one or more processors cause the one or more processors to2. The system of claim 1 , wherein the non-transitory computer-readable storage medium stores further instructions that when executed by the one or more processors ...

Tracker For A Head-Mounted Display

A tracker for a Head-Mounted Display, HMD, unit is provided. The tracker comprises a carrier element carrying one or more markers that are configured to permit determining a position of the tracker . The carrier element comprises at least one magnetic element configured to cooperate with at least one magnetic element provided on the HMD unit , or on a base element that is to be fixed to the HMD unit , for detachably attaching the carrier element to the HMD unit 1. A tracker for a Head-Mounted Display , HMD , unit , the tracker comprisinga carrier element carrying one or more markers that are configured to permit determining a position of the tracker, the carrier element being configured to be detachably attachable to the HMD unit;wherein the carrier element comprises at least one magnetic element configured to cooperate with at least one magnetic element provided on the HMD unit, or on a base element to be fixed to the HMD unit, for detachably attaching the carrier element to the HMD unit.2. The tracker of claim 1 , whereintwo or more markers are provided, wherein at least some of the two or more markers are arranged along a longitudinal extension of the carrier element, andwherein the carrier element has a curved shape in its longitudinal extension.3. The tracker of claim 1 , whereinthe carrier element has an alignment element configured to cooperate with a corresponding alignment element provided on the HMD unit or on the base element to be fixed to the HMD unit.4. The tracker of claim 1 , further comprisingthe base element configured to be fixed to the HMD unit so as to extend in a circumferential direction of the HMD unit when being fixed to the HMD unit, andwherein the carrier element and the base element each comprise the respective at least one magnetic element.5. The tracker of claim 4 , whereinthe carrier element and the base element each comprise an alignment element configured to cooperate with each other so as to positionally align the carrier element ...

POSITION ESTIMATION APPARATUS, POSITION ESTIMATION METHOD, AND PROGRAM

A position estimation apparatus, a position estimation method, and a program that can estimate a position of a tracker with high accuracy while the power consumption is reduced is provided. An estimation result storage section () stores a result of estimation of a position of a tracker (). An acceleration data acquisition section () repeatedly acquires acceleration data indicative of an acceleration of the tracker () from a sensor that detects an acceleration of the tracker (). An estimation result updating section () updates the result of estimation on the basis of the result of estimation of a position of the tracker stored in the estimation result storage section () and the acquired latest acceleration data. A velocity information acquisition section () acquires information of the velocity of the tracker (). The estimation result updating section () updates, when the velocity information is acquired by the velocity information acquisition section (), the estimation result of a position of the tracker () stored in the estimation result storage section () on the basis of the velocity information and the acquired latest acceleration data. 1. A position estimation apparatus , comprising:an estimation result storage section configured to store a result of estimation of a position of a tracker;an acceleration data acquisition section configured to repeatedly acquire acceleration data indicative of an acceleration of the tracker from a sensor that detects an acceleration of the tracker;an estimation result updating section configured to update the result of estimation on a basis of the result of estimation of the position of the tracker stored in the estimation result storage section and the acquired latest acceleration data; anda velocity information acquisition section configured to acquire information of a velocity of the tracker, wherein the estimation result updating section updates, when the velocity information is acquired by the velocity information acquisition ...

Display Screen Front Panel of HMD for Viewing by Users Viewing the HMD Player

Method for providing image of HMD user to a non-HMD user includes, receiving a first image of a user including the user's facial features captured by an external camera when the user is not wearing a head mounted display (HMD). A second image capturing a portion of the facial features of the user when the user is wearing the HMD is received. An image overlay data is generated by mapping contours of facial features captured in the second image with contours of corresponding facial features captured in the first image. The image overlay data is forwarded to the HMD for rendering on a second display screen that is mounted on a front face of the HMD.

MULTI-DIMENSIONAL REFERENCE ELEMENT FOR MIXED REALITY ENVIRONMENTS

Approaches provide for controlling, managing, and/or otherwise interacting with mixed (e.g., virtual and/or augmented) reality content in response to input from a user, including voice input, device input, among other such inputs, in a mixed reality environment. For example, a mixed reality device, such as a headset or other such device can perform various operations in response to a voice command or other such input. In one such example, the device can receive a voice command and an application executing on the device or otherwise in communication with the device can analyze audio input data of the voice command to control the view of content in the environment, as may include controlling a user's “position” in the environment. The position can include, for example, a specific location in time, space, etc., as well as directionality and field of view of the user in the environment. A reference element can be displayed as an overlay to the mixed reality content, and can provide a visual reference to the user's position in the environment. 1. A mixed reality display system , comprising:a display;a device processor; display virtual reality content on the display;', 'display a multi-dimensional reference element as an overlay to the virtual reality content, the multi-dimensional reference element operable to provide a first visual reference to a first position within a virtual environment and a first field of view (orientation) in the virtual environment;', 'receive audio input data, the audio input data corresponding to an utterance received by a microphone of the virtual reality display device;', 'use automatic speech recognition (ASR) techniques on the audio input data to generate text data that represents words;', 'use natural language understanding (NLU) techniques on the text data to identify a command to display virtual reality content for a second position within the virtual environment and a second field of view within the virtual environment;', 'update the ...

PRIVACY ZONING AND AUTHORIZATION FOR AUDIO RENDERING

Example devices and methods are disclosed. An example device includes a memory configured to store a plurality of audio streams and an associated level of authorization for each of the plurality of audio streams. The device also includes one or more processors implemented in circuitry and communicatively coupled to the memory. The one or more processors are configured to select, based on the associated levels of authorization, a subset of the plurality of audio streams, the subset of the plurality of audio streams excluding at least one of the plurality of audio streams. 1. A device comprising:a memory configured to store a plurality of audio streams and an associated level of authorization for each of the plurality of audio streams; andone or more processors implemented in circuitry and communicatively coupled to the memory, and configured to:select, based on the associated levels of authorization, a subset of the plurality of audio streams, the subset of the plurality of audio streams excluding at least one of the plurality of audio streams.2. The device of claim 1 , wherein the memory is further configured to store location information associated with coordinates of an acoustical space in which a corresponding one of the plurality of audio streams was captured or synthesized.3. The device of claim 2 , wherein the device comprises an extended reality headset claim 2 , and wherein the acoustical space comprises a scene represented by video data captured by a camera.4. The device of claim 2 , wherein the device comprises an extended reality headset claim 2 , and wherein an acoustical space comprises a virtual world.5. The device of claim 2 , wherein the device comprises an extended reality headset claim 2 , and wherein an acoustical space comprises a physical world.6. The device of claim 2 , wherein the selected subset of the plurality of audio streams is further based on the location information.7. The device of claim 1 , wherein excluded audio streams are ...

ENVIRONMENTAL INTERRUPT IN A HEAD-MOUNTED DISPLAY AND UTILIZATION OF NON FIELD OF VIEW REAL ESTATE

A wearable computing device includes a head-mounted display (HMD) that generates a virtual reality environment. Through the generation and tracking of positional data, a the virtual environment may be interrupted or paused. Upon pausing the environment, a user may access a number of ancillary menus and controls not otherwise available during normal operation of the virtual environment. 1. A method for executing a function within a virtual environment , the method comprising:storing information in memory regarding at least one control setting that associates a function with a change in position of a head-mounted display,calibrating a neutral position for the head-mounted display, wherein the calibration is performed using one or more sensors,monitoring positional data associated with the head-mounted display, wherein the monitored positional data is obtained via the one or more sensors,evaluating the monitored positional data of the head-mounted display against one or more thresholds specified in the at least one control setting, andexecuting the function associated with the at least one control setting based on the evaluated monitored positional data of the head-mounted display.2. The method of claim 1 , wherein the one or more sensors used for calibrating the neutral position includes three-dimensional axis gyroscopes claim 1 , accelerometers and magnetometers.3. The method of claim 1 , wherein the neutral position for the head-mounted display is with respect to a vertical plane.4. The method of claim 1 , wherein the neutral position for the head-mounted display is with respect to a horizontal plane.5. The method of claim 1 , wherein the neutral position for the head-mounted display is with respect to both a vertical plane and a horizontal plane.6. The method of claim 1 , wherein the one or more thresholds includes movement of the head-mounted display that exceeds a pre-determined distance from the neutral position.7. The method of claim 6 , wherein the one or more ...

SYSTEMS, METHODS, APPARATUSES AND DEVICES FOR DETECTING FACIAL EXPRESSION AND FOR TRACKING MOVEMENT AND LOCATION IN AT LEAST ONE OF A VIRTUAL AND AUGMENTED REALITY SYSTEM

Systems, methods, apparatuses and devices for detecting facial expressions according to EMG signals for a virtual and/or augmented reality (VR/AR) environment, in combination with a system for simultaneous location and mapping (SLAM), are presented herein. 1. A facial expression determination system for determining a facial expression on a face of a user comprising:an apparatus comprising a plurality of EMG (electromyography) electrodes configured for contact with the face of the user, said apparatus comprising an electrode interface;a mask which contacts an upper portion of the face of the user, said mask including an electrode plate attached to at least eight EMG electrodes and one reference electrode such that said EMG electrodes contact said upper portion of the face of the user, wherein said electrode interface is operatively coupled to said EMG electrodes and a hardware processor, said electrode interface for providing said EMG signals from said EMG electrodes to said hardware processor; and receive said EMG signals;', 'process said EMG signals to form processed EMG signals and to determine at least one feature of said EMG signals in said processed EMG signals;', 'determine a roughness of said processed EMG signals according to a defined window, said determining a roughness comprising calculating an EMG-dipole and determining a movement of said processed EMG signals according to said EMG-dipole, and performing a nonlinear transformation of said processed EMG signals to enhance high-frequency contents of said processed EMG signals;', 'classify, using a classifier, a facial expression of said user according to said at least one feature of said EMG signals and according to said roughness., 'a computational device configured to receive a plurality of EMG signals from said EMG electrodes, and comprising said hardware processor and a memory having instructions thereon operable by said hardware processor to cause the computational device to2. The system of claim 1 , ...

Multi-human tracking system and method with single kinect for supporting mobile virtual reality application

The invention discloses a multi-human tracking system and method with single Kinect for supporting mobile virtual reality applications. The system can complete the real-time tracking of users occluded in different degrees with a single Kinect capture device to ensure smooth and immersive experience of players. The method utilizes the principle that the user's shadow is not occluded when the user is occluded under certain lighting conditions, and converts the calculation of the motion of the occluded user into a problem of solving the movement of the user's shadow, and can accurately detect the position of each user, rather than just predicting the user's position, thereby actually realizing tracking.

Position Determination and Alignment of a Virtual Reality Headset and Fairground Ride with a Virtual Reality Headset

A method for determining a position and to a method for aligning at least one virtual reality headset in amusement rides. The virtual reality headset is a mobile virtual reality headset and has at least one receiver or at least one apparatus. The receiver receives a position signal of a position transmitter as a received signal, and the apparatus receives an alignment signal of an alignment transmitter. The disclosure additionally relates to an amusement ride with which a method according to the disclosure can be carried out. 1. A method for determining a position of a virtual reality headset in a vehicle moving along a travel stretch of a fairground ride , comprising:receiving a passenger on whom the virtual reality headset is placed during a trip with the vehicle,generating, during an operation of the fairground ride in an actual reality, a virtual reality corresponding to the fairground ride with the vehicle, wherein the virtual reality is represented on the virtual reality headset,wherein the fairground ride comprises a sensor from which a position signal emanates,wherein the virtual reality headset is a mobile virtual reality headset,wherein the virtual reality headset comprises a receiver which evaluates the position signal of the sensor for the determination of the position of the virtual reality headset in the vehicle relative to the sensor,and the virtual reality headset comprises a communication interface via which a data connection takes place between the virtual reality headset and a data processing device of the fairground ride, wherein the communication interface is detachable, wireless, or a combination thereof.2. The method according to claim 1 , further comprising determining the position and an alignment of the virtual reality headset by utilizing a motion capture system.3. The method according to claim 1 , wherein the sensor is a position sensor and an alignment sensor.4. The method according to claim 1 , wherein the fairground ride comprises at ...

NAVIGATION METHODS FOR THREE DIMENSIONAL (3D) MODELS IN MIXED REALITY (MR) ENVIRONMENTS

A mixed reality (MR) system that implements navigation methods for 3D models in MR environments. The navigation methods allow a user to navigate themselves about a 3D model (such as an MR environment including one or more buildings/structures) while in an MR environment. The navigation methods include an elevator feature or navigation mode that allows the user to move accurately between “floors” of the 3D model used to provide the MR environment (move to a next upper or lower floor from their present position). The navigation techniques include a jump feature or navigation mode. The jump feature allows the user to move to any point in an MR environment created using a 3D model that they can presently see in the display of the MR environment or a location that may be partially obscured from the user's view. 1. A system for providing a mixed reality (MR) environment with enhanced user navigation , comprising:a display element;memory storing a three dimensional (3D) model; anda processor running a navigation application and operating the display element to display a first image generated from the 3D model based on a first user location within the 3D model,wherein the navigation application receives a navigation request,wherein the navigation application processes the 3D model using the navigation request to calculate a second user location within the 3D model that differs from the first user location,wherein the processor operates the display element to display a second image generated from the 3D model based on the second user location within the 3D model,wherein the navigation request includes an elevator navigation request along with an upward navigation request or a downward navigation request,wherein, when the navigation request includes the upward navigation request, the second user location is on a higher horizontal surface above the first user location in the 3D model,wherein, when the navigation request includes the downward navigation request, the second user ...

UNDERWATER VIRTUAL REALITY SYSTEM

Embodiments are directed to virtual reality methods and/or apparatus for providing a user with an underwater virtual reality experience that causes the user to experience the virtual reality environment, and to interact with the virtual reality environment in addition to experiencing the sensations of the real underwater environment. 1. A method of providing a user with an underwater virtual reality experience , the method comprising:providing a user equipped with an underwater breathing apparatus and a virtual reality system located in an underwater environment with a virtual reality environment modeling an underwater setting, the underwater setting being provided by a virtual reality system comprising a computer or mobile computing device connected to a head-mounted display system, and at least one sensor for identifying the position of the user's head and at least one sensor for detecting movement of the user's body, the virtual reality system inhibiting visual perception by the user of items outside of the virtual reality environment;dynamically modifying the virtual underwater setting in relation to the user's body and audio to simulate user interaction with the virtual underwater environment.2. A virtual reality apparatus for providing a user with an underwater virtual reality experience , the apparatus comprising:an underwater breathing apparatus; anda computer or mobile computing device implemented virtual reality system configured to present to the user a virtual reality environment modeling an underwater setting, the virtual reality system inhibiting visual perception by the user of items outside of the virtual reality environment, wherein said computer or mobile computing device implemented virtual reality system includes a head-mounted display system, such display system worn by the user including at least one sensor to identify at least one position of the user's head, and movement of the user's body which corresponds with the movement of the ...

VIRTUAL REALITY IMMERSION WITH AN ARCHITECTURAL DESIGN SOFTWARE APPLICATION

A virtual reality system can comprise a virtual reality headset, one or more position tracking sensors, and a computerized architectural design environment. The position tracking sensors can identify a relative location of the user with respect to a physical space. The computerized architectural design environment can comprise a computer-implemented method for displaying a three-dimensional view of an architectural design to a user through the one or more virtual reality components. The method can comprise receiving a communication from the one or more position tracking sensors, rendering at least a portion of an architectural design file, and transmitting to the virtual reality headset the rendered portion of the architectural design file. 1. A virtual reality module , comprising:a display screen;one or more position tracking sensors, wherein the one or more position tracking sensors identify a relative location of a user with respect to a physical space; and display a three-dimensional image of an architectural design on the display screen, wherein the architectural design comprises a design for a particular physical space;', 'display a starting point for the user within the three-dimensional image, wherein the starting point corresponds to a first bookmark corresponding to a pre-determined virtual physical position and virtual view direction within the displayed architectural design;', 'receive indicia from one or more position tracking sensors on the virtual reality module that indicate a change in physical position and orientation of the display screen within the physical space;', 'display a new virtual physical position and a new virtual view corresponding to the first bookmark within the displayed three-dimensional image of the architectural design;', 'receive additional indicia from the user the virtual reality module; and', 'display a new virtual physical position and a new virtual view corresponding to a second bookmark within the displayed three- ...

Animated faces using texture manipulation

A method and system is provided to create animated faces using texture manipulation. A face template is provided to enable a user to define features of the face. A composite face is created from multiple layers that include a bottom layer, an animation layer, and a static layer. The composite face is animated by selectively animating one or more of the layers.

CONTENT POSITIONING IN EXTENDED REALITY SYSTEMS

Methods, devices, and apparatuses are provided to facilitate a positioning of an item of virtual content in an extended reality environment. For example, a placement position for an item of virtual content can be transmitted to one or more of a first device and a second device. The placement position can be based on correlated map data generated based on first map data obtained from the first device and second map data obtained from the second device. In some examples, the first device can transmit the placement position to the second device. 1. An apparatus comprising:a memory; and obtain, from a first device, first map data;', 'obtain, from a second device, second map data;', 'produce correlated map data based on the first map data and the second map data; and', 'transmit a placement position for an item of virtual content in an extended reality environment to one or more of the first device and the second device, the placement position being based on the correlated map data., 'a processor coupled to the memory, the processor configured to2. The apparatus of claim 1 , wherein the apparatus is a server in communication with the first device and the second device.3. The apparatus of claim 1 , wherein the apparatus is the first device claim 1 , and wherein the placement position is transmitted to the second device.4. The apparatus of claim 3 , wherein the placement position is transmitted to the second device via a server.5. The apparatus of claim 3 , wherein the processor is further configured to receive a user input to invoke a placement of the item of virtual content.6. The apparatus of claim 5 , wherein the user input is a hand gesture.7. The apparatus of claim 6 , wherein the hand gesture is received via a touch-sensitive surface.8. The apparatus of claim 7 , wherein the user input is associated with a position in the extended reality environment and the placement position is further based on the user input.9. The apparatus of claim 8 , wherein the placement ...

Dynamic Entering and Leaving of Virtual-Reality Environments Navigated by Different HMD Users

Systems and methods for processing operations for head mounted display (HMD) users to join virtual reality (VR) scenes are provided. A computer-implemented method includes providing a first perspective of a VR scene to a first HMD of a first user and receiving an indication that a second user is requesting to join the VR scene provided to the first HMD. The method further includes obtaining real-world position and orientation data of the second HMD relative to the first HMD and then providing, based on said data, a second perspective of the VR scene. The method also provides that the first and second perspectives are each controlled by respective position and orientation changes while viewing the VR scene. 1. A computer-implemented method for processing operations for head mounted display (HMD) users to join virtual reality (VR) scenes , comprising:providing a first perspective of a VR scene to a first HMD of a first user;receiving an indication that a second user of a second HMD is requesting to join the VR scene provided to the first HMD;obtaining real-world position and orientation data of the second HMD relative to the first HMD; andproviding, based on the real world position and orientation data, a second perspective of the VR scene in response to the request to join the VR scene by the second user;wherein the first perspective and the second perspective of the VR scene are each controlled by respective position and orientation changes while viewing the VR scene.2. The computer-implemented method as recited in claim 1 , wherein the first user of the first HMD and the second user of the second HMD are co-located in a real-world location.3. The computer-implemented method as recited in claim 1 , wherein the first user of the first HMD controls a progression of the VR scene claim 1 , the progression of the VR scene depending upon changes in real-world position and orientation of the first user while viewing the VR scene.4. The computer-implemented method as ...

DISPLAY CONTROL METHOD AND SYSTEM FOR EXECUTING THE DISPLAY CONTROL METHOD

A display control method for execution by a system including a head-mounted device. The display control method includes generating virtual space data for defining a three-dimensional virtual space. The display control method further includes displaying a visual-field image on the head-mounted device based on a visual field of the virtual space data. The display control method further includes updating the visual-field image in response to a detected movement of the head-mounted device exceeding a threshold. Updating the visual-field image includes changing a scale of an object in the virtual space by adjusting an angular range of the visual-field image. 19-. (canceled)10. A method , comprising:defining a three-dimensional virtual space including a virtual camera;displaying a visual-field image on a head-mounted device based on a visual field of the virtual camera; andupdating the visual-field image in response to a detected movement of the head-mounted device; changing a scale of the virtual camera in response to a predetermined condition is satisfied;changing a rendering range of the virtual camera in response to changing a scale of the virtual camera;updating the visual-field image in response to changing a rendering range of the virtual camera.11. The method according to claim 10 , defining a center position of the virtual camera in the three-dimensional virtual space;', 'defining a position of an image acquisition unit of the virtual camera for defining the rendering range in the three-dimensional virtual space;', 'changing a position of the image acquisition unit in response to changing a scale of the virtual camera;', 'correcting the center position of the virtual camera for remaining the position of the image acquisition unit in a visual-axis direction of the virtual camera constant during the updating of the visual-field image., 'further comprising;'}12. The method according to claim 10 , further comprising;defining a center position of the virtual camera in ...

CONTEXTUAL AUGMENTED REALITY DEVICES COLLABORATION

A first display device determines a first device configuration of the first display device and a first context of a first AR application operating on the first display device. The first display device connects to a second display device and determines a second device configuration of the second display device and a second context of a second AR application operating on the second display device. The first display device generates a first collaboration configuration for the display device and a second collaboration configuration for the second display device. The first and second collaborations are based on the first device configuration, the first context, the second device configuration, and the second context. The second display device modifies an operation of the second AR application based on the second collaboration configuration. The first display device modifies an operation of the first AR application based on the first collaboration configuration. 1. A first display device comprising:one or more hardware processors; anda memory storing instructions that, when executed by the one or more hardware processors, configure the first display device to perform operations comprising:determining a first device configuration of the first display device and a first context of a first AR application operating on the first display device;connecting to a second display device;determining a second device configuration of the second display device and a second context of a second AR application operating on the second display device;generating a first collaboration configuration for the first display device and a second collaboration configuration for the second display device, the first and second collaboration configurations being generated based on the first device configuration, the first context, the second device configuration, and the second context, the second display device being configured to modify an operation of the second AR application based on the second ...

AUGMENTED REALITY FAILSAFE MODE

A device executes an augmented reality (AR) application at the device. The device detects a failure of the AR application and enters a failsafe mode in response to detecting the failure of the AR application. The failsafe mode includes a minimum set of AR content and AR application functionalities relative to the full set of AR content and AR application functionalities when the AR application is being executed. 1. A device comprising:one or more hardware processors; anda memory storing instructions that, when executed by the one or more hardware processors, configure the device to perform operations comprising:executing an augmented reality (AR) application in the device;detecting that an operation of the AR application is crashing; andentering a failsafe mode of the AR application in response to detecting that the operation of the AR application is crashing, the failsafe mode enabling the AR application to operate with a basic set of AR content and AR application functionalities relative to a full set of AR content and AR application functionalities when the AR application is being executed.2. The device of claim 1 , further comprising:a sensor;a display,wherein operating the AR application further comprises:identifying a physical object with the sensor;rendering first AR content in the display obscuring a portion of the physical object, the first AR content including a three-dimensional model of a first virtual object associated by a database with the identified physical object;adjusting a rendering of the first AR content based on a relative position between the device and the physical object; andgenerating a first virtual user interface in the display, the first virtual user interface providing a first set of functions to control the first AR content.3. The device of claim 2 , wherein the operations further comprise:replacing the first virtual user interface in the display with a second virtual user interface, the second virtual user interface providing a ...

METHOD AND SYSTEM FOR IMAGE PROCESSING

Provided is an image processing method and system. More particularly, provided are an image processing method and system that may map a facial expression of a user, a specific event, etc., to a character in real time by mapping an object such as a face of a user included in an input image to a virtual object such as a face of a specific character, and by applying an event corresponding to a combination of components defined for the object to the virtual object, and a non-transitory computer-readable recording medium that, when executed by a processor, cause the processor to implement the image processing method in conjunction with a computer device. 1. A non-transitory computer-readable recording medium storing instructions that , when executed by a processor , cause the processor to implement an image processing method in conjunction with a computer , the image processing method comprising:extracting landmark information of an object included in an input image;mapping the object included in the input image to a virtual object based on the landmark information;determining values of a plurality of components with respect to the object based on the landmark information;determining an event corresponding to a combination of the determined values of the plurality of components; andapplying the determined event to the virtual object.2. The non-transitory computer-readable recording medium of claim 1 , wherein the plurality of components comprises a plurality of landmark components included in the object and a plurality of angular components representing a rotation of the object.3. The non-transitory computer-readable recording medium of claim 2 , wherein:the image processing method further comprises setting and managing a plurality of shape values representing two or more different shapes for one or more of the plurality of landmark components; andthe determining a values of the plurality of components comprises determining a shape value corresponding to a shape of a ...

LAST-LEVEL PROJECTION METHOD AND APPARATUS FOR VIRTUAL AND AUGMENTED REALITY

An apparatus and method for efficient image reprojection in a virtual reality system. For example, one embodiment of an apparatus comprises: a sensor interface to collect motion data from one or more sensors during a virtual reality session; graphics circuitry to execute graphics program code to render an image frame during the virtual reality session; a processor to generate motion transform data using the motion data, the motion transform data specifying how the image frame is to be adjusted prior to display; a reprojection engine to perform an in-line reprojection of the frame using the motion transform data to generate a reprojected image frame; and display circuitry to display the reprojected frame. 1. An apparatus comprising:an interface to couple to a memory to receive a first image frame rendered at a first point in time by a graphics processing unit (GPU) in response to execution of a virtual reality (VR) application; andprocessing circuitry coupled to the interface to access the first image frame, the processing circuitry to perform a reprojection operation comprising a transformation of the first image frame in accordance with view data indicating a current view of a user captured at a second point in time subsequent to the first point in time to produce a reprojected image frame.2. The apparatus of wherein the processing circuitry is to perform performs lens distortion correction operations on the first image frame.3. The apparatus of wherein the view data comprises sensor data generated by head and/or eye tracking sensors.4. The apparatus of wherein the view data indicates a new orientation of a user's head and/or eyes at the second point in time.5. The apparatus of wherein the processing circuitry and GPU are integrated within a single chip.6. The apparatus of further comprising:an interconnect to couple the interface to the memory and/or the GPU.7. The apparatus of wherein the interconnect comprises an on-chip system fabric.8. The apparatus of wherein ...

SYSTEM FOR SIMULATING A VIRTUAL FITNESS PARTNER

A method for execution by a mobile communications device includes generating display data, where a display device of eyewear, worn by a user performing a fitness activity, displays a virtual fitness partner in accordance with the display data to virtually superimpose the virtual fitness partner upon physical surroundings viewed by the user through at least one translucent lens of the eyewear to simulate performance of the fitness activity by the virtual fitness partner along a virtual transit within a physical vicinity of the user throughout the transit of the user. The virtual fitness partner is displayed in accordance with perspective data generated based on the user's transit velocity and head orientation, determined based on sensor data collected by at least one motion sensor. The virtual fitness partner is further depicted to perform one of a plurality of fitness partner actions in accordance with event trigger monitoring data. 1. Eyewear worn by a user while performing a fitness activity , comprising:at least one sensor;at least one lens, wherein a surface of the at least one lens includes a display region; a transit velocity vector corresponding to a transit of the user during the fitness activity that indicates a transit pace and a transit direction; and', 'head tracking data corresponding to head movement of the user that indicates a head orientation direction relative to the transit direction;, 'a motion data generator that processes input data collected by the at least one sensor to generate motion data, wherein the motion data indicatesa communication interface that receives fitness partner profile data, wherein the fitness partner profile data is configured by the user by utilizing a client application associated with the eyewear via input to an interactive interface displayed by a client device associated with the user; appearance data corresponding to an appearance of the virtual fitness partner, wherein the appearance data is determined based on the ...

Robotic Avatar Apparatus and System

A robotic avatar apparatus is provided. The robotic avatar apparatus includes a robotic avatar unit. The robotic avatar unit includes a camera, a processor and a unit signal transceiver. The camera is connected to the processor. The processor is connected to the unit signal transceiver. The robotic avatar apparatus also includes a user interface device. The user interface device includes a user signal transceiver and a display. The robotic avatar unit is designed to connect to the user interface device via a connection between the unit signal transceiver and the user signal transceiver. 1) A robotic avatar apparatus , comprising:a robotic avatar unit;the robotic avatar unit comprising a camera;the robotic avatar unit comprising a processor;the robotic avatar unit comprising a unit signal transceiver;the camera in operable connection with the processor;the processor in operable connection with the unit signal transceiver;a user interface device;the user interface device comprising a user signal transceiver;the user interface device comprising a display;the robotic avatar unit configured to connect to the user interface device via a connection between the unit signal transceiver and the user signal transceiver.2) The robotic avatar apparatus of claim 1 , wherein the robotic avatar unit further comprises a motor claim 1 , wherein the motor is configured to manipulate the position of the camera.3) The robotic avatar apparatus of claim 1 , wherein the robotic avatar unit further comprises a unit microphone in operable connection with the unit signal transceiver.4) The robotic avatar apparatus of claim 1 , wherein the robotic avatar unit further comprises a unit speaker.5) The robotic avatar apparatus of claim 1 , wherein the user interface device comprises a user microphone in operable connection with the user signal transceiver.6) The robotic avatar apparatus of claim 1 , wherein the user interface device comprises a user speaker.7) The robotic avatar apparatus of claim ...

Head pose mixing of audio files

Examples of wearable devices that can present to a user of the display device an audible or visual representation of an audio file comprising a plurality of stem tracks that represent different audio content of the audio file are described. Systems and methods are described that determine the pose of the user; generate, based on the pose of the user, an audio mix of at least one of the plurality of stem tracks of the audio file; generate, based on the pose of the user and the audio mix, a visualization of the audio mix; communicate an audio signal representative of the audio mix to the speaker; and communicate a visual signal representative of the visualization of the audio mix to the display.

DATA PROCESSING SYSTEMS

In a data processing system that includes a field sequential colour display, when displaying a frame that is to be transformed based on a predicted view orientation on the field sequential colour display, each colour field to be displayed on the display for the frame is transformed based on a predicted view orientation for that particular colour field, such that each colour field will be subjected to a different view orientation transformation to the other colour fields for the frame. The so-transformed colour fields are then displayed sequentially on the display to display the frame. 1. A method of operating a data processing system , the data processing system comprising:a field sequential colour display operable to display frames for display;a producer processor operable to generate frames for display on the display;a display processor operable to provide frames generated by the producer processor to the display for display; anda transformation engine operable to transform a frame generated by the producer processor based on a view orientation to provide a view orientation transformed frame for display on the display;the method comprising:when displaying a frame generated by the producer processor that is to be transformed based on a predicted view orientation on the field sequential colour display:the transformation engine transforming each colour field to be displayed on the display for the frame based on a predicted view orientation for that particular colour field, such that each colour field will be subjected to a different view orientation transformation to the other colour fields for the frame; anddisplaying the so-transformed colour fields sequentially on the display to display the frame.2. The method of claim 1 , wherein the producer processor is a graphics processor.3. The method of claim 1 , wherein the predicted view orientation for a colour field is based on a sensed view orientation for the display and the time when the colour field will be ...

Web-like hierarchical menu display configuration for a near-eye display

Technology is described for web-like hierarchical menu interface which displays a menu in a web-like hierarchical menu display configuration in a near-eye display (NED). The web-like hierarchical menu display configuration links menu levels and menu items within a menu level with flexible spatial dimensions for menu elements. One or more processors executing the interface select a web-like hierarchical menu display configuration based on the available menu space and user head view direction determined from a 3D mapping of the NED field of view data and stored user head comfort rules. Activation parameters in menu item selection criteria are adjusted to be user specific based on user head motion data tracked based on data from one or more sensors when the user wears the NED. Menu display layout may be triggered by changes in head view direction of the user and available menu space about the user's head.

IMAGE PROCESSING APPARATUS, IMAGE PROCESSING METHOD, AND STORAGE MEDIUM

An image processing apparatus is described comprising a first acquisition unit configured to acquire image data from an image capturing apparatus and a second acquisition unit configured to acquire display installation information indicating a configuration of one or more display screens in a display installation to be simulated. A generation unit is provided to generate, based on the display installation information and the acquired image data, a simulation image that simulates display of the image on the display installation. A display control unit is provided to display the simulation image on a display device. 1. An image processing apparatus comprising:a first acquisition unit configured to acquire image data from an image capturing apparatus;a second acquisition unit configured to acquire display installation information indicating a configuration of one or more display screens in a display installation to he simulated;a generation unit configured to generate, based on the display installation information and the acquired image data, a simulation image that simulates display of the image on the display installation; anda display control unit configured to display the simulation image on a display device.2. The image processing apparatus according to claim 1 , wherein the generation unit is configured to simulate a display installation that has a curved screen.3. The image processing apparatus according to claim 1 , wherein the generation unit is configured to simulate a display installation that includes a combination of a plurality of fiat display screens.4. The image processing apparatus according to claim 1 , wherein the generation unit is configured to generate claim 1 , as the simulation image claim 1 , an image based on a viewing angle at which the one or more display screens display an image to a viewer.5. The image processing apparatus according to claim 1 , wherein the generation unit is configured to generate claim 1 , by performing rendering claim 1 ...

SYSTEM FOR DETERMINING ANATOMICAL FEATURE ORIENTATION

The systems and methods disclosed herein provide determination of an orientation of a feature towards a reference target. As a non-limiting example, a system consistent with the present disclosure may include a processor, a memory, and a single camera affixed to the ceiling of a room occupied by a person. The system may analyze images from the camera to identify any objects in the room and their locations. Once the system has identified an object and its location, the system may prompt the person to look directly at the object. The camera may then record an image of the user looking at the object. The processor may analyze the image to determine the location of the user's head and, combined with the known location of the object and the known location of the camera, determine the direction that the user is facing. This direction may be treated as a reference value, or “ground truth.” The captured image may be associated with the direction, and the combination may be used as training input into an application. 1. An apparatus for determining an orientation of an anatomical feature , comprising: identify an anatomical feature of a subject within an environment; and', 'determine a location of the anatomical feature of the subject within the environment; and, 'anatomical feature location determination logic to prompt the subject to orient the anatomical feature toward a target within the environment;', 'determine a location of the target within the environment; and', 'determine an orientation of the anatomical feature based on the location of the anatomical feature and the location of the target., 'orientation reference logic to2. The apparatus of claim 1 , wherein the orientation reference logic is further to cause a camera to capture an image of the anatomical feature of the subject oriented toward the location of the target.3. The apparatus of claim 2 , wherein the orientation reference logic is further to:determine metadata based upon the determined orientation of ...

WEARABLE EYE TRACKING SYSTEM WITH SLIPPAGE DETECTION AND CORRECTION

Apparatus, systems and methods configured for eye tracking with slippage detection and correction are disclosed. In one example, a method comprises: obtaining a first image and a second image of a person's eye, with an imaging sensor; determining a position relationship between an eyeball center of the eye and the imaging sensor based on the first image and the second image. 1. A method comprising:obtaining a first image and a second image of a person's eye, with an imaging sensor;determining a position relationship between an eyeball center of the eye and the imaging sensor based on the first image and the second image.2. The method of claim 1 , further comprising determining a relative movement of the imaging sensor with respect to the eyeball center based on the position relationship.3. The method of claim 1 , wherein the first image and the second image are different.4. The method of claim 1 , wherein determining the position relationship comprises extracting contours of a pupil or a corneal limbus from the first image and the second image claim 1 , and determining the position relationship based on the contours.5. The method of claim 1 , wherein the first image and the second image are obtained during an intentional or random movement of the eye.6. The method of claim 1 , further comprising determining an orientation relationship between the imaging sensor and the person's head.7. The method of claim 6 , wherein determining the orientation relationship comprises:obtaining a first set of gaze vectors while the person's head rotates about a first axis and the eye maintains a first gaze point;obtaining a second set of gaze vectors while the person's head rotates about a second axis and the eye maintains a second gaze point, or obtaining a third gaze vector while the eye maintains a third gaze point;determining the orientation relationship based on the first set of gaze vectors, and based on the second set of gaze vectors or the third gaze vector.8. The method of ...

METHOD, ELECTRONIC DEVICE, AND STORAGE MEDIUM FOR INFORMATION PROCESSING

A method for information processing includes displaying a display object at a first display location in a first space area of a physical scene; detecting a first user-related event; and in response to the first user-related event satisfying a preset condition, performing a display operation in connection to the display object with respect to a second space area of the physical scene. 1. A method for information processing , comprising:displaying a display object at a first display location in a first space area of a physical scene;detecting a first user-related event; andin response to the first user-related event satisfying a preset condition, performing a display operation in connection to the display object with respect to a second space area of the physical scene.2. The method according to claim 1 , wherein the preset condition comprises:a separation distance between the first space area of the physical scene and the second space area of the physical scene exceeds a predetermined threshold distance.3. The method according to claim 2 , wherein in response to the first user-related event satisfying the preset condition claim 2 , performing the display operation in connection to the display object with respect to the second space area of the physical scene includes: 'displaying the display object at a second display location in the second space area.', 'responsive to the separation distance between the first space area of the physical scene and the second space area of the physical scene as determined exceeding the predetermined threshold distance4. The method according to claim 2 , wherein in response to the first user-related event satisfying the preset condition claim 2 , performing the display operation in connection to the display object with respect to the second space area of the physical scene includes: generating prompt information, the prompt information prompting whether to display the display object at a second display location in the second space area ...