СПОСОБ, УСТРОЙСТВО И СИСТЕМА ДЛЯ ДОСТУПА К УСТАНАВЛИВАЕМОМУ В ГЛАЗ УСТРОЙСТВУ С ПОМОЩЬЮ ПОЛЬЗОВАТЕЛЬСКОГО ИНТЕРФЕЙСА

Группа изобретений относится к области медицины, а именно к офтальмологии. Для управления процессом аккомодации обеспечивают доступ к способному к аккомодации устанавливаемому в или на глаз устройству с помощью пользовательского интерфейса вспомогательного устройства. В одном варианте осуществления пользовательский интерфейс предлагает рекомендации пользователю устанавливаемого в глаз устройства выполнять различные зрительные действия, причем устанавливаемое в глаз устройство принимает от вспомогательного устройства сообщения, указывающие на соответствующие моменты зрительных действий. На основании сообщений устанавливаемое в глаз устройство генерирует конфигурационную информацию, указывающую на соответствие соответствующих состояний устанавливаемого в глаз устройства соответствующим характеристикам зрительных действий. В другом варианте осуществления на основании конфигурационной информации определяют рабочие режимы устанавливаемого в глаз устройства. Группа изобретений облегчает доступ ...

НЕВИЗУАЛЬНАЯ ОБРАТНАЯ СВЯЗЬ ВИЗУАЛЬНОГО ИЗМЕНЕНИЯ В СПОСОБЕ И УСТРОЙСТВЕ ОТСЛЕЖИВАНИЯ ВЗГЛЯДА

Изобретение относится к устройству и способу вычисления, обеспечивающим невизуальную обратную связь. Технический результат заключается в обеспечении невизуальной обратной связи для предупреждения наблюдателя устройства отображения о том, что было запущено визуальное изменение, с использованием данных отслеживания взгляда от системы отслеживания взгляда для определения того, что наблюдатель меняет местоположение взгляда. Такой результат достигается за счет того, что на основе определения того, что наблюдатель меняет местоположение взгляда, запускается визуальное изменение и наблюдателю обеспечивается невизуальная обратная связь, указывающая на запуск визуального изменения. Если в пределах заранее определенного временного интервала принимается ввод отмены изменения, визуальное изменение не отображается. Если в пределах временного интервала не принимается ввод отмены изменения, визуальное изменение отображается посредством устройства отображения. 3 н. и 16 з.п. ф-лы, 8 ил.

ИСПОЛЬЗОВАНИЕ ВТОРОГО ЭКРАНА В КАЧЕСТВЕ ПЕРСОНАЛЬНОГО ПРОЕКЦИОННОГО ДИСПЛЕЯ НА ОЧКАХ

Изобретение относится к способу расширения области обзора дисплейного устройства для отображения контента. Технический результат заключается в расширении ограниченной экранной области, имеющейся в вычислительных устройствах, в целях обеспечения возможности работы в многозадачном режиме для пользователей. Сущность изобретения заключается в том, что способ включает этапы, на которых определяют наличие окулярного вычислительного устройства вблизи первичного вычислительного устройства. Первичное вычислительное устройство имеет первичный дисплей, который выполнен с возможностью отображения контента. Окулярное вычислительное устройство включает вторичный дисплей. Окулярное вычислительное устройство соединяют с первичным вычислительным устройством. Определяют направление обзора для окулярного вычислительного устройства относительно первичного дисплея. Отображают на вторичном дисплее дополнительный контент, который является дополнением к контенту, отображаемому на первичном дисплее. Вторичный дисплей ...

СПОСОБ И УСТРОЙСТВО УПРАВЛЕНИЯ ДИСПЛЕЕМ

Изобретение относится к способу управления экранным дисплеем. Технический результат заключается в расширении арсенала средств. Способ управления экранным дисплеем (550) электронного устройства (500) включает в себя этапы, на которых определяют (110) посредством электронного устройства разрешение камеры (720) путем выбора одного из высокого разрешения и низкого разрешения, распознают лицо пользователя с использованием камеры, работающей с определенным разрешением (120), и управляют экранным дисплеем посредством распознавания лица. При этом, когда камера работает с низким разрешением, распознают позицию лица пользователя, а когда камера работает с высоким разрешением, получают позицию взгляда пользователя, и при этом определение разрешения содержит этапы, на которых определяют состояние питания электронного устройства; и выбирают одно из высокого разрешения или низкого разрешения в зависимости от состояния питания электронного устройства. 2 н. и 16 з.п. ф-лы, 12 ил.

СПОСОБ ВЫПОЛНЕНИЯ ФУНКЦИИ УСТРОЙСТВА И УСТРОЙСТВО ДЛЯ ВЫПОЛНЕНИЯ СПОСОБА

Изобретение относится к области реализации на мобильных устройствах приложений, услуг и содержимого контента. Технический результат – обеспечение гарантированной доступности приложений, услуг и содержимого контента на мобильных устройствах. Предложен способ выполнения функции устройства на основании информации движения устройства в заданном рабочем режиме (например, активном или режиме ожидания) устройства и устройство для выполнения способа. Устройство включает в себя датчик, выполненный с возможностью обнаружения перемещения устройства в режиме ожидания (или активном режиме) устройства; хранилище, выполненное с возможностью сохранения информации движения на основе информации, которая относится к перемещению, и по меньшей мере одного фрагмента информации функции, соответствующего информации движения; и процессор, выполненный с возможностью управления устройством для выполнения функции, соответствующей информации движения, в заданном рабочем режиме на основе информации, которая относится ...

УСТРОЙСТВО, СПОСОБ И СИСТЕМА ОБЕСПЕЧЕНИЯ УВЕЛИЧЕННОГО ДИСПЛЕЯ С ИСПОЛЬЗОВАНИЕМ ШЛЕМА-ДИСПЛЕЯ

Шлем-дисплей содержит логику формирования, прозрачный дисплей и модуль презентации. Прозрачный дисплей выполнен с возможностью предоставления пользователю шлема-дисплея просматривать по меньшей мере один виртуальный дисплей и физический дисплей в сцене реального мира. Указанный по меньшей мере один виртуальный дисплей представлен на прозрачном дисплее в качестве дополнительного объекта в сцене реального мира, и указанный по меньшей мере один виртуальный дисплей используется в соединении с физическим дисплеем для представления сформированной компьютером информации на расширенном рабочем столе с визуальной точки зрения пользователя. Технический результат заключается в обеспечении мобильности развертывания одного или более дисплеев для портативных устройств для вывода на экран сформированной компьютером информации в соединении с физическим дисплеем вычислительной системы. 3 н. и 15 з.п. ф-лы, 5 ил.

БИОСЕНСОРЫ, КОММУНИКАТОРЫ И КОНТРОЛЛЕРЫ ДЛЯ МОНИТОРИНГА ДВИЖЕНИЯ ГЛАЗ И СПОСОБЫ ИХ ПРИМЕНЕНИЯ

Группа изобретений относится к области медицины. Варианты устройств включают устройство, приспособленное для ношения на голове человека; источник излучения для направления излучения на глаза человека во время ношения устройства; первый и второй световоды для передачи изображений, содержащие первые концы, закрепленные на устройстве, первый световод для передачи изображений, расположенный для рассмотрения первого глаза человека, носящего устройство, второй световод для передачи изображений, расположенный для рассмотрения второго глаза человека, носящего устройство; и камеру, связанную с указанными первым и вторым световодами для получения изображений первого и второго глаза, камера включает детектор типа ПЗС или КМОП, и в котором изображения от первого и второго световодов для передачи изображений одновременно принимаются на активную область детектора. При этом способ мониторинга движения глаз заключается в подаче излучения от источника излучения на глаз; мониторинге движения глаза с помощью ...

Система и способ (варианты) управления интерфейсом дополненной реальности

Изобретение относится к системам и способам управления интерфейсом дополненной реальности (AR). Техническим результатом является расширение функциональных возможностей системы дополненной реальности путем возможности управления интерфейсом AR с помощью биоэлектрических сигналов, регистрируемых с помощью контактных линз, повышение компактности носимой части системы управления. Система управления содержит контактную линзу, блок проецирования интерфейса дополненной реальности в глаз пользователя, сенсор для регистрации биоэлектрических сигналов пользователя, блок обработки и блок управления, выполненный с возможностью изменения интерфейса дополненной реальности в соответствии с управляющими сигналами. Указанный сенсор содержит, по меньшей мере, два основных электрода, интегрированных в указанную контактную линзу. Блок обработки выполнен с возможностью формирования управляющих сигналов на основе данных этого сенсора, отражающих изменение во времени разности потенциалов между его электродами ...

УСТРОЙСТВО ПОЛЬЗОВАТЕЛЬСКОГО ТЕРМИНАЛА И СПОСОБ УПРАВЛЕНИЯ ИМ

... 1. Карманное пользовательское устройство, содержащее:первый пользовательский интерфейс;второй пользовательский интерфейс;воспринимающее устройство, которое воспринимает рабочее условие карманного пользовательского устройства; иконтроллер, который активирует упомянутый первый пользовательский интерфейс и деактивирует упомянутый второй пользовательский интерфейс, если рабочее условие соответствует первому условию, и активирует упомянутый второй пользовательский интерфейс и деактивирует упомянутый первый пользовательский интерфейс, если рабочее условие соответствует второму условию,при этом рабочее условие является по меньшей мере одним из внешней рабочей среды карманного пользовательского устройства и движения карманного пользовательского устройства.2. Карманное пользовательское устройство по п. 1, дополнительно содержащее:третий пользовательский интерфейс,при этом контроллер активирует два из упомянутого первого пользовательского интерфейса, упомянутого второго пользовательского интерфейса ...

КОМПАКТНЫЙ НАГОЛОВНЫЙ ДИСПЛЕЙ С ОТСЛЕЖИВАНИЕМ ДВИЖЕНИЯ ГЛАЗ

... 1. Наголовный дисплей с отслеживанием движения глаз, содержащий:микродисплей для формирования изображения для просматривания пользователем, имеющий связанные с ним оптический путь дисплея и выходной зрачок;первую плоскость на микродисплее и вторую плоскость на выходном зрачке;датчик изображения для приема из второй плоскости оптического излучения, отраженного от глаза пользователя, имеющий связанный с ним оптический путь датчика; иоптическое устройство дисплея, находящееся в оптической связи с микродисплеем по оптическому пути дисплея и с датчиком изображения по оптическому пути датчика, причем оптическое устройство дисплея имеет выбранную поверхность, расположенную ближе всего к микродисплею и к датчику изображения, и оптическое устройство дисплея расположено относительно микродисплея и датчика изображения таким образом, что оптические пути дисплея и датчика изображения попадают на соответствующие разные части выбранной поверхности.2. Наголовный дисплей по п. 1, в котором оптические пути ...

Устройство определения координат линии взора наблюдателя в режиме реального времени

Изобретение относится к области физики, а именно к обработке данных, в частности к получению в режиме реального времени точных угловых координат линии взора наблюдателя относительно заданной точки на изображении и координат пересечения линии взора наблюдателя с плоскостью изображения. Устройство содержит оптические блоки для правого глаза и левого глаза наблюдателя, расположенные на одной плоскости таким образом, что их оптические оси параллельны и проходят в непосредственной близости от линии взора наблюдателя; устройство регулирования расстояния между упомянутыми оптическими блоками; контроллер регулирования расстояния между оптическими блоками; сенсоры определения движения головы; контроллер связи с внешними устройствами и внешние устройства. При этом каждый оптический блок состоит из оптической системы. Оптическая система состоит по меньшей мере из одной линзы, зеркала со спектральным делением каналов, устройства вывода визуальной информации и отслеживающего устройства на базе монокристальной ...

СИСТЕМА ПОДКЛЮЧЕНИЯ

ОТСЛЕЖИВАНИЕ ВЗЛЯДА ЧЕРЕЗ ОЧКОВУЮ ОПТИКУ

СИСТЕМА УВЕДОМЛЕНИЙ ТРАНСПОРТНОГО СРЕДСТВА, ВКЛЮЧАЮЩАЯ В СЕБЯ ПРОЗРАЧНЫЕ И ЗЕРКАЛЬНЫЕ УСТРОЙСТВА ОТОБРАЖЕНИЯ

ОСНОВАННОЕ НА ВЗГЛЯДЕ РАЗМЕЩЕНИЕ ОБЪЕКТОВ В СРЕДЕ ВИРТУАЛЬНОЙ РЕАЛЬНОСТИ

СИСТЕМА ОТОБРАЖЕНИЯ ВИРТУАЛЬНОЙ РЕАЛЬНОСТИ

... 1. Устройство, содержащее:процессор (216) изображений, сконфигурированный для приема информации (226) о положении первого глаза (234) на голове (236) пользователя (206) и информации (228) о положении второго глаза (240) на голове (236) пользователя (206);создания первого изображения (252) виртуальной реальности (204) для первого глаза (234) на основе информации (226) о положении первого глаза (234);создания второго изображения (254) виртуальной реальности (204) для второго глаза (240) на основе информации (228) о положении второго глаза (240); ипередачу первого изображения (252) и второго изображения (254) для отображения для пользователя (206).2. Устройство по п.1, дополнительно содержащее:контроллер (214), сконфигурированный для создания виртуальной реальности (204);средство (405) слежения за головой, сконфигурированное для отслеживания положения (264) головы (236) и получения информации (230) о положении головы (236); и,причем, процессор (216) изображений дополнительно сконфигурирован ...

СПОСОБ И УСТРОЙСТВО ОТОБРАЖЕНИЯ С ОПТИМИЗАЦИЕЙ ПЕРЕРАСПРЕДЕЛЕНИЯ ПИКСЕЛЕЙ

РОБОТИЗИРОВАННАЯ ХИРУРГИЧЕСКАЯ СИСТЕМЫ С УСОВЕРШЕНСТВОВАННЫМ УПРАВЛЕНИЕМ

... 1. Роботизированная хирургическая система (10), содержащая по меньшей мере одну роботизированную руку (11), действующую под управлением пульта (12) управления, предназначенного для хирурга, отличающаяся тем, что пульт (12) содержит отслеживающая систему (21) "айтрекинг" для распознавания направления взгляда хирурга и ввода команд, зависящих от направлений взгляда, которые были распознаны.2. Система по п.1, отличающаяся тем, что указанный пульт (12) содержит экран (22), имеющий по меньшей мере одну область (23) для просмотра рабочей зоны, причем указанная отслеживающая система (21) вырабатывает команду, запрещающую перемещение по меньшей мере одной роботизированной руки (11) при распознавании направления взгляда за пределы экрана или по меньшей мере указанной области (23) экрана.3. Система по п.1, отличающаяся тем, что указанный пульт (12) содержит экран (22), имеющий по меньшей мере одну область для просмотра рабочей зоны и интерфейс «человек-машина», который отображает на экране области ...

СПОСОБ И АППАРАТ УПРАВЛЕНИЯ ОТОБРАЖЕНИЕМ, ЭЛЕКТРОННОЕ УСТРОЙСТВО

СИСТЕМА И СПОСОБ ДЛЯ УДАЛЕННОГО ИЗМЕРЕНИЯ ОПТИЧЕСКОГО ФОКУСА

... 1. Измерительная система для удаленного измерения оптического фокуса удаленной системы формирования оптического изображения, при этом удаленная система формирования оптического изображения содержит объектив и чувствительную поверхность, причем измерительная система содержит:светопроекционное устройство (P) для проецирования световой тест-картины в оптическом излучении таким образом, чтобы тест-картина находилась в фокусе в известной фокальной плоскости (FPL1) перед объективом удаленной системы формирования оптического изображения,камеру (CAM), имеющую оптическую ось (AXCAM), которая между светопроекционным устройством (P) и известной фокальной плоскостью (FPL1), по меньшей мере, частично совпадает с оптической осью (AXP) светопроекционного устройства (P), для записи изображения отражения световой тест-картины на чувствительной поверхности удаленной системы формирования оптического изображения, иблок (IP) обработки изображений для определения резкости записанной световой тест-картины, отраженной ...

БИОСЕНСОРЫ, КОММУНИКАТОРЫ И КОНТРОЛЛЕРЫ ДЛЯ МОНИТОРИНГА ДВИЖЕНИЯ ГЛАЗ И СПОСОБЫ ИХ ПРИМЕНЕНИЯ

... 1. Устройство для мониторинга движения глаз человека, включающее устройство, приспособленное для ношения на голове человека; источник излучения для направления излучения на глаза человека во время ношения устройства; первый и второй световоды для передачи изображений, содержащие первые концы, закрепленные на устройстве, первый световод для передачи изображений, расположенный для рассмотрения первого глаза человека, носящего устройство, второй световод для передачи изображений, расположенный для рассмотрения второго глаза человека, носящего устройство; камеру, связанную с первым и вторым жгутами для получения изображений первого и второго глаза. 2. Устройство по п.1, в котором камера включает детектор типа CCD или CMOS и в котором изображения от первого и второго световодов для передачи изображений одновременно принимаются на активную область детектора. 3. Устройство по п.1, в котором источник света включает один или большее число инфракрасных источников излучения, сконфигурированных для ...

Vorrichtung zum Bestimmen eines Fixationspunktes

Vorrichtung zum Bestimmen eines Fixationspunktes (3) eines anthropomorph aufgebauten Auges (5) beim Betrachten eines Gegenstandes (1) mit einer Augenreferenzpunktsdetektionseinrichtung, mit der das Auge (5) innerhalb eines vorbestimmten Raumbereiches abbildbar ist, mit einem als Flächendetektor ausgebildeten Augenreferenzpunktsdetektor (6), mit dem ein zu dem Auge (5) in fester räumlicher Beziehung stehender Augenreferenzpunkt (49) detektierbar ist, mit einer Blickrichtungsdetektionseinrichtung, mit der eine Blickrichtung des Auges als Blickrichtungsfunktion bestimmbar ist, wobei die Blickrichtungsdetektionseinrichtung über einen als Flächendetektor ausgebildeten Blickrichtungsdetektor (9) verfügt, der mit einem in zwei überlagerbaren Richtungen schwenkbaren und neigbaren Schwenk/Neigeelement (11) auf das Auge (5) ausrichtbar ist und mit dem wenigstens die Pupille (13) sowie die unmittelbare Pupillenumgebung (14, 15) abbildbar ist, und wobei das Schwenk/Neigeelement (11) von einer mit Steuersignalen ...

Sichtlinienmessvorrichtung

Eine Sichtlinienmessvorrichtung beinhaltet eine Abbildungseinheit (110) und eine Sichtlinienmesseinheit (140). Die Abbildungseinheit beinhaltet einen variablen Belichtungspegel und ist konfiguriert, um ein Bild eines Subjekts aufzunehmen. Die Sichtlinienmesseinheit misst eine Sichtlinienrichtung des Subjekts basierend auf dem Bild, das durch die Abbildungseinheit aufgenommen wird. Die Abbildungseinheit ist konfiguriert, um kontinuierlich zwischen einer Aufnahme eines ersten Bilds (1101), das ein gesamtes Gesicht des Subjekts zeigt, mit einem ersten Belichtungspegel und einer Aufnahme eines zweiten Bilds (1102), das eine Bereich um die Augen des Subjekts herum zeigt, mit einem zweiten Belichtungspegel, der höher als der erste Belichtungspegel festgelegt ist, abzuwechseln. Die Sichtlinienmesseinheit ist konfiguriert, um eine Positionsabweichung, die mit einer Bewegung des Subjekts verknüpft ist, zwischen einem beliebigen Bild und einem nächsten Bild von dem ersten Bild und dem zweiten Bild ...

SYSTEME UND VERFAHREN FÜR EINEN AUFGABENBASIERTEN EINSTELLBAREN FOKUSABSTAND FÜR HEADS-UP-DISPLAYS

Es werden Systeme und Verfahren für die aufgabenbasierte Modifikation visueller Medien in einem Heads-Up-Display bereitgestellt. Ein beispielhaftes System umfasst eine Heads-Up-Display-Baugruppe, die so konfiguriert ist, dass sie von einem Benutzer getragen werden kann, wobei die Heads-Up-Display-Baugruppe eine Kopfhalterung und einen Präsentationsgenerator umfasst, der so konfiguriert ist, dass er dem Benutzer visuelle Medien anzeigt. Das beispielhafte System enthält ferner einen Speicher und einen Prozessor, der so konfiguriert ist, dass er mit dem Präsentationsgenerator und dem Speicher verbunden werden kann. Der Prozessor identifiziert eine aktuelle Aufgabe, die mit dem Benutzer assoziiert ist, und erhält einen Fokusabstand, bei dem die visuellen Medien dem Benutzer während der Ausführung der aktuellen Aufgabe angezeigt werden sollen. Der Prozessor veranlasst die Präsentation, die visuellen Medien bei dem erhaltenen Fokusabstand während der Ausführung der aktuellen Aufgabe anzuzeigen ...

Arrangement for automatic viewing point analysis with image recognition for computer control

The arrangement has one or more cameras near the monitor that continuously acquire images of the head or of parts of the head of the person. One or both pupils and if appropriate the head position are determined with respect to a spatial coordinate system using an electronic data processing system, e.g. a PC, by image processing and recognition using the digitized camera images. One or both viewing lines are derived from knowledge of the data. The image point is computed from the intercept of the sight lines.

Anzeigevorrichtung für ein Kraftfahrzeug und Verfahren zum Betreiben einer solchen Anzeigevorrichtung

Anzeigevorrichtung für ein Kraftfahrzeug, mit wenigstens einem ersten Anzeigebereich (14) und wenigstens einem zweiten Anzeigebereich (16), mit einer Einrichtung (24) zum Erkennen der Blickrichtung eines Nutzers des Kraftfahrzeugs, mittels welcher der durch einen Blick des Nutzers ausgewählte Anzeigebereich (14, 16, 18) derart aktivierbar ist, dass eine dem ausgewählten Anzeigebereich (14, 16, 18) zugeordnete Bedienhandlung vornehmbar ist, und mit einem Bedienelement (26) zum Vornehmen der Bedienhandlung, wobei in jedem der Anzeigebereiche (14, 16, 18) eine Mehrzahl voneinander verschiedener Inhalte anzeigbar und durch Betätigen des Bedienelements (26) die jeweils voneinander verschiedenen Inhalte nacheinander in dem jeweiligen Anzeigebereich (14, 16, 18) aufrufbar sind, dadurch gekennzeichnet, dass die Inhalte nicht gleichzeitig, sondern nacheinander in dem Anzeigebereich (14, 16, 18) vorhanden sind, und wobei die Anzeigevorrichtung Mittel zum optischen Kenntlichmachen des durch den Blick ...

Fortbewegungsmittel, Vorrichtung und Verfahren zur Aktivierung einer Empfangsfunktion eines Assistentensystems in einem Fortbewegungsmittel

Es werden ein Fortbewegungsmittel, eine Vorrichtung sowie ein Verfahren zur Aktivierung einer Empfangsfunktion eines Assistentensystems in einem Fortbewegungsmittel (10) vorgeschlagen. Das Verfahren umfasst die Schritte: – Ermitteln einer vordefinierten Blickrichtung eines Insassen (1) des Fortbewegungsmittels (10) und im Ansprechen darauf – Aktivieren der Empfangsfunktion des Assistentensystems.

TRAINIEREN EINES NEURONALEN NETZES ZUM BESTIMMEN VON FUSSGÄNGERN

Die Offenbarung stellt Trainieren eines neuronalen Netzes zum Bestimmen von Fußgängern bereit. Es werden ein Trainingssystem für ein neuronales Netzsystem und ein Trainingsverfahren offenbart. Das Verfahren kann Folgendes umfassen: Empfangen eines Einzelbildes von einem Sensor, das aufgenommen wurde, während ein Fahrzeugführer ein Fahrzeug steuert; Verwenden eines dem Sensor zugeordneten Augennachverfolgungssystems, das die Augen des Fahrzeugführers überwacht, um Augapfelblickdaten zu bestimmen; Bestimmen einer Vielzahl von Fußgängern aus dem Einzelbild; und iteratives Trainieren des neuronalen Netzsystems, um unter Verwendung der Augapfelblickdaten und eines Antwortdatensatzes, der auf den Augapfelblickdaten basiert, den einen oder die mehreren Zielfußgänger aus der Vielzahl von Fußgängern zu bestimmen, wobei der bestimmte eine oder die bestimmten mehreren Zielfußgänger eine relativ höhere Wahrscheinlichkeit einer Kollision mit dem Fahrzeug aufweisen als ein Rest der Vielzahl von Fußgängern ...

BLICKSCHÄTZUNG UNTER VERWENDUNG EINES ODER MEHRERER NEURONALER NETZWERKE

Es werden Geräte, Systeme und Verfahren zur Schätzung des Benutzerblicks vorgestellt. In mindestens einer Ausführungsform werden ein oder mehrere neuronale Netzwerke verwendet, um grobe und feine Blickschätzungen für einen oder mehrere Benutzer zu bestimmen.

Gerät und Verfahren für eine Auslöseanordnung für die Eye-Tracking-Datenerfassung

Ein System und ein Verfahren verfolgen ein Auge eines Benutzers für eine Auslöseanordnung für die Datenerfassung nach. Das Verfahren umfasst das Gewinnen erster Bilddaten in einem ersten Sichtfeld eines ersten Datenerfassungssensors. Das Verfahren umfasst das Bestimmen eines vom Auge eines Benutzers betrachteten ersten Bereichs in Abhängigkeit von den ersten Bilddaten. Das Verfahren umfasst das Gewinnen zweiter Bilddaten in einem zweiten Sichtfeld eines zweiten Sensors. Das Verfahren umfasst das Bestimmen eines in dem zweiten Sichtfeld erfassten zweiten Bereichs in Abhängigkeit von den zweiten Bilddaten. Das Verfahren umfasst das Bestimmen von Lagedaten des ersten Bereichs in dem zweiten Bereich. Das Verfahren umfasst das Erfassen einer maschinenlesbaren Darstellung von Daten (mlDvD) unter Verwendung des zweiten Sensors in Abhängigkeit von den Lagedaten, wenn ein Auslösebefehl erfasst wird.

Fahrerüberwachungskamera zur Steuerung von Beleuchtungsmerkmalen

Leuchteneinschaltsystem für ein Kraftfahrzeug, gekennzeichnet durch:ein bewegliches Beleuchtungselement, das beweglich ist, um eine Richtung des Lichts einzustellen;einen Aktor zum Bewegen des beweglichen Beleuchtungselements,eine Fahrererfassungsvorrichtung; undeine Steuerung, die dazu konfiguriert ist, den Betrieb des Aktors zum Bewegen des beweglichen Beleuchtungselements auf Grundlage einer erfassten Blickrichtung des Fahrers zu befehlen.

Gestapelte Medienelemente mit selektiven Parallax-Effekten

Gemäß einer Ausführungsform schließt ein Verfahren das Erhalten einer Medienkomposition zum Anzeigen auf einer elektronischen Anzeigevorrichtung ein. Die Medienkomposition schließt eine Vielzahl von Schichten ein, wobei jede Schicht ein visuelles Element einschließt. Das Verfahren schließt auch das Auswählen von mindestens einigen der Schichten der Medienkomposition, um einen Parallax-Effekt darauf anzuwenden, und das Bestimmen eines Ausmaßes des gesamten Parallax-Effekts ein, der auf die ausgewählten Schichten angewendet werden soll. Das Verfahren schließt auch das Bestimmen eines geeigneten Versatzausmaßes, das individuell auf jede der ausgewählten Schichten angewendet werden soll, und das Verschieben der ausgewählten Schichten in einer oder mehreren Richtungen um ihre jeweils geeigneten Ausmaße ein. Darüber hinaus schließt das Verfahren das Anzeigen der Medienkomposition, das den Parallax-Effekt zeigt, auf der elektronischen Anzeigevorrichtung ein.

SYSTEM UND VORRICHTUNG FÜR BENUTZERGESTEUERTE VIRTUELLE KAMERA FÜR VOLUMETRISCHES VIDEO

Vorrichtung und Verfahren zum Bestimmen eines Qualitäts-Score für virtuelle Videokameras. Zum Beispiel umfasst eine Ausführungsformen Folgendes: einen Bereich von Interesse (ROI) -Detektor zum Erkennen von Bereichen von Interesse innerhalb eines ersten Bildes, das von einer ersten physikalischen Kamera (PCAM) erzeugt wird, die an ersten Koordinaten positioniert ist; virtuelle Kameraschaltung und/oder -logik zum Erzeugen eines zweiten Bildes, das an den ersten Koordinaten positioniert ist; Bildvergleichsschaltung und/oder -logik zum Einrichten einer Pixel-zu-Pixel-Entsprechung zwischen dem ersten Bild und dem zweiten Bild; einen Bildqualitätsbeurteiler zum Bestimmen eines Qualitätswerts für das zweite Bild durch Beurteilen des zweiten Bildes im Hinblick auf das erste Bild.

Bildaufnahmegerät

HERSTELLEN VON VERBINDUNGEN ÜBER HEAD-UP-DISPLAYS MITHILFE VON AUGENKONTAKTEN

Gemäß einer Ausführungsform der vorliegenden Erfindung ermittelt ein System zur Einheiteninteraktion über ein Head-up-Display einen Sichtbereich einer ersten Einheit, die ein erstes Head-up-Display nutzt, und stellt eine Interaktion zwischen mindestens einer ersten Einheit und dem ersten Head-up-Display als Reaktion darauf her, dass mindestens ein Einfluss des ermittelten Sichtbereichs auf mindestens eine zweite Einheit erkannt wurde. Ausführungsformen der vorliegenden Erfindung enthalten ferner ein Verfahren und ein Computerprogrammprodukt, um im Wesentlichen in derselben Weise wie oben beschrieben mit einer Einheit über ein Head-up-Display zu interagieren.

System und Verfahren zur Anpassung einer Anfrageantwort eines elektronischen digitalen Assistenten als Funktion von einer zuvor erfassten Benutzeraufnahme von zugehörigen Videoinformationen

Ein Verfahren bei einer elektronischen Datenverarbeitungsvorrichtung, das eine von einem elektronischen digitalen Assistenten erzeugte Anfrageantwort als Funktion von zuvor erfassten Benutzeraufnahmen von zugehörigen Informationen beinhaltet das Empfangen, von einer Videoerfassungsvorrichtung eingerichtet zum Verfolgen einer Blickrichtung eines ersten Benutzers, eines Videostreams mit einem ersten Sichtfeld des ersten Benutzers. Ein Objekt ist dann in dem Videostream mit dem erstem Sichtfeld identifiziert, das in dem ersten Sichtfeld für eine vorbestimmte Schwellenwertzeitspanne verbleibt, und das Objekt wird durch einen Videoverarbeitungsalgorithmus verarbeitet, um Objektinformationen zu produzieren, die dann gespeichert wird. Anschließend wird eine Anfrage von dem ersten Benutzer nach Informationen empfangen und es wird bestimmt, dass die Anfrage die Objektinformationen betrifft. Der elektronische digitale Assistent stellt dann eine Antwort auf die Anfrage als Funktion der Objektinformationen ...

Managementsystem für Audioinhalt und visuellen Inhalt

Computerlesbares Speichermedium mit Befehlen, die, wenn sie ausgeführt werden, bewirken, dass mindestens ein Prozessor eines Rechensystems das Managen von Nachrichteninhalt durchführt, mit den Schritten: Empfangen einer Nachricht mit Audioinhalt und visuellem Medieninhalt durch eine oder mehrere Rechenvorrichtungen; Senden eines ersten Satzes von Daten, die den Audioinhalt beschreiben, zu einer Audiovorrichtung durch die eine oder die mehreren Rechenvorrichtungen, wobei die Audiovorrichtung dazu konfiguriert ist, den Audioinhalt zu einem Benutzer der Audiovorrichtung zu übermitteln; Senden eines zweiten Satzes von Daten, die den visuellen Medieninhalt beschreiben, zu einer Anzeigevorrichtung durch die eine oder die mehreren Rechenvorrichtungen, wobei die Anzeigevorrichtung dazu konfiguriert ist, den visuellen Medieninhalt für den Benutzer anzuzeigen; und Liefern einer Benachrichtigung zum Benutzer der Audiovorrichtung durch die eine oder die mehreren Rechenvorrichtungen, um den visuellen ...

Vorrichtung und Verfahren zur Auswahl eines Gerätes

Die Erfindung betrifft die Auswahl eines Geräts (5, 6, 7) für die Ausführung einer auf das Gerät (5, 6, 7) bezogenen Aktion. Dabei wird eine Erfassung der Blickrichtung eines Benutzers vorgenommen und eine Überprüfung durchgeführt, ob durch den Blick des Benutzers ein Gerät (5, 6, 7) anvisierten wird. Im Falle der Anvisierung eines Geräts (5, 6, 7) erfolgt eine Festlegung des anvisierten Geräts (5, 6, 7) für die Ausführung einer auf das Gerät (5, 6, 7) bezogenen Aktion.

VERFAHREN UND VORRICHTUNG ZUR ÜBERWACHUNG EINES ZIELS

Verfahren und Vorrichtung zur augenfolgegesteuerten Fernladung

Haushaltsgerät mit einer Anzeigeeinheit und Verfahren zur Anzeige einer Information auf einer Anzeigeeinheit eines Haushaltsgeräts

Haushaltsgerät mit einer Anzeigeeinheit (4), gekennzeichnet durch eine Bilderfassungseinrichtung (5), welche zur Detektion des Blickwinkels eines die Anzeigeeinheit (4) beobachtenden Beobachters (11) und/oder der Augenbewegung des Beobachters (11) der Anzeigeeinheit (4) ausgebildet ist, und einer Steuereinheit (7), welche abhängig von den detektierten Informationen der Bilderfassungseinrichtung (5) zur von dem momentanen Blickwinkel und/oder der Augenbewegung abhängigen Steuerung der Bewegung von auf der Anzeigeeinheit (4) angezeigten Informationen ausgebildet ist.

Image processing method and apparatus

Image processing

Virtual reality

Control apparatus and related method

Apparatus and method

Dual display immersive screen technology

Improvements in and relating to displays

High resolution display

Holographic display

Using headset movement for compression

Automatic view generation from recording photographer's eye movements

Automatic view generation, such as rostrum view generation, may be used beneficially for viewing of still or video images on low resolution display devices such as televisions or mobile. However, the generation of good quality automatic presentations such as rostrum presentations presently requires skilled manual intervention. By recording important parts of the picture at the time of capture time based on conscious and subconscious user actions at the time of capture, extra information may be derived from the capturing process which helps to guide or determine a suitable automatic view generation for presentation of the captured image.

Controlling the operation of a head-up display apparatus

User device with gaze tracing to effect control

A device includes a display, a speaker, a gaze tracking system operative to track and identify points on the display to which a user of the device is directing their gaze over time, and a face tracking system operative to determine a distance of the face of the user of the device from the display. A processor determines the duration of gaze fixation, and dynamics of gaze movement, based, at least in part, on the received locations of the points on the display and the determined distance. The processor outputs an instruction that modifies at least one output of the device on the display and/or the speaker. E.g. if the user is scanning over the display, accompanying audio could be changed or the appearance of an image or chart could be modified. Depending on where the user is looking a video could be activated if stopped or a pop-up menu could be opened.

Apparatus and method for displaying information

A method comprising: obtaining information associated with a vehicle or an external image data of a region proximal to the vehicle; and displaying on a wearable display device a graphical representation of a component of the vehicle which depicts a characteristic related to the obtained information or displaying the external image; wherein the display device overlies a portion of the vehicle and makes said portion appear at least partially transparent. The wearable display device may be a spectacle based display. Both the graphical representation of the component and the external image may be displayed to overlap a portion of the vehicle. A delay may be introduced to the external image data before it is displayed. The portion of the vehicle which is overlapped may be the bonnet 520. The graphical representation may be the wheels of the vehicle 550 and the information may be the steering angle of the wheels. The external image may be obtained by one or more downwards facing cameras.

Using eye-tracking for authentication in payment systems

A method of unlocking a pattern lock comprises transmitting a pattern diagram from a server and displaying the diagram to a user, allowing the user to trace a movement path on the displayed diagram, and determining whether the traced movement path matches a pre-stored unlock movement path. The movement path may be traced by swiping a touch screen (not shown). Preferably, the movement path is traced using eye-tracking means to determine movement of the user's eye whilst gazing at the diagram. A payment method using the pattern-unlocking method as a means of authenticating a user when making payments at retail sites or online is also claimed. The payment method preferably includes additional security steps comprising creating and displaying a one-time or single use authentication code on a user device which can be read by a merchant device in order to authenticate the user, and/or providing the merchant with a stored image of a registered user such that the merchant can visually verify the ...

Control apparatus and related methods for addressing driver distraction

Presentation of data on an at least partially transparent display based on user focus

Vehicle notification system including transparent and mirrored displays

A vehicle notification system including a transparent display, which may be a head up display 210 on a windscreen 110, a display 220, 230, 240 in a mirror such as a rear view mirror 120 or side view mirrors 130,140, an eye tracker 250 and a controller 205. The controller uses the eye tracker to determine which display the driver is gazing at and displays a notification in that display. If a notification source, such as a pothole (400, fig3C), is viewable in the display then it is shown in the display and if the driver looks at another display then that display is used to show the notification. The notification may take the form of a warning and may suggest a course of action to respond to the warning which may be based in part on feedback from sensors such as RADAR, LIDAR, IR, camera or other vehicle data.

Ocular video stabilization

A method for ocular stabilization of video images, suitable for use in augmented reality head mounted displays, has the steps of obtaining binocular eye-gaze directions of the left 501-L and right 501-R eyes of a user of a wearable head-mountable device (HMD) (102, figure 1a) using an eye-tracking device, while simultaneously capturing video images in a forward field of view with a forward-facing video camera 502 of the HMD. Based on the obtained binocular eye-gaze directions of left and right eyes of the user of the HMD, convergent gaze directions of the user 503-L, 503-R are determined as a function of time during an interval concurrent with the capturing of the video images. The captured video images are then stabilized by compensating for motion of the forward-facing video camera with an intersection of the convergent gaze directions of the user with an image plane of the forward-facing video camera. In another aspect of the invention, only one eye is tracked. A further aspect uses ...

Methods and apparatuses for handling visual virtual reality content

A method, associated apparatus and computer-readable medium, are described for the optimization of the display of visual virtual reality (VR) content (such as, for instance a VR headset). The method comprises receiving a first portion 21 of an image (or video) at a first display quality and a second portion 22 of the picture at a second, lower, quality. A third portion 24 of the image (within the field of view (FOV) of the observer) is generated by a neural network, trained on the first or second portions, this third region being part of the second region. The quality of the third region may be different from that of the first and second sections, but higher than the one of the second portion. A change in the viewing direction of a user may be predicted, and the generation of the third region may be based on this estimation. Indicators of quality might be the image resolution, bit depth and/or rate or the frame rate. The neural network might be a Generative Adversarial Network (GAN).

Telepresence system

A system for telepresencing is provided with a user station at a remote user's location and a telepresence device at a local subject's location. Distance of objects around the local subject is sensed such that a selected distance between the telepresence device and the local subject is maintained. Eye features and gazes of the local subject and the remote user are sensed. The height of the telepresence device's display is adjusted automatically such that the eye level of the local subject matches with the eye level of the user's image in the display. When an intended eye contact is sensed, positions of the telepresence device, its display and a sensor hub that houses cameras and other sensors are adjusted, and a face image may be recreated at a different viewing angle to resemble an eye contact between the local subject and the remote user while preserving all other facial expressions.

Rendering a mixed reality scene using a combination of multiple reference viewing points

Gaze tracking system

Holograms

Holograms

Video image processing

Display

Data processing systems

In a data processing system, when displaying a foveated image, an image producer processing unit generates plural different resolution versions of the frame to be displayed. A display processor then generates a view orientation transformed output version of the frame to be displayed using data from the plural different resolution versions of the frame to be displayed generated by the producer processing unit based on data indicative of which resolution version of the frame is to be used for respective regions of the view orientation transformed output version of the frame to be displayed provided to the display processor.

Cursor movement device

Apparatus, method and system for determining a viewed status of a document

Controlling the orientation of a device display based on usage context

A method for controlling an orientation of data on a display (117) of a device (100) comprises determining a device angular position for the device, determining a usage context parameter for the device, and controlling the orientation of the data on the display based on the device angular position and the usage context parameter. A change in the orientation of the data on the display, for instance from portrait to landscape, may be suppressed based on the usage context parameter. Usage context parameters may include user facial orientation angular position detected by a camera (140), or user grip position detected by grip sensors (145). A usage context parameter may also comprise user input data, such as a user input regarding the locking of the orientation of the data. In a further embodiment the usage context parameter may consist of a reversal display orientation change event, such as when a user reacts to an orientation change by changing the device position to force a change back to ...

Display control apparatus and method

Visual communications apparatus

Controlling the operation of a head-up display apparatus

Method and system for displaying a virtual object

Virtual reality headset

Production and packaging of entertainment data for virtual reality

Method and apparatus for colour imaging

Method and apparatus for depth estimation relative to a mobile terminal

Systems and methods for task-based adjustable focal distance for heads-up displays

Method of displaying an image

A method of displaying an image on a vehicles rearward view display is disclosed which involves capturing first image data with a first frame of reference from a first imaging means located at a first location of the vehicle proximal to the display device, and capturing second image data with a second frame of reference from a second imaging means located at a second location of the vehicle. A transformation is then applied to the second image data to form third image data having the first frame of reference. A composite image is then formed from the first third image data which image appears to a user to have been captured at the first location. The method may also include determining a restricted portion of the first image data, e.g. obstructions such as headrests, passengers, and luggage, and may further involve overlaying a portion of the third image data corresponding to the restricted portion to provide an unrestricted composite image. The imaging means may be cameras. The method ...

Autonomously acting robot changing pupils

An autonomously acting robot is provided in which a monitor is installed in an eye of the robot, and an eye image 174 is displayed on the monitor. The robot extracts a feature quantity of an eye of a user from a captured image of the user. The feature quantity of the eye of the user is reflected on the eye image 174. Examples of the feature quantity are the sizes of a pupillary zone 158 and a pupil image 164 and the shape of an eyelid image 176. A frequency of blinking may also be reflected as the feature quantity. A degree of intimacy is set for each user, and which user's feature quantity is to be reflected may be determined in accordance with the degree of intimacy.

An image processing system and method thereof

A method and system for obtaining images suitable for eye-tracking. An imaging module 104 captures at least one image while an illumination module 108 is on (204, fig.2), and at least one consecutive image while the illumination module is off (206, fig.2). The at least two consecutive frames are processed to produce a resultant image containing at least one eye across a selected wavelength range (optionally 380-800nm RGB, 700-1000nm, NIR, or 400-700nm, visible light). The pixel intensity of the “on” and “off” images may be calculated, and the ratio of the average value used to control an analog dimming function of the imaging module. The images may be aligned by matching identical features across the frames and the difference between the pixel intensity may be calculated to produce a resultant image in the chosen wavelength. The eye-tracking may be an estimated line-of-sight for virtual reality (VR) applications, an estimated eye-position for augmented reality (AR) applications, or a user-state ...

Method for detecting user input to a breath input configured user interface

A method of breath input recognition, including an electronic device displaying a breath input enabled user interface, BIEUI on a display whilst in a breath input receiving state or operational mode. Detecting, using a camera, a position of a head of a user relative to the camera and determining, if at least part of a face of the user is directed towards the displayed BIEUI. Detecting an audio signal using a microphone and determining if the detected audio signal is an intentional breath input to the BIEUI based on the audio signal and the detected position of the head of the user. In a second aspect a method for training a user to provide breath input includes determining characteristics of the detected audio input to be in conformance with characteristics of a predetermined type of intentional breath input training BIEUI and causing a visual indicator of the conformance to be presented.

Eye tracking data filtering

Gaze tracking apparatus and systems

Advertisement unit and a method thereof

Advertisement unit and a method thereof

Advertisement unit and a method thereof

EYE PURSUIT LIGHTING

Apparatus and method for eye monitoring

Die Erfindung betrifft eine Vorrichtung und ein Verfahren zur Augenüberwachung umfassend ein Sichtglas (3) und einen dieses aufnehmenden Rahmen (2), wobei das Sichtglas (3) ein flächiger Lichtwellenleiter (4) ist oder zumindest einen flächigen Lichtwellenleiter (4) aufweist, wobei der flächige Lichtwellenleiter (4) zumindest eine Schicht mit lumineszenten Partikeln aufweist, wobei am Sichtglas (3) oder am flächigen Lichtwellenleiter (4) zumindest ein Photosensor (5) angeordnet ist, welcher die Intensität des im flächigen Lichtwellenleiter (4) und/oder im Sichtglas (3) durch totale interne Reflexion (TIR) geleiteten Lumineszenzlichts erfasst.

Apparatus and method for eye monitoring

Die Erfindung betrifft eine Vorrichtung und ein Verfahren zur Augenüberwachung umfassend ein Sichtglas (3) und einen dieses aufnehmenden Rahmen (2), wobei das Sichtglas (3) ein flächiger Lichtwellenleiter (4) ist oder zumindest einen flächigen Lichtwellenleiter (4) aufweist, wobei der flächige Lichtwellenleiter (4) zumindest eine Schicht mit lumineszenten Partikeln aufweist, wobei am Sichtglas (3) oder am flächigen Lichtwellenleiter (4) zumindest ein Photosensor (5) angeordnet ist, welcher die Intensität des im flächigen Lichtwellenleiter (4) und/oder im Sichtglas (3) durch totale interne Reflexion (TIR) geleiteten Lumineszenzlichts erfasst.

Display system and method

Disclosed is a method of operation in a virtual image presentation system, the method comprising: detecting a head movement of an end user; determining respective resolutions for each of at least two portions of a frame in response to the detected head movement of the end user; modifying a first resolution of a first portion of the frame such that it is different from a second resolution of a second portion of the frame by adjusting a first slope of a first drive signal corresponding to the first portion of the frame such that it is different from a second slope of a second drive signal corresponding to the second portion of the frame; and displaying a virtual object to the end user based on the first and second resolutions of the respective first and second portions of the frame.

A system for recorded e-book digital content playout

A system allows for audio playout of e-book content data using a playout electronic device recorded by a remote recording electronic device. The system may analyse the e-book content to infer XY on¬ screen display coordinates for each word of the e-book and speech recognition may correlate the timing of the spoken words to the XY coordinates. As such, a read along display animation may be generated by the digital display of the playout electronic device in time with the audio and the respective on-screen position of each word. Eye tracking may be employed by the playout electronic device for the display of a gaze position indicator to the recording electronic device in substantial real-time. The system may further employ machine learning to optimise a trained machine to output at least one prosodic features for user profile specific speech emulation using a speech emulator.

Information System and Method for Providing Information Using a Holographic Element

An information system and a method for providing information in correlation with light that is incident on an eye includes a holographic element disposed in front of the eye and a device capable of recording optical signals which detects light that is incident on the eye via the holographic element. The device capable of recording optical signals detects light which is diffracted by the holographic element before the light impinges on the eye such that the diffracted light does not enter the eye.

Tools for Use within a Three Dimensional Scene

Tools for use within a 3D scene. The 3D scene may be presented by at least one display, which includes displaying at least one stereoscopic image of the 3D scene by the display(s). The 3D scene may be presented according to a first viewpoint. User input may be received to the 3D scene using one or more tools. The 3D scene may be updated in response to the use of the one or more tools.

Eye gaze user interface and method

A software controlled user interface and method for an eye gaze controlled device, designed to accommodate angular accuracy versus time averaging tradeoffs for eye gaze direction sensors. The method can scale between displaying a small to large number of different eye gaze target symbols at any given time, yet still transmit a large array of different symbols to outside devices with minimal user training. At least part of the method may be implemented by way of a virtual window onto the surface of a virtual cylinder, with eye gaze sensitive symbols that can be rotated by eye gaze thus bringing various groups of symbols into view, and then selected by continual gazing. Specific examples of use of this interface and method on an eyeglasses-like head-mountable, vision-controlled, device are disclosed, along with various operation examples including sending and receiving text messages, control of robotic devices and control of remote vehicles.

Qualified command

A method for executing a qualified command including detecting a hand gesture input for identifying a command, detecting one or more non-hand gesture inputs to qualify the command, and configuring a processor to execute the qualified command on a machine in response to the hand gesture input and one or more non-hand gesture inputs.

Systems and methods for aiding traffic controllers and/or pilots

Gaze based systems and methods are used to aid traffic controllers and/or pilots. A gaze line of an eye of the user viewing the display is tracked using an eyetracker. An intersection of the gaze line of the eye with the display is calculated to provide continuous feedback as to where on the display the user is looking A trace of the gaze line of the eye is correlated with elements of a situation. The user's awareness of the situation is inferred by verifying that the user has looked at the elements of the situation. In an embodiment, the user is notified of the situation when it is determined that the user has not looked at the elements of the situation for a predetermined period of time. The notification is automatically removed once it is determined that the user has looked at the elements of the situation.

Three Dimensional User Interface Effects on a Display by Using Properties of Motion

The techniques disclosed herein use a compass, MEMS accelerometer, GPS module, and MEMS gyrometer to infer a frame of reference for a hand-held device. This can provide a true Frenet frame, i.e., X- and Y-vectors for the display, and also a Z-vector that points perpendicularly to the display. In fact, with various inertial clues from accelerometer, gyrometer, and other instruments that report their states in real time, it is possible to track the Frenet frame of the device in real time to provide a continuous 3D frame-of-reference. Once this continuous frame of reference is known, the position of a user's eyes may either be inferred or calculated directly by using a device's front-facing camera. With the position of the user's eyes and a continuous 3D frame-of-reference for the display, more realistic virtual 3D depictions of the objects on the device's display may be created and interacted with by the user.

Remote telepresence gaze direction

A method and apparatus for controlling a live avatar include features for directing the live avatar gaze to be consistent with a remote operator's view of the live avatar's surroundings. Indication of gaze direction may be continuously provided to the live avatar using movable optics, for example a heads-up display, or a movable optical prism or lens. Gaze direction or operator commands may be communicated using a haptic device. Video and control streams from multiple live avatars may be aggregated through a host server and allocated via an online interface to multiple prospective remote operators.

Apparatus and method for controlling interface

In an apparatus and method for controlling an interface, a user interface (UI) may be controlled using information on a hand motion and a gaze of a user without separate tools such as a mouse and a keyboard. That is, the UI control method provides more intuitive, immersive, and united control of the UI. Since a region of interest (ROI) sensing the hand motion of the user is calculated using a UI object that is controlled based on the hand motion within the ROI, the user may control the UI object in the same method and feel regardless of a distance from the user to a sensor. In addition, since positions and directions of view points are adjusted based on a position and direction of the gaze, a binocular 2D/3D image based on motion parallax may be provided.

Mobile terminal and operation control method thereof

Disclosed herein is a mobile terminal and an operation control method thereof in which a delay time of the screen lock execution is controlled according to the type of application, thereby improving the inconvenience of a user interface and effectively managing a battery according to an interrupt when required to continuously receive an input from the user or continuously provide visual information to the user. For this purpose, a mobile terminal according to an embodiment of the present disclosure may include an input unit configured to receive a user input; an execution controller configured to execute screen lock if the user input is not received for a predetermined time; and a change controller configured to change the predetermined time based on a type of application.

Navigation of endoscopic devices by means of eye-tracker

A method for controlling image capturing and spatial control in a device monitored by imaging, in particular an endoscopic instrument, for examination and manipulation purposes, wherein the work region and the position and action of the device are monitored by means of one of the display means displaying the observed work region via optics. The device is held and moved by motor-driven equipment depending on the display, the respective position and intended action. The eye movements of the operating person or the line of sight during observation of the display is registered according to the eye-tracking method and used to control the image capturing and tracking and the display.

Contextual information encoded in a formed expression

Embodiments include a method, a manual device, a handheld manual device, a handheld writing device, a system, and an apparatus. An embodiment provides a manual device operable in a context. The manual device includes a writing element operable to form a mark on a surface in response to a movement of the writing element with respect to the surface. The manual device also includes a controller operable to encode information corresponding to the context of the manual device by regulating the formation of the mark.

Mobile terminal and operation control method thereof

A mobile terminal and an operation control method thereof are provided. The operation control method includes displaying a stereoscopic 3D image, which is created based on the disparity between left- and right-eye images, on a display module; determining a position of a user's gaze on the stereoscopic 3D image by performing eye tracking; and varying a 3D effect of a portion of the stereoscopic 3D image corresponding to the determined gaze position. Therefore, it is possible to effectively control various operations performed by a mobile terminal in connection with a stereoscopic 3D image by using eye tracking.

Method and system for continuous notifications using subliminal and supraliminal cues

A method of notifying a person using a computer includes providing a computer system including at least one sensory cueing component. The method includes defining a subliminal and a supraliminal range of at least one human sense. A cue may be defined and includes a variable attribute, and the cue is directed to the at least one human sense. An alert level may be associated with the message and the alert level may be classified as being in the subliminal or supraliminal range. The method may include a presentation methodology of the cue and the message. The cue may be initiated and presented in either the subliminal range or the supraliminal range and directed to the at least one human sense in the user's sensory environment. The user acknowledges the cue and the message in either the subliminal or supraliminal range, before the cue is removed from the user's sensory environment.

Calibration free, motion tolerent eye-gaze direction detector with contextually aware computer interaction and communication methods

Eye tracking systems and methods include such exemplary features as a display device, at least one image capture device and a processing device. The display device displays a user interface including one or more interface elements to a user. The at least one image capture device detects a user's gaze location relative to the display device. The processing device electronically analyzes the location of user elements within the user interface relative to the user's gaze location and dynamically determine whether to initiate the display of a zoom window. The dynamic determination of whether to initiate display of the zoom window may further include analysis of the number, size and density of user elements within the user interface relative to the user's gaze location, the application type associated with the user interface or at the user's gaze location, and/or the structure of eye movements relative to the user interface.

Automatic focus improvement for augmented reality displays

An augmented reality system provides improved focus of real and virtual objects. A see-through display device includes a variable focus lens a user looks through. A focal region adjustment unit automatically focuses the variable focus lens in a current user focal region. A microdisplay assembly attached to the see-through display device generates a virtual object for display in the user's current focal region by adjusting its focal region. The variable focus lens may also be adjusted to provide one or more zoom features. Visual enhancement of an object may also be provided to improve a user's perception of an object.

Comprehension and intent-based content for augmented reality displays

A method and system that enhances a user's experience when using a near eye display device, such as a see-through display device or a head mounted display device is provided. The user's intent to interact with one or more objects in the scene is determined. An optimized image is generated for the user based on the user's intent. The optimized image is displayed to the user, via the see-through display device. The optimized image visually enhances the appearance of objects that the user intends to interact with in the scene and diminishes the appearance of objects that the user does not intend to interact with in the scene. The optimized image can visually enhance the appearance of the objects that increase the user's comprehension. The optimized image is displayed to the user, via the see-through display device.

Information processing apparatus, method, and storage medium storing program

A display unit 11 has a display area of a predetermined shape and displays an image on the display area. An image capturing unit 12 generates data of a captured image by capturing an image of an eye 2 of a user in which the display area is reflected. A reference detection unit 21 detects a moving reference point that moves along with the movement of user's eye-gaze and an unmoving reference point that can be assumed as approximately stationary regardless of the user's eye-gaze movement, and generates a vector drawn from the unmoving reference point to the moving reference point as a shift amount V(x,y). An eye-gaze detection unit 22 detects a movement vector ΔV(Δx,Δy) as the user's eye-gaze movement amount based on a reference shift amount Vo(xo,yo) generated in the past and the shift amount V(x,y) currently generated.

Ar glasses with sensor and user action based control of external devices with feedback

This disclosure concerns an interactive head-mounted eyepiece with an integrated processor for handling content for display and an integrated image source for introducing the content to an optical assembly through which the user views a surrounding environment and the displayed content, wherein the eyepiece includes sensor and user action based control of external devices with feedback.

Ar glasses with state triggered eye control interaction with advertising facility

This disclosure concerns an interactive head-mounted eyepiece with an integrated processor for handling content for display and an integrated image source for introducing the content to an optical assembly through which the user views a surrounding environment and the displayed content, wherein the eyepiece includes a state triggered eye control interaction with advertising facility.

Methods, systems, and computer program products for managing attention of a user of a portable electronic device

Methods and systems are described for managing attention of a user of a portable electronic device. A detection is made that a portable electronic device is in motion relative to a first object separate from the portable electronic device An interaction is detected between a user and the portable electronic device while the portable electronic device is in the motion, wherein in the interaction the portable electronic device is a first source of sensory input for the user. A determination is made, based on detecting the motion, that a specified attention criterion is met in response to detection of the interaction. In response to detecting that the attention criterion is met, attention information is sent for presenting an attention output, via an output device, for directing the user to an object, not included in the portable electronic device, as a second source of sensory input for the user.

Ar glasses with event and sensor input triggered user action capture device control of ar eyepiece facility

This disclosure concerns an interactive head-mounted eyepiece with an integrated processor for handling content for display and an integrated image source for introducing the content to an optical assembly through which the user views a surrounding environment and the displayed content, wherein the eyepiece includes event and sensor input triggered user action capture device control.

See-through near-eye display glasses with a small scale image source

This disclosure concerns an interactive head-mounted eyepiece with an integrated processor for handling content for display and an integrated image source for introducing the content to an optical assembly through which the user views a surrounding environment and the displayed content, wherein the image source comprises a lighting system that directs light from a light source to a curved polarizing film that reflects a portion of the light to a reflective image display. The ratio of the height of the curved polarizing film to the width of the reflective image display is less than 1:1.

Advanced remote control of host application using motion and voice commands

A head mounted microdisplay (HMD) device that uses hand, body, head movement or gesture and voice commands to control interaction with a local processor or host computer. The local processor overlays information, such as onto an operating system desktop or application program menu, indicating the motion, gesture, and/or voice input needed to activate an associated action.

See-through near-eye display glasses including an auto-brightness control for the display brightness based on the brightness in the environment

This disclosure concerns an interactive head-mounted eyepiece with an integrated processor for handling content for display and an integrated image source for introducing the content to an optical assembly through which the user views a surrounding environment and the displayed content. The image source includes an auto-brightness control that controls the brightness of the displayed content based on the brightness in the environment.

Login method based on direction of gaze

A method of authenticating a user of a computing device is proposed, together with computing device on which the method is implemented. A plurality of objects is displayed on a display screen. The plurality of objects includes at least objects that make up a sequence of objects pre-selected as the user's passcode. In response to a trigger signal an image of the user's face is captured while looking at one of the objects on the display screen. A determination of which object is in the direction of the user's gaze is made from the photograph and whether or not the gaze is on the correct object in the sequence of the passcode. This is repeated for each object in the sequence of the passcode.

Methods for predicting affective response from stimuli

Creating a machine learning-based affective response predictor to predict a user's emotional state after being exposed to tokens representing stimuli that influence the user's affective state, comprising: receiving samples comprising temporal windows of token instances to which the user was exposed; the token instances are spread over a long period of time, and a subset of the token instances originate from same source and have overlapping instantiation periods; receiving target values, which represent affective response annotations of the user and correspond to the temporal windows of token instances; and creating the machine learning-based affective response predictor for the user, which compensates for non-linear effects resulting from the user being exposed to the subset of token instances originating from the same source and having overlapping instantiation periods, by running a machine learning training procedure on input data comprising the samples and the corresponding target values.

System for switching displays based on the viewing direction of a user

Embodiments of the present invention provide a method, system, and computer program product for automatically assigning a display as a primary display in a multi-display computer. In an embodiment of the invention, a method for configuring a primary display in a multi-display environment is provided and can include tracking eye movements of an end user interacting with a multi-display computer. A display principally viewed by the end user , can be identified based upon the eye movements amongst at least two displays of the multi-display computing environment. Thereafter, the identified display can be configured as a primary display in the multi-display computing environment in response to determining that the end user has viewed the identified display for a threshold period of time.

System and method for tracking the point of gaze of an observer

A system for tracking the point of gaze of an observer observing an object comprises a camera for recording an image of an eye of the observer, comprises a means for providing a luminous marker, and means for analyzing the image of the eye to determine the reflection of the marker on the eye and the centre of the pupil. The relative positions of the corneal reflection of the marker and the pupil centre are measured. The marker is repositioned, in dependence on the determined relative positions, to improve correspondence between the corneal reflection of the marker and the pupil centre.

Sessionless pointing user interface

A method, including receiving, by a computer, a sequence of three-dimensional maps containing at least a hand of a user of the computer, and identifying, in the maps, a device coupled to the computer. The maps are analyzed to detect a gesture performed by the user toward the device, and the device is actuated responsively to the gesture.

Method and system for correcting gaze offset

A method of correcting gaze offset in an image of at least one individual having eyes is disclosed. The method comprises: processing the image to extract location of at least one eye over the image, processing the image to replace imagery data associated with each location of each eye with replacement data thereby providing a corrected image, and transmitting the corrected image to a display device. The replacement data are preferably previously-recorded imagery data which respectively correspond to the same eye but a different gaze.

3dimension stereoscopic display device

Disclosed is a 3Dimension stereoscopic display device including a screen composition processing unit for generating a right-eye image or video image and a left-eye image or video image for planar display or for three-dimensional stereoscopic display of an image or video image, a video image playback device 5 for playing back the images or video images, and a stereoscopic display monitor unit 6 for displaying the images or video images played back by the playback device 5 in a three-dimensional stereoscopic manner, in which the screen composition processing unit 4 switches between outputting the right-eye image or video image and the left-eye image or video image for planar display to the playback device 5 and outputting the right-eye image or video image and the left-eye image or video image for three-dimensional stereoscopic display to the playback device 5 according to state of a moving object.

Automatic adjustment logical positions of multiple screen

A method and a computer system are provided for automatically setting the logical positions of multiple screen displays. A computer system may comprise a plurality of display devices, at least one image capturing device, and a controller. The controller may be coupled to the display devices and image capturing devices. The adjustment module may be adapted to adjust the plurality of the display settings.

Eye Gaze Detection to Determine Speed of Image Movement

A wearable computing device or a head-mounted display (HMD) may be configured to track the gaze axis of an eye of the wearer. In particular, the device may be configured to observe movement of a wearer's pupil and, based on the movement, determine inputs to a user interface. For example, using eye gaze detection, the HMD may change a tracking rate of a displayed virtual image based on where the user is looking. Gazing at the center of the HMD field of view may, for instance, allow for fine movements of the virtual display. Gazing near an edge of the HMD field of view may provide coarser movements.

Adaptive Projector

An apparatus for processing data includes a projector, which is configured to project content onto at least a part of a scene. A processor is configured to detect a location of an eye of a person in the scene and to control the projector so as to reduce an intensity of the projected content in an area of the eye.

System and Method for Augmented and Virtual Reality

One embodiment is directed to a system for enabling two or more users to interact within a virtual world comprising virtual world data, comprising a computer network comprising one or more computing devices, the one or more computing devices comprising memory, processing circuitry, and software stored at least in part in the memory and executable by the processing circuitry to process at least a portion of the virtual world data; wherein at least a first portion of the virtual world data originates from a first user virtual world local to a first user, and wherein the computer network is operable to transmit the first portion to a user device for presentation to a second user, such that the second user may experience the first portion from the location of the second user, such that aspects of the first user virtual world are effectively passed to the second user.

VIDEO DISPLAY MODIFICATION BASED ON SENSOR INPUT FOR A SEE-THROUGH NEAR-TO-EYE DISPLAY

This disclosure concerns a near field communication (NFC) device which includes a wrist-worn NFC-enabled electronics device, wherein the wrist-worn NFC enabled electronics device includes a first communications link for communicating with a second NFC-enabled electronics device via NFC protocols, and a second communications link for communicating with an eyepiece via a medium-range communications protocol and receiving control commands. The wrist-worn NFC-enabled electronics device facilitates the transfer of data between the eyepiece and the second NFC-enabled electronics device. The eyepiece comprises optics enabling a see-through display on which is displayed the data. 1. A system comprising:an interactive head-mounted eyepiece worn by a user, wherein the eyepiece includes an optical assembly through which the user views a surrounding environment and displayed content,an integrated processor for handling content for display to the user,an integrated image source for introducing the content to the optical assembly;the processor adapted to modify the content, wherein the modification is made in response to a sensor input.2. The system of wherein the content is a video image.3. The system of wherein the modification is at least one of adjust brightness claim 2 , adjust color saturation claim 2 , adjust color balance claim 2 , adjust color hue claim 2 , adjust video resolution claim 2 , adjust transparency claim 2 , adjust compression rate claim 2 , adjust frames per second rate claim 2 , isolate part of the video claim 2 , stop the video from playing claim 2 , pause the video claim 2 , or restart the video.4. The system of wherein sensor input is derived from at least one of a charge-coupled device claim 1 , a black silicon sensor claim 1 , an IR sensor claim 1 , a acoustic sensor claim 1 , a induction sensor claim 1 , a motion sensor claim 1 , an optical sensor claim 1 , an opacity sensor claim 1 , a proximity sensor claim 1 , an inductive sensor claim 1 , an Eddy- ...

Electronic Devices With Gaze Detection Capabilities

An electronic device may have gaze detection capabilities that allow the device to detect when a user is looking at the device. The electronic device may implement a power management scheme using the results of gaze detection operations. When the device detects that the user has looked away from the device, the device may dim a display screen and may perform other suitable actions. The device may pause a video playback operation when the device detects that the user has looked away from the device. The device may resume the video playback operation when the device detects that the user is looking towards the device. Gaze detector circuitry may be powered down when sensor data indicates that gazed detection readings will not be reliable or are not needed. 1. A method for using a portable electronic device having an accelerometer and gaze detection circuitry , the method comprising:with the accelerometer, determining whether a measured acceleration level for the portable electronic device has exceeded a given threshold value; andwhen it has been determined that the acceleration level of the portable electronic device has exceeded the given threshold value, disabling the gaze detection circuitry.2. The method defined in further comprising:after it has been determined that the acceleration level of the portable electronic device has exceeded the given threshold value, determining whether the acceleration level of the portable electronic device has dropped below a second threshold value; andwhen it has been determined that the acceleration level of the portable electronic device has dropped below the second threshold value, enabling the gaze detection circuitry.3. The method defined in wherein the gaze detection circuitry comprises a camera that is used to determine whether the user's gaze is directed towards the portable electronic device and wherein disabling the gaze detection circuitry comprises turning off the camera.4. The method defined in wherein determining ...

Head-Angle-Trigger-Based Action

Methods and devices for applying at least one manipulative action to a selected content object are disclosed. In one aspect, a head-mounted-device (HMD) system includes at least one processor and data storage with user-interface logic executable by the at least one processor to apply at least one manipulative action to a displayed content object based on received data that indicates a first direction in which the HMD is tilted and an extent to which the HMD is tilted in the first direction. The at least one manipulative action is applied to a degree corresponding to the indicated extent to which the HMD is tilted in the first direction.

Systems and methods for a gaze and gesture interface

A system and methods for activating and interacting by a user with at least a 3D object displayed on a 3D computer display by at least the user's gestures which may be combined with a user's gaze at the 3D computer display. In a first instance the 3D object is a 3D CAD object. In a second instance the 3D object is a radial menu. A user's gaze is captured by ahead frame containing at least an endo camera and an exo camera worn by a user. A user's gesture is captured by a camera and is recognized from a plurality of gestures. User's gestures are captured by a sensor and are calibrated to the 3D computer display.

New kind of multi-touch input device

A multi-touch method includes, in response to at least one finger of a user pointing toward a computer screen, recognizing an eye and at least one finger in an image frame taken by a camera; mapping a first coordinate of the recognized finger in the image frame to a second coordinate of the finger on the computer screen; mapping a third coordinate the recognized eye in the image frame to a fourth coordinate of the eye on the computer screen; determining whether the fourth coordinate is in the proximity of the second coordinate; in response to a positive decision, determining the second coordinate as a multi-touch point.

SYSTEMS AND METHODS FOR ENHANCING TELECONFERENCING COLLABORATION

An advanced video teleconferencing system facilitates an engaging and realistic video conferencing experience. Key design elements and video, audio, and control capabilities are provided for a video conferencing experience that cannot be attained with conventional methods, which elements and capabilities include careful design of the table and room at each site, supplementary displays showing imagery in a mirrored manner, networked environmental control, an AutoDirector with keyword and gesture recognition, and audio reflection from a display or beam splitter. 1. An auto director apparatus comprising:means for recognizing and reacting to gestures made by participants in a teleconference either to compute gaze direction or to determine therefrom a focus of attention at a teleconference site;wherein gaze direction or focus of attention is considered in selecting an appropriate signal for transmission.2. An auto director apparatus , operating at a local conference site , for emulating behavior of a human video director during a video teleconference between a plurality of conference sites , comprising any of:means for automatically selecting, from among one or more local video camera feeds and other local video inputs, an outbound video signal for transmission to remote video conferencing sites;means for automatically selecting, from among one or more inbound video signals received from remote sites, a primary video signal for display on a primary display at said local conference site; andmeans for automatically selecting, from among said plurality of local video camera feeds, said local video inputs, and said inbound video signals, one or more supplementary video signals for display on one or more supplementary displays,wherein at least one of said means for automatically selecting responds to both of audio signals obtained from one or more microphones at one or more conference sites and to content of said audio signals.3. The apparatus of claim 2 , wherein at least ...

ELECTRONIC DEVICE AND METHOD FOR RELIEVING VISUAL FATIGUE USING THE ELECTRONIC DEVICE

An electronic device includes a visual sensor and a display screen. The visual sensor senses whether a user is looking at the display screen. If the user is looking at the display screen, the electronic device adjusts a font size of a font being displayed on the display screen. If the user looks at the display screen for not less than a first predefined time, the electronic device prompts the user to have a rest and turn off the display screen. After the electronic device has been turned off for more than a second predefined time, the display screen is turned on again automatically. 1. An electronic device , comprising:a storage system;at least one processor; andone or more programs stored in the storage system and executed by the at least one processor, the one or more programs comprising:a controlling module that controls a visual sensor of the electronic device to capture an image of eyes of a user, and detects whether the user is looking at a display screen of the electronic device by determining whether the image of the eyes matches an eyes model stored in the storage system;an adjusting module that adjusts a font size of a font on a page displayed on the display screen to a preset font size when the user is looking at the display screen; anda prompting module that prompts the user to have a rest and controls the display screen to turn off when the user has looked at the display screen not less than a first predefined time.2. The electronic device as described in claim 1 , wherein the controlling module determines that the user is looking at the display screen when the image of the eyes matches one of the eyes models.3. The electronic device as described in claim 1 , wherein the one or more programs further comprise a first timing module that sets an initial time when the visual sensor determines that the user is looking at the screen.4. The electronic device as described in claim 1 , wherein the one or more programs further comprise an opening module that ...

ELECTRONIC EQUIPMENT

A mobile phone which is an example of electronic equipment includes an infrared camera and an infrared LED. The infrared camera is arranged above a display and the infrared LED is arranged below the display. A user, by an eye-controlled input, designates a button image or a predetermined region on a screen. When a line of sight is to be detected, an infrared ray (infrared light) emitted from the infrared LED arranged below the display is irradiated to a lower portion of a pupil. Accordingly, even in a state that the user slightly closes his/her eyelid, the pupil and a reflected light of the infrared light can be imaged. 1. Electronic equipment provided with a display portion , comprising:an infrared light detecting portion which is arranged above the display portion and detects an infrared light; anda first infrared light output portion which is arranged below the display portion.2. The electronic equipment according to claim 1 , wherein the infrared light detecting portion and the first infrared light output portion are arranged on a first line which is in parallel with a vertical direction of the display portion.3. The electronic equipment according to claim 2 , further comprising a second infrared light output portion claim 2 , whereinthe second infrared light output portion is arranged on a second line which is in parallel with a horizontal direction of the display portion at an opposite side of the infrared light detecting portion in the horizontal direction of the display portion, the infrared light detecting portion being arranged on the second line.4. The electronic equipment according to claim 1 , wherein the infrared light detecting portion and the first infrared light output portion are arranged at diagonal positions sandwiching the display portion.5. The electronic equipment according to claim 1 , further comprising:a gaze area detecting portion which detects a gaze area on a screen of the display portion at which a user is gazing, based on a pupil of ...

MOBILE TERMINAL AND CONTROL METHOD THEREOF

A mobile terminal and a control method of the mobile terminal are disclosed. A mobile terminal and a control method of the mobile terminal of the present invention comprises a first camera; a touchpad; and a controller, if a touch input of a user is obtained through the touchpad, configured to carry out operation based on a gaze direction of the user obtained through the first camera. According to the present invention, operability for the mobile terminal is improved by controlling the mobile terminal based on a touch input through a touchpad and a gaze direction of the user. 1. A mobile terminal , comprising:a first camera;a touchpad; anda controller, if a touch input of a user is obtained through the touchpad, configured to carry out operation based on a gaze direction of the user obtained through the first camera.2. The mobile terminal of claim 1 , further comprising a display claim 1 , wherein the touchpad is disposed in a bezel area outside of the display.3. The mobile terminal of claim 2 , wherein the touchpad indicates position of the obtained touch input or further comprises a lighting unit for indicating a position requiring the touch input.4. The mobile terminal of claim 1 , wherein a gaze direction of the user is determined by including the user's facial movements obtained through the first camera.5. The mobile terminal of claim 1 , wherein the controller claim 1 , if the touch input is obtained claim 1 , is configured to carry out the operation by actually displaying an item corresponding to the gaze direction in a center region of a display.6. The mobile terminal of claim 1 , wherein the controller claim 1 , if the touch input is obtained claim 1 , is configured to carry out the operation by scrolling an item displayed in a display along a direction corresponding to the gaze direction.7. The mobile terminal of claim 7 , wherein the controller is configure to change the scrolling speed according to a contact area of the obtained touch input.8. The mobile ...

HUMAN-COMPUTER INTERFACE SYSTEM HAVING A 3D GAZE TRACKER

An apparatus for interfacing a person with a computer, the apparatus comprising a gaze tracker having a 3D camera and a picture camera that image the person and a controller that processes images acquired by the cameras to determine a gaze direction and point of regard of the person. 1. A system for interfacing a person with a computer:a video display on which the computer generates an image to interact with the person;a 3D camera that acquires a range image of the person located in a field of view (FOV) of the camera;a picture camera that acquires a picture of the person in the FOV of the 3D camera; anda controller that processes the range image and the picture to determine spatial coordinates for features of the person's head and an eye of the person, determines a gaze direction and origin for a gaze vector of the eye responsive to the determined spatial coordinates and modifies the image displayed on the video display responsive to the gaze vector.2. A system according to wherein modifying the displayed image comprises determining a point of regard (POR) responsive to the gaze vector and modifying the displayed image responsive to the POR.3. A system according to wherein modifying the displayed image responsive to the POR comprises generating an icon on the video display that indicates a location of the POR in the display.4. A system according to wherein the icon is configured to indicate a region of uncertainty (ROU) for the POR.5. A system according to claim 1 , wherein the FOV extends from a distance from the gaze tracker equal to about 0.3 m.6. A system according to claim 1 , wherein the FOV extends to a distance from the gaze tracker equal to or greater than about 2 m.7. A system according to and comprising a light source that illuminates at least a portion of the FOV.8. A system according to wherein the controller adjusts intensity of light provided by the light source responsive to a spatial coordinate of the spatial coordinates determined by the ...

Image Adjusting

An apparatus including a display configured to display an image; and a system for adjusting the image on the display based upon location of a user of the apparatus relative to the apparatus. The system for adjusting includes a camera and an orientation sensor. The system for adjusting is configured to use signals from both the camera and the sensor to determine the location of the user relative to the display.

Executable virtual objects associated with real objects

Embodiments for interacting with an executable virtual object associated with a real object are disclosed. In one example, a method for interacting with an executable virtual object associated with a real object includes receiving sensor input from one or more sensors attached to the portable see-through display device, and obtaining information regarding a location of the user based on the sensor input. The method also includes, if the location includes a real object comprising an associated executable virtual object, then determining an intent of the user to interact with the executable virtual object, and if the intent to interact is determined, then interacting with the executable object.

EYE TRACKING METHOD AND DISPLAY APPARATUS USING THE SAME

A display apparatus employs two tracking units to track the gaze of a user. The display apparatus includes a first tracking unit to generate position information on a user positioned relative to a displayed image; and a second tracking unit to track a gaze of the user upon the displayed image, based on the position information. A method of eye tracking using the display apparatus includes steps of displaying an image; generating position information on a user positioned relative to the displayed image; and tracking a gaze of the user upon the displayed image, based on the position information. 1. A display apparatus comprising:a first tracking unit to generate position information on a user positioned relative to a displayed image; anda second tracking unit to track a gaze of the user upon the displayed image, based on the position information.2. The display apparatus of claim 1 , wherein the first tracking unit comprises:a sensor to detect a distance between the user and the displayed image and to output the position information based on the detected distance.3. The display apparatus of claim 1 , wherein the first tracking unit comprises:a first camera to capture a first image of the user; anda position tracking unit to derive the position information from the captured first image,wherein the position information includes coordinate information indicative of a position of the head of the user within the first image.4. The display apparatus of claim 1 , wherein the position information includes coordinate information indicative of a position of the head of the user.5. The display apparatus of claim 4 , wherein the second tracking unit comprises:a second camera to capture a second image of the user by referencing the coordinate information; andan eye tracking unit to track the gaze of the user by analyzing the second image.6. The display apparatus of claim 5 , wherein the second image corresponds to a portion of the first image.7. A display apparatus comprising:a ...

Systems and Methods for the Manipulation of Captured Light Field Image Data

Systems and methods for the manipulation of captured light fields and captured light field image data in accordance with embodiments of the invention are disclosed. In one embodiment of the invention, a system for manipulating captured light field image data includes a processor, a display, a user input device, and a memory, wherein a depth map includes depth information for one or more pixels in the image data, and wherein an image manipulation application configures the processor to display a first synthesized image, receive user input data identifying a region within the first synthesized image, determine boundary data for the identified region using the depth map, receive user input data identifying at least one action, and perform the received action using the boundary data and the captured light field image data.

GAZE DETECTING HEADS-UP DISPLAY SYSTEMS

A display system for a head-mounted device comprising a display for displaying information. The display system includes a glance detector comprising a light source, disposed proximate the display, for transmitting light toward an eye of a user of the head-mounted device, a light detector, disposed proximate the light source, for detecting light reflected from the eye of the user and generating a voltage based on the detected light. The display system also includes a processor operably coupled to the display, and the gaze detector and configured to control the display to turn-on the display based on the voltage received from the glance detector. 1. A display system for a head-mounted device comprising:a display for displaying information; a light source for transmitting light toward a pupil of an eye of a user of the head-mounted device; and', 'a light detector for detecting light reflected from the pupil of the eye of the user and generating a voltage based on the detected light; and, 'a glance detector disposed proximate the display and comprisinga processor for controlling the display and receiving the voltage from the glance detector, the processor configured to turn-on the display based on the voltage received from the glance detector.2. The display system according to claim 1 , wherein the processor is configured to turn on the display when the voltage received from the glance detector is less than a threshold voltage.3. The display system according to claim 1 , wherein the processor is configured to turn on the display when the voltage received from the glance detector is greater than a threshold voltage.4. The display system according to claim 1 , wherein the light detector generates a voltage based on the detected light in a first time period claim 1 , and generates a voltage based on the detected light in a second time period claim 1 , and the processor is configured to determine a change in a percentage of the voltage generated in the first and second time ...

Image Capture Based on Gaze Detection

Methods and systems for capturing an image are provided. In one example, a head-mounted device (HMD) having an image capturing device, a viewfinder, a gaze acquisition system, and a controller may be configured to capture an image. The image capturing device may be configured to have an imaging field of view including at least a portion of a field of view provided by the viewfinder. The gaze acquisition system may be configured to acquire a gaze direction of a wearer. The controller may be configured to determine whether the acquired gaze direction is through the viewfinder and generate an image capture instruction based on a determination that the acquired gaze direction indicates a gaze through the viewfinder. The controller may further be configured to cause the image capturing device to capture an image.

SCANNING MIRROR LASER AND PROJECTOR HEAD-UP DISPLAY GLASSES

This application consists of a method for wearable heads-up display devices to display image or video that does not require the user to focus his eyes on the display of the device. 1. I claim a method for wearable display devices that displays images or video to the person who wears the device wherein the improvement comprises that the person wearing the device does not have to focus their eyes on the display , instead the eyes can remain relaxed and focused at an arbitraty distance , and this is achieved by directing beams of light toward the eye from various angles.2. The method of claim 1 , wherein a laser beam is scanned and reflected off a surface placed in front of the eye.3. The method of claim 1 , wherein a projector produces the image that is reflected off a surface placed in front of the eye.4. The method of claim 1 , wherein transparent light emitting elements such as laser diodes emit the light into the eye without the need for a reflecting surface.5. The method of claim 1 , wherein the angles of the beams are automatically corrected to match the position of the eyes by using a camera or motion detection device that tracks the position of the eyes. This non-provisional utility patent application claims the benefit of the filing date of the provisional patent application with application Ser. No. 61/607,626 with filing date of Mar. 7, 2012. This non-provisional utility patent application bears the same title as the prior provisional patent application.This application pertains to the field of optics and wearable devices.Current image or video display systems face a challenge when placed close to the eye of a person because the eye cannot focus on the display if the display is placed too close to the eye.The system described herein overcomes the issue of a display placed too close to the eye by directing the rays of light directly towards the eye in the form of beams, each beam corresponding to a pixel of the image.The solution to the problem of having a ...

Control apparatus, electronic device, control method, and program

There is provided a control apparatus including a control unit configured to control information to be displayed on a display screen in a manner that a different type of information is visually recognized according to a positional relationship between the display screen and a predetermined portion of a user viewing the display screen. The information to be displayed on the display screen under control of the control unit is information extracted from information capable of being displayed on the display screen as information having an attribute satisfying a predetermined condition set for each positional relationship.

IMAGE PROCESSING DEVICE AND IMAGE PROCESSING METHOD

An image processing device includes a memory unit, a candidate pupil detecting unit and a pupil determining unit. The memory unit is used in storing information regarding a pupil size. The candidate pupil detecting unit detects noncircular candidate pupils from an image in which an eye area is captured. The pupil determining unit extrapolates the shapes of the candidate pupils that are detected by the candidate pupil detecting unit and, based on the pupil size stored in the memory unit, determines a pupil from among the candidate pupils. 1. An image processing device comprising:a memory unit that is used in storing information regarding a pupil size;a candidate pupil detecting unit that detects noncircular candidate pupils from an image in which an eye area is captured; anda pupil determining unit that extrapolates the shapes of the candidate pupils that are detected by the candidate pupil detecting unit and, based on the pupil size stored in the memory unit, determines a pupil from among the candidate pupils.2. The image processing device according to claim 1 , further comprising a pupil information registering unit that stores the size of the detected pupil in the memory unit.3. The image processing device according to claim 2 , further comprising a pupil information learning unit that claim 2 , when a circular pupil is detected from the image in which an eye area is captured claim 2 , stores brightness pupil information claim 2 , in which the size of the detected pupil and the brightness of the eye area is held in a corresponding manner claim 2 , in the memory unit claim 2 , whereinin the memory unit, the pupil information registering unit stores a detection timing at which a pupil is detected and the size of a pupil in a corresponding manner, andwhen the time difference between the timing at which a candidate pupil is detected and the detection timing stored in the memory unit is equal to or greater than a predetermined amount of time, the pupil determining unit ...

Mobile Device Light Guide Display

Light guide techniques are described. In one or more implementations, an apparatus includes a housing, a light guide supported by the housing, a light engine disposed within the housing and optically coupled to the light guide, and one or more modules disposed within the housing and implemented at least partially in hardware. The one or more modules are configured to cause the light engine to output a user interface for display using the light guide along an image plane focused at infinity.

Eye Gaze Based Location Selection for Audio Visual Playback

In response to the detection of what the user is looking at on a display screen, the playback of audio or visual media associated with that region may be modified. For example, video in the region the user is looking at may be sped up or slowed down. A still image in the region of interest may be transformed into a moving picture. Audio associated with an object depicted in the region of interest on the display screen may be activated in response to user gaze detection.

Gaze Based Communications for Locked-In Hospital Patients

Effective patient-centered care in a hospital relies heavily on the ability of patients to communicate their physical needs to care givers. If a patient is unable to speak, he has limited means of communicating at a time when he needs it the most. The embodiments presented here, generally referred to as EyeVoice, include unobtrusive eye-operated communication systems for locked-in hospital patients who cannot speak or gesture. EyeVoice provides an alternate means of communication, allowing hospital patients to communicate with their care givers using their eyes in place of their voices. Simply by looking at images and cells displayed on a computer screen placed in front of them, patients are able to: answer questions posed by caregivers; specify locations, types and degrees of pain and discomfort; request specific forms of assistance; ask or answer care related questions, and help direct his own care. 1. A system for communicating with locked-in patients , including:a visual display for displaying information to a patient in a healthcare setting;an eyetracker to monitor a gazepoint trace of the patient's eyes within the visual display;means to correlate the gazepoint trace of the patient's eyes with elements of the visual display; andmeans to infer the patient's awareness of a situation by verifying that the patient has looked at one or more designated elements of the visual display.2. The system of claim 1 , wherein the visual display is for the patient claim 1 , and the system further comprising a second visual display for a caregiver.3. The system of wherein a monitor of the visual display for the patient and a second monitor of the second visual display for the caregiver are mounted back to back in an integrated unit.4. A gaze-based system for administering a test to a person claim 2 , including:a visual display for displaying alternative answers to a plurality of questions;an eyetracker that monitors the person's gazepoint within the visual display; andmeans ...

Skin input via tactile tags

The invention is directed to systems, methods and computer program products for determining an input instruction based on a touch associated with a tactile tag. An exemplary method includes detecting, by a device in contact with a user's body, a touch on the user's skin by a tactile tag. The tactile tag comprises a three-dimensional pattern, and the three-dimensional pattern causes a vibration in the user's body when a surface of the tactile tag touches the user's skin. The method additionally includes determining at least one characteristic associated with the touch; and determining an input instruction based at least partially on the at least one characteristic associated with the touch.

DIGITAL CAMERA AND IMAGE CAPTURING METHOD

A digital camera and an image capturing method for photographing at least one object in the digital camera. An image is sensed, and an eye-gazing detection process is accordingly preformed on the image to detect an eye-gazing direction of at least one pair of eyes of the at least one object. It is determined whether the eye-gazing direction meets a gazing criterion. If the eye-gazing direction meets the gazing criterion, an application of the digital camera is triggered. 1. An image processing method for a digital camera , comprising:sensing an image;performing an eye-gazing detection process on the image to detect an eye-gazing direction of at least one eye of at least one object in the image;determining whether the eye-gazing direction meets a gazing criterion, wherein the gazing criterion defines a specific angle of the eye-gazing direction of the at least one eye of the at least one object with respect to the digital camera, and wherein the eye-gazing direction is determined by analyzing pupils of the at least one eye; andtriggering an application of the digital camera if the eye-gazing direction meets the gazing criterion.2. The image processing method as claimed in claim 1 , further comprising claim 1 , asserting an indication signal if the gazing criterion is not met thereby allowing the at least one object to react for meeting the gazing criterion.3. The image processing method as claimed in claim 1 , wherein the indication signal is a voice signal claim 1 , a blinking LED or a displayed character.4. The image processing method as claimed in claim 1 , further comprising configuring the gazing criterion prior to performing the eye-gazing detection process.5. The image processing method as claimed in claim 1 , wherein the gazing criterion defines a number of the at least one of eyes gazing at the digital camera.6. The image processing method as claimed in claim 1 , wherein the eye-gazing detection process is performed when a trigger button of the digital ...

Electronic-payment authentication process with an eye-positioning method for unlocking a pattern lock

This invention discloses an eye-positioning method for unlocking a pattern lock by a user, comprising: showing to the user a pattern diagram on a display, the pattern diagram being pre-stored in a storage device at a back-end server, and is transmitted to the display via a network; capturing a facial image of the user by a video-taking equipment coupled to the display; an on-screen marker real-time tracking an eye of the payer, enabling the marker to move on the pattern diagram to form a marker-movement path according to movement of the eye relative to the video-taking equipment, the eye being either a left eye or a right eye as pre-agreed; and determining if the marker-movement path includes an entirety of a pre-set action path. This invention also provides an electronic-payment authentication method using the aforementioned pattern-unlocking method, for payments at retail sites and for online payments.

Multi-Media Systems, Controllers and Methods for Controlling Display Devices

A multi-media system includes a display device and a controller. The display device receives a video signal indicating image content and displays the image content according to the video signal. The controller captures a user's information, identifies a user's sensory state according to captured user information, and generates a control signal according to the sensory state for controlling the display device. The user information includes image information. The sensory state includes eyeball movement.

Method And System For Monitoring An Industrial System Involving An Eye Tracking System

A method for notifying at least a first condition in an industrial system by way of a monitoring system in order to draw the attention of an operator monitoring the industrial system to the at least one condition. The method includes displaying the first condition in a first portion of a display screen of the monitoring system, determining by eye tracking an area where a user focuses on the display screen, the area where the user focuses differing from the first portion, and alerting the user of the first condition in order to allow the user to be notified of the first condition in the industrial system. A monitoring system is also presented herein. 1. A method for notifying at least a first condition in an industrial system , the first condition being abnormal behavior in the industrial system , by means of a monitoring system , the method comprising:displaying the first condition in a first portion of a display screen of the monitoring system,determining by means of eye-tracking an area where a user focuses on the display screen, and if the area where the user focuses differs from the first portion,alerting the user of the first condition in order to allow the user to be notified of the first condition in the industrial system, wherein the alerting comprises magnifying at least one object displayed in the first portion of the display screen.2. The method as claimed in claim 1 , comprising:verifying that the user has been notified of the first condition by detecting a user action, andreturning to a normal viewing mode of the display screen.3. The method as claimed in claim 1 , wherein the alerting comprises setting off an audible alarm.4. The method as claimed in claim 1 , wherein the alerting comprises setting off a visual alarm on the display screen.5. The method as claimed in claim 1 , comprising:displaying a second condition of the industrial system in a second portion of the display screen, the second portion differing from the area where the user focuses, ...

SYSTEM AND METHOD FOR TRACKING GAZE AT DISTANCE

A system and method for tracking a gaze at a distance are provided. A remote gaze tracking system may include an infrared lighting unit including a plurality of infrared lightings to emit an infrared light toward a user, a gaze tracking module to track a position of a face of the user, and to collect, from the tracked position of the face, an eye image including at least one reflected light among a plurality of corneal reflected lights and a lens-reflected light, the corneal reflected lights being reflected from a cornea by the emitted infrared light, and the lens-reflected light being reflected from a lens of glasses, and a processor to compare a magnitude of the lens-reflected light with a threshold in the collected eye image, and when the magnitude of the lens-reflected light is equal to or less than the threshold, to detect coordinates of a center of each of the plurality of corneal reflected lights, and to calculate a gaze position. 1. A remote gaze tracking system , comprising:an infrared lighting unit comprising a plurality of infrared lightings to emit an infrared light toward a user;a gaze tracking module to track a position of a face of the user, and to collect, from the tracked position of the face, an eye image comprising at least one reflected light among a plurality of corneal reflected lights and a lens-reflected light, the corneal reflected lights being reflected from a cornea by the emitted infrared light, and the lens-reflected light being reflected from a lens of glasses; anda processor to compare a magnitude of the lens-reflected light with a threshold in the collected eye image, and when the magnitude of the lens-reflected light is equal to or less than the threshold, to detect coordinates of a center of each of the plurality of corneal reflected lights, and to calculate a gaze position.2. The remote gaze tracking system of claim 1 , wherein claim 1 , when the magnitude of the lens-reflected light is equal to or less than the threshold claim 1 , ...

GAZE DETECTION IN A SEE-THROUGH, NEAR-EYE, MIXED REALITY DISPLAY

The technology provides various embodiments for gaze determination within a see-through, near-eye, mixed reality display device. In some embodiments, the boundaries of a gaze detection coordinate system can be determined from a spatial relationship between a user eye and gaze detection elements such as illuminators and at least one light sensor positioned on a support structure such as an eyeglasses frame. The gaze detection coordinate system allows for determination of a gaze vector from each eye based on data representing glints on the user eye, or a combination of image and glint data. A point of gaze may be determined in a three-dimensional user field of view including real and virtual objects. The spatial relationship between the gaze detection elements and the eye may be checked and may trigger a re-calibration of training data sets if the boundaries of the gaze detection coordinate system have changed. 1. A method for determining gaze in a near-eye mixed reality display device comprising:determining boundaries of a three dimensional gaze detection coordinate system for each eye based on positions of glints detected on the respective eye, positions on the near-eye display device of illuminators for generating the glints, and a position of at least one sensor for capturing reflected eye data of the respective eye from which the positions of glints are detected;automatically checking periodically the reflected eye data for any change along a depth axis of the gaze detection coordinate system, the depth axis extending from a respective display optical system of the near-eye, mixed reality display device toward the respective eye;responsive to detecting any change along at least one depth axis, triggering redetermination of the boundaries of the three dimensional gaze detection coordinate system;determining a gaze vector for each eye based on reflected eye data including the glints and the boundaries of the three dimensional gaze detection coordinate system; ...

APPARATUS AND METHOD OF CONTROLLING MOBILE TERMINAL BASED ON ANALYSIS OF USER'S FACE

An apparatus and method of controlling a mobile terminal by detecting a face or an eye in an input image are provided. The method includes performing face recognition on an input image facing and being captured by an image input unit equipped on the front face of the mobile terminal; determining, based on the face recognition, user state information that includes whether a user exists, a direction of the user's face, a distance from the mobile terminal, and/or a position of the user's face; and performing a predetermined function of the mobile terminal according to the user state information. According to the method, functions of the mobile terminal may be controlled without direct inputs from the user.

User input processing with eye tracking

A system determines which user of multiple users provided input through a single input device. A mechanism captures images of the one or more users. When input is detected, the images may be processed to determine which user provided an input using the input device. The images may be processed to identify each users head and eyes, and determine the focus point for each user's eyes. The user which has eyes focused at the input location is identified as providing the input. When the input mechanism is a touch screen, the user having eyes focused on the touch screen portion which was touched is identified as the source of the input.

USER INTERFACE CONTROL DEVICE, USER INTERFACE CONTROL METHOD, COMPUTER PROGRAM AND INTEGRATED CIRCUIT

A user interface control device provides a GUI allowing a depth of a graphic to be easily set when composing the graphic with a stereoscopic image. The device comprises: a graphic information obtaining unit that specifies an area occupied by the graphic when the graphic is arranged on one of two viewpoint images forming a stereoscopic image; a depth information analyzing unit that acquires a depth of a subject appearing within the specified area occupied by the graphic in the one viewpoint image; and a depth setting presenting unit that presents a first alternative and a second alternative for setting a depth of the graphic, the first alternative corresponding to the depth of the subject, and the second alternative corresponding to a depth differing for the depth of the subject. 1. A user interface control device that provides a user interface for setting , when composing a graphic with a stereoscopic image , a depth at which to position the graphic in the stereoscopic image , the user interface control device comprising:a planar position specifying unit configured to specify with respect to a graphic arranged on one of two viewpoint images forming a stereoscopic image, an area occupied by the graphic in the one viewpoint image;a viewpoint image depth acquiring unit configured to acquire a depth of a subject appearing within the area occupied by the graphic in the one viewpoint image; anda presenting unit configured to present a first alternative and a second alternative for setting a depth of the graphic in the stereoscopic image, the first alternative corresponding to the depth of the subject, and the second alternative corresponding to a depth differing from the depth of the subject.2. The user interface control device in claim 1 , whereinthe first alternative corresponds to a depth of a subject of smallest depth among one or more subjects appearing within the area occupied by the graphic, andthe second alternative corresponds to a smaller depth than the depth of ...

Method for determining gaze direction and device for same

The present disclosure relates to a device ( 10 ) for supporting, in motion, a mobile terminal ( 20 ) having a display surface ( 21 ), such as a mobile phone or a similar object. The invention is characterised in that it comprises: a supporting element ( 11 ) forming a hand grip for the device; and a plate ( 12 ) mounted such that it can move in relation to the supporting element ( 11 ), said plate ( 12 ) forming a housing that is intended to receive the mobile terminal in the use position, in which the display surface ( 21 ) can be observed by a user. The device comprises a plurality of reference light sources ( 16 a , 16 b , 16 c , 16 d ) linked to the supporting element ( 11 ), which can emit light towards the user's eye and which are disposed in the environment of the housing in a two-dimensional array coplanar with the display surface ( 21 ) of the mobile terminal that is intended to be received in the housing.

AUDIO USER INTERACTION RECOGNITION AND APPLICATION INTERFACE

Disclosed is an application interface that takes into account the user's gaze direction relative to who is speaking in an interactive multi-participant environment where audio-based contextual information and/or visual-based semantic information is being presented. Among these various implementations, two different types of microphone array devices (MADs) may be used. The first type of MAD is a steerable microphone array (a.k.a. a steerable array) which is worn by a user in a known orientation with regard to the user's eyes, and wherein multiple users may each wear a steerable array. The second type of MAD is a fixed-location microphone array (a.k.a. a fixed array) which is placed in the same acoustic space as the users (one or more of which are using steerable arrays). 1. A method for displaying information , the method comprising:detecting a gaze direction line-of-sight for a user based on at least an audio detection component from a fixed array and at one steerable array;determining if a display device is in the line-of-sight of the user;if the display device is in the line-of-sight of the user, then displaying a first image on the display device; andif the display device is not in the line-of-sight of the user, then displaying a second image on the display.2. The method of claim 1 , wherein the display device is a personal display device that is movable into the line-of-sight of the user.3. The method of claim 2 , wherein the first image specifically pertains to that which the user is looking at based on the gaze direction.4. The method of claim 1 , wherein the second image is one of a blank display or a non-specific display.5. The method of claim 1 , wherein the line-of-sight determination comprises both a horizontal component corresponding to a target and a vertical component corresponding to whether the device is being held up to be viewed claim 1 , wherein both components must be true in order for the determination to be true.6. A method for displaying ...

Unlocking a Screen Using Eye Tracking Information

Methods and systems for unlocking a screen using eye tracking information are described. A computing system may include a display screen. The computing system may be in a locked mode of operation after a period of inactivity by a user. Locked mode of operation may include a locked screen and reduced functionality of the computing system. The user may attempt to unlock the screen. The computing system may generate a display of a moving object on the display screen of the computing system. An eye tracking system may be coupled to the computing system. The eye tracking system may track eye movement of the user. The computing system may determine that a path associated with the eye movement of the user substantially matches a path associated with the moving object on the display and switch to be in an unlocked mode of operation including unlocking the screen. 1. A method comprising:receiving eye-tracking information;based on the eye-tracking information, determining that a gaze characteristic substantially matches a predetermined gaze characteristic; andin response to determining that the gaze characteristic substantially matches the predetermined gaze characteristic, causing a computing device to switch from operating in a first mode of operation to operating in a second mode of operation, wherein the second mode of operation comprises a state of increased responsiveness to user-input as compared to the first mode of operation.2. The method of claim 1 , wherein the computing device includes a head-mounted display (HMD) claim 1 , and wherein the eye-tracking information is indicative of eye movement with respect to the head-mounted display.3. The method of claim 1 , further comprising:adjusting the eye-tracking information based on a location of a gaze axis of an eye with respect to a reference axis.4. The method of claim 3 , wherein adjusting the eye-tracking information comprises:applying a transform to the eye-tracking information, and wherein the transform comprises ...

GAZE DETECTION IN A SEE-THROUGH, NEAR-EYE, MIXED REALITY DISPLAY

The technology provides various embodiments for gaze determination within a see-through, near-eye, mixed reality display device. In some embodiments, the boundaries of a gaze detection coordinate system can be determined from a spatial relationship between a user eye and gaze detection elements such as illuminators and at least one light sensor positioned on a support structure such as an eyeglasses frame. The gaze detection coordinate system allows for determination of a gaze vector from each eye based on data representing glints on the user eye, or a combination of image and glint data. A point of gaze may be determined in a three-dimensional user field of view including real and virtual objects. The spatial relationship between the gaze detection elements and the eye may be checked and may trigger a re-calibration of training data sets if the boundaries of the gaze detection coordinate system have changed. 1. A gaze vector determination system for use in a mixed reality display system comprising:at least one image generation unit for generating at least one virtual image for display by a near-eye, mixed reality display device; a set of illuminators for generating glints on the at least one eye each illuminator positioned on the display device at a respective predetermined position and generating illumination about a predetermined wavelength, and', 'at least one sensor for capturing light reflected from the at least one eye and generating data representing the captured reflected light, the at least one sensor positioned at a predetermined position in relation to predetermined positions of the set of illuminators on the display device;, 'a respective arrangement of gaze detection elements positioned on the display device for forming a spatial relationship between the gaze detection elements and at least one eye, the gaze detection elements including'}a memory for storing software and the data; and in a time period, using a first technique that comprises determining ...

DISPLAY CONTROL DEVICE AND DISPLAY CONTROL METHOD

A display control device is provided, which suppresses a display content from being scrolled when a viewer does not consciously pay attention to the display content. The display control device includes: a display unit ; an image pickup unit that detects an eye position of a viewer from a taken image of the viewer; an attention state determination unit that determines an attention state to the display content, based on movement of the detected viewer's eye position; a gaze position specifying unit that specifies a gaze position on the display content, based on the detected viewer's eye position and the display content; and a display control unit that causes the display unit to display a non-displayed portion of displayed program information, based on the determined attention state and the specified gaze position, when the viewer pays attention to the non-displayed portion. 1. A display control device , comprising:a display unit that displays, on a display screen, a display content, and non-display information indicating that the display content has a non-displayed portion;an image pickup unit that takes an image of a viewer, and detects an eye position of the viewer from the taken image of the viewer;an attention state determination unit that determines an attention state to the display content on the display screen, based on movement of the detected viewer's eye position;a gaze position specifying unit that specifies a gaze position on the display content, based on the detected viewer's eye position, and the display content on the display screen; anda display control unit that causes the display unit to display the non-displayed portion, in a case where the attention state determination unit has determined the attention state to the display content on the display screen, and the display control unit has determined that the non-display information is gazed, based on the gaze position specified by the gaze position specifying unit.2. The display control device ...

ARRANGEMENT, METHOD AND COMPUTER PROGRAM FOR CONTROLLING A COMPUTER APPARATUS BASED ON EYE-TRACKING

A computer apparatus is associated with a graphical display presenting at least one GUI-component adapted to be manipulated based on user-generated commands. An event engine is adapted to receive an eye-tracking data signal that describes a user's point of regard on the display. Based on the signal, the event engine produces a set of non-cursor controlling event output signals, which influence the at least one GUI-component. Each non-cursor controlling event output signal describes a particular aspect of the user's ocular activity in respect of the display. Initially, the event engine receives a control signal request from each of the at least one GUI-component. The control signal request defines a sub-set of the set of non-cursor controlling event output signals which is required by the particular GUI-component. The event engine delivers non-cursor controlling event output signals to the at least one GUI-component in accordance with each respective control signal request. 1. An arrangement for controlling a computer apparatus associated with a graphical display , the display displaying a representation of at least one GUI-component , the arrangement comprising:an event engine configured to receive an eye-tracking data signal describing a user's point of regard on the display, and configured to influence the representation of the at least one GUI-component at least based on the eye-tracking data signal,wherein upon the user's point of regard remaining fixated within bounds of the representation of the at least one GUI-component for a predetermined period of time, the event engine influences the computer apparatus to expand the at least one GUI-component such that more information than was initially visible in the at least one GUI-component is displayed.2. The arrangement according to claim 1 , wherein the at least one GUI-component is a button claim 1 , icon claim 1 , a text field claim 1 , or a menu.3. The arrangement according to claim 2 , wherein the at least one ...

Gaze-Based Display Control

A method includes receiving an image including an eye of a user of a computerized system and identifying, based the image of the eye, a direction of a gaze performed by the user. Based on the direction of the gaze, a region on a display coupled to the computerized system is identified, an operation is performed on content presented in the region. 1. A method , comprising:receiving an image including an eye of a user of a computerized system;identifying, based the image of the eye, a direction of a gaze performed by the user;identifying, based on the direction of the gaze, a region on a display coupled to the computerized system; andperforming an operation on content presented in the region.2. The method according to claim 1 , wherein the image of the eye is selected from a list comprising a two dimensional image and a three dimensional map.3. The method according to claim 1 , wherein the content comprises an image claim 1 , and performing the operation comprises bringing the region into focus.4. The method according to claim 1 , and comprising identifying a target point on the display in the direction of the gaze claim 1 , identifying an interactive item positioned in proximity to the target point claim 1 , and calculating a calibration coefficient based on the proximity of the target point to the interactive item.5. A method claim 1 , comprising:presenting, by a computerized system, multiple interactive items on a display coupled to the computer;receiving, from a sensing device coupled to the computer, an input representing a gaze direction of a user;identifying a target point on the display based on the gaze direction;associating the target point with a first interactive item appearing on the display; andresponsively to the target point, opening one or more second interactive items on the display.6. The method according to claim 5 , wherein presenting the multiple interactive items comprises arranging the multiple items in a concentric pattern on the display claim ...

ARRANGEMENT, METHOD AND COMPUTER PROGRAM FOR CONTROLLING A COMPUTER APPARATUS BASED ON EYE-TRACKING

A computer apparatus is associated with a graphical display presenting at least one GUI-component adapted to be manipulated based on user-generated commands. An event engine is adapted to receive an eye-tracking data signal that describes a user's point of regard on the display. Based on the signal, the event engine produces a set of non-cursor controlling event output signals, which influence the at least one GUI-component. Each non-cursor controlling event output signal describes a particular aspect of the user's ocular activity in respect of the display. Initially, the event engine receives a control signal request from each of the at least one GUI-component. The control signal request defines a sub-set of the set of non-cursor controlling event output signals which is required by the particular GUI-component. The event engine delivers non-cursor controlling event output signals to the at least one GUI-component in accordance with each respective control signal request. 1. An arrangement for controlling a computer apparatus associated with a graphical display , the graphical display displaying a representation of at least one GUI-component , the arrangement comprising:an event engine configured to receive both an eye-tracking data signal describing a user's point of regard on the display and at least one auxiliary input signal, and configured to produce a non-cursor controlling event based on a combination of the eye-tracking data signal and the at least one auxiliary input signal,wherein the non-cursor controlling event causes the computer apparatus to alter the display of the at least one GUI-component.2. The arrangement according to claim 1 , wherein the auxiliary input signal originates from a button claim 1 , a switch claim 1 , a speech signal claim 1 , eye-tracking data claim 1 , a movement pattern of an input member claim 1 , a camera registered gesture pattern claim 1 , a camera image claim 1 , a keyboard signal claim 1 , a mouse signal claim 1 , a facial ...

EYE DIRECTION DETECTING APPARATUS AND EYE DIRECTION DETECTING METHOD

An eye direction detecting apparatus includes a display unit, an imaging unit, a first detecting unit, a second detecting unit, and a display control unit. The imaging unit captures a subject. The first detecting unit detects a position of an eye of the subject from an image captured by the imaging unit. The second detecting unit detects a distance from the imaging unit to the position of the eye of the subject. The display control unit displays the image indicating the position of the eye of the subject on the display unit with a display manner being changed according to the distance detected by the second detecting unit. 1. An eye direction detecting apparatus comprising:a display unit;an imaging unit configured to image a subject;a first detecting unit configured to detect a position of an eye of the subject from an image captured by the imaging unit;a second detecting unit configured to detect a distance to the position of the eye of the subject from the imaging unit; anda display control unit configured to display an image representing the position of the eye of the subject corresponding to the position of the eye detected by the first detecting unit onto the display unit with a display manner being changed according to the distance detected by the second detecting unit.2. The eye direction detecting apparatus according to claim 1 , whereinthe display control unit displays the image representing the position of the eye of the subject onto the display unit with the size of the image being changed according to the distance detected by the second detecting unit.3. The eye direction detecting apparatus according to claim 2 , wherein the display unit to display a marker corresponding to a size of the image indicating the position of the eye of the subject on a reference distance set beforehand;', 'the display unit to change the image indicating the position of the eye of the subject to be smaller than the marker and to display the resultant, when the distance ...

Eyewear

A problem related to a known eyewear-type electro-oculogram measuring apparatus which detects the eye potential using a pair of electrodes positioned outside both the eyes of a user and a pair of electrodes respectively positioned above and below one eye is that the two pairs of electrodes have had an impact on the skins of users, and discomfort on them. Besides, the electrodes are not excellent in design. The present invention provides eyewear including: a frame; a pair of nose pads; and a first electrode and a second electrode respectively provided on the surface of the pair of nose pads, the first electrode and the second electrode detecting eye potential

METHOD FOR DETECTING POINT OF GAZE AND DEVICE FOR DETECTING POINT OF GAZE

A gaze point detection device has four cameras , light sources , control circuits , and an image processor . The image processor calculates vectors r, each of which is from a corneal reflection point to the center of a pupil, on a plane that is vertical to base lines. The image processor also calculates angles θ of the line of sight on the basis of the vectors r and by using a function f including M parameters. Moreover, the image processor determines the M parameters based on the angles θ, and detects a point of gaze Q based on the line of sight direction calculated using the determined parameters. The number of cameras is set at M×1/2 or higher. 1. A gaze point detection method , comprising:a face image generation step of generating face images of a subject by using an N number of cameras (N is a natural number of 2 or more) and a plurality of light sources;a vector calculation step of calculating vectors r based on the face images generated by the N number of cameras, the vectors r each representing an actual distance between a center of a pupil of the subject and a corneal reflection point on a cornea of the subject on which light from the light sources reflects;{'sub': '0', 'claim-text': {'br': None, 'i': f', 'r−r, 'sub': '0', 'θ=(||)\u2003\u2003(1)'}, 'a line of sight direction calculation step of calculating angles θ of a gaze of the subject with respect to base lines connecting the center of the pupil and the N number of cameras, based on the vectors r corresponding to the N number of cameras, by using the following formula (1) in use of a function f and an M number of undetermined constants (M is a natural number of 3 or more) including at least an offset vector rof each of the vectors r;'}an undetermined constant determination step of determining the M number of undetermined constants included in the function f, by using a plurality of relational expressions that are derived based at least on the angles θ calculated with respect to the N number of cameras; ...

Control of displayed content in virtual environments

A system and method are disclosed for controlling content displayed to a user in a virtual environment. The virtual environment may include virtual controls with which a user may interact using predefined gestures. Interacting with a virtual control may adjust an aspect of the displayed content, including for example one or more of fast forwarding of the content, rewinding of the content, pausing of the content, stopping the content, changing a volume of content, recording the content, changing a brightness of the content, changing a contrast of the content and changing the content from a first still image to a second still image.

Device and method for eye tracking data trigger arrangement

A system and method tracks a user's eye for a data trigger arrangement. The method includes obtaining first image data within a first vision field of a first data capturing sensor. The method includes determining a first area viewed by a user's eye as a function of the first image data. The method includes obtaining second image data within a second vision field of a second sensor. The method includes determining a second area captured within the second field of vision as a function of the second image data. The method includes determining disposition data of the first area within the second area. When a trigger command is captured, the method includes data capturing a machine readable representation of data (MRRoD) using the second sensor as a function of the disposition data.

Method of Determining Reflections of Light

A method of filtering glints by processing an image of a user's cornea to obtain coordinates of desired glints from a configuration of light sources, comprising processing an image, in a first image space, of a user's cornea to determine coordinates of respective multiple positions of glints; and iteratively: selecting from the coordinates a first and a second set of coordinates; computing from the first set of coordinates a transformation that transforms the first set of coordinates into first coordinates of a predetermined spatial configuration; and testing whether the transformation transforms also the second set into positions that match second positions of the predetermined configuration. The coordinates of the desired glints are selected as those first and second sets which are transformed into coordinates that match the first and second coordinates of the predetermined configuration. The method is based on a geometrical homography and is expedient for robust gaze estimation in connection with e.g. eye tracking. 1. A method of determining reflections of light to obtain coordinates of desired reflections from a predefined configuration of light , comprising:processing an image of an individual's cornea to determine multiple coordinates of reflections of light, respectively; selecting from the coordinates a first and a second set of coordinates;', 'computing from the first set of coordinates a transformation that transforms the first set of coordinates into first coordinates of a predetermined spatial configuration; and', 'testing whether the transformation transforms also the second set into positions that match second positions of the predetermined configuration;, 'iterativelyselecting the coordinates of the desired reflections to be those first and second sets which are transformed into coordinates that match the first and second coordinates of the predetermined configuration.2. A method according to claim 1 , further comprising:applying the transformation ...

EYE TRACKING BASED SELECTIVE ACCENTUATION OF PORTIONS OF A DISPLAY

Systems, apparatus, articles, and methods are described including operations for eye tracking based selective accentuation of portions of a display. 126.-. (canceled)27. A computer-implemented method for selectively accentuating a focus area on a display of a computer , comprising:receiving eye movement data of one or more users;performing eye tracking for at least one of the one or more users based at least in part on the received eye movement data;determining a region of interest associated with a portion of the display of the computer system based at least in part on the performed eye tracking; andselectively accentuating the focus area, wherein the focus area corresponds with the portion of the display associated with the determined region of interest.28. The method of claim 27 , wherein the selective accentuation of the focus area comprises zooming in on the focus area.29. The method of claim 27 , wherein the selective accentuation of the focus area comprises out scaling the focus area.30. The method of claim 27 , wherein the selective accentuation of the focus area comprises highlighting the focus area claim 27 , wherein the highlighting the focus area comprises framing the focus area claim 27 , re-coloring the focus area claim 27 , and/or framing and re-coloring the focus area.31. The method of claim 27 , wherein the selective accentuation of the focus area comprises selectively accentuating the focus area based at least in part on a default area size.32. The method of claim 27 , wherein the selective accentuation of the focus area comprises selectively accentuating the focus area based at least in part on associating the region of interest with a discrete display element claim 27 , wherein the discrete display element comprises a text box claim 27 , a paragraph of text claim 27 , a default number of text lines claim 27 , a picture claim 27 , and/or a menu.33. The method of claim 27 , further comprising:selectively accentuating one or more subsequent focus ...

METHOD AND APPARATUS FOR GAZE POINT MAPPING

An apparatus for mapping a gaze point of a subject on a scene image to a gaze point in a reference image, wherein said scene image and said reference image have been taken by a camera from a different position, said apparatus comprising: A module for executing a feature detection algorithm on said reference image to identify a plurality of characteristic features and their locations in said reference image; a module for executing said feature detection algorithm on said scene image to re-identify said plurality of characteristic features and their locations in said scene image; a module for determining a point transfer mapping that transforms point positions between said scene image and said reference image based on the locations of said plurality of characteristic features detected in said reference image and said scene image; a module for using said point transfer mapping to map a gaze point which has been determined in said scene image to its corresponding point in said reference image. 1. An apparatus for mapping a gaze point of a subject on a scene image to a gaze point in a reference image , wherein said scene image and said reference image have been taken by a camera from a different position , said apparatus comprising:A module for executing a feature detection algorithm on said reference image to identify a plurality of characteristic features and their locations in said reference image;a module for executing said feature detection algorithm on said scene image to re-identify said plurality of characteristic features and their locations in said scene image;a module for determining a point transfer mapping that transforms point positions between said scene image and said reference image based on the locations of said plurality of characteristic features detected in said reference image and said scene image;a module for using said point transfer mapping to map a gaze point which has been determined in said scene image to its corresponding point in said ...

METHOD AND SYSTEM FOR ROTATING DISPLAY IMAGE

A method and a system for rotating a display image applied to an electronic display device which includes a display image are provided. The method includes at least the following steps: enabling a rotation mode; capturing user information relative to the electronic display device via an image capturing unit; receiving the user information via an analysis unit and generating analysis information according to the user information; and receiving the analysis information via a processing unit and generating a control signal to control the display image to rotate according to the analysis information. 1. A method for rotating a display image , applied to an electronic display device which includes a display image , the method comprising:enabling a rotation mode;capturing user information relative to the electronic display device via an image capturing unit;receiving the user information via an analysis unit and generating analysis information according to the user information; andreceiving the analysis information via a processing unit and generating a control signal to control the display image to rotate according to the analysis information.2. The method for rotating the display image according to claim 1 , further including:receiving the control signal and making the display image rotate by an angle via an execution unit.3. The method for rotating the display image according to claim 1 , wherein the image capturing unit is an eye tracking component.4. The method for rotating the display image according to claim 1 , wherein the user information includes one or a combination of an eye image claim 1 , an eye coordinate claim 1 , a face image claim 1 , a head image claim 1 , an upper part of a body.5. The method for rotating the display image according to claim 1 , wherein the step of generating the analysis information according to the user information includes:analyzing a relative position between a nose and a lip in a head image;determining a tilt angle of the head ...

ARRANGEMENT, METHOD AND COMPUTER PROGRAM FOR CONTROLLING A COMPUTER APPARATUS BASED ON EYE-TRACKING

A computer apparatus is associated with a graphical display presenting at least one GUI-component adapted to be manipulated based on user-generated commands An event engine is adapted to receive an eye-tracking data signal that describes a user's point of regard on the display. Based on the signal, the event engine produces a set of non-cursor controlling event output signals, which influence the at least one GUI-component. Each non-cursor controlling event output signal describes a particular aspect of the user's ocular activity in respect of the display. Initially, the event engine receives a control signal request from each of the at least one GUI-component. The control signal request defines a sub-set of the set of non-cursor controlling event output signals which is required by the particular GUI-component. The event engine delivers non-cursor controlling event output signals to the at least one GUI-component in accordance with each respective control signal request. 1. An arrangement for controlling a computer apparatus associated with a graphical display , the graphical display displaying a representation of at least one GUI-component , the arrangement comprising:an event engine comprising a library of pre-defined GUI-components capable of being manipulated by eye-tracking data,wherein the event engine is configured to provide each of the GUI-components to a respective application on the computer apparatus, andwherein the event engine is configured to manipulate the GUI-components based on a user's ocular activity.2. The arrangement according to claim 1 , wherein the library comprises at least eye-buttons claim 1 , scroll-bars claim 1 , multiview toolbars claim 1 , text input fields claim 1 , expandable text input fields and scroll windows.3. The arrangement according to claim 2 , wherein the multiview toolbar comprises at least one eye-button.4. The arrangement according to claim 3 , wherein at least one eye-button comprises a thumbnail image of an ...

METHOD AND SYSTEM FOR CONTROLLING EYE TRACKING

A method for controlling an eye tracking applied to an electronic display device which includes a display screen and executes at least one application program is provided. The method includes at least the following steps: enabling an eye tracking control mode; capturing an eye image via an eye tracking detection unit and generating eye position information; receiving the eye position information via an analysis unit and mapping the eye position information to the display screen to generate analysis information; receiving the analysis information via a processing unit and generating an eye tracking cursor according to the analysis information, wherein the processing unit generates a preview window on the display screen according to a position of the application program corresponding to the eye tracking cursor. 1. A method for controlling an eye tracking , applied to an electronic display device which includes a display screen and executes at least one application program , the method comprising:enabling an eye tracking control mode;capturing an eye image via an eye tracking detection unit and generating eye position information;receiving the eye position information via an analysis unit and mapping the eye position information to the display screen to generate analysis information; andreceiving the analysis information via a processing unit and generating an eye tracking cursor according to the analysis information, wherein the processing unit generates a preview window on the display screen according to a position of the application program corresponding to the eye tracking cursor.2. The method for controlling the eye tracking according to claim 1 , wherein the preview window is selected from image information of the application program.3. The method for controlling the eye tracking according to claim 1 , wherein the preview window includes at least one control item.4. The method for controlling the eye tracking according to claim 3 , further including:executing the ...

Methods and systems for synchronization and distribution of multiple physiological and performance measures

Methods and systems are disclosed including receiving, by circuitry of a computer, physiological data comprising physiological information gathered by a physiological sensor regarding a subject, the subject having physiological reactions to an event that is external to the subject, wherein at least one of the physiological data also comprises a first time at which the physiological data was recorded, wherein the first time is based on a global reference time frame; receiving non-physiological data comprising information regarding the external event, wherein at least one of the non-physiological data also comprises a second time at which the non-physiological data was recorded, wherein the second time is based on the global reference time frame; correlating the physiological data with the non-physiological data based on the first and second times; and generating physiological reaction data indicative of a physiological reaction of the subject to the external event.

Method for correcting user's gaze direction in image, machine-readable storage medium and communication terminal

A method and mobile terminal for correcting a gaze of a user in an image includes setting eye outer points that define an eye region of the user in an original image, transforming the set eye outer points to a predetermined reference camera gaze direction, and transforming the eye region of the original image based on the transformed eye outer points.

Display apparatus control system and method and apparatus for controlling a plurality of displays

A display apparatus control system and a method and apparatus for controlling a plurality of display apparatuses are provided. The method of controlling a plurality of display apparatuses includes capturing a region at which a user is located, generating a user-captured image, determining which display apparatus the user is looking at from among the plurality of display apparatuses, based on the user-captured image, and determining the determined display apparatus to be a control target display apparatus, and transmitting a control signal which corresponds to a user's command to be the control target display apparatus when the user's command is input.

Speech and gesture recognition enhancement

The recognition of user input to a computing device is enhanced. The user input is either speech, or handwriting data input by the user making screen-contacting gestures, or a combination of one or more prescribed words that are spoken by the user and one or more prescribed screen-contacting gestures that are made by the user, or a combination of one or more prescribed words that are spoken by the user and one or more prescribed non-screen-contacting gestures that are made by the user.

GAZE DETECTION IN A 3D MAPPING ENVIRONMENT

A method, including receiving a three-dimensional (3D) map of at least a part of a body of a user () of a computerized system, and receiving a two dimensional (2D) image of the user, the image including an eye () of the user. 3D coordinates of a head () of the user are extracted from the 3D map and the 2D image, and a direction of a gaze performed by the user is identified based on the 3D coordinates of the head and the image of the eye. 1. A method , comprising:receiving a three-dimensional (3D) map of at least a part of a body of a user of a computerized system;receiving a two dimensional (2D) image of the user, the image including an eye of the user;extracting, from the 3D map and the 2D image, 3D coordinates of a head of the user; andidentifying, based on the 3D coordinates of the head and the image of the eye, a direction of a gaze performed by the user.2. The method according to claim 1 , wherein identifying the direction of the gaze comprises analyzing light reflected off an element of the eye.3. The method according to claim 2 , wherein the element is selected from a list comprising a pupil claim 2 , an iris and a cornea.4. The method according to claim 1 , wherein extracting the 3D coordinates of the head comprises segmenting the 3D map in order to extract a position of the head along a horizontal axis claim 1 , a vertical axis claim 1 , and a depth axis.5. The method according to claim 1 , wherein extracting the 3D coordinates of the head comprises identifying claim 1 , from the 2D image a first position of the head along a horizontal axis and a vertical axis claim 1 , and segmenting the 3D map in order to identify claim 1 , from the 3D map claim 1 , a second position of the head along a depth axis.6. The method according to claim 1 , and comprising controlling a function of the computerized system responsively to the direction of the gaze.7. The method according to claim 6 , wherein controlling the function of the computerized system comprises identifying ...

IMAGE MANIPULATION BASED ON TRACKED EYE MOVEMENT

The disclosure relates to controlling and manipulating an image of an object on a display device based on tracked eye movements of an observer. When the object is displayed according to an initial view, the observer's eye movement is tracked and processed in order to determine the focus of the observer's attention or gaze on the image. Thereafter, the displayed image is modified to provide a better view of the part of the object in which the observer is most interested. This is accomplished by modifying at least one of the spatial positioning of the object within the viewing area, the angle of view of the object, and the viewing direction of the object. 1. A method , comprising:obtaining a data point of a tracked eye gaze at a predetermined sampling interval of a displayed image of an object in a viewing area, the data point representing a detected location of the tracked eye gaze within the viewing area;determining a range of the viewing area in which the data point is located;magnifying the displayed image when the determined range is smaller than a first predetermined range; andshrinking the displayed image when the determined range is larger than a second predetermined range.2. The method of claim 1 , further comprising analyzing the data point to identify a region of the displayed image corresponding to a high degree of eye attentiveness based on the tracked eye gaze.3. The method of claim 2 , further comprising modifying the displayed image by moving the object to thereby move the identified region of the object to a different location of the viewing area in the modified displayed image.4. The method of claim 3 , wherein the modifying includes moving the identified region of the object to the center of the viewing area.5. The method of claim 1 , wherein the first and second predetermined ranges include predetermined viewing angles.6. The method of claim 5 , further comprising computing a viewing angle of the viewing area based on the tracked eye gaze.7. The ...

FAST WAKE-UP IN A GAZE TRACKING SYSTEM

A gaze tracking system, leaving a low power mode in response to an activation signal, captures an initial burst of eye pictures in short time by restricting the image area of a sensor, with the purpose of enabling an increased frame rate. Subsequent eye pictures are captured at normal frame rate. The first gaze point value is computed memorylessly based on the initial burst of eye pictures and no additional imagery, while subsequent values may be computed recursively by taking into account previous gaze point values or information from previous eye pictures. The restriction of the image area may be guided by a preliminary overview picture captured using the same or a different sensor. From the gaze point values, the system may derive a control signal to be supplied to a computer device with a visual display. 1. A method in a gaze tracking system for controlling a computer device associated with a visual display , said eye tracking system comprising at least one sensor with a light-sensitive surface arranged to receive light from the head of a viewer of the visual display , wherein:in response to a gaze tracking activation signal from input means of said computer device at said gaze tracking system, leaving a low power mode of said gaze tracking system;capturing an overview picture of at least a portion of the head of the viewer and locating an eye region of the viewer in the overview picture;capturing, at a first frame rate, a plurality of pictures of at least a portion of the viewer's head using said at least one sensor, from which data are read out in restricted regions, which are positioned in accordance with the eye region location and the union of which occupies at most 10% of the area of the light-sensitive surface;determining a first gaze point value of said viewer on the visual display based on information from said plurality of pictures only and optionally on additional information from the overview picture;providing a first control signal to the computer ...

INPUT DEVICE, INPUT METHOD, AND COMPUTER PROGRAM

An input device detects a sight line position using an elliptical parameter method based on captured data of two cameras of a left eye and a right eye. In the elliptical parameter method, since the accuracy becomes low as an ellipse approximates a circle, weighting is performed in advance for a section on a display image indicated by a sight line. For example, a weight for a section D5 in the nearest distance from a normal line position is set to 0.3. Next, a weight for sections, which distances from the normal line position are next to the section D5, is set to 0.5. Last, a weight for sections in the remotest distance from the normal line position is set to 0.8. Last, a center sight line coordinate value according to both eyes is calculated based on a determined sight line coordinate value corresponding to both cameras. 1. An input device comprising:a weighting unit which divides a display screen area of a display device into a plurality of sections, and applies weighted values indicating accuracy to the plurality of sections in advance, the weighted values corresponding to a right eye sight line position and a left eye sight line position of a user;a right eye sight line position detection unit which detects the right eye sight line position of the user on a display screen of the display device based on captured data captured by a camera for a right eye, which is arranged in order to detect the right eye sight line position of the user, and displays the right eye sight line position using coordinates;a right eye sight line position determination unit which determines a coordinate value of a right eye sight line determination position by integrating a weight indicating the accuracy and corresponding to the detected right eye sight line position into a coordinate value of the right eye sight line position;a left eye sight line position detection unit which detects the left eye sight line position of the user on a display screen of the display device based on ...

System and method for reducing the effects of inadvertent touch on a touch screen controller

A method and system are provided for determining if a control button on a touch screen controller (TSC) was inadvertently touched. A user gaze point is determined using gaze detection apparatus, and the level of functional significance of the control button is identified. User gaze boundary values for the control button are then determined, and the gaze boundary values are altered to reflect the level of functional significance.

Mobile communicator device including user attentiveness detector

Disclosed herein is a mobile communicator that includes a speed detection system for determining whether the mobile communicator is moving faster than a threshold speed. The mobile communicator further includes a user attentiveness detector configured to determine whether a user is inattentive and a sensor element configured to determine orientation of the mobile communicator. The mobile communicator further includes a circuit configured to ensure that a functionality of the mobile communicator is in a turned off state when the speed detection system determines that the mobile communicator is moving faster than the threshold speed and at least one of: (a) the user attentiveness detector determines that the user is inattentive; and (b) the sensor element determines that the mobile communicator is in a substantially vertical position.

EYE GAZE USER INTERFACE AND CALIBRATION METHOD

A software controlled user interface and calibration method for an eye gaze controlled device, designed to accommodate angular accuracy versus time averaging tradeoffs for eye gaze direction sensors. The method can scale between displaying a small to large number of different eye gaze target symbols at any given time, yet still transmit a large array of different symbols to outside devices with minimal user training. At least part of the method may be implemented by way of a virtual window onto the surface of a virtual cylinder, with eye gaze sensitive symbols that can be rotated by eye gaze thus bringing various groups of symbols into view, and then selected by continual gazing. Specific examples of use of this interface and method on an eyeglasses-like head-mountable, vision-controlled, device are disclosed, along with various operation examples including sending and receiving text messages, control of robotic devices and control of remote vehicles. 1. A method of controlling a head-mountable , vision-controlled , device for transmitting and receiving information from a human user with at least one eye , said device having a virtual display and at least one eye position tracking sensor , method comprising:displaying a plurality of visual targets on said virtual display, said plurality of visual targets each comprising a visible element embedded within a eye position zone with an area that is equal to or larger than said visible element;using said at least one eye position tracking sensor to determine when said at least one eye is on average gazing in a direction that is within the eye position zone of at least one of said plurality of visible elements for a time period exceeding a hovering time interval, signaling when said hovering time interval has been exceeded, and if said gaze remains within the eye position zone of said at least one of said plurality of visible elements for a time period exceeding a keypress time interval, registering that a virtual key ...

OCULAR BIOMETRIC AUTHENTICATION WITH SYSTEM VERIFICATION

A biometric user authentication method and computer program product includes receiving asserted user credentials from a user into a biometric authentication system, and obtaining a digitally-stored image key and ocular biometric data both associated with the asserted user credentials from memory within the biometric authentication system. The biometric authentication system is verified by simultaneously displaying the image key and at least one image other than the image key to the user and detecting that the user has selected the image key. The user is authenticated by scanning an eye of the user to obtain ocular biometric data and matching the scanned ocular biometric data to the digitally stored ocular biometric data. If the biometric system is verified and the user is authenticated, then the user is provided access to a protected area. 1. A biometric user authentication method , comprising:receiving asserted user credentials from a user into a biometric authentication system;obtaining a digitally-stored image key and ocular biometric data both associated with the asserted user credentials from memory within the biometric authentication system;verifying the biometric authentication system by simultaneously displaying the image key and at least one image other than the image key to the user and detecting that the user has selected the image key;authenticating the user by scanning an eye of the user to obtain ocular biometric data and matching the scanned ocular biometric data to the digitally stored ocular biometric data; andproviding user access to a protected area in response to both successfully verifying the biometric system and authenticating the user.2. The biometric user authentication method of claim 1 , further comprising:performing the step of verifying the biometric authentication system as a prerequisite to the step of scanning the eye of the user in authenticating the user.3. The biometric user authentication method of claim 1 , further comprising: ...

Control system and method thereof

A control system for controlling an external device based on images obtained by a control device comprises a detecting module, an obtaining module, and a determining module. The detecting module generates an obtaining signal when the control device establishes a communication with the external device. The obtaining module obtains an image of the current environment around the external device in response to the obtaining signal. The determining module determines whether at least one predefined object contained in the obtained image. When there is no predefined object contained in the obtained image, the determining module further determines whether the external device is in a first state and generates a first control signal when the external device is not in the first state. The external device switches to a first state for powering off the backlight of the external device in response to the first control signal.

GAZE DETECTION APPARATUS AND GAZE DETECTION METHOD

A gaze detection apparatus includes: a first imaging unit which has a first angle of view and generates a first image; a second imaging unit which has a second angle of view narrower than the first angle of view, and generates a second image; a face detection unit which detects from the first image a face region; a coordinate conversion unit which identifies on the second image a first region corresponding to the face region or to an eye peripheral region containing the user's eye; a Purkinje image detection unit which detects a corneal reflection image of a light source and the center of the user's pupil from within an eye region, identified based on the first region; and a gaze detection unit which detects the user's gaze direction or gaze position based on a positional relationship between the center of the pupil and the corneal reflection image. 1. A gaze detection apparatus comprising:a light source which illuminates a user's eye;a first imaging unit which has a first angle of view, and generates a first image by capturing an image of the user's face;a second imaging unit which has a second angle of view narrower than the first angle of view, and generates a second image by capturing an image of at least a portion of the user's face;a face detection unit which detects from the first image a face region containing the user's face;a coordinate conversion unit which identifies on the second image a first region that corresponds to the face region or to an eye peripheral region detected from within the face region as containing the user's eye;a Purkinje image detection unit which detects a corneal reflection image of the light source and the center of the user's pupil from within an eye region, identified based on the first region, that contains the user's eye on the second image; anda gaze detection unit which detects the user's gaze direction or gaze position based on a positional relationship between the center of the pupil and the corneal reflection image.2. ...