Cognitive-neural method for image analysis

A self adapting cognitive-neural method and system for image analysis is disclosed. The method comprises detecting a set of potential items of interest in an image using a cognitive algorithm, refining the set of potential items of interest using a neural analysis, and adapting the cognitive algorithm based on information gained from the results of the neural analysis. The method may further comprise fusing the results of the cognitive and neural algorithms with a fusion classifier. The neural analysis can include eye tracked EEG, RSVP, or presentation to an operator for visual inspection.

Method and apparatus for augmented reality hybrid tracking system with fiducial-based heading correction

An apparatus and method is disclosed for enabling accurate registration of virtual information upon real objects in selected environments from the perspective of a moveable observation platform having a position and an orientation within an observation space positioned within the environment, the position and orientation defining the registration of the virtual information within the observation space within the environment, which may comprise: a plurality of at least three tracking fiducials selectively each respectively located in fixed predetermined locations in the observation space; the observation platform including a position sensor apparatus adapted to sense an estimated position of the observation platform in the observation space and a heading sensor apparatus adapted to sense an estimated heading of the observation platform within the observation space; the observation platform further including a wide view fiducial detection apparatus mounted on the observation platform and adapted to detect the relative direction from the observation platform to each of the plurality of tracking fiducials, a heading error detection apparatus adapted to detect the difference between the estimated heading and the present heading based upon the observed relative directions to the respective tracking fiducials, and forming a corrected heading from the estimated heading and the detected heading error.

Systems, methods, and apparatus for neuro-robotic tracking point selection

Systems, methods, and apparatus for neuro-robotic tracking point selection are disclosed. A described example robot control system includes a feature and image presenter, a classifier, a visual-servo controller, and a robot interface. The feature and image presenter is to display an image of an object, emphasize one of more potential trackable features of the object, receive a selection of the emphasized feature, and determine an offset from the selected feature as a goal. The classifier is to classify a mental response to the emphasized features, and to determine that the mental response corresponds to the selection of one of the emphasized features. The visual-servo controller is to track the emphasized feature corresponding to an identified brain signal. The robot interface is to generate control information to effect a robot action based on the emphasized feature, the visual-servo controller to track the emphasized feature while the robot action is being effected.

Beacon-Augmented Pose Estimation

A beacon-augmented pose estimation system includes positionable beacons that can determine their own 3-D geospatial coordinates and transmit those coordinates back to the pose object. An imaging sensor images the field-of-view of the pose object to provide 2-D image coordinates for any of the beacons in the FOV. A pose object controller processes the sets of 3-D geospatial coordinates and 2-D image coordinates to refine a pose estimate of the pose object. The positionable beacons may include both static beacons that are pre-positioned and mobile beacons that are command-guided to position themselves in the pose object's likely or actual FOV.

Method and apparatus for geographic shape preservation for identification

A method for transforming a map based on associated geospatial data. A map is segmented into logical segments representing map-based entities. The geospatial data is then associated with the logical segments. The logical segments are scaled based upon the associated data. The logical segments are then repositioned to avoid overlapping other logical segments and a new map display based on the scaled and repositioned logical segments is provided.

Distributed display composed of active fiducials

An apparatus configured to obtain, process, and relay data to a user in a coherent and useful manner. An active fiducial is equipped with an interface for receiving and transmitting data. The fiducial may transmit its position using a satellite-based position sensing device such as a GPS. Active fiducials may also be equipped with battery power pack regenerated with solar cells. Similarly, the fiducials can be equipped with at least one video camera or other device having a focal plane array and a computer software system, configured to recognize shapes. The fiducials may also be equipped with inductive coils or other means for sensing metal containing compounds. The active fiducials may be equipped with a gas chromatograph. The active fiducials may use a variety of propulsion means including motor driven tracks, motor driven wheels, propellers, or other device or a combination of devices.

Geo-coded spatialized audio in vehicles

A three dimensional audio playback system in which the audio clips are determined by location. The audio playback system being located within a vehicle to aid in navigation or for entertainment or informational or safety purposes.

System for electronically-mediated collaboration including eye-contact collaboratory

A system and method for producing and receiving electronically-mediated collaboration is presented. The electronically-mediated collaboration system and method include a high-fidelity audio system as well as application and cursor sharing, and a session manager module to synchronize among participants. In one embodiment, the system includes an eye contact collaboratory that manipulates video and eye position input to synthesize virtual camera images which simulate ordinary gaze behavior in a collaboratory system. An audio component optionally spatializes the voices of the participants to appear to coincide with their images.

Teleconferencing system

A system and method for encoding the speech of multiple speakers for transmission on a limited medium such as ISDN or ADSL. The system and method encode one or more "dominant" speakers on a first channel, and merges all other speakers into a second channel. A "dominant" speaker may be defined according to the needs of the application, but intuitively it is a speaker in a collaboration session who has the "floor". The method and system transmit speech through a teleconferencing system having an available bandwidth. Speech from a speaking one, or ones, of the plurality of participants is passed while speech from non-speaking participants is rejected. The system responds to the speech from the one, or ones of the speaking participants passed by the speech detector, and determines which of the detected speaking one, or ones, thereof is a dominant speaker. The system allocates one portion of the available bandwidth to the determined dominant speaker and merges the non-dominant participant(s) ...

Handheld virtual overlay system

A handheld virtual overlay system (HVOS) device is presented that includes a global positioning system (GPS) receiver for detecting a global position of the HVOS device; a compass/tilt sensor for detecting a direction and tilt of the HVOS device; a communication module for transmitting the global position, the direction, and the tilt to a data store; and an input/output display configured to receive and display a rendered image. The data store is configured to receive the global position, the direction, and the tilt, and to generate geo-coded data for transmission to a server, where the server is configured to receive the geo-coded data and to transform and render the geo-coded data into a rendered image. A user can point the HVOS device at a scene, with the HVOS device producing a rendered image representing the scene.

System for intelligent goal-directed search in large volume imagery and video using a cognitive-neural subsystem

A system for intelligent goal-directed search in large volume visual imagery using a cognitive-neural subsystem is disclosed. The system comprises an imager, a display, a display processor, a cognitive-neural subsystem, a system controller, and operator controls. The cognitive-neural subsystem comprises a cognitive module, a neural module, and an adaptation module. The cognitive module is configured to extract a set of regions of interest from the image using a cognitive algorithm. The neural module is configured to refine the set of regions of interest using a neural processing algorithm. The adaptation module is configured to bias the cognitive algorithm with information gained from the neural module to improve future searches. The system functions in a plurality of operating modes, including: batch mode, semi-interactive mode, real-time mode, and roaming mode.

Beacon-augmented pose estimation

A beacon-augmented pose estimation system includes positionable beacons that can determine their own 3-D geospatial coordinates and transmit those coordinates back to the pose object. An imaging sensor images the field-of-view of the pose object to provide 2-D image coordinates for any of the beacons in the FOV. A pose object controller processes the sets of 3-D geospatial coordinates and 2-D image coordinates to refine a pose estimate of the pose object. The positionable beacons may include both static beacons that are pre-positioned and mobile beacons that are command-guided to position themselves in the pose object's likely or actual FOV.

Recall system using spiking neuron networks

Described is a recall system that uses spiking neuron networks to identify an unknown external stimulus. The system operates by receiving a first input signal (having spatial-temporal data) that originates from a known external stimulus. The spatial-temporal data is converted into a first spike train. A first set of polychronous groups (PCGs) are generated as a result of the first spike train. Thereafter, a second input signal originating from an unknown external stimulus is received. The spatial-temporal data of the second input signal is converted into a second spike train. A second set of PCGs are then generated as a result of the second spike train. Finally, the second set of PCGs is recognized as being sufficiently similar to the first set of PCGs to identify the unknown external stimulus as the known external stimulus.

Method and apparatus for sharing geographically significant information

A three dimensional audio playback system in which the audio clips are determined by location. The audio playback system being located within a vehicle to aid in navigation or for entertainment or informational or safety purposes.

Acoustic ranging system using atmospheric dispersion

A method and apparatus for processing sound from a sound source. The sound from the sound source is detected. Harmonics in the sound from the sound source are identified. A distance to the sound source is identified using the harmonics and a number of atmospheric conditions.

Converting video metadata to propositional graphs for use in an analogical reasoning system

An automatic approach is described for the semantic analysis and conversion of video annotation metadata (e.g., Video Event Markup Language or "VEML" metadata) to propositional graphs suitable for analysis by an analogical reasoning system (ARS). Also as described herein, a system architecture implements this conversion while maintaining semantic consistency from annotation through results reporting. Further, the techniques herein leverage the same ontology to populate options for an annotation tool, provide the rules for the metadata-to-propositional graph mapping, and provide the basis for comparison in an analogical reasoning algorithm.

Converting Video Metadata to Propositional Graphs for Use in an Analogical Reasoning System

An automatic approach is described for the semantic analysis and conversion of video annotation metadata (e.g., Video Event Markup Language or “VEML” metadata) to propositional graphs suitable for analysis by an analogical reasoning system (ARS). Also as described herein, a system architecture implements this conversion while maintaining semantic consistency from annotation through results reporting. Further, the techniques herein leverage the same ontology to populate options for an annotation tool, provide the rules for the metadata-to-propositional graph mapping, and provide the basis for comparison in an analogical reasoning algorithm.

System for detecting an object of interest in a scene

The present invention relates to a system for detecting an object of interest in a scene. The system operates by receiving an image frame of a scene and extracting features from the image frame, the features being descriptors. The descriptors are quantized to generate PHOW features. A sliding window protocol is implemented to slide a window over the image and analyze the PHOW features that fall inside the window. Finally, the system determines if the PHOW features represent the object of interest and, if so, then designates the window as a location in the image with a detected object of interest.

Multi-user real-time augmented reality system and method

A high resolution real-time multi-user augmented reality system broadcasts a wide FOV video signal from a plurality of image sources over a distribution bus to a plurality of users that independently select and view different portions of the video signal such that the angular resolution of the displayed video signal is constant over the range of possible viewing orientations. This is accomplished by assuming that each user's position is fixed at the center of a virtual sphere, the image sources' positions are known relative to the center of the virtual sphere, and that the users are looking at a portion of the inner surface of the sphere. As a result, the "flat" images generated by the image sources and the "flat" images viewed by the users can be efficiently mapped onto the virtual sphere and represented as index pairs (d,n). Thus, each user extracts those pixels on the virtual sphere corresponding to the user's current FOV and remaps them to a flat display. The video signals can be augmented ...

System and method for deep packet inspection and intrusion detection

The present invention relates to a system for deep packet inspection and intrusion detection. The system uses a pattern matching module receiving as an input a data stream in a neural network. Neurons are activated such that when active, the neuron fires to all connecting output neurons to form a neuron spike, each neuron spike from the assigned neuron to a connecting output neuron having a delay. A delay is associated with each input character in the pattern, such that a position of each input character relative to an end of the pattern is stored in an alphabet-pattern-delay matrix (APDFM). An activation matrix (AM) is used to match each input character with a stored pattern to generate a similarity match and determine if the string of characters is the stored pattern.

System for representing, storing, and reconstructing an input signal

Described is a system for representing, storing, and reconstructing an input signal. The system constructs an index of unique polychronous groups (PCGs) from a spiking neuron network. Thereafter, a basis set of spike codes is generated from the unique PCGs. An input signal can then be received, with the input signal being spike encoded using the basis set of spike codes from the unique PCGs. The input signal can then be reconstructed by looking up in a reconstruction table, for each unique PCG in the basis set in temporal order according to firing times, anchor neurons. Using a neuron assignment table, an output location can be looked up for each anchor neuron to place a value based on the firing times of each unique PCG. Finally, the output locations of the anchor neurons can be compiled to reconstruct the input signal.

Virtual display screen system

Disclosed is a computerized data display system including a computer operating in accord with a window display management system for the display and control of a plurality of data windows on a display screen of a display device. The data windows are displayed on the display screen in a spatial relation corresponding to the field of view seen from a preselected viewing location selected by means of a control signal provided as an input to the computer. A head coupled image display device, coupled to the display screen of the display device, is adapted to display the data windows appearing on the display screen of the display device separately to each eye of a user to create a binocular, stereoscopic virtual screen image to the user that has a virtual screen size independent of the size of the display screen of the display device. A user controlled input position sensor, coupled to the computer, is adapted for generating the position control signal as an input to the computer to selectively ...

Systems, methods, and apparatus for neuro-robotic tracking point selection

Systems, methods, and apparatus for neuro-robotic tracking point selection are disclosed. A described example method to control a robot arm includes presenting one or more potential trackable features of a target object, emphasizing at least one of the potential trackable features, determining a selection of one of the emphasized features as a tracking point by monitoring a first mental response to emphasizing the feature, and effectuating an end goal via a robot by tracking the selected feature and effectuating a position based on the selected feature.

Acoustic Ranging System Using Atmospheric Dispersion

A method and apparatus for processing sound from a sound source. The sound from the sound source is detected. Harmonics in the sound from the sound source are identified. A distance to the sound source is identified using the harmonics and a number of atmospheric conditions. 1. A method for processing sound from a sound source , the method comprising:detecting the sound from the sound source;identifying harmonics in the sound from the sound source; andidentifying a distance to the sound source using the harmonics and a number of atmospheric conditions.2. The method of further comprising:identifying a location of the sound source using the distance to the sound source and at least one of an azimuth to the sound source and an elevation to the sound source.3. The method of further comprising:identifying at least one of an azimuth and an elevation to the sound source from the sound.4. The method of claim 1 , wherein identifying the distance to the sound source using the harmonics and the number of atmospheric conditions comprises:identifying a time delay between a first harmonic and a second harmonic in the harmonics; andidentifying the distance to the sound source using the time delay between the first harmonic and the second harmonic and atmospheric dispersion caused by the number of atmospheric conditions.5. The method of claim 4 , wherein identifying the distance to the sound source using the time delay between the first harmonic and the second harmonic and the atmospheric dispersion caused by the number of atmospheric conditions comprises:identifying the distance to the sound source using the time delay between the first harmonic and the second harmonic and an atmospheric dispersion factor, wherein the atmospheric dispersion factor takes into account dispersion caused by the number of atmospheric conditions for the first harmonic and the second harmonic.6. The method of claim 4 , wherein identifying the time delay between the first harmonic and the second harmonic ...

NEURO-COGNITIVE DRIVER STATE PROCESSING

A driver state module for interfacing with a vehicle, with a surroundings vicinity of the vehicle and with a driver of the vehicle, the driver state module comprising: (i) a frame memory for storing representations of behaviors with related context; (ii) an evaluation system for ranking the frames based on goals and rewards; (iii) a working memory comprising a foreground sub-memory and a background sub-memory, the working memory for holding and sorting frames into foreground and background frames, and (iv) a recognition processor for identifying salient features relevant to a frame in the foreground memory ranked highest by the evaluation system. 1. A driver state module for interfacing with a vehicle , with a surrounding vicinity of the vehicle and with a driver of the vehicle , the driver state module comprising:(i) a frame memory for storing representations of behaviors with related context;(ii) an evaluation system for ranking the frames based on goals and rewards; 'a foreground sub-memory, a background sub-memory, and a control for sorting frames into the foreground sub-memory or the background sub-memory; and', '(iii) a working memory comprising'}(iv) a recognition processor for identifying salient features relevant to a frame in the foreground sub-memory or the background sub-memory ranked highest by the evaluation system.2. The driver state module of claim 1 , configured for modeling focus of attention and awareness of the driver and for predicting imminent actions of the driver.3. The driver state module of wherein said interfacing with the vehicle claim 1 , the surrounding vicinity of the vehicle and the driver of the vehicle is via sensors.4. A vehicle comprising the driver state module of .5. A driver assistance system for assisting a driver of a vehicle within a surrounding vicinity of the vehicle claim 1 , the driver assistance system comprising:{'claim-ref': {'@idref': 'CLM-00001', 'claim 1'}, '(i) the driver state module of ;'}(ii) a vehicle state ...

Systems and methods for predicting entity behavior

Systems and method are provided for controlling a vehicle. In one embodiment, a method includes: receiving sensor data sensed from an environment associated with the vehicle; processing, by a processor, the sensor data to determine observation data, the observation data including differential features associated with an agent in the environment; determining, by the processor, a context associated with the agent based on the observation; selecting, by the processor, a first probability model associated with the context; processing, by the processor, the observation data with the selected first probability model to determine a set of predictions; processing, by the processor, the set of predictions with a second probability model to determine a final prediction of interaction behavior associated with the agent; and selectively controlling, by the processor, the vehicle based on the final prediction of interaction behavior associated with the agent.

VIRTUAL LANE ESTIMATION USING A RECURSIVE SELF-ORGANIZING MAP

A virtual lane estimation system includes a memory device, a sensor and a computer. The memory device is configured to store a road map that corresponds to a portion of a road ahead of a vehicle. The sensor is configured to observe a plurality of trajectories of a plurality of neighboring vehicles that traverse the portion of the road. The computer is configured to initialize a recursive self-organizing map as a plurality of points arranged as a two-dimensional grid aligned with the road map, train the points in the recursive self-organizing map in response to the trajectories, generate a directed graph that contains one or more virtual lanes through the road map in response to the points trained to the trajectories, and generate a control signal that controls navigation of the vehicle through the portion of the road in response to the virtual lanes in the directed graph. 1. A virtual lane estimation system comprising:a memory device configured to store a road map that corresponds to a portion of a road ahead of a vehicle;a sensor configured to observe a plurality of trajectories of a plurality of neighboring vehicles that traverse the portion of the road; anda computer configured to initialize a recursive self-organizing map as a plurality of points arranged as a two-dimensional grid aligned with the road map, train the plurality of points in the recursive self-organizing map in response to the plurality of trajectories, generate a directed graph that contains one or more virtual lanes through the road map in response to the points trained to the plurality of trajectories, and generate a control signal that controls navigation of the vehicle through the portion of the road in response to the one or more virtual lanes in the directed graph.2. The virtual lane estimation system according to claim 1 , wherein the training of the recursive self-organizing map uses the plurality of trajectories one at a time.3. The virtual lane estimation system according to claim 2 , ...

NAVIGATION TRAJECTORY USING REINFORCEMENT LEARNING FOR AN EGO VEHICLE IN A NAVIGATION NETWORK

An ego vehicle includes decider modules and a grader module coupled to a resolver module. The decider modules generate trajectory decisions at a current time, generate a current two-dimensional slice of a flat space around the ego vehicle, generate future two-dimensional slices of the flat space by projecting the current two-dimensional slice of the flat space forward in time, and generate a three-dimensional state space by stacking the current two-dimensional slice and the future two-dimensional slices. The grader module generates rewards for the trajectory decisions based on a recent behavior of an ego vehicle. The resolver module selects a final trajectory decision for the ego vehicle from the trajectory decisions based on the three-dimensional state space and the rewards. The current two-dimensional slice includes a current ego vehicle location and current neighboring vehicle locations. The future two-dimensional slices include future ego vehicle locations and future neighboring vehicle locations. 1. An ego vehicle comprising: generate a plurality of trajectory decisions at a current time,', 'generate a current two-dimensional slice of a flat space around the ego vehicle,', 'generate a plurality of future two-dimensional slices of the flat space around the ego vehicle by projecting the current two-dimensional slice of the flat space forward in time, and', 'generate a three-dimensional state space by stacking the current two-dimensional slice and the plurality of future two-dimensional slices; and, 'a plurality of decider modules coupled to a resolver module, and configured to'} the resolver module is configured to select a final trajectory decision for the ego vehicle from the plurality of trajectory decisions based on the three-dimensional state space and the plurality of rewards,', 'the current two-dimensional slice includes a current ego location of the ego vehicle and a plurality of current neighboring locations of a plurality of neighboring vehicles at the ...

System, method and controller for graph-based path planning for a host vehicle

A method of path planning for a host vehicle includes: receiving host vehicle, environmental and obstacle information; calculating one or more projected host vehicle locations; computing a projected obstacle location for each obstacle; and determining a collision potential between each projected host vehicle location and each projected obstacle location. Until a maximum number of steps is reached, and while at least one projected host vehicle location has an associated collision potential below a collision threshold, the method further includes repeating the calculating, computing and determining steps.

Method and apparatus for autonomous system performance and grading

The present application generally relates to methods and apparatus for evaluating and assigning a performance metric to a driver response to a driving scenario. More specifically, the application teaches a method and apparatus for breaking a scenario into features, assigning each feature a grade and generating an overall grade in response to a weighted combination of the grades.

Method And Apparatus For Scenario Generation And Parametric Sweeps For The Development & Evaluation Of Autonomous Driving Systems

The present application generally relates to methods and apparatus for evaluating driving performance under a plurality of driving scenarios and conditions. More specifically, the application teaches a method and apparatus for testing a driving scenario repetitively while altering a parametric variation, such as fog level, in order to evaluate driving system performance under changing conditions. 1. An apparatus comprising:a sensor interface for generating sensor data for coupling to a vehicle control system;a control system interface for receiving control data from the vehicle control system;a memory for storing a first scenario wherein the first scenario is associated with a first parametric variation and a second parametric variation; anda simulator for simulating a first driving environment in response to the first scenario, the first parametric variation and the control data and assigning a first performance metric in response to the simulation of the first driving environment, the simulator further operative to simulate a second driving environment in response to the first scenario, the second parametric variation and the control data and assigning a second performance metric in response to the simulation of the second driving environment.2. The apparatus of wherein the first parametric variation and the second parametric variation relate to intensity of weather.3. The apparatus of wherein the memory is operative to store a second scenario associated with the first parametric variation and the second parametric variation.4. The apparatus of wherein the apparatus is implemented in software.5. The apparatus of wherein the apparatus is implemented in hardware.6. The apparatus of wherein the simulator is further operative to generate a success rate in response to the first performance metric and the second performance metric.7. The apparatus of wherein the vehicle control system is an autonomous vehicle control system.8. A method comprising:receiving a driving ...

Method And Apparatus For Autonomous System Performance And Benchmarking

The present application generally relates to methods and apparatus for evaluating and assigning a complexity metric to a driving scenario. More specifically, the application teaches a method and apparatus for breaking a scenario into subtasks, assigning each subtask a complexity value and generating an overall complexity metric in response to a weighted combination of the subtask complexities as well as human-perceived task complexity. 1. An apparatus comprising:a sensor interface for generating sensor data for coupling to a vehicle control system;a control system interface for receiving control data from the vehicle control system;a memory for storing a first scenario having a first overall complexity wherein the first scenario is divided into a first subtask and a second subtask and wherein the first subtask has a first complexity and the second subtask has a second complexity and wherein the first overall complexity is determined in response to the first complexity and the second complexity; anda simulator for simulating a driving environment in response to the first scenario and the control data, the simulator further operative to control the sensor interface and to generate performance data in response to the control data.2. The apparatus of wherein the first complexity and the second complexity are weighted in response to a human complexity such that the first overall complexity correlates with the human complexity.3. The apparatus of wherein the memory is operative to store a second scenario having a second overall complexity.4. The apparatus of wherein the apparatus is implemented in software.5. The apparatus of wherein the apparatus is implemented in hardware.6. The apparatus of wherein the simulator is further operative to generate a success rate in response to the performance data.7. The apparatus of wherein the vehicle control system is an autonomous vehicle control system.8. A method comprising:receiving a driving scenario;segmenting the driving ...

EFFICIENT SITUATIONAL AWARENESS BY EVENT GENERATION AND EPISODIC MEMORY RECALL FOR AUTONOMOUS DRIVING SYSTEMS

A system and method is taught for data processing in an autonomous vehicle control system. Using information is acquired from the vehicle, network interface, and sensors mounted on the vehicle, the system can perceive situations around it with much less complexity in computation without losing crucial details, and then make navigation and control decisions. The system and method are operative to generate situation aware events, store them, and recall to predict situations for autonomous driving. 1. A method comprising:receiving a first plurality of information via a vehicle sensor;receiving a second plurality of information via a network interface;generating an episode in response to a temporal ordering of the first plurality of information and the second plurality of information;receiving a third plurality of information via the vehicle sensor;receiving a fourth plurality of information via the network interface;generating an event in response to the third plurality of information and the fourth plurality of information;comparing the episode to the event to generate a predicted outcome of the event; andgenerating a control signal in response to the predicted outcome.2. The method of wherein the event corresponds to a first temporal portion of the episode.3. The method of wherein the control signal is used to guide an autonomous vehicle.4. The method of further comprising generating a header in response to the temporal ordering of the first plurality of information and the second plurality of information wherein the header includes environment type claim 1 , host intent claim 1 , and host vehicle status.5. The method of further comprising generating a header in response to the temporal ordering of the first plurality of information and the second plurality of information wherein the header is used to characterize the episode.6. The method of wherein the header is used as part of a tree structure based classification system for storing the episode.7. The method of ...

System and method to cue specific memory recalls while awake

Described is a system for cueing a specific memory in a waking state. The system sends an initiation signal to a memory recall controller to select a stored stimulation pattern previously associated with a specific memory of an event. The system signals to a memory recall controller to initiate delivery of the selected stimulation pattern to a brain in a waking state for a duration of the event via a brain stimulation system. Following completion of the event, the system signals for the memory recall controller to stop the brain stimulation system from delivering the selected stimulation pattern.

ONLINE DRIVING PERFORMANCE EVALUATION USING SPATIAL AND TEMPORAL TRAFFIC INFORMATION FOR AUTONOMOUS DRIVING SYSTEMS

An autonomous vehicle, system and method of operating the autonomous vehicle. The system includes a performance evaluator, a decision module and a navigation system. The performance evaluator determines a performance grade for each of a plurality of decisions for operating the autonomous vehicle. The decision module selects a decision have a greatest performance grade. The navigation system operates the autonomous vehicle using the selected decision. 1. A method of operating an autonomous vehicle , comprising;receiving a plurality of decisions for operating the autonomous vehicle at a decision resolver of a cognitive processor associated with the autonomous vehicle;determining a performance grade for each of the plurality of decisions;selecting a decision have a greatest performance grade; andoperating the autonomous vehicle using the selected decision.2. The method of claim 1 , wherein the performance grade is a combination of an instantaneous performance grade and a temporal performance grade.3. The method of claim 2 , wherein the instantaneous performance grade is based on a compliance with a traffic rule and a compliance with a flow of traffic.4. The method of claim 2 , wherein the temporal performance grade is determined over a time period extending from a start time in the past to an end time in the future.5. The method of claim 4 , wherein the start time is the most recent of (i) a start time of a new event; and (ii) a time indicated by a selected time interval prior to a current time.6. The method of claim 2 , further comprising using a standard deviation of grades in the temporal performance grade to weight the contribution of each of the instantaneous performance grade and the temporal performance grade in the performance grade.7. The method of claim 2 , wherein the temporal performance grade is a combination of a mean grade over a time interval and a minimum grade over the time interval.8. A system for operating an autonomous vehicle claim 2 , comprising: ...

COGNITIVE PROCESSOR FEEDFORWARD AND FEEDBACK INTEGRATION IN AUTONOMOUS SYSTEMS

An autonomous vehicle, cognitive system for operating an autonomous vehicle and method of operating an autonomous vehicle. The cognitive system includes one or more hypothesizer modules, a hypothesis resolver, one or more decider modules, and a decision resolver. Data related to an agent is received at the cognitive system. The one or more hypothesizer modules create a plurality of hypotheses for a trajectory of the agent based on the received data. The hypothesis resolver selects a single hypothesis for the trajectory of the agent from the plurality of hypotheses based on a selection criteria. The one or more decider modules create a plurality of decisions for a trajectory of the autonomous vehicle based on the selected hypothesis for the agent. The decision resolver selects a trajectory for the autonomous vehicle from the plurality of decisions. The autonomous vehicle is operated based on the selected trajectory. 1. A method of operating an autonomous vehicle , comprising;receiving data related to an agent at a processor of a cognitive system;creating, at one or more hypothesizer modules of the cognitive system, a plurality of hypotheses for a trajectory of the agent based on the received data;selecting, at a hypothesis resolver of the cognitive system, a single hypothesis for the trajectory of the agent from the plurality of hypotheses based on a selection criteria;creating, at one or more decider modules of the cognitive system, a plurality of decisions for a trajectory of the autonomous vehicle based on the selected hypothesis for the agent.selecting, at a decision resolver of the cognitive system, a trajectory for the autonomous vehicle from the plurality of decisions for the trajectory of the autonomous vehicle;operating the autonomous vehicle based on the selected trajectory for the autonomous vehicle.2. The method of claim 1 , further comprising projecting claim 1 , at a virtual controller claim 1 , a future state of the autonomous vehicle resulting from ...

Hyperassociation in episode memory

An autonomous vehicle, system and method of operating the autonomous vehicle. The system includes an episodic memory, a hyper-association module and a navigation system. The episodic memory stores a plurality of episodes, recalls a plurality of candidate episodes in response to receiving a partial prefix and recalls a hypothesis episode in response to receiving an intermediate episode. The hyper-association module receives the plurality of candidate episodes from the episodic memory and obtains the intermediate episode from the plurality of candidate episodes. The navigation system navigates the autonomous vehicle using the hypothesis episode.

SYSTEMS AND METHODS FOR AUTONOMOUS DRIVING USING NEURAL NETWORK-BASED DRIVER LEARNING ON TOKENIZED SENSOR INPUTS

In various embodiments, methods, systems, and autonomous vehicles are provided. In one exemplary embodiment, a method is provided that includes obtaining first sensor inputs pertaining to one or more actors in proximity to an autonomous vehicle; obtaining second sensor inputs pertaining to operation of the autonomous vehicle; obtaining, via a processor, first neural network outputs via a first neural network, using the first sensor inputs; and obtaining, via the processor, second neural network outputs via a second neural network, using the first network outputs and the second sensor inputs, the second neural network outputs providing one or more recommended actions for controlling the autonomous vehicle.

System and Method for Controlling an Autonomous Vehicle

An automotive vehicle includes an actuator configured to control vehicle acceleration, steering, or braking. The vehicle additionally includes at least one sensor configured to detect an object in the vicinity of the vehicle. The vehicle further includes at least one controller configured to automatically control the actuator according to an automated driving system algorithm. The at least one controller is configured to define a first vehicle route, detect at least one object in the vicinity of the vehicle; calculate a probabilistic forecast of a future location of the at least one detected object, define a second vehicle route in response to the probabilistic forecast indicating proximity of the at least one detected object to the first route, and to control the actuator to execute the second vehicle route.

Apparatus and method for self-calibrating visual time-to-contact sensor

The invention relates to a method and a self-calibrating visual time-to-contact (TTC) sensor for a mobile manned or autonomous unmanned vehicle which operates in a high speed manner and which permits the continuous, adaptive motion of the vehicle through the vehicle's environment. More specifically, the invention relates to an apparatus and method for novel use of active sensor control for aligning a camera, mounted on the vehicle, to track the direction of motion of the vehicle in order to successfully navigate in complex environments and to simplify difficult processing steps previously attempted through algorithmic means. Rather than attempting to explicitly find a focus of expansion (FOE) from image data, the invention continuously calibrates the sensor to point in the direction of the FOE, using weighted global average of the horizontal and vertical component of the optical flow. The pan and tilt angles of the visual TTC sensor are iteratively changed by the scanning mechanism in the opposite direction by small increments, biased by the global magnitude of the optical flow. In this way, the visual TTC sensor is not constrained to point in any direction at the start, and after several iterations it will point near the FOE and continue to do so as the vehicle moves. By actively centering the FOE in the image sequence, better accuracy is possible. By avoiding the focus of expansion calculation, the method becomes self-calibrating and it is faster and more robust to vibrations and sensor misalignments occurring in changing and complex indoor or outdoor environments.

A multi-user real-time virtual reality system and method

A high resolution real-time multi-user augmented reality system 10 broadcasts a wide FOV video signal 22 from a plurality of image sources 12 over a distribution channel 24 to a plurality of users 25 that independently select and view different portions 42 of the video signal such that the angular resolution of the displayed video signal is constant over the range of possible viewing orientations. This is accomplished by assuming that each user's position is fixed at the center 58 of a virtual sphere 54 , the image sources' positions are known relative to the center of the virtual sphere, and that the users are looking at a portion of the inner surface of the sphere. As a result, the flat " images generated by the image sources and the flat " images viewed by the users can be efficiently mapped onto the virtual sphere and represented as index pairs (d,n). Thus, each user extracts those pixels on the virtual sphere corresponding to the user's current FOV 46 and remaps them to a flat display 40 . The video signals can be augmented with synthetic point-of-interest data such as visual overlays 26 or audio messages 28 that are registered to the video.

Acoustic ranging system using atmospheric dispersion

A method and apparatus for processing sound (218) from a sound source (210). The sound (218) from the sound source (210) is detected. Harmonics (224) in the sound (218) from the sound source (210) are identified. A distance (222) to the sound source (210) is identified using the harmonics (224) and a number of atmospheric conditions (226).

Method and apparatus for scenario generation and parametric sweeps for the development and evaluation of autonomous driving systems

The present application generally relates to methods and apparatus for evaluating driving performance under a plurality of driving scenarios and conditions. More specifically, the application teaches a method and apparatus for testing a driving scenario repetitively while altering a parametric variation, such as fog level, in order to evaluate driving system performance under changing conditions.

Method and apparatus for display of geospatially-based data

System and method for robot supervisory control with an augmented reality user interface

Described is a system for robot supervisory control. The system includes an operator device that receives camera imagery from a camera mounted on a robot and three-dimensional (3D) data from a 3D sensor mounted on the robot. The user interface displays a two-dimensional (2D) view of the scene from the camera and 3D sensor data. One or more object markers of objects in the scene are overlaid on the 2D view or 3D sensor data. Viewing the scene, the user can choose a robotic action from a library of actions, which generates a simulation of the robot performing the selected action. The simulation is then rendered and overlaid on top of the 3D sensor data. Moreover, the simulation shows the expected variability of the robot's action. Based on the simulation's outcome, the user can approve and trigger the execution of the action in the real robot.

Teleconferencing system

A system and method for encoding the speech of multiple speakers for transmission on a limited medium such as ISDN or ADSL. The system and method encode one or more 'dominant' speakers on a first channel, and merges all other speakers into a second channel. A 'dominant' speaker may be defined according to the needs of the application, but intuitively it is a speaker in a collaboration session who has the 'floor'. The method and system transmit speech through a teleconferencing system having an available bandwidth. Speech from a speaking one, or ones, of the plurality of participants is passed while speech from non-speaking participants is rejected. The system responds to the speech from the one, or ones of the speaking participants passed by the speech detector, and determines which of the detected speaking one, or ones, thereof is a dominant speaker.

Acoustic ranging system using atmospheric dispersion

A method and apparatus for processing sound (218) from a sound source (210). The sound (218) from the sound source (210) is detected. Harmonics (224) in the sound (218) from the sound source (210) are identified. A distance (222) to the sound source (210) is identified using the harmonics (224) and a number of atmospheric conditions (226).

METHOD AND DEVICE FOR AUTONOMOUS SYSTEM PERFORMANCE AND CLASSIFICATION

Die vorliegende Anmeldung bezieht sich im Allgemeinen auf Verfahren und Vorrichtungen zum Bewerten und Zuweisen einer Leistungsmetrik zu einer Fahrerreaktion auf ein Fahrszenario. Im Besonderen lehrt die Anmeldung ein Verfahren und eine Vorrichtung, um ein Szenario in Merkmale zu zerlegen, jedem Merkmal eine Einstufung zuzuordnen und als Reaktion auf eine gewichtete Kombination der Einstufungen eine Gesamteinstufung zu erzeugen. The present application generally relates to methods and apparatus for evaluating and assigning a performance metric to a driver response to a driving scenario. In particular, the application teaches a method and apparatus for decomposing a scenario into features, classifying each feature, and generating an overall rating in response to a weighted combination of the classifications.

Teleconferencing system

A system and method for encoding the speech of multiple speakers for transmission on a limited medium such as ISDN or ADSL. The system and method encode one or more "dominant" speakers on a first channel, and merges all other speakers into a second channel. A "dominant" speaker may be defined according to the needs of the application, but intuitively it is a speaker in a collaboration session who has the "floor". The method and system transmit speech through a teleconferencing system having an available bandwidth. Speech from a speaking one, or ones, of the plurality of participants is passed while speech from non-speaking participants is rejected. The system responds to the speech from the one, or ones of the speaking participants passed by the speech detector, and determines which of the detected speaking one, or ones, thereof is a dominant speaker.

Teleconferencing system

A system and method for encoding the speech of multiple speakers for transmission on a limited medium such as ISDN or ADSL. The system and method encode one or more "dominant" speakers on a first channel, and merges all other speakers into a second channel. A "dominant" speaker may be defined according to the needs of the application, but intuitively it is a speaker in a collaboration session who has the "floor". The method and system transmit speech through a teleconferencing system having an available bandwidth. Speech from a speaking one, or ones, of the plurality of participants is passed while speech from non-speaking participants is rejected. The system responds to the speech from the one, or ones of the speaking participants passed by the speech detector, and determines which of the detected speaking one, or ones, thereof is a dominant speaker. The system allocates one portion of the available bandwidth to the determined dominant speaker and merges the non-dominant participant(s) onto a second bandwidth portion of the available bandwidth.

Unsupervised velocity prediction and correction for urban driving entities from sequence of noisy position estimates

A method using unsupervised velocity prediction and correction for urban driving from sequences of noisy position estimates includes: performing a vehicle velocity prediction for one or more other vehicles in a vicinity of a host automobile vehicle; calculating a first heuristic based on a uniformity test; calculating a second heuristic based on a vehicle speed of the one or more other vehicles; combining the first heuristic and the second heuristic using a weighted sum; determining an uncertainty mask applying the combined first heuristic and the second heuristic and a heuristic threshold; and applying the uncertainty mask to identify a velocity correction for use by the host automobile vehicle.

Methods for cognitive situation awareness using an attention-based event structure

A method of using perception-inspired event generation for situation awareness for a vehicle, including receiving perception input data from a sensor of the vehicle and processing the perception input data to classify and generate parameters related to an external entity in a vicinity of the vehicle. The method includes generating a hierarchical event structure that classifies and prioritizes the perception input data by classifying the external entity into an attention zone and prioritizing the external entity within the attention zone according to a risk level value for the external entity. A higher risk level value indicates a higher priority within the attention zone. The method further includes developing a behavior plan for the vehicle based on the hierarchical event structure.

A system for detecting an object of interest in a scene

Virtual lane estimation using a recursive self-organizing map

A virtual lane estimation system includes a memory device, a sensor and a computer. The memory device is configured to store a road map that corresponds to a portion of a road ahead of a vehicle. The sensor is configured to observe a plurality of trajectories of a plurality of neighboring vehicles that traverse the portion of the road. The computer is configured to initialize a recursive self-organizing map as a plurality of points arranged as a two-dimensional grid aligned with the road map, train the points in the recursive self-organizing map in response to the trajectories, generate a directed graph that contains one or more virtual lanes through the road map in response to the points trained to the trajectories, and generate a control signal that controls navigation of the vehicle through the portion of the road in response to the virtual lanes in the directed graph.

Hyper-assoziation im episodenspeicher

Ein autonomes Fahrzeug, System und Verfahren zum Betrieb des autonomen Fahrzeugs. Das System umfasst einen episodischen Speicher, ein Hyper-Assoziationsmodul und ein Navigationssystem. Der episodische Speicher speichert eine Vielzahl von Episoden, ruft in Erwiderung auf den Empfang eines Teilpräfixes eine Vielzahl von Kandidatenepisoden ab und ruft in Erwiderung auf den Empfang einer Zwischenepisode eine Hypothesenepisode ab. Das Hyper-Assoziationsmodul erhält die Vielzahl der Kandidatenepisoden aus dem episodischen Speicher und erhält die Zwischenepisode aus der Vielzahl der Kandidatenepisoden. Das Navigationssystem navigiert das autonome Fahrzeug anhand der Hypothesenepisode.

Navigationstrajektorie mit verstärkendem lernen für ein ego-fahrzeug in einem navigationsnetzwerk

Ein Ego-Fahrzeug enthält Entscheidermodule und ein Bewertungsmodul, das mit einem Lösermodul gekoppelt ist. Die Entscheidermodule generieren Trajektorienentscheidungen zu einem aktuellen Zeitpunkt, generieren einen aktuellen zweidimensionalen Ausschnitt eines flachen Raums um das Ego-Fahrzeug herum, generieren zukünftige zweidimensionale Ausschnitte des flachen Raums durch Projektion des aktuellen zweidimensionalen Ausschnitts des flachen Raums in der Zeit nach vorne, und generieren einen dreidimensionalen Zustandsraum durch Stapeln des aktuellen zweidimensionalen Ausschnitts und der zukünftigen zweidimensionalen Ausschnitte. Das Bewertungsmodul generiert Benotungen für die Trajektorienentscheidungen basierend auf dem aktuellen Verhalten eines Ego-Fahrzeugs. Das Lösermodul wählt eine endgültige Trajektorienentscheidung für das Ego-Fahrzeug aus den Trajektorienentscheidungen basierend auf dem dreidimensionalen Zustandsraum und den Benotungen aus. Der aktuelle zweidimensionale Ausschnitt enthält einen aktuellen Ego-Fahrzeugstandort und aktuelle Nachbarfahrzeugstandorte. Die zukünftigen zweidimensionalen Ausschnitte enthalten zukünftige Ego-Fahrzeugstandorte und zukünftige Nachbarfahrzeugstandorte.

Beacon-augmented pose estimation

A beacon-augmented pose estimation system includes positionable beacons that can determine their own 3-D geospatial coordinates and transmit those coordinates back to the pose object. An imaging sensor images the field-of-view of the pose object to provide 2-D image coordinates for any of the beacons in the FOV. A pose object controller processes the sets of 3-D geospatial coordinates and 2-D image coordinates to refine a pose estimate of the pose object. The positionable beacons may include both static beacons that are pre-positioned and mobile beacons that are command-guided to position themselves in the pose object's likely or actual FOV.

System und verfahren für ein vorhersage-entitätenverhalten

Verfahren zur Steuerung eines Fahrzeugs (10), umfassend:Empfangen von Sensordaten, die von einer dem Fahrzeug (10) zugeordneten Umgebung erfasst werden;Verarbeiten der Sensordaten durch einen Prozessor (44) zum Bestimmen von Beobachtungsdaten, wobei die Beobachtungsdaten unterschiedliche Merkmale beinhalten, die einem Agenten in der Umgebung zugeordnet sind;Bestimmen eines Kontextes (114), der dem Agenten aufgrund der Beobachtung zugeordnet ist, durch den Prozessor (44);Auswählen eines ersten Wahrscheinlichkeitsmodells, das dem Kontext (114) zugeordnet ist, durch den Prozessor (44);Verarbeiten der Beobachtungsdaten mit dem ausgewählten ersten Wahrscheinlichkeitsmodell durch den Prozessor (44), um einen Satz von Vorhersagen (116) zu bestimmen;Verarbeiten des Satzes von Vorhersagen (116) mit einem zweiten Wahrscheinlichkeitsmodell durch den Prozessor (44), um eine endgültige Vorhersage des mit dem Agenten verbundenen Interaktionsverhaltens zu bestimmen; undselektives Steuern des Fahrzeugs (10) durch den Prozessor (44) basierend auf der endgültigen Vorhersage des mit dem Agenten verbundenen Interaktionsverhaltens;wobei das erste Wahrscheinlichkeitsmodell ein Gaußsches Mischmodell - Hidden-Markov-Modell ist; undwobei das zweite Wahrscheinlichkeitsmodell ein spärlich korreliertes Hidden-Markov-Modell ist;ferner umfassend das Durchführen eines unbeaufsichtigten Trainings des Gaußschen Mischmodells - Hidden-Markov-Modells basierend auf dem Kontext (114), undumfassend das Durchführen eines unbeaufsichtigten Trainings des spärlich korrelierten Hidden-Markov-Modells basierend auf dem Kontext (114).

Beacon-augmented pose estimation

A beacon-augmented pose estimation system includes positionable beacons that can determine their own 3-D geospatial coordinates and transmit those coordinates back to the pose object. An imaging sensor images the field-of-view of the pose object to provide 2-D image coordinates for any of the beacons in the FOV. A pose object controller processes the sets of 3-D geospatial coordinates and 2-D image coordinates to refine a pose estimate of the pose object. The positionable beacons may include both static beacons that are pre-positioned and mobile beacons that are command-guided to position themselves in the pose object's likely or actual FOV.

System, verfahren und steuergerät für graphenbasierte wegplanung für ein host-fahrzeug

Ein Verfahren zur Wegplanung für ein Host-Fahrzeug umfasst: Empfangen von Host-Fahrzeug-, Umgebungs- und Hindernisinformationen; Ausrechnen einer oder mehrerer projizierter Host-Fahrzeugpositionen; Berechnen einer projizierten Hindernisposition für jedes Hindernis; und Bestimmen eines Kollisionspotentials zwischen jeder projizierten Host-Fahrzeugposition und jeder projizierten Hindernisposition. Bis eine maximale Anzahl von Schritten erreicht ist und während mindestens ein projizierter Standort des Host-Fahrzeugs ein zugehöriges Kollisionspotenzial unterhalb eines Kollisionsschwellenwerts aufweist, umfasst das Verfahren ferner die Wiederholung der Schritte des Ausrechnens, des Berechnens und des Bestimmens.

Unüberwachte geschwindigkeitsvorhersage und -korrektur für städtische fahrzeuge aus einer folge verrauschter positionsschätzungen

Ein Verfahren, das eine unüberwachte Geschwindigkeitsvorhersage und -korrektur für das Fahren in der Stadt aus Sequenzen von verrauschten Positionsschätzungen verwendet, beinhaltet: Durchführen einer Fahrzeuggeschwindigkeitsvorhersage für ein oder mehrere andere Fahrzeuge in der Nähe eines Host-Automobilfahrzeugs; Berechnen einer ersten Heuristik auf der Grundlage eines Gleichmäßigkeitstests; Berechnen einer zweiten Heuristik auf der Grundlage einer Fahrzeuggeschwindigkeit des einen oder der mehreren anderen Fahrzeuge; Kombinieren der ersten Heuristik und der zweiten Heuristik unter Verwendung einer gewichteten Summe; Bestimmen einer Unsicherheitsmaske unter Anwendung der kombinierten ersten Heuristik und der zweiten Heuristik und eines Heuristik-Schwellenwerts; und Anwenden der Unsicherheitsmaske, um eine Geschwindigkeitskorrektur zur Verwendung durch das Host-Automobilfahrzeug zu identifizieren.

Verfahren zur kognitiven situationswahrnehmung unter verwendung einer aufmerksamkeitsbasierten ereignisstruktur

Ein Verfahren zum Verwenden von wahrnehmungsinspirierter Ereignisgenerierung zur Situationswahrnehmung für ein Fahrzeug, einschließlich Empfangen von Wahrnehmungseingabedaten von einem Sensor des Fahrzeugs und Verarbeiten der Wahrnehmungseingabedaten zum Klassifizieren und Erzeugen von Parametern, die sich auf eine externe Entität in der Nähe des Fahrzeugs beziehen. Das Verfahren umfasst das Erzeugen einer hierarchischen Ereignisstruktur, die die Wahrnehmungseingabedaten klassifiziert und priorisiert, indem die externe Entität in eine Beachtungszone klassifiziert wird und die externe Entität innerhalb der Beachtungszone gemäß einem Risikoniveauwert für die externe Entität priorisiert wird. Ein höherer Risikoniveauwert bedeutet eine höhere Priorität innerhalb der Beachtungszone. Das Verfahren beinhaltet ferner das Entwickeln eines Verhaltensplans für das Fahrzeug auf der Grundlage der hierarchischen Ereignisstruktur.

3d-objekterkennungsverfahren unter nutzung der synergie heterogener sensoren für autonomes fahren

Verfahren zum Durchführen einer Objekterkennung während des autonomen Fahrens, das Folgendes umfasst: Durchführen einer 3D-Objekterkennung in einem 3D-Objekterkennungssegment; Hochladen einer Ausgabe mehrerer Sensoren, die mit dem 3D-Objekterkennungssegment in Verbindung stehen, in mehrere Punktwolken; Übertragen von Punktwolkendaten aus den mehreren Punktwolken in ein Region Proposal Network (RPN); unabhängiges Durchführen einer 2D-Objekterkennung in einem 2D-Objektdetektor parallel zu der 3D-Objekterkennung in dem 3D-Objekterkennungssegment; und Nehmen eines gegebenen Eingangsbildes und gleichzeitiges Lernen von Feldkoordinaten und Klassenetikettenwahrscheinlichkeiten in einem 2D-Objekterkennungsnetzwerk, das so arbeitet, dass die Objekterkennung als ein Regressionsproblem behandelt wird.

Online-fahrleistungsbewertung mit räumlichen und temporären verkehrsinformationen für autonome fahrsysteme

Ein autonomes Fahrzeug, System und Verfahren zum Betrieb des autonomen Fahrzeugs. Das System umfasst einen Leistungsbewerter, ein Entscheidungs-Modul und ein Navigationssystem. Der Leistungsbewerter bestimmt für jede von mehreren Entscheidungen zum Betrieb des autonomen Fahrzeugs eine Leistungsnote. Das Entscheidungs-Modul wählt eine Entscheidung aus, die den größten Leistungsnote hat. Das Navigationssystem bedient das autonome Fahrzeug anhand der gewählten Entscheidung.

Teleconferencing system

3D object detection method using synergy of heterogeneous sensors for autonomous driving

A method for performing object detection during autonomous driving includes: performing 3D object detection in a 3D object detection segment; uploading an output of multiple sensors in communication with the 3D object detection segment to multiple point clouds; transferring point cloud data from the multiple point clouds to a Region Proposal Network (RPN); independently performing 2D object detection in a 2D object detector in parallel with the 3D object detection in the 3D object detection segment; and taking a given input image and simultaneously learning box coordinates and class label probabilities in a 2D object detection network operating to treat object detection as a regression problem.

Skalierbares system zur interpretation von strassenschildern für das autonome fahren

Ein System zur Interpretation von Straßenschildern umfasst eine nach vorne gerichtete Kamera, die an oder in einem Fahrzeug angebracht ist und Bilddaten von mehreren Straßenschildern erfasst. Ein erstes gefaltetes neuronales Netzwerk (Convolutional Neural Network, CNN) empfängt die Bilddaten von der nach vorne gerichteten Kamera und liefert einen Satz von Schildervorhersagen, die eine oder mehrere Schildtextexemplare enthalten. Ein zweites CNN, das einen Textextraktor definiert, empfängt die Bilddaten von der nach vorne gerichteten Kamera und extrahiert Textkandidaten, die mehrere Zeichentextinstanzen enthalten. Die Lokalisierung von Zeichen und Zeichendaten ist im zweiten CNN vorgesehen, um eine Textreihenfolge aus den mehreren Zeichentextinstanzen zu berechnen. Ein Zeichentextsynthesemodul empfängt einzelne Zeichentextinstanzen vom ersten CNN und einzelne der Zeichentextinstanzen in digitalisierter Form von einem optischen Zeichenerkenner (OCR). Ein semantisches Kodierungs- und Interpretationsmodul empfängt die Schildtextexemplare und identifiziert die Semantik der verschiedenen Verkehrszeichen.

Schätzung einer virtuellen fahrspur mittels einer rekursiven selbstorganisierenden karte

Ein virtuelles Fahrspur-Schätzsystem umfasst eine Speichervorrichtung, einen Sensor und einen Computer. Die Speichervorrichtung ist so konfiguriert, dass sie eine Straßenkarte speichert, die einem Abschnitt einer Straße vor einem Fahrzeug entspricht. Der Sensor ist so konfiguriert, dass er eine Vielzahl von Trajektorien einer Vielzahl von benachbarten Fahrzeugen beobachtet, die den Abschnitt der Straße überqueren. Der Computer ist so konfiguriert, dass er eine rekursive selbstorganisierende Karte als eine Vielzahl von Punkten initialisiert, die als zweidimensionales Gitter angeordnet sind, das mit der Straßenkarte ausgerichtet ist, die Punkte in der rekursiven selbstorganisierenden Karte als Reaktion auf die Trajektorien trainiert, einen gerichteten Graphen erzeugt, der eine oder mehrere virtuelle Fahrspuren durch die Straßenkarte als Reaktion auf die auf die Trajektorien trainierten Punkte enthält, und ein Steuersignal erzeugt, das die Navigation des Fahrzeugs durch den Straßenabschnitt als Reaktion auf die virtuellen Fahrspuren in dem gerichteten Graphen steuert.

Kognitive prozessor vorsteuerung- und rückkopplung integration in autonome systeme

Ein autonomes Fahrzeug, ein kognitives System zum Betreiben eines autonomen Fahrzeugs und ein Verfahren zum Betreiben eines autonomen Fahrzeugs. Das kognitive System umfasst ein oder mehrere Hypothesengeber-Module, einen Hypothesen-Resolver, ein oder mehrere Entscheidungs-Module und einen Entscheidungs-Resolver. Daten, die mit einem Agenten in Verbindung stehen, werden am kognitiven System empfangen. Das mindestens eine Hypothesengeber-Modul erstellt eine Vielzahl von Hypothesen für eine Trajektorie des Agenten auf der Basis der empfangenen Daten. Der Hypothesen-Resolver wählt aus der Vielzahl der Hypothesen anhand eines Auswahlkriteriums eine einzige Hypothese für die Trajektorie des Agenten aus. Das mindestens eine Entscheidungs-Module erzeugen eine Vielzahl von Entscheidungen für eine Trajektorie des autonomen Fahrzeugs basierend auf der gewählten Hypothese für den Agenten. Der Entscheidungs-Resolver wählt aus der Vielzahl der Entscheidungen eine Trajektorie für das autonome Fahrzeug aus. Das autonome Fahrzeug wird basierend auf der gewählten Trajektorie betrieben.

Scalable road sign interpretation system for autonomous driving

A road sign interpretation system includes a front-facing camera mounted on or in a vehicle collecting image data of multiple road signs. A first convolutional neural network (CNN) receives the image data from the front-facing camera and yields a set of sign predictions including one or more sign text instances. A second CNN defining a text extractor receives the image data from the front-facing camera and extracts text candidates including the multiple sign text instances. Sign and sign data localization is provided in the second CNN to compute a text order from the multiple sign text instances. A sign text synthesizer module receives individual sign text instances from the first CNN and individual ones of the sign text instances in digitized forms from an optical character recognizer (OCR). A semantic encoding and interpretation module receives the sign text instances and identifies semantics of the multiple road signs.

Systems, methods, and apparatus for neuro-robotic goal selection

Systems, methods, and apparatus for neuro-robotic goal selection are disclosed. An example method to control a robot is described, including presenting a target object to a user, the target object corresponding to a goal to be effectuated by a robot, emphasizing a portion of the target object, identifying a first brain signal corresponding to a first mental response of the user to the emphasized portion, determining whether the first mental response corresponds to a selection of the emphasized portion by the user, and effectuating the robot with respect to the goal based on the emphasized portion.