MODEL-INDEPENDENT FACE LANDMARK RECOGNIZING DEVICE IN SPACE AUGMENTED REALITY

Semiconductor memory of the present invention embodiment of the present invention achieved by the present invention and the purpose of the present invention preferred embodiment members of example hereinafter with reference to describe it in a heat chamber.

First, a term used in a particular application only is used to account for in the embodiment, the present invention is defining which are not intending to be, it is apparent that a single representation of the differently in order not providing language translators, comprising plurality of representation can be. In addition in the application, the term "comprising" or "having disclosed" specification of articles feature, number, step, operation, components, parts or specify a combination not present included, another aspect of one or more moveable number, step, operation, component, component or a combination of these is understood to presence of or additionally pre-times those possibility should not number.

In describing the present invention, publicly known or a function of the specific subject matter of invention related description is the description if a haze can be decided to be dispensed to each other.

In the embodiment according to Figure 1 shows a land mark recognizing device of the present invention also augmented reality spatial model at one of stand-alone face are disclosed.

The reference also 1, special issue one in the embodiment according to of the present invention stand-alone face land mark recognizing device for imaging spatial model at reality acquiring unit (100), facial feature detector (200), land generation unit (300) a landmark tracking unit (400) made of disclosed.

Said Image acquisition section (100) includes a color (RGB: red-a green-a blue) Image and depth (Depth) camera RGB-a D camera in the color Image and depth can be acquiring an Image substrate.

Said color (RGB) Image capture the Image depth (Depth) for liquid water-depth camera RGB camera RGB-a D (Depth) or separate depth (Depth) camera can be configured.

General camera RGB Image obtained only but, in one embodiment of the present invention embodiment can obtain information used RGB-a D camera depth (Depth) the slide substrate.

Said depth (Depth) camera design condition for the SwissRanger 4000, PMD CamCube, D provided IMager, using Microsoft yarn Kinect disapproval.

Digital video acquisition section (100) during a predetermined time interval includes a plurality of input images each other.

On the other hand, Image acquisition section (100) includes a plurality of input images are analyzed each single channel can be obtained. For example, gray (Gray) scale the Image can be changing. Or an input channel when the multichannel Image 'RGB' same one channel value changing may be filled. The, respect to the input Image (Intensity) by values into one channel in coding, the color can be represent brightness distribution for hereinafter.

Then, facial feature detector (200) comprises a plurality of input Image from each detecting facial Image, Image is arranged around face detects a feature of other.

To this end facial feature detector (200) also includes a reference surface 2, face detector (210), local binary pattern calculator for (220), surface normal calculator for (230), each calculator for local (240) and a mixer (250) having a predetermined wavelength.

Said face detector (210) comprises a plurality of input Image from each detecting facial Image could be bonded each other. Face detector (210) includes a coarse face face after detection in each input images from certain components of the eyes, nose, mouth or the like to extract, the basis can be predetermined face area.

For example, if the state of the position is detected, determines the distance eyes can be. Face detector (210) distance between the eyes is facial model may be extracted face region is connected to an input, the input Image can be a defect to background or variation of human head style.

In addition, face detector (210) is the size of the extracted face region information face area capable of normalizing. Normalizing the size of face region by the distance between two eyes in a facial region, the distance between the eye and nose unique characteristics of the same scale level can be calculated.

Next, local binary pattern calculator for (220) is the number of facial Image with considering the brightness for each pixel to determine the local binary pattern.

Local binary pattern calculator for (220) includes a brightness LBP (Local Binary Pattern) by techniques can be calculated local binary pattern.

Thus, by applying LBP technique, face recognition etc. replacing (robust) illumination can be robust against external interference. Only, local binary pattern calculator for (220) includes illumination of an external interference insignificantly and may be omitted if disapproval. In addition, local binary pattern calculator for (220) local to respective facial Image binary pattern may be calculated, to calculate the local binary pattern formed on at least one of the predetermined face may be filled.

Said LBP technique and an illuminated visual effect of change in high durability and simple operation the Image recovery, biometric Image analysis, facial Image analysis and recognition method is applied to the intensity representative binary pattern conversion are disclosed. LBP neighboring pixel value of the pixel value difference is current position value of 0 and 1 exhibits.

LBP is greater as compared to a clockwise direction central pixel values 1, otherwise 0 value of the converting method, compared to 8 adjacent pixels because the represented by 8 bit.

On the other hand, surface normal calculator for (230) is 3 dimensional object information based on the specifications of the depth images from local surface normal vector (Local Surface Normal Vector) where it is extracting, camera and a local face coordinate conversion between rotating for matrix using a local surface normal vector Image consistency despite switching matrix face rotation point extracted.

Here, surface normal calculator for (230) also includes a reference surface 3, with reference to the camera coordinate face major landmarks (amount eyebrows, protrusion amount intermediate) local face coordinate system transformation matrix (Transformation Matrix) Tr inputs with each other.

Transformation matrix (Transformation Matrix) (rotation) value linear motion (translation) of rotation determined by the torso equations 1 through 3 dimensional coordinates camera coordinate system transformation of the face direction can be consistent.

(1 Expressions)

Here, (L_1X , L_1Y , L_1Z ) In local face coordinate right and landmarks of the eyebrows, (L_2X , L_2Y , L_2Z ) Land mark left in local face coordinate eyebrows, (L_3X , L_3Y , L_3Z ) In local face coordinate intermediate landmarks and right jaw, (L_4X , L_4Y , L_4Z ) Left in local face coordinate exhibits intermediate landmarks of protrusion.

And, (L'_1X , L'_1Y , L'_1Z ) In a camera coordinate right and landmarks of the eyebrows, (L'_2X , L'_2Y , L'_2Z ) Land mark left in camera coordinate eyebrows, (L'_3X , L'_3Y , L'_3Z ) In a camera coordinate intermediate landmarks and right jaw, (L'_4X , L'_4Y , L'_4Z ) Left in camera coordinate exhibits landmarks of the intermediate protrusion.

On the other hand, transformation matrix Tr 2 is arched as shown in the expressions (internal calibration matrix) on internal calibration matrix camera A, T R and transition vector (translation vector) diagonal matrix (rotation matrix) consisting disclosed.

(2 Expressions)

As described below 3 on expressions using camera coordinate conversion between a local face coordinate rotation of converting totally or matrix (R^-1 ) Surface of camera coordinate tangential vector Image by applying a local surface normal vector Image consistency despite face rotation produce a topical face coordinate conversion point.

(3 Expressions)

Here, (SN_Ix , SN_Iy , SN_Iz ) (Here, n is from 1 as the [...] represents i) surface of a camera coordinate and tangential vector, (SN '_Ix , SN '_Iy , SN '_Iz ) (Here, n is from 1 as the [...] represents i) face coordinate system of local tangential vector disclosed.

On the other hand, each calculator for local (240) is obtained local surface normal vector at each pixel (Locality Principle) local principle applying to each pattern to determine the local (LAP: Local Angular Pattern).

In residual same more particularly, each calculator for local (240) is pivotal (pivot) local surface normal vectors obtained in each pixel dot through local surface normal vector (inner product) between angle less than 0 when threshold (threshold), when taking into account the depth and direction 1 large considering local each pattern (LAP) is defined.

Each calculator for said local (240) is below 0 and 1 8 sites between 0 - 255 through expressions 4 consisting of a result local each pattern formed on the substrate.

(4 Expressions)

Here, v_Pivot Pivot local surface normal vector whose, v_i The system of the normal vector and pixel i, thresholds for the threshold which is, LAP [x, y] has a coordinate (x, y) represents each pattern having local pixel i, lap to lap of each pattern indicate components local, lap is representative is 0 to 7 quantizes i components are disclosed.

The definition is local (LAP) pixels each pattern generally (holistic) can be used for detecting and tracking face part (patch provided based) can be utilized even detecting a landmark face recognition by anti-inflammatory effect.

Next, mixer (250) includes a local binary pattern calculator for (220) calculated by a local binary pattern each calculator for local (240) calculated by the local each patterns face feature formed on the substrate.

Next, land generation unit (300) includes a tub portion of face Image extracted.

As shown in fig. 5 for land generation unit (300) includes a face topology former (310), potential regression [...] learning device (320) and recognizer (330) having a predetermined wavelength.

Said face topology former (310) in specific positions to the front and the landmarks are part of respective face face face topology tree design such as 6 are received in positional relationship to each other.

An upper face to the lower two landmark set into a set of tree produced face phase relationship tree structure, each leaf node in order to create a root node 10 landmark set of face washing tub (Leaf Node) less than 1000.

Then, potential regression [...] learning device (320) includes a root node of the tree including the plurality of tub ablaze matches said face include for facial Image (specifically said facial feature detector (220) test face Image detected in the face feature of entered by a learner) in each node topology tree having learning potential regression [...] subset including portion of face to have face along landmark landmark Image learning other.

As a result, potential regression [...] learning device (320) includes a washing tub including Image data include for root node matches ablaze after learning [...] has in each node of the subset has a landmark face along face topology including Image data.

Figure 7 shows a one in the embodiment according to of the present invention representing the potential regression [...] also are disclosed.

The split (split) node potential regression [...], facilitating the binary tree leaf node and nodes can be defined. Split node perform against the input test function, determine whether right or left child progress as children in the can.

Performing test function facilitating node does not. Instead, the split node facilitating arrival samples from two portions from the samples is defined. Single site of objects corresponding to the leaf node can output samples.

A general regression [...] potential regression [...] alternatively root node input samples can be divided into various portions. For example, root node of input predetermined potential tree model divided into various portions, divided portions along parallel tree leaf nodes different propagated to other. A test sample is divided facilitating root node input node, divided portions can be propagated to a leaf node. While, the general regression [...] root node of input does not dividing. Along one of input is entirely tree leaf node propagated to other.

Splitter node data comparing (Split) separating face feature in the substrate. When the largest dispersion value subset data separating can be tested several times between and separating into two sets (Split), for dispersion in difference current split node 50 to 95 with previously split node is large enough and not separating unit (Division) executable module. Separation (Division) topology data divided into independent of each tree node of child node corresponding to the node a selected partial region are disclosed. Child node corresponding to the node topology is preferable and facilitating node landmarks set of position value difference (Offset Vectors) for sparse subtrees.

Sides of the leaf node include for automatically change over only the set potential regression [...] leaf node to leaf node representing partial land position value difference overnight.

Recognizer (330) is said facial feature calculating unit (200) face feature of the input value learned when potential regression [...] interleave (Traverse) facilitating node and the leaf then stored at the node mark calculates a difference apparent landmark of results accumulated in face area.

On the other hand, M having face topology tree node, represents p (i) is preferable, on the children nodes is r l (i) (i) exhibit, here i∈M and belong to, i=0, 1, 2.. |M|are disclosed.

Here, p_i^l Each node i corresponding Image I with toe roll landmarks set from the central position of big.

Each potential regression [...] tree topology of each corresponding step out with each other. The former has an extent the root node topology tree i=0 in the model corresponding to the landmark face with each other.

Tree is grown by, landmark of leaf-node range on the children nodes until r l (i) (i) divided according. A learning data at each node in the subset of S S 2^l And S^r To separation function f_i On randomly chosen threshold τ_i Using apart from each other.

The learning process using the topology node i processing advances. Separation function f_i On subset defined as follows.

(5 Expressions)

(6 Expressions)

At this time, a device for imaging a pixel value I (,) I represents, on the normalized offset (offset) randomly u v exhibits. Wherein f_i The separation function are disclosed.

Separation function f_i In information and has the largest gain value, a gain value is not increased in previous node information, the learning process advances to a separate process. The following expressions i gain information in topology node is defined.

(7 Expressions)

Here, ∑_Im Χ logic value vector θ_m Sample covariance matrix of the set of are disclosed. 2 Subset of current center position used for offset vector disclosed.

In the center of the offset vector of a given facial Image learning data selected apart from each other by separation. The children nodes with corresponding learning data the server processing continues to a fine range. The offset vector stored in node separation.

For separating the topology node divided leaf node are repeated until one of the end landmark enter and reach.

In each leaf node, offset from the center of the landmarks of the preferable node is stored disclosed.

Next, landmark tracking unit (400) is detected t p landmarks of the frame Image^l_t (X^l_t , Y^l_t ) Based on (portion landmark), density optical flow (dense optical flow) G=(u_t , V_t ) (Convolution) (median filtering kernel) on a map of the landmark [...] 37a filtering kernel M p t + 1 frame^l_{T + 1} (Landmark landmark portion of) the position of a substrate.

(8 Expressions)

Here,, which is meaning [...], (x^-l_t , Y^-l_t ) Is (x^l_t , Y^l_t ) Round position are disclosed.

Then landmark tracking unit (400) is rapid motion or rotation during abrupt face configured landmark tracking of water to a landmark from around threshold error landmark (outlier landmark) whose one end, their neighboring configuration for fixing the landmark of track-pull other.

Specifically, landmark tracking unit (400) includes a first land marks (eye, nose, mouth) semantic space area neighboring configuration defined considering 2000.

The landmark tracking unit (400) includes a landmark error occurence, configuration and a direction value averages the distance of movement of neighboring land marks, and amplifiers landmarks error correcting substrate.

The same reference also 8, left eye [sep second configuration different configurations of land marks out of landmarks are around left eye [sep landmarks are error when it moving distance and direction of paired configuration land marks averaging amplifiers landmarks error correcting substrate.

The landmark tracking techniques the present invention according to RGB camera motion opens the front face by applying loose face number initial tracking result therefrom.

(A) of Figure 9 does not use a learning model of Figure 9 (b) is using the present invention and as a result of use Sparse optical flow based KLT tracker that at or superior performance.

In this way, the present invention refers to the van without face model learning, robust tracking performance than existing KLT tracking technique and viscoelasticity.

Future, severe deformation input, robustness of the eye motion of tracking the detected landmark information or to use constant tracking error is higher than initialization, it became grudge topology defined number can be the same.

In addition, tracking results in computer graphics module can be integrated with a seal number based system to augmented reality.