VISUAL SPECIAL EFFECT IMAGE OR VIDEO PAIR AND GENERATION METHOD AND DEVICE THEREFOR

1. Field of the Invention

The present invention relates to a visual special effect image or video pair suitable for binocular viewing, and a method and device for generating the visual special effect image or video pair suitable for binocular viewing.

2. Relevant Background

Currently, market-oriented 3D display approach mainly presents two-channel visual content to left and right eyes respectively. The viewer can stably fuse the two-channel visual content and generate three-dimensional sensation. Two eyes have a distance of about 6 cm, and thus see an object from two different positions and views. The difference in position of images projected to two eyes respectively is called binocular disparity. Generally, it is thought that two-channel visual content only have binocular disparity, but no other differences, such as difference in brightness, color and so on.

There are many ways to find correspondence between left and right view images for stereoscopic display. Two regions in correspondence are called corresponding regions for two views. The following are several common ways to find corresponding regions: 1) Depth map based method: Assume a point A in one view with location (mx, my). if the view's depth map and camera parameters are already known, we can find the corresponding point A′ in the other view through formulation

mx′=mx+ftxZ,

where f is camera focal length and tx is the distance between two cameras. The location of point A′ in the other view is (mx′, my). Corresponding regions can be found based on corresponding points. 2) Content feature based matching algorithm: firstly, significant features are extracted from the two images. Local features are corner, edge, line, etc. Global features are polygon, image structure descriptor, etc. Then, features are represented by parameters. Finally, match the two images by the feature parameters and find corresponding regions. 3) Greyscale based matching algorithm: images are regarded as 2D signals. Matching relationship is found by calculating the correlation function of the two signals. Thus, corresponding regions are found for two images. The way to find corresponding regions is not limited to the above.

When difference of binocular images is little, our brain can fuse left and right views and have stable texture perception and determinate depth perception. The physiological and psychological study of binocular vision reveals that when the difference of binocular images is large, our brain can't fuse left and right views and thus have unstable texture perception and uncertain depth perception. Especially, when regions in binocular images have opposite brightness contrast polarities, and the absolute value of difference between two brightness contrasts exceeds a critical value, these regions are perceived with fluctuating lustre. This phenomenon is called binocular rivalry. The fluctuating lustre sensation is due to the nonuniform, unstable and time varying brightness sensation of the surface.

The present invention provides a visual special effect image or video pair suitable for binocular viewing, and a method and device for generating the visual special effect image or video pair suitable for binocular viewing.

The present invention employs the following technical solution:

The first object of the present invention is to provide a visual special effect image or video pair, in which the brightness contrasts (D1 and D2) of corresponding regions (Z1 and Z2) in left and right view for at least one region to perimeter regions (ZZ1 and ZZ2) have opposite polarities (P1 and P2), and the absolute value of the difference between two brightness contrasts of the regions in left and right views exceeds a critical value. So that when the image or video pair is viewed separately using both eyes, a visual special effect of fluctuating lustre sensation appears in that region.

The second object of the present invention is to a method for generating visual special effect image or video, comprising: modifying the brightness of at least one region Z1 in at least one view of left and right views, so that the polarity P1 of a brightness contrast D1 of Z1 to its perimeter region ZZ1 is opposite to the polarity P2 of a brightness contrast D2 of a corresponding region Z2 in the other view to the perimeter region ZZ2 of left and right views, and the absolute value of the difference between the brightness contrast of the region in the view to the perimeter region and the brightness contrast of the corresponding region in the other view to the perimeter region exceeds a critical value; and obtaining a new image or video pair.

The preferred brightness modification further comprises: Modifying the brightness of pixels in both the region Z1 and at least one neighboring region ZA1 in the perimeter.

The preferred brightness modification is one of the following ways:

1) Assume that the region to be modified in brightness is called Z1, region Z2 is the corresponding region Z1 in the other view, the average brightness of perimeter region of Z1 is valued L_S1, and the average brightness of perimeter region of Z2 is valued L_S2; for a pixel in region Z1 (x₁, y₁)εZ₁, its brightness value L_i(x₁, y₁) adds A(x₁, y₁) to be L*_i(x₁, y₁), and the average brightness of region Z1 becomes L*_o(Z₁); for a pixel in region Z2 (x₂, y₂)εZ₂, do not modify its brightness value or the brightness value L_i(x₂, y₂) adds B(x₂, y₂) to be L*_i(x₂, y₂), and the average brightness of region Z2 becomes L*_o(Z₂); so that after the brightness modification, (L*_o(Z₁)−L_S1)(L*_o(Z₂)−L_S2)<0, that is the polarity of the brightness contrast of region Z1 is opposite to the polarity of the brightness contrast of region Z2, and the absolute value of difference between two brightness contrasts exceeds a critical value; at least one of the added value A(x₁, y₁) and B(x₂, y₂) of corresponding regions should not be zero;

2) Assume that the region to be modified in brightness is called Z1, region Z2 is the corresponding region Z1 in the other view, the average brightness of perimeter region of Z1 is valued L_S1, and the average brightness of perimeter region of Z2 is valued L_S2; for a pixel in region Z1 (x₁, y₁)εZ₁, its brightness value L_i(x₁, y₁) multiplies A(x₁, y₁) to be L*_i(x₁, y₁), and the average brightness of region Z1 becomes L*_o(Z₁); for a pixel in region Z2 (x₂, y₂)εZ₂, do not modify its brightness value or the brightness value L_i(x₂, y₂) multiplies B(x₂, y₂) to be L*_i(x₂, y₂), and the average brightness of region Z2 becomes L*_o(Z₂); so that after the brightness modification, (L*_o(Z₁)−L_S1)(L*_o(Z₂)−L_S2)<0, that is the polarity of the brightness contrast of region Z1 is opposite to the polarity of the brightness contrast of region Z2, and the absolute value of difference between two brightness contrasts exceeds a critical value; at least one of the multiplied value A(x₁, y₁) and B(x₂, y₂) of corresponding regions should not be zero.

The preferred brightness modification further comprises: Keeping brightness of pixels on the frontiers of corresponding regions Z1 and Z2 unchanged and only modifying the brightness of pixels inside corresponding regions Z1 and Z2.

The third object of the present invention is to provide a device for generating visual special effect image or video pair, comprising:

An input module of left view and right view that inputs images of left view and right view to be processed;

A brightness contrast polarity reversal module that modifies brightness of at least one region Z1 in at least one view of left and right views in image or video pair, so that the polarity P1 of a brightness contrast D1 of Z1 to its perimeter region ZZ1 is opposite to the polarity P2 of a brightness contrast D2 of a corresponding region Z2 in the other view to the perimeter region ZZ2 of left and right views, and the absolute value of the difference between the brightness contrast of the region in the view to the perimeter region and the brightness contrast of the corresponding region in the other view to the perimeter region exceeds a critical value.

The preferred brightness modification, in the brightness contrast reversal module, comprises one of the following submodules:

1) Addition submodule: Assume that the region to be modified in brightness is called Z1, region Z2 is the corresponding region Z1 in the other view, the average brightness of perimeter region of Z1 is valued L_S1, and the average brightness of perimeter region of Z2 is valued L_S2; for a pixel in region Z1 (x₁, y₁)εZ₁, its brightness value L_i(x₁, y₁) adds A(x₁, y₁) to be L*_i(x₁, y₁), and the average brightness of region Z1 becomes L*_o(Z₁); for a pixel in region Z2 (x₂, y₂)εZ₂, do not modify its brightness value or the brightness value L_i(x₂, y₂) adds B(x₂, y₂) to be L*_i(x₂, y₂), and the average brightness of region Z2 becomes L*_o(Z₂); so that after the brightness modification, (L*_o(Z₁)−L_S1)(L*_o(Z₂)−L_S2)<0, that is the polarity of the brightness contrast of region Z1 is opposite to the polarity of the brightness contrast of region Z2, and the absolute value of difference between two brightness contrasts exceeds a critical value; at least one of the added value A(x₁, y₁) and B(x₂, y₂) of corresponding regions should not be zero;

2) Multiplication submodule: assume that the region to be modified in brightness is called Z1, region Z2 is the corresponding region Z1 in the other view, the average brightness of perimeter region of Z1 is valued L_S1, and the average brightness of perimeter region of Z2 is valued L_S2; for a pixel in region Z1 (x₁, y₁)εZ₁, its brightness value L_i(x₁, y₁) multiplies A(x₁, y₁) to be L*_i(x₁, y₁), and the average brightness of region Z1 becomes L*_o(Z₁); for a pixel in region Z2 (x₂, y₂)εZ₂, do not modify its brightness value or the brightness value L_i(x₂, y₂) multiplies B(x₂, y₂) to be L*_i(x₂, y₂), and the average brightness of region Z2 becomes L*_o(Z₂); so that after the brightness modification, (L*_o(Z₁)−L_S1)(L*_o(Z₂)−L_S2)<0, that is the polarity of the brightness contrast of region Z1 is opposite to the polarity of the brightness contrast of region Z2, and the absolute value of difference between two brightness contrasts exceeds a critical value; at least one of the multiplied value A(x₁, y₁) and B(x₂, y₂) of corresponding regions should not be zero.

The preferred brightness modification, in the brightness contrast reversal module, further comprises: Keeping brightness of pixels on the frontiers of corresponding regions Z1 and Z2 unchanged and only modifying the brightness of pixels inside corresponding regions Z1 and Z2.

The beneficial effect of the present invention lies in:

When the image or video pair generated by the method or device described in the present invention is viewed in stereoscopic display, the view can get:

1) a metallic shine surface perception effect;

2) a nonuniform and time varying brightness perception effect;

3) a dynamic perception effect provided in static images.

The embodiment of the present invention provides a visual special effect image or video pair.

The visual special effect image or video pair comprises:

the brightness contrasts (D1 and D2) of corresponding regions (Z1 and Z2) in left and right view for at least one region to perimeter regions (ZZ1 and ZZ2) have opposite polarities (P1 and P2), and the absolute value of the difference between two brightness contrasts of the regions in left and right views exceeds a critical value. So that when the image or video pair is viewed separately using both eyes, a visual special effect of fluctuating lustre sensation appears in that region.

The corresponding regions can be either regions occupied in left and right view images by an object and inside the object boundary, or regions occupied in left and right view images by part of an object. Corresponding regions in left and right view images are not confined to the above two definitions. Briefly speaking, corresponding regions in left and right view images are target regions for generating special effect by this invention.

The following two concrete examples elaborate the visual special effect image or video pair.

FIG. 3 shows left and right view images, where regions occupied by the sun are corresponding regions for left and right view images. The perimeter regions are peripheral regions neighbor to the sun border. In left view, average brightness of the region occupied by the sun are greater than that of perimeter region; while in right view, average brightness of the region are smaller than that of perimeter region, so that polarities of brightness contrasts are opposite. And when the absolute value of difference between two brightness contrasts of average brightness of regions occupied by the sun to average brightness of perimeter regions in left and right view exceeds a critical value, a visual special effect of fluctuating lustre sensation appears in regions occupied by the sun due to binocular rivalry in stereoscopic display of left and right view. But regions that have fluctuating lustre effect don't have fluctuating lustre texture, or time and space varying complicated texture in each image of present image pair.

FIG. 5 shows left and right view images, where regions occupied by diamond facets are corresponding regions for left and right view images, e.g. facet Z in FIG. 5. The perimeter regions of facet Z are neighboring facet ZZ1, ZZ2 and ZZ3. In left and right view, the differences between average brightness of facet Z1 and average brightness of neighboring facet ZZ1, ZZ2 and ZZ3 have opposite polarities, so that polarities of brightness contrasts are opposite. When the absolute value of difference between brightness contrasts of average brightness of region Z1 to average brightness of perimeter regions ZZ1, ZZ2 and ZZ3 in left and right view exceeds a critical value, a visual special effect of fluctuating lustre sensation appears in facet Z due to binocular rivalry in stereoscopic display of left and right view. But regions that have fluctuating lustre effect don't have fluctuating lustre texture, or time and space varying complicated texture in each image of present image pair.

The obtained multi-view video pair, in which at least one frame is the visual special effect image pair mentioned above, is regarded as visual special effect video pair. Therefore, this invention is also applicable to videos.

The embodiment of the present invention provides a method for generating visual special effect image or video.

The method specifically comprises:

modifying the brightness of at least one region Z1 in at least one view of left and right views, so that the polarity P1 of a brightness contrast D1 of Z1 to its perimeter region ZZ1 is opposite to the polarity P2 of a brightness contrast D2 of a corresponding region Z2 in the other view to the perimeter region ZZ2 of left and right views, and the absolute value of the difference between the brightness contrast of the region in the view to the perimeter region and the brightness contrast of the corresponding region in the other view to the perimeter region exceeds a critical value; and obtaining a new image or video pair.

The corresponding regions can be either regions occupied in left and right view images by an object and inside the object boundary, or regions occupied in left and right view images by part of an object. Corresponding regions in left and right view images are not confined to the above two definitions. Briefly speaking, corresponding regions in left and right view images are target regions for generating special effect by this invention.

The following two concrete examples elaborate the visual special effect image or video pair.

FIG. 2 shows left and right view images before processing, and FIG. 3 shows left and right view images after processing. Regions occupied by the sun are corresponding regions in left and right view images. The perimeter regions are peripheral regions neighbor to the sun border. Before processing, in both left and right view, average brightness of the region occupied by the sun are greater than that of perimeter region. Modify the brightness of region occupied by the sun in the right view, to make average brightness of region occupied by the sun greater than that of perimeter region. So that after modification, brightness contrasts of regions occupied by the sun to perimeter regions in the left and right view have opposite polarities, and the absolute value of difference between two brightness contrasts of average brightness of regions occupied by the sun to average brightness of perimeter regions in left and right view exceeds a critical value. When the obtained left and right view image pair is viewed in stereoscopic display, a visual special effect of fluctuating lustre sensation appears in regions occupied by the sun due to binocular rivalry. But regions that have fluctuating lustre effect don't have fluctuating lustre texture, or time and space varying complicated texture in each image of present image pair.

FIG. 4 shows left and right view images before processing, and FIG. 5 shows left and right view images after processing. Regions occupied by diamond facets are corresponding regions in left and right view images, e.g. facet Z in FIG. 5. The perimeter regions of facet Z are neighboring facet ZZ1, ZZ2 and ZZ3. Before processing, all diamond facets have same average brightness in left and right view, so brightness contrasts of facet Z in both left and right view equal zero. Modify the brightness of each facet in left and right view, so that brightness contrasts of average brightness of facet Z1 to average brightness of perimeter regions ZZ1, ZZ2 and ZZ3 in left and right view have opposite polarities, and the absolute value of difference between two brightness contrasts exceed a critical value. When the obtained left and right view image pair is viewed in stereoscopic display, a visual special effect of fluctuating lustre sensation appears in regions occupied by the sun due to binocular rivalry. But regions that have fluctuating lustre effect don't have fluctuating lustre texture, or time and space varying complicated texture in each image of present image pair.

The obtained multi-view video pair, in which at least one frame is the visual special effect image pair mentioned above, is regarded as visual special effect video pair. Therefore, this invention is also applicable to videos.

The embodiment of the present invention provides a method for generating visual special effect image or video.

The method specifically comprises:

modifying the brightness of at least one region Z1 in at least one view of left and right views, so that the polarity P1 of a brightness contrast D1 of Z1 to its perimeter region ZZ1 is opposite to the polarity P2 of a brightness contrast D2 of a corresponding region Z2 in the other view to the perimeter region ZZ2 of left and right views, and the absolute value of the difference between the brightness contrast of the region in the view to the perimeter region and the brightness contrast of the corresponding region in the other view to the perimeter region exceeds a critical value; and obtaining a new image or video pair.

Before modification processing, if input is 2D image or video, duplicate it to be at least two view images or videos; if input are two channel 3D images or videos, the two are regarded as left and right view; if input are multi-view images or videos, select any two channel images among multi-view contents to be left and right view; if input are depth image based rendering contents, select any two channel images among depth image based rendering contents to be left and right view.

Detailed modification is:

Modify the brightness of pixels in both the region Z1 and at least one neighboring region ZA1 in the perimeter;

Specifically, as shown in FIG. 6, region Z1 and region ZA1 are neighboring regions in a view, and region Z2 and region ZA2 are corresponding regions in the other view of region Z1 and region ZA1 respectively. Assume that L_Z1and L_ZA1are average brightness of region Z1 and region ZA1 respectively, L_S1is the average brightness of region Z1 and region ZA1, and L_Z2and L_ZA2are average brightness of region Z2 and region ZA2. Before brightness modification, brightness contrasts of region Z1 and Z2 to region ZA1 and ZA2 in left and right view have same polarity, namely (L_Z1−L_ZA1)(L_Z2−L_ZA2)>0, so fluctuating lustre sensation would not appear. Reverse the brightness of pixels in region Z1 with the average brightness L_S1of both region Z1 and region ZA1, to make average brightness of region Z1 be L_Z1*=2L_S1−L_Z1=L_ZA1; reverse the brightness of pixels in region ZA1 with the average brightness L_S1of both region Z1 and region ZA1, to make average brightness of region ZA1 be L_ZA1*=2L_S1−L_ZA1=L_Z1; after brightness modification, (L_Z1*−L_ZA1)(L_Z2−L_ZA2)<0, where L_Z2and L_ZA2are average brightness of region Z2 and region ZA2, so that the polarity of brightness contrast of region Z1 to region ZA1 is opposite to the polarity of brightness contrast of region Z2 to region ZA2, and the absolute value of difference between two brightness contrasts exceeds a critical value.

This brightness modification method guarantees the total average brightness of target region unchanged.

This example is only a realization of brightness modification method. There are other different realizations, but just another one or a combination realization should be the method of patent subject matter.

For multi-view videos, modify brightness of at least one frame in left and right view video by the above method. The obtained video pair is also viewed with fluctuating lustre in stereoscopic display. Therefore, the invention is also applicable to videos.

The embodiment of the present invention provides a method for generating visual special effect image or video.

The method specifically comprises:

modifying the brightness of at least one region Z1 in at least one view of left and right views, so that the polarity P1 of a brightness contrast D1 of Z1 to its perimeter region ZZ1 is opposite to the polarity P2 of a brightness contrast D2 of a corresponding region Z2 in the other view to the perimeter region ZZ2 of left and right views, and the absolute value of the difference between the brightness contrast of the region in the view to the perimeter region and the brightness contrast of the corresponding region in the other view to the perimeter region exceeds a critical value; and obtaining a new image or video pair.

Before modification processing, if input is 2D image or video, duplicate it to be at least two view images or videos; if input are two channel 3D images or videos, the two are regarded as left and right view; if input are multi-view images or videos, select any two channel images among multi-view contents to be left and right view; if input are depth image based rendering contents, select any two channel images among depth image based rendering contents to be left and right view.

Under normal circumstances, in unprocessed image pair, brightness contrasts of corresponding regions to perimeter regions in left and right view are of same polarity. That is to say average brightness of the regions in left and right view are both greater or both smaller than average brightness of perimeter regions. Therefore, the invention needs to modify brightness of at least one region in at least one view image, to make brightness contrasts of the regions in left and right view have opposite polarities, and the absolute value of difference between two brightness contrasts exceeds a critical value.

Detailed modification is:

Assume that the region to be modified in brightness is called Z1, region Z2 is the corresponding region Z1 in the other view, the average brightness of perimeter region of Z1 is valued L_S1, and the average brightness of perimeter region of Z2 is valued L_S2; for a pixel in region Z1 (x₁, y₁)εZ₁, its brightness value L_i(x₁, y₁) adds A(x₁, y₁) to be L*_i(x₁, y₁), and the average brightness of region Z1 becomes L*_o(Z₁); for a pixel in region Z2 (x₂, y₂)εZ₂, do not modify its brightness value or the brightness value L_i(x₂, y₂) adds B(x₂, y₂) to be L*_i(x₂, y₂), and the average brightness of region Z2 becomes L*_o(Z₂); so that after the brightness modification, (L*_o(Z₁)−L_S1)(L*_o(Z₂)−L_S2)<0, that is the polarity of the brightness contrast of region Z1 is opposite to the polarity of the brightness contrast of region Z2, and the absolute value of difference between two brightness contrasts exceeds a critical value. At least one of the added value A(x₁, y₁) and B(x₂, y₂) of corresponding regions should not be zero;

A(x₁, y₁) and B(x₂, y₂) can be constants. That is, brightness of every pixel in region Z1 adds or extracts a unified offset A, and brightness of every pixel in corresponding region Z2 adds or extracts a unified offset B. A(x₁, y₁) and B(x₂, y₂) can also be monotonous functions along a certain direction. The scope of function A(x₁, y₁) and B(x₂, y₂) can be either all pixels or part pixels in region Z1 and its corresponding region Z2. It should keep the texture in corresponding regions in left and right view unchanged after modification, and satisfy that brightness contrasts of corresponding regions to their perimeter regions have opposite polarities and the absolute value of difference between two brightness contrasts exceeds a critical value.

This brightness modification method guarantees the texture of target regions unchanged.

Specifically, as shown in FIG. 7, assume that target regions Z1 and Z2 are corresponding regions in left and right view, regions ZZ1 and ZZ2 are perimeter regions of Z1 and Z2 respectively, L_o(Z₁) and L_o(Z₂) are average brightness of target regions Z1 and Z2 respectively, and L_S1, L_S2are average brightness of perimeter regions ZZ1 and ZZ2 respectively. Before brightness modification, brightness contrasts of region Z1 and Z2 to region ZA1 and ZA2 in left and right view have same polarity, namely (L_o(Z₁)−L_S1)(L_o(Z₂)−L_S2)>0, so fluctuating lustre sensation would not appear. Brightness of pixels in region Z1 extracts an offset A, to make the average brightness be L*_o(Z₁)=L_o(Z₁)−A; brightness of pixels in region Z2 adds an offset B, to make the average brightness be L*_o(Z₂)=L_o(Z₂)+B. The original texture of region Z1 and Z2 are preserved. After brightness modification of corresponding regions Z1 and Z2, brightness contrasts of these corresponding regions in left and right view have opposite polarities, namely (L*_o(Z₁)−L_S1)(L*_o(Z₂)−L_S2)<0, and the absolute value of difference between two brightness contrasts exceeds a critical value.

This example is only a realization of brightness modification method. There are other different realizations, but just another one or a combination realization should be the method of patent subject matter.

For multi-view videos, modify brightness of at least one frame in left and right view video by the above method. The obtained video pair is also viewed with fluctuating lustre in stereoscopic display. Therefore, the invention is also applicable to videos.

The embodiment of the present invention provides a method for generating visual special effect image or video.

The method specifically comprises:

modifying the brightness of at least one region Z1 in at least one view of left and right views, so that the polarity P1 of a brightness contrast D1 of Z1 to its perimeter region ZZ1 is opposite to the polarity P2 of a brightness contrast D2 of a corresponding region Z2 in the other view to the perimeter region ZZ2 of left and right views, and the absolute value of the difference between the brightness contrast of the region in the view to the perimeter region and the brightness contrast of the corresponding region in the other view to the perimeter region exceeds a critical value; and obtaining a new image or video pair.

Before modification processing, if input is 2D image or video, duplicate it to be at least two view images or videos; if input are two channel 3D images or videos, the two are regarded as left and right view; if input are multi-view images or videos, select any two channel images among multi-view contents to be left and right view; if input are depth image based rendering contents, select any two channel images among depth image based rendering contents to be left and right view.

Under normal circumstances, in unprocessed image pair, brightness contrasts of corresponding regions to perimeter regions in left and right view are of same polarity. That is to say average brightness of the regions in left and right view are both greater or both smaller than average brightness of perimeter regions. Therefore, the invention needs to modify brightness of at least one region in at least one view image, to make brightness contrasts of the regions in left and right view have opposite polarities, and the absolute value of difference between two brightness contrasts exceeds a critical value.

Detailed modification is:

Assume that L_S1is the average brightness of region ZZ1 on the perimeter of region Z1 and L_S2is the average brightness of region ZZ2 on the perimeter of region Z2. For a pixel in region Z1 (x₁, y₁)εZ₁, its brightness value L_i(x₁, y₁) multiplies A(x₁, y₁) to be L*_i(x₁, y₁), and the average brightness of region Z1 becomes L*_o(Z₁); for a pixel in region Z2 (x₂, y₂)εZ₂, do not modify its brightness value or the brightness value L_i(x₁, y₂) multiplies B(x₂, y₂) to be L*_i(x₂, y₂), and the average brightness of region Z2 becomes L*_o(Z₂); so that after the brightness modification, (L*_o(Z₁)−L_S1)(L*_o(Z₂)−L_S2)<0, that is the polarity of the brightness contrast of region Z1 is opposite to the polarity of the brightness contrast of region Z2, and the absolute value of difference between two brightness contrasts exceeds a critical value; at least one of the multiplied value A(x₁, y₁) and B(x₂, y₂) of corresponding regions should not be zero.

A(x₁, y₁) and B(x₂, y₂) can be constants. That is, brightness of every pixel in region Z1 multiplies a unified value A, and brightness of every pixel in corresponding region Z2 multiplies a unified value B. A(x₁, y₁) and B(x₂, y₂) can also be monotonous functions along a certain direction. The scope of function A(x₁, y₁) and B(x₂, y₂) can be either all pixels or part pixels in region Z1 and its corresponding region Z2. It should keep the texture in corresponding regions in left and right view unchanged after modification, and satisfy that brightness contrasts of corresponding regions to their perimeter regions have opposite polarities and the absolute value of difference between two brightness contrasts exceeds a critical value.

This brightness modification method enhances the texture contrast of target regions.

Specifically, as shown in FIG. 8, assume that target regions Z1 and Z2 are corresponding regions in left and right view, regions ZZ1 and ZZ2 are perimeter regions of Z1 and Z2 respectively, L_o(Z₁) and L_o(Z₂) are average brightness of target regions Z1 and Z2 respectively, and L_S1, L_S2are average brightness of perimeter regions ZZ1 and ZZ2 respectively. Before brightness modification, brightness contrasts of region Z1 and Z2 to region ZA1 and ZA2 in left and right view have same polarity, namely (L_o(Z₁)−L_S1)(L_o(Z₂)−L_S2)>0, so fluctuating lustre sensation would not appear. Brightness of pixels in region Z1 multiplies a value A, to make average brightness be L*_o(Z₁)=A·L_o(Z₁); brightness of pixels in region multiplies a value B, to make average brightness be L*_o(Z₂)=B·L_o(Z₂). The original texture of region Z1 and Z2 are preserved. After brightness modification of corresponding regions Z1 and Z2, brightness contrasts of these corresponding regions in left and right view have opposite polarities, namely (L*_o(Z₁)−L_S1)(L*_o(Z₂)−L_S2)<0, and the absolute value of difference between two brightness contrasts exceeds a critical value.

This example is only a realization of brightness modification method. There are other different realizations, but just another one or a combination realization should be the method of patent subject matter.

For multi-view videos, modify brightness of at least one frame in left and right view video by the above method. The obtained video pair is also viewed with fluctuating lustre in stereoscopic display. Therefore, the invention is also applicable to videos.

The embodiment of the present invention provides a method for generating visual special effect image or video.

The method specifically comprises:

modifying the brightness of at least one region Z1 in at least one view of left and right views, so that the polarity P1 of a brightness contrast D1 of Z1 to its perimeter region ZZ1 is opposite to the polarity P2 of a brightness contrast D2 of a corresponding region Z2 in the other view to the perimeter region ZZ2 of left and right views, and the absolute value of the difference between the brightness contrast of the region in the view to the perimeter region and the brightness contrast of the corresponding region in the other view to the perimeter region exceeds a critical value; and obtaining a new image or video pair.

Under normal circumstances, in unprocessed image pair, brightness contrasts of corresponding regions to perimeter regions in left and right view are of same polarity. That is to say average brightness of the regions in left and right view are both greater or both smaller than average brightness of perimeter regions. Therefore, the invention needs to modify brightness of at least one region in at least one view image, to make brightness contrasts of the regions in left and right view have opposite polarities, and the absolute value of difference between two brightness contrasts exceeds a critical value.

Visually establishing corresponding regions needs strong clue for binocular matching. Before brightness modification, keep brightness of pixels on the frontiers of corresponding regions unchanged, and only modify the brightness of pixels inside corresponding regions. Two regions with the same frontier can be easily matched in vision, so that after brightness modification, when the obtained image or video pair is viewed separately using both eyes, a visual special effect of fluctuating lustre sensation appears inside that region.

The embodiment of the present invention provides a method for generating visual special effect image or video.

The method specifically comprises:

modifying the brightness of at least one region Z1 in at least one view of left and right views, so that the polarity P1 of a brightness contrast D1 of Z1 to its perimeter region ZZ1 is opposite to the polarity P2 of a brightness contrast D2 of a corresponding region Z2 in the other view to the perimeter region ZZ2 of left and right views, and the absolute value of the difference between the brightness contrast of the region in the view to the perimeter region and the brightness contrast of the corresponding region in the other view to the perimeter region exceeds a critical value; and obtaining a new image or video pair.

Before modification processing, if input is 2D image or video, duplicate it to be at least two view images or videos; if input are two channel 3D images or videos, the two are regarded as left and right view; if input are multi-view images or videos, select any two channel images among multi-view contents to be left and right view; if input are depth image based rendering contents, select any two channel images among depth image based rendering contents to be left and right view.

Detailed modification is:

Modify the brightness of pixels in both the region Z1 and at least one neighboring region ZA1 in the perimeter.

Brightness modification of region Z1 and region ZA1 adopts methods same as embodiment 4 or embodiment 5.

Visually establishing corresponding regions needs strong clue for binocular matching. Before brightness modification, keep brightness of pixels on the frontiers of corresponding regions unchanged, and only modify the brightness of pixels inside corresponding regions. Two regions with the same frontier can be easily matched in vision, so that after brightness modification, when the obtained image or video pair is viewed separately using both eyes, a visual special effect of fluctuating lustre sensation appears inside that region.

The embodiment of the present invention provides a device for generating visual special effect image or video. FIG. 1 is a structure diagram of a device for generating visual special effect image or video pair of this embodiment. The device comprises two modules:

1) An input module of left view and right view: that inputs images of left view and right view;

2) A brightness contrast polarity reversal module: input are output of the input module of left view and right view, and output are left and right view image or video pair after processing. Its function is to modify the brightness of at least one region Z1 in at least one view of left and right views, so that the polarity P1 of a brightness contrast D1 of Z1 to its perimeter region ZZ1 is opposite to the polarity P2 of a brightness contrast D2 of a corresponding region Z2 in the other view to the perimeter region ZZ2 of left and right views, and the absolute value of the difference between the brightness contrast of the region in the view to the perimeter region and the brightness contrast of the corresponding region in the other view to the perimeter region exceeds a critical value; and obtaining a new image or video pair.

The embodiment of the present invention provides a device for generating visual special effect image or video. The device comprises two modules:

1) An input module of left view and right view: input can be one of the following four sorts of content: {circle around (1)} 2D content; {circle around (2)} two channel 3D content; {circle around (3)} multi-view content; {circle around (4)} depth image based rendering content. Output are left and right view image or video pair to be processed. Its function is that before brightness modification, if input is 2D image or video, duplicate it to be at least two view images or videos, as shown in FIG. 9; if input are two channel 3D images or videos, the two are regarded as left and right view, as shown in FIG. 10; if input are multi-view images or videos, select any two channel images among multi-view contents to be left and right view, as shown in FIG. 11; if input are depth image based rendering contents, select any two channel images among depth image based rendering contents to be left and right view, as shown in FIG. 12.

2) A brightness contrast polarity reversal module: input are output of the input module of left view and right view, and output are left and right view image or video pair after processing. Its function is to modify the brightness of at least one region Z1 in at least one view of left and right views, so that the polarity P1 of a brightness contrast D1 of Z1 to its perimeter region ZZ1 is opposite to the polarity P2 of a brightness contrast D2 of a corresponding region Z2 in the other view to the perimeter region ZZ2 of left and right views, and the absolute value of the difference between the brightness contrast of the region in the view to the perimeter region and the brightness contrast of the corresponding region in the other view to the perimeter region exceeds a critical value; and obtaining a new image or video pair.

The embodiment of the present invention provides a device for generating visual special effect image or video. The device comprises two modules:

1) An input module of left view and right view: input can be one of the following four sorts of content: {circle around (1)} 2D content; {circle around (2)} two channel 3D content; {circle around (3)} multi-view content; {circle around (4)} depth image based rendering content. Output are left and right view image or video pair to be processed. Its function is that before brightness modification, if input is 2D image or video, duplicate it to be at least two view images or videos, as shown in FIG. 9; if input are two channel 3D images or videos, the two are regarded as left and right view, as shown in FIG. 10; if input are multi-view images or videos, select any two channel images among multi-view contents to be left and right view, as shown in FIG. 11; if input are depth image based rendering contents, select any two channel images among depth image based rendering contents to be left and right view, as shown in FIG. 12.

2) A brightness contrast polarity reversal module: input are output of the input module of left view and right view, and output are left and right view image or video pair after processing. Its function is to modify the brightness of at least one region Z1 in at least one view of left and right views and the brightness of one of its perimeter regions ZA1. After brightness modification, the polarity P1 of a brightness contrast D1 of Z1 to its perimeter region ZZ1 is opposite to the polarity P2 of a brightness contrast D2 of a corresponding region Z2 in the other view to the perimeter region ZZ2 of left and right views, and the absolute value of the difference between the brightness contrast of the region in the view to the perimeter region and the brightness contrast of the corresponding region in the other view to the perimeter region exceeds a critical value; and obtain a new image or video pair.

The embodiment of the present invention provides a device for generating visual special effect image or video. The device comprises two modules:

1) An input module of left view and right view: input can be one of the following four sorts of content: {circle around (1)} 2D content; {circle around (2)} two channel 3D content; {circle around (3)} multi-view content; {circle around (4)} depth image based rendering content. Output are left and right view image or video pair to be processed. Its function is that before brightness modification, if input is 2D image or video, duplicate it to be at least two view images or videos; if input are two channel 3D images or videos, the two are regarded as left and right view; if input are multi-view images or videos, select any two channel images among multi-view contents to be left and right view; if input are depth image based rendering contents, select any two channel images among depth image based rendering contents to be left and right view.

2) A brightness contrast polarity reversal module: input are output of the input module of left view and right view, and output are left and right view image or video pair after processing. It comprises addition submodule, multiplication submodule or combination of the two, as shown in FIG. 13-15.

The function of addition submodule is that assume that L_S1is the average brightness of region ZZ1 on the perimeter of region Z1 and L_S2is the average brightness of region ZZ2 on the perimeter of region Z2. For a pixel in region Z1 (x₁, y₁)εZ₁, its brightness value L_i(x₁, y₁) adds A(x₁, y₁) to be L*_i(x₁, y₁), and the average brightness of region Z1 becomes L*_o(Z₁); for a pixel in region Z2 (x₂, y₂)εZ₂, do not modify its brightness value or the brightness value L_i(x₂, y₂) adds B(x₂, y₂) to be L*_i(x₂, y₂), and the average brightness of region Z2 becomes L*_o(Z₂); so that after the brightness modification, (L*_o(Z₁)−L_S1)(L*_o(Z₂)−L_S2)<0 that is the polarity of the brightness contrast of region Z1 is opposite to the polarity of the brightness contrast of region Z2, and the absolute value of difference between two brightness contrasts exceeds a critical value; at least one of the added value A(x₁, y₁) and B(x₂, y₂) of corresponding regions should not be zero.

The function of multiplication submodule is that assume that L_S1is the average brightness of region ZZ1 on the perimeter of region Z1 and L_S2is the average brightness of region ZZ2 on the perimeter of region Z2. For a pixel in region Z1 (x₁, y₁)εZ₁, its brightness value L_i(x₁, y₁) multiplies A(x₁, y₁) to be L*_i(x₁, y₁), and the average brightness of region Z1 becomes L*_o(Z₁); for a pixel in region Z2 (x₂, y₂)εZ₂, do not modify its brightness value or the brightness value L_i(x₂, y₂) multiplies B(x₂, y₂) to be L*_i(x₂, y₂), and the average brightness of region Z2 becomes L*_o(Z₂); so that after the brightness modification, (L*_o(Z₁)−L_S1)(L*_o(Z₂)−L_S2)<0 that is the polarity of the brightness contrast of region Z1 is opposite to the polarity of the brightness contrast of region Z2, and the absolute value of difference between two brightness contrasts exceeds a critical value; at least one of the multiplied value A(x₁, y₁) and B(x₂, y₂) of corresponding regions should not be zero.

The embodiment of the present invention provides a device for generating visual special effect image or video. The device comprises two modules:

1) An input module of left view and right view: input can be one of the following four sorts of content: {circle around (1)} 2D content; {circle around (2)} two channel 3D content; {circle around (3)} multi-view content; {circle around (4)} depth image based rendering content. Output are left and right view image or video pair to be processed. Its function is that before brightness modification, if input is 2D image or video, duplicate it to be at least two view images or videos; if input are two channel 3D images or videos, the two are regarded as left and right view; if input are multi-view images or videos, select any two channel images among multi-view contents to be left and right view; if input are depth image based rendering contents, select any two channel images among depth image based rendering contents to be left and right view.

2) A brightness contrast polarity reversal module: input are output of the input module of left view and right view, and output are left and right view image or video pair after processing. Its function is to modify the brightness of at least one region Z1 in at least one view of left and right views and the brightness of one of its perimeter regions ZA1. After brightness modification, the polarity P1 of a brightness contrast D1 of Z1 to its perimeter region ZZ1 is opposite to the polarity P2 of a brightness contrast D2 of a corresponding region Z2 in the other view to the perimeter region ZZ2 of left and right views, and the absolute value of the difference between the brightness contrast of the region in the view to the perimeter region and the brightness contrast of the corresponding region in the other view to the perimeter region exceeds a critical value; and obtain a new image or video pair.

The brightness modification of region Z1 and one of its perimeter regions ZA1 comprises addition submodule, multiplication submodule or combination of the two. Assume that the region to be modified in brightness is called Z1, region Z2 is the corresponding region Z1 in the other view.

The function of addition submodule is that assume that L_S1is the average brightness of region ZZ1 on the perimeter of region Z1 and L_S2is the average brightness of region ZZ2 on the perimeter of region Z2. For a pixel in region Z1 (x₁, y₁)εZ₁, its brightness value L_i(x₁, y₁) adds A(x₁, y₁) to be L*_i(x₁, y₁), and the average brightness of region Z1 becomes L*_o(Z₁); for a pixel in region Z2 (x₂, y₂)εZ₂, do not modify its brightness value or the brightness value L_i(x₂, y₂) adds B(x₂, y₂) to be L*_i(x₂, y₂), and the average brightness of region Z2 becomes L*_o(Z₂); so that after the brightness modification, (L*_o(Z₁)−L_S1)(L*_o(Z₂)−L_S2)<0, that is the polarity of the brightness contrast of region Z1 is opposite to the polarity of the brightness contrast of region Z2, and the absolute value of difference between two brightness contrasts exceeds a critical value; at least one of the added value A(x₁, y₁) and B(x₂, y₂) of corresponding regions should not be zero.

The function of multiplication submodule is that assume that L_S1is the average brightness of region ZZ1 on the perimeter of region Z1 and L_S2is the average brightness of region ZZ2 on the perimeter of region Z2. For a pixel in region Z1 (x₁, y₁)εZ₁, its brightness value L_i(x₁, y₁) multiplies A(x₁, y₁) to be L*_i(x₁, y₁), and the average brightness of region Z1 becomes L*_o(Z₁); for a pixel in region Z2 (x₂, y₂)εZ₂, do not modify its brightness value or the brightness value L_i(x₂, y₂) multiplies B(x₂, y₂) to be L*_i(x₂, y₂), and the average brightness of region Z2 becomes L*_o(Z₂); so that after the brightness modification, (L*_o(Z₁)−L_S1)(L*_o(Z₂)−L_S2)<0 that is the polarity of the brightness contrast of region Z1 is opposite to the polarity of the brightness contrast of region Z2, and the absolute value of difference between two brightness contrasts exceeds a critical value; at least one of the multiplied value A(x₁, y₁) and B(x₂, y₂) of corresponding regions should not be zero.

It should be emphasized that, though embodiments mentioned above take images as example, it doesn't mean the invention is only applicable to images. This invention is also applicable to videos. Moreover, method for brightness modification of corresponding regions in left and right view can be a combination method of embodiment 3, embodiment 4 and embodiment 5.

It should be noted that: the above embodiments are merely provided for describing the technical solutions of the present invention, but not intended to limit it. Although reference to the embodiments of the present inventions has been described in detail, those skilled in the art will appreciate: it can still modify the foregoing technical solutions described in the embodiments, or equivalently replace some technical features; and such modifications or replacements do not make the essence of corresponding technical solutions depart from the scope of the technical solutions of the embodiments of the present invention.