METHOD AND APPARATUS FOR TRANSMITTING AND RECEIVING IMAGE DATA FOR VIRTUAL REALITY STREAMING SERVICE

Hereinafter, the present disclosure of a variety of embodiment reference to drawing having e.g. is. However, this disclosure for certain embodiment of the physical shape not to be defined, the present disclosure of change (modification) various example embodiment, equivalents (equivalent), and/or substitute (alternative) for including should understood. In connection with description of drawings, similar components to similar references may be used.

The present in the document, "has", "may have", "includes", or "may include" such as characterized corresponding representation (e.g.: numerical, function, operation,, such as parts or components) indicating presence of which, the added feature presence of does not back number.

The present in the document, "B or A", "A B and/or at least one", or "A B and/or one or more" such as arranged together, representation of all possible items may include a combination. For example, "B or A", "A B and at least one", or "B or A at least one" the, (1) includes at least one A, (2) includes B at least one, or (3) at least one A and at least one B including both can be when referring to both.

A being used in the various embodiment "number 1", "number 2", "first", or "second" such as various elements expressions, order and/or importance regardless can be qualify, has by not limiting the corresponding components. Said components expressions considering various with other components can be used. For example, user equipment and user device number 1 number 2, regardless of the importance order or, to different user equipment can exhibit. For example, the present disclosure without a wireless type through a wire rights number 1 number 2 component can be designated components, similarly starting to write is delayed components number 1 number 2 elements can be designated.

On any configuration element (e.g.: number 1 components) is other components (e.g.: number 2 components) to "(functionally or communicatively) is connected (coupled with/to (operatively or communicatively))" know "is connected (connected to)" when-mentioned that, said is any component said other components or directly linked to, other components (e.g.: number 3 components) and is coupled through the wall of the rectangular. to be understood. While, on any configuration element (e.g.: number 1 components) is other components (e.g.: number 2 components) to "is directly connected with the" know "are directly connected" when-mentioned that, on any configuration element and said other components said components is captured in another (e.g.: components number 3) there is no can be understood.

Constitution: the invention concerns a method used in representation "- configured to (set as a function selection) (configured to)" according to in a situation with an, for example, "suitable - (suitable for)", "- a (having the capacity to) the ability to", "designed to - (designed to)", "(adapted to) a is modified so the system -", "- designed to (made to)", or "- (capable of) that is capable of making a two" and can be used a desired position by utilizing a. Terms "- configured to (or set of functions) a" the in hardware, thereby "specifically designed (specifically designed to)" necessarily only those may not KIPO & means. Instead, in certain circumstances, "device configured to -" as the representation, which, the same device or parts to be packaged with the other device "the -" can be means. For example, phrase "A, B, and C configured to execute at (or set of functions) a processor" receives an dedicated processor to perform an operation (e.g.: an embedded processor), or memory device stored in the one or more software program according to the execution of a, corresponding may perform operations to as a universal processor (generic-allots an authority to write the processor) (e.g.: application processor or CPU) can be mixture by the addition of an initiator.

Constitution: the invention concerns a method used in the embodiment a particular only terms are used to describe the e.g. the keypad are inputted to a keyboard, other embodiment, or limit the scope of, e.g. can be is not intended be. Contextually it is apparent that a plurality of representation without the carelessly differently, may include a multiple representations. Technical or scientific for a term including all to the disclosure the present terms are upwardly urged by person with skill in the art in the technical field of equivalent to those that would have been understood may have the meanings. Of the commonly used predefined terms are contextually purpose: the same, or similar, semantics and having photopolymerization initiator represented by chemical can be interpreted to, transparently constitution: the invention concerns a method is not defined, is ideal for the widest sense of the formal excessively or does not interpreted. Optionally, constitution: the invention concerns a method defined in terms of disclosure the present even interpreted to the embodiment examples and cannot back number.

The present disclosure of a variety of embodiment examples according to the electronic device, for example, electronic device has (smartphone) smart phone, tablet PC (tablet personal computer), mobile telephone (mobile phone), video phone, electronic book reader (e-book reader), desktop PC (desktop personal computer), laptop PC (laptop personal computer), net-type computer (netbook computer), workstation (workstation), server, PDA (personal digital assistant), PMP (portable multimedia player), MP3 player, mobile medical device, camera (camera), or wearable device (wearable device) (e.g.: smart glasses, head wearable device (head-of a mother member-constitution: (HMD)), electronic garment, electronic bracelet, electronic necklace, electronic [...] (appcessory), electronic tattoo, smart mirror, or smart watch (smart watch)) may comprise an at least one.

Examples any embodiment, electronic device the smart household number article can be (smart home appliance). Smart home the number, for example, television, DVD (digital video disk) player, audio, refrigerator, air conditioner, cleaner, oven, microwave oven, washing machine, air cleaner, set-top box (set-top box), home auto [...] control panel (home automation control panel), security control panel (security control panel), TV box (e.g.: HomeSyncTM ternary, TVTM applicator, or google TVTM), game console (e.g.: XboxTM, PlayStationTM), electronic dictionary, electronic key, camcorder (camcorder), or electronic frame may comprise an at least one.

In other embodiment, the electronic device, various medical instruments (e.g.: various portable medical measuring device (the blood sugar measuring member is provided to, heart measuring instrument, blood pressure measuring instrument, rate or temperature measuring machine or the like and converts the), MRA (magnetic resonance angiography), MRI (magnetic resonance imaging), CT (computed tomography), camera, or ultrasound cell, machine or the like and converts the), navigation (navigation) device, GPS receiver (global positioning system receiver), (event data recorder) EDR, FDR (flight data recorder), vehicle Infotainment (infotainment) device, electronic equipment for ship (e.g.: for ship navigation device, such as gyrocompasses), avionics device (avionics), security device, head unit (head unit), industrial or household robot, ATM of financial institution (automatic teller's machine), the shop's POS (point of sales), or things (internet of things) Internet device (e.g.: bulb, various sensor, electrical or gas meter, sprinkler device, fire alarm, temperature controller (thermostat), street, toaster (toaster), exercise device, hot water tank, heater, a boiler) may comprise an at least one.

According to any embodiment e.g., electronic device/or furniture has part of structure (furniture), electronic copyboards (electronic board), electronic sine receiving device (electronic signature receiving device), projector (projector), or of various metrology instrument (e.g.: may, electric, gas, such as metrology instrument or radio wave) may comprise an at least one. In various embodiment, electronic device has various of a plaque one or more device each can be a combination of.. According to any embodiment e.g. electronic device can be of a flexible electronic device. In addition, the present disclosure according to e.g. embodiment of electronic device has of a plaque which is not limited to the devices, according to new power generation techniques may comprise an device.

Hereinafter, reference to attached drawing, various embodiment described are according to e.g. electronic device. The present in the document, an electronic device user term device or electronic a person using the device using (for example, artificial intelligence electronic device) can be, " refers to.

VR under public affairs number service of the present invention in the embodiment in an electronic device a person using the current time based on the so head tracking information and acceleration information, included in an Image event (event) information, and the like viewing angle's future user site uses a predicting and information, predicted information Image of quality parameters each view based on a method and a device number into a number under public affairs.

Acceleration information method that utilize a the current viewing angle acceleration direction information of the user subsequently visits in a correcting direction to a be with the field of view of an transparent angle which, for example, to predict a, view number in a higher picture quality Image in the direction are moved into mating engagement with the can be under public affairs. In addition of the method for utilizing a the event information, such as object information sound specific picture-in utilizes a method may be proposed. A rear sound for example a user and being lifted from and a have heard sensing part measures a rear, and a, so in this case and rearwardly of the cam groove can be under public affairs number in a higher picture quality Image. In addition from Image appearing in a body new what point the user if new natural feeling appearing in a body or follows a direction of a sight, will be directed search result to the a field-of-view, in a higher picture quality number can be under public affairs. The streaming technique in addition event based prediction scheme VR in addition local (local) while being subjected to the down content selectively rendering (rendering) and counts the. is connected.

In addition according to an embodiment of the present invention VR under public affairs number service an electronic device according to situation bandwidth is currently streaming VR under public affairs (Adaptive VR streaming) service can be number. Adaptive VR streaming acceleration, and more information, location information, based on various bandwidth information, and the like for improving picture quality of an. under public affairs number. For example, user's current viewing angle ball are provided to include the user with a high probability portions of the gate electrode layer in a higher picture quality, providing low probability viewable by the user to be sent to or with that picture quality or neutralizing portion, current user viewing angle can be moved away from an Image for second width within the scope transmits only the field of view of the digital value exceeds a predetermined range displaceable an Image projected angles may be not transferred to the.

Method determining that streaming VR adaptive according to an embodiment of the present invention an electronic device utilizes a buffer level of method, bandwidth-available method :. These move independently or method can be are of value for use in combination.

First, buffer level for use in a method of an electronic device based on state buffer level by a rope. speculate width condition band. I.e., buffer level is for higher the all expected to receiving/transmitting high definition Image Image send buffer level is low when the source signal and control adaptation of to narrow bandwidth chamber is area viewing angle, for which are expected to low Image quality Image can be transmission Image. For example, buffer level is under in at assuming a back-is divided into three phases, a current buffer different levels and also all visual angles 360 if Image both transmitting and in a higher picture quality, during buffer level is current if the current viewing angle including a viewing-angle expected to be (hereinafter, region of interest) transmitting and in a higher picture quality for an other region than the region of interest (hereinafter, non region of interest) neutralize Image information communication network transmitting and, current buffer level is the region of interest ramyon transmission in a higher picture quality for an for an region of interest and non can be transmission with that picture quality.

Method measuring the bandwidth in addition the chunk (Chunk) measuring bandwidth average unit said measurements all visual angles (Chunk size/Chunk download time) is required to high definition Image of number under public affairs bandwidth if it is less than non interest with that picture quality or neutralize Image information can be transmission.

Then, adaptive VR under public affairs number streaming service according to an embodiment of the present invention relates to a method.

According to an embodiment of the present invention adaptive VR under public affairs a method number streaming service plurality of one Image whether the encoding symbol greatly according to. can be decomposed into two.

One server symbol rate plurality on the laundry dry hanger encoded with server not in ratio by or sign view by Image also 360 or managing electronic device constituting Image data based on request for packet data including information on can be. The server through a 360 - degree camera quality multiple the motion is sensed by the sensing unit chunk by encoding the divided difference 1, again each angle each chunks so that the (Angle) (View) separated in units of a view or a angle block (Angle block) in stores the repository the glass fiber bundles, splitting the difference 2.

Also according to an embodiment of the present invention Figure 1 shows a. timing an example of data partitioning. In Figure 1 the 9 of non-repetitive patterns (cam 1 to cam 9) each with the monitoring camera using chunks so that the block angle of 9 to partitioning of a number of camera but modifiable recorded at the memory angle [...]. In addition in Figure 1 has to a two-step quality, i.e. high definition and to a encoding with that picture quality to three or more but also quality of molding material having a thermal expansion rate KIPO &.

The manner in which the divided into blocks angle chunks so that the uniform angle divided into divided into angle and irregular a manner manner in which the use can be made of, for both.

Uniform (Symmetric) the manner in which the divided into angle, for example 45 set a PCB 8 360 degree of angle block 12 set a PCB 30 or into angle of a means to two. In addition a non-uniform (Asymmetric) the manner in which the divided into angle, for example a user is viewing a relatively small portion providing low probability to a size which is angle form a block be seen user with a high probability portions of the gate electrode layer to form blocks angle to relatively large dimensions or, with a relatively high a major portion movement to form blocks angle to a size a minor portion of movement, to a size which is a relatively small can be form a block angle. In addition angle size of the blocks of probability in proportion to is visible to as differently setting the size of the blocks of all angles may be. In addition sport video, movie, natural Image such as size of the blocks of angle according to of regions and outputs region information of emission display device includes a resolution level or may be loaded with. For example, such as a pressure gage in the case of Image operates seamlessly screen to be viewed

On the other hand, electronic device network state current his/her capability according to server to able to request a service KIPO & streaming VR. To this end electronic device has every chunk each chunk index and the corresponding chunk quality information to deliver an a server, chunks so that the quality the corresponding server, and transmits the electronic device. In addition according to an embodiment of the present invention angle in a block unit and device for requesting chunks so that the electrons to the server an Image in the when the user requests it chunk index and angle with quality information is received and a digital-information block..

In addition of the present invention in the embodiment according to view different lengths in their quality Image when ball number, electronic device has the segmented Image signal with view by from the server receives data performs a coupled (aggregation) must decide. View is divided or server a modulation signal by a modulation classified images 360 picture followed for enabling a user to transmit KIPO & electronic device. Or server and an electronic device between establishing a connection process for negotiating capability information Image processing of terminal in (negotiation) negotiated and coupled principal Image based on the result of. may be determined. E.g. an e-device Image processing ability is low output powers server is made to an electronic device to a method number can be e. under public affairs.

While, smart CODEC method such as compression of the ROI by differently encoding when the, electronic device and buffer level is, head track information be based on future by the user of the lower side from the request information to the server use in an area of interest, an electronic server based on information received from the device user of regions of interest an Image of the zoomed-in an appropriate compression ratio based on changes lowered so as transmitting and in a higher picture quality, non with that picture quality increase an appropriate compression ratio based on changes region of interest by sending a data a vibrator is connected with the first KIPO &. Pre-determined region of interest and a determining unit for determining or block user or assigning next, electronic device or according to buffer level of, electronic device which is coupled to a camera focus in certain objects through a for automatically setting a region of interest or a specific region may be loaded with setting.

Also Figure 2 shows a number streaming service according to an embodiment of the present invention adaptive VR an electronic under public affairs. timing configuration of device.

2 also refers to surface, according to an embodiment of the present invention electronic device (200) the head tracking information extraction section (210) and a, adaptive VR streaming judging section (220) and a, adaptive VR streaming part (230) and a, Image request part (240) may comprise an.

Head tracking information extraction section (210) comprises an electronic device mounted gyro sensor of sensing movement of a acceleration sensor or a sensor, camera based indicative of the information received from the user may accelerated to are determined from the extracted area information.

Adaptive VR streaming judging section (220) band width condition according to a predetermined value and decides the streaming VR. Specifically, electronic device is measured at 360 bandwidth high definition both picture number Image under public affairs bandwidth request definition for all if the reacquisition time is greater than receiver in a higher picture quality Image adaptive VR streaming judges whether a without performing, said bandwidth measured when the less than the band request definition said adaptive VR streaming of the PLD to washing to simplify the pump, said bandwidth measured low Image quality both picture number Image 360 under public affairs smaller than the bandwidth request low Image quality for all Image when an annealing process is adaptive VR with that picture quality receive without performing streaming. judges that. Each judgment result is and adaptive VR streaming part (230) and a video request part (240) transmits it.

Adaptive VR streaming part (230) the head tracking information extraction section (210) extracted from acceleration information and position information, adaptive VR streaming judging section (220) adaptive streaming VR to based on the determination of the determining to perform said acceleration information and location information server based on and decides the information chunk request.

Image request part (240) has adaptive VR streaming judging section (220) and adaptively, streaming VR part (230) of VR to servers on the basis of that decision a request streaming. The Image request part (240) the server from an received manipulator pest (manifest) based on file and angle quality information/n ratio, capable of supporting a obtains index information block, obtained information based request message to request the traffic encryption key. transmission to the server. Said request message as the above - mentioned a chunk index, quality information, may comprise an index block angle.

Then, differential quality number according to an embodiment of the present invention relates to ball method.

First, and to influence the user reference a time point at which the present ball the rear high definition Image user with a high probability the peripheral portion, and the remaining part can request the device KIPO & low Image quality Image. The position information and acceleration of information the pixel elements are using the two approaches may be proposed.

Position information using user when the packaged view viewing at that adjacent to can be request in a higher picture quality view.

In addition position information and acceleration information are bound every utilize acceleration view in the direction is receiving user with a high probability is moved to a specific sight the judges that the, high definition Image a movement direction of acceleration current view can request a KIPO &.

Figure 3 shows a degree position, of the rotary drum Image of quality differentiated converted into the information requesting Figure 4 and this shown e.g. position information and acceleration information by considering simultaneously the MSE Image of quality differential. timing e.g. requesting.

Also 3 with a, when a current to both view and burn 5 including burn 5 5 4 6-th and the adjacent time burn, and a ceiling 9 requesting and in a higher picture quality burn, remaining view (burn 1 - 3, 7 - 8 burn) can be request with that picture quality Image of.

Refers to surface 4 also in addition, current 5 left and is burn information in the presence of an acceleration direction, including burn 5 5 4 6-th and the adjacent time burn, the acceleration direction adjacent to burn 5 3 burn, and a ceiling 9 requesting and in a higher picture quality burn, remaining view (1 - 2 burn, burn 7 - 8) can be request with that picture quality Image of.

Said high definition and 3 and also in Figure 4 has the subjective video quality but array block, and by dividing 6 110,130 two-step, one or more step 3 transmitted array block, and by dividing 6 may be loaded with. For example in Figure 3 the 4 burn, burn 5, 6 burn, requesting and in a higher picture quality burn 9, remaining view are currently 3-th and the adjacent relatively view, neutralizing once communication network 7, 8-th and the up as a reference point and the relatively, far with that picture quality 1, 2 can be request once. In addition in Figure 4 the 3 - 6 9-th and the once in a higher picture quality, remaining view in neutralizing once acceleration member accommodates the communication network 1 - 2, the opposite direction acceleration once can be request with that picture quality 7 - 8.

Prior in addition as described above included in an Image event (event) site uses information, and the like user's future viewing angle information even when the mother pipe having a predicted information based on 2 or more differential according each view Image of quality can be under public affairs number.

Then, and to influence the user the rear reference a time point at which the present ball user with a high probability the peripheral located between only, while the remaining portions selectively request does not request the anti-reflecting layer is formed can be by bandwidth savings. Method request Image selected home automation view reticle also quality Image request position information and similarly with an acceleration of information the pixel elements are using the two approaches may be proposed.

Position information using user when the packaged view viewing at that adjacent to, while the remaining portions requesting Image when view may not KIPO & requested.

In addition position information and acceleration information are bound every utilize acceleration view in the direction is receiving user with a high probability is moved to a specific sight the judges that the, current view in the area of Image when acceleration direction, while the remaining portions requesting may not KIPO & requested.

Figure 5 shows a degree position, of the rotary drum Image selected converted into the information requesting the e.g. Figure 6 and shown in the acceleration position information and Image selected by considering the information requesting the e.g. timing..

Also 5 refers to surface, when a current to both view and burn 5 including burn 5 5 4 6-th and the adjacent time burn, a ceiling and code and burn 9, remaining view (burn 1 - 3, 7 - 8 burn) does not request the scene of the.

With a 6 also in addition, current 5 left and is burn information in the presence of an acceleration direction, including burn 5 5 4 6-th and the adjacent time burn, the acceleration direction adjacent to burn 5 3 burn, and a ceiling 9 code and burn, remaining view (1 - 2 burn, burn 7 - 8) does not request the scene of the.

Prior in addition as described above included in an Image event (event) information, and the like viewing angle's future user site uses even when information based on information predicted Image according under public affairs each view can be selectively number.

According to adaptive VR streaming service of the present invention embodiment to also Figure 7 shows a ball procedure shown in the order number.

With a 7 also, electronic device the server to manipulator pest (Manifest) and which receives it and devices to request file (701,702) encoding and chunk information acquires (703). The electron device for his/her hyperlink decoding capability including capability information can be it sends to the server. The current electronic device the measured bandwidth adaptively based bandwidth resin to streaming VR. (704). I.e., electronic device is measured at 360 bandwidth high definition both picture number Image under public affairs bandwidth request definition for all if the reacquisition time is greater than receiver in a higher picture quality Image adaptive VR streaming judges whether a without performing, said bandwidth measured when the less than the band request definition said adaptive VR streaming of the PLD to washing to simplify the pump, said bandwidth measured low Image quality both picture number Image 360 under public affairs smaller than the bandwidth request low Image quality for all Image when an annealing process is adaptive VR with that picture quality receive without performing streaming. judges that. Adaptive VR performs to perform a streaming, motion sensors mounted electronic device, to from camera received head tracking information server based on acceleration information and position information and decides the information chunk request (705). And said VR to servers on the basis of that decision a request streaming (706). The electron the server device received manipulator pest (manifest) from an based on file and angle quality information/n ratio, capable of supporting a obtains index information block, obtained information based request message to request the traffic encryption key. transmission to the server (708). Said index a chunk request message, quality information, may comprise an index block angle. According to request from the server has the same, and electronic device: this recorder receives chunks so that the (708). The, server 701 received at a terminal based on capability information terminal in a range that enables of decoding of the etching the gate can be of.

Constitution: the invention concerns a method "- part" terms used in the, for example, hardware, software or firmware (firmware) including for or combination of two or more one unit (unit) can be mixture by the addition of an initiator. ". ". "". "- Part" the, for example, unit (unit), logic (logic), logic block (logical block), component (component), or circuit (circuit) gave her a set of terms, such as (interchangeably use) can be used. ". ". "". The "- part", the minimum unit of components integrally comprises may be part or.. ". ". "". "- Part" has mechanical or electronically may be embodied in. For example, "- part" the, may be developed forward the alphas1 -, alphas2 known, what actions if any for performing ASIC (application-specific integrated circuit) chip, FPGAs (field-programmable gate arrays) or a programmable logic device (programmable-logic device) may comprise an at least one.

Various embodiment e.g. according to device (e.g.: functions, or module) or method (e.g.: operations) at least a portion of, e.g., in the form of program modules computer-readable storage medium (storage media computer the plaintext value and a second value) stored in the may be embodied in instructions for automatically routing an integrated circuit. A discard logic said, processor (e.g.: processor (120)) when executed by, said one or more processors is corresponding to said instruction can be in the stage of carrying out the. A storage medium computer-readable, for example, said memory (130) can be is.

Said a recording medium computer-readable, hard disk, floppy, magnetic medium (magnetic media) (e.g.: magnetic tape), optical recording medium (optical media) (e.g.: CD-ROM(compact disc read only memory), DVD (digital versatile disc), magnetic - optical media (media deposited topography) (e.g.: flop mote curl disk (floptical disk)), hardware device (e.g.: ROM (read only memory), RAM (random access memory), or the flash such as memory) such as may include a. In addition, the compiler program instructions created by interpreter as well as code 2000 such as the computer use the purpose: a high-grade and executed by may comprise an language code. The above-mentioned hardware device different embodiment example to perform the join operation software modules in one or more can be configured to operate as, . similarly and vice versa.

According to module or program modules various embodiment e.g. the aforementioned components include or at least more than one of said, part achieved due to the omission or, other or additionally can be further include components to. Various embodiment e.g. according to module, program modules or with other components by the operations performed by the devices by a sequential order, parallel, heuristics or repeatedly (heuristic) .can be carried out as an a method. In addition, some other operation or by a sequence suitable, dropped or, or other can be additional operation.

According to various embodiment e.g., a storage storing instructions medium, said instructions are at least one processor when executed by the at least one said to output relating one operation by processors in a and if a bit in the complimentary performed with the same, said at least one operation, electronic device process for performing mode virtual reality in; at least one of the event generating according to interrupt sensor; said event which has occurred related information of events associated with said virtual reality mode modifying the score data can shaped corresponding to the; and said modified event related information may be carried out responsive to the virtual reality mode on a display screen; may include a.

The drawing and specification examples according to e.g. a invention to an alarm for warning techniques of the present invention embodiment of the present invention embodiment of the present invention embodiment and described to assist the comprehension of e.g. particular instance and the user makes a it is timing number, of the present invention embodiment e.g., or limit the scope of, . without intending to be. Thus various of the present invention invention herein range example embodiment a embodiment of the present invention addition examples various embodiment e.g. which is derived based on the idea of an altered form modification or all various embodiment of the present invention is included within the scope of e.g. should interpreted to.