CN103197757A - Immersion type virtual reality system and implementation method thereof - Google Patents
Immersion type virtual reality system and implementation method thereof Download PDFInfo
- Publication number
- CN103197757A CN103197757A CN2012100063389A CN201210006338A CN103197757A CN 103197757 A CN103197757 A CN 103197757A CN 2012100063389 A CN2012100063389 A CN 2012100063389A CN 201210006338 A CN201210006338 A CN 201210006338A CN 103197757 A CN103197757 A CN 103197757A
- Authority
- CN
- China
- Prior art keywords
- data
- data processing
- processing equipment
- display
- user
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Processing Or Creating Images (AREA)
Abstract
The invention provides an immersion type virtual reality system. The system comprises at least one display, at least one motion capture device and at least one data processing device, wherein the display can be worn on the body of a user, and is used for capturing the head turning motion of the user, sending the head turning motion data to the as least one data processing device, receiving data sent by at least one data processing device, and transmitting a video or audio of a virtual environment to the user; the motion capture device captures motion trails of main node parts of the user, and sends the motion trail data to the at least one data processing device; and the data processing device is used for processing data acquired from the at least one display and the at least one motion capture device and transmitted to the data processing device, and sending the processed result to the at least one display. The invention further provides an implementation method of the immersion type virtual reality system.
Description
Technical field
The present invention relates to a kind of virtual reality system and method, in more detail, the present invention relates to a kind of immersive VR system and method.
Background technology
Virtual reality (Virtual Reality, VR) technology is a kind of advanced person, digitized human-machine interface technology, its characteristics are that computing machine produces a kind of artificial virtual environment, generate one based on visual experience, the artificial environment that comprises the comprehensive perception of the sense of hearing, sense of touch, people can pass through the virtual world of polyesthesia passage perception computer simulations such as vision, the sense of hearing, sense of touch and acceleration sense, it is mutual also can to pass through the most natural mode and virtual world such as movement, voice, expression, gesture and sight line, thereby produces experience on the spot in person.Virtual reality technology is the comprehensive development of multiple technologies such as computer technology, sensor technology, human-computer interaction technology, artificial intelligence technology, be applied in various aspects such as military affairs, medical science, education, amusement, manufacturing industry, engineering training at present, it is considered to current and influences one of important technology of people's life in the future.
Virtual reality system is divided into immersion system and non-immersion system according to the degree difference of the property immersed.The immersive VR system realizes that by some special external units, high-performance computer and corresponding software it is immersed in the graphics world of computing machine creation, just as experiencing real world the people fully.The immersive VR system is a kind of senior virtual reality system, and it provides an experience of immersing fully, allows the user that a kind of sensation of placing oneself in the midst of among the virtual boundary is arranged.It utilizes helmet-mounted display or miscellaneous equipment, participant's vision, the sense of hearing and other are felt to be closed, and new, a virtual sense space is provided, and it is a kind of on the spot in person, with single-hearted devotion and immerse wherein sensation to utilize position tracker, data glove, other manual input equipment, sound etc. to make that the participant produces.
The immersive VR system comprises cave (CAVE) immersive VR system, giant-screen 3 D stereo immersive VR system, cylinder ring curtain projection immersive VR system etc., these immersive VR systems are display mode with the virtual real stereographic projection of large format or even super large breadth, for the participant provides the feeding type Virtual Space environment with height telepresenc, allow the mutual high fidelity ground, the virtual three-dimensional world of wanting appear in one's mind in the participant at the moment.Yet these immersion virtual systems need fixed-site that large-scale display device is installed could to use, limit its usable range.
But the appearance of wearable display has solved the huge problem of display device, and this display mode volume is little, easy to carry and transport, is most possibly popularized widely.Yet the tracking of human body and time delay of data transmission are the technical barriers that the immersive VR system of employing Wearable display faces.
For example the space tracking system (STS) of CN97198214.7 (Spatial tracking system) provides the solution that human body is followed the tracks of, the MVN inertia motion capture product produced of 3DSuit motion capture product and Xsens Technologies for example, adopt means such as inertial sensor, the motion of the main skeletal sites of human body is measured in real time.This kind equipment advantage be the user to be immersed in the virtual world whole-heartedly go, shortcoming is the system equipment complexity, and is expensive, is difficult to penetration and promotion.
Summary of the invention
Because the extensive and deep research of the inventor, the inventor has proposed according to technical scheme of the present invention, makes the problems referred to above to be solved.
The invention provides a kind of immersive VR system, described system comprises at least:
At least one can wear the display on user's health, at least one motion capture device, and at least one data processing equipment;
Wherein,
Described display is used for showing image and sound to the user, and the head rotation that is used for the seizure user is moved, and the head rotation action data is sent at least one described data processing equipment, and receive the data that at least one described data processing equipment sends, and transmit the video and/or audio of virtual environment to the user;
Described motion capture device catches the movement locus at user's main node position, and motion trace data is sent at least one described data processing equipment;
Described data processing equipment is for the treatment of the data of obtaining and transfer to described data processing equipment from least one described display and at least one described motion capture device, and the result after will handling is sent at least one display.
According to immersive VR of the present invention system, preferred described display comprises at least one angle induction device.Described angle induction device can be caught at least around the spinning movement of z axle, and preferred described angle induction device is digital compass.Also preferred described angle induction device can be responded to around the spinning movement of x, y and three axles of z, and more preferably described angle induction device is three-axis gyroscope.
According to immersive VR of the present invention system, preferred described action acquisition equipment comprises at least one infrared transmitter and at least one infrared pick-up head.More preferably described action acquisition equipment is body sense video camera, and most preferably described action acquisition equipment is Kinect body sense video camera and/or WAVI Xtion body sense video camera.
According to immersive VR of the present invention system, preferred described data processing equipment is computer, for example PC, notebook computer; Personal digital assistant (Personal Digital Assistant, PDA), the Dell Axim X51V palm PC produced of Dell for example; Panel computer, for example TouchPad of the iPad of Apple's production and iTouch, Hewlett-Packard's production; Mobile phone, the iPhone that produces of Apple for example, the Nokia N85 that Nokia produces; Audio frequency processing terminal, for example MP3 and iPod; Video processing terminal, for example MP4; And/or game machine, for example PS2, PS3, PSP, NGC, NDSL, XBOX, Wii, GBA, 3DS and NDS.
According to immersive VR of the present invention system, at least one display (COM1), at least one action acquisition equipment (COM2) and VRS of at least one data processing equipment (COM3) composition.Also can be constituted by a plurality of VRS according to virtual reality system of the present invention.
According to immersive VR of the present invention system, it also comprises at least one DEU data exchange unit, for the treatment of the data transmission between a plurality of VRS.Preferred described DEU data exchange unit is computer, personal digital assistant, panel computer, mobile phone, audio frequency processing terminal, Video processing terminal and/or game machine.
According to immersive VR of the present invention system, transmission speed between described display and described motion capture device, described display and described data processing equipment and/or described motion capture device and the described data processing equipment preferably is not less than 18432000bps (bit per second, bits per second), more preferably be not less than 55296000bps, more preferably be not less than 55296000bps, also more preferably be not less than 147456000bps.The preference data transmitting device adopts the WHDI transmission mode.
The present invention also provides a kind of method that realizes immersive VR, and described method comprises at least with the lower part:
Catch the movement locus at user's main node position by at least one action acquisition equipment, and motion trace data be sent at least one data processing equipment,
Catch user's head rotation by at least one display and move, and the head rotation action data is sent at least one described data processing equipment,
By the data of at least one described data processing equipment processing from described action acquisition equipment and described display, and the data after will handling are sent at least one described display;
Data after the processing that will receive from least one described data processing equipment by at least one described display are converted to video and/or audio, the reality environment that the user can be experienced follow oneself action to change.
According to the method for realization immersive VR of the present invention, preferred described display comprises at least one angle induction device.Described angle induction device can be caught at least around the spinning movement of z axle, and preferred described angle induction device is digital compass.Also preferred described angle induction device can be responded to around the spinning movement of x, y and three axles of z, and more preferably described angle induction device is three-axis gyroscope.
According to the method for realization immersive VR of the present invention, preferred described action acquisition equipment comprises at least one infrared transmitter and at least one infrared pick-up head.More preferably described action acquisition equipment is body sense video camera, and most preferably described action acquisition equipment is Kinect camera and/or WAVI Xtion camera.
According to the method for realization immersive VR of the present invention, preferred described data processing equipment is computer, for example PC, notebook computer; Personal digital assistant (Personal Digital Assistant, PDA), the Dell Axim X51V palm PC produced of Dell for example; Panel computer, for example TouchPad of the iPad of Apple's production and iTouch, Hewlett-Packard's production; Mobile phone, the iPhone that produces of Apple for example, the Nokia N85 that Nokia produces; Audio frequency processing terminal, for example MP3 and iPod; Video processing terminal, for example MP4; And/or game machine, for example PS2, PS3, PSP, NGC, NDSL, XBOX, Wii, GBA, 3DS and NDS.
According to the method for realization immersive VR of the present invention, at least one display (COM1), at least one action acquisition equipment (COM2) and at least one data processing equipment (COM3) are formed a VRS.Also can be constituted by a plurality of VRS according to virtual reality system of the present invention.
According to the method for realization immersive VR of the present invention, it also comprises by at least one DEU data exchange unit handles data transmission between a plurality of VRS.Preferred described DEU data exchange unit is computer, personal digital assistant, panel computer, mobile phone, audio frequency processing terminal, Video processing terminal and/or game machine.
Method according to realization immersive VR of the present invention, transmission speed between described display and described motion capture device, described display and described data processing equipment and/or described motion capture device and the described data processing equipment preferably is not less than 18432000bps (bit per second, bits per second), more preferably be not less than 55296000bps, more preferably be not less than 36864000bps, also more preferably be not less than 147456000bps.The preference data transmitting device adopts the WHDI transmission mode.。
Description of drawings
Accompanying drawing 1 is the structural drawing according to display of the present invention;
Accompanying drawing 2a-2d is the synoptic diagram according to x, y and z axes of the present invention;
Accompanying drawing 3 is the fundamental diagrams according to display of the present invention;
Accompanying drawing 4 is the structural drawing according to action acquisition equipment of the present invention;
Accompanying drawing 5 is human body main node position synoptic diagram that action acquisition equipment according to the present invention can be caught;
Accompanying drawing 6 is the structural drawing according to data processing equipment of the present invention;
Accompanying drawing 7 is the fundamental diagrams according to data processing equipment of the present invention;
Accompanying drawing 8 is the fundamental diagrams according to DEU data exchange unit of the present invention;
Accompanying drawing 9 is the topology diagrams according to the immersive VR system of embodiments of the invention 1;
Accompanying drawing 10 is the topology diagrams according to the immersive VR system of embodiments of the invention 2;
Accompanying drawing 11 is the topology diagrams according to the immersive VR system of embodiments of the invention 3;
Accompanying drawing 12 is the topology diagrams according to the immersive VR system of embodiments of the invention 4;
Accompanying drawing 13 is the topology diagrams according to the immersive VR system of embodiments of the invention 5;
Accompanying drawing 14 is the topology diagrams according to the immersive VR system of embodiments of the invention 6.
Embodiment
Below, will set forth the present invention by reference to the accompanying drawings in more detail.The example that occurs in the instructions only is used for explaining, setting forth and explanation the present invention, has the implication that includes but not limited to, and does not mean that having any type of restriction in of the present invention.
Display (COM1) is used for transmitting to the user video and/or audio of virtual environment, makes the user produce impression on the spot in person; And comprise at least one angle induction device, described angle induction device is used for catching the head rotation action, and described head rotation action comprises the spinning movement of head and deflection action by a small margin.The voice that display (COM1) can also be used between realization user and other user are linked up.
With reference to accompanying drawing 1, display (COM1) has display device (101), microphone (102), earphone (103), angle induction device (104), signal processing apparatus (106), and data transmission device (105).
Display device (101) is for the part that shows image, experiences more vivid visual effect for the ease of the user, and near people's eye, this display device can be any display devices such as CRT, LCD usually in the position of display device (101).Microphone (102) is be used to catching sound, and voice signal is converted to the device of simulating signal, generally is placed near people's the oral area.Earphone (103) is to be voice signal with analog signal conversion, and with the device that sound broadcasts, generally is placed near people's the ear.
Angle induction device (104) can be responded to the spinning movement around the z axle, and better, angle induction device (104) can be responded to around the spinning movement of x, y and three axles of z.As shown in Figure 2, described spinning movement around the z axle refers to that user's head is the spinning movement of axle with the vertical direction, namely, (yaw) (α) action of angle variation rotates, for example user motion or basic stationary state head portion left-right rotation action comprises also that for example the user uprightly stands or the almost head left-right rotation action under the very little situation of non-displacement or displacement of sitting state head portion and health.Described around x, the spinning movement of y and three axles of z refers to except around the spinning movement of z axle, also comprise the axially-movable action vertically of user's head, namely, (yaw) angle (α) rotates, deflection (pitch) angle (β) and pitching (roll) angle (γ), for example any head rotation action of user, comprise the head rotation action, nodding action, backward wing, yaw actions etc. certainly comprise also that for example the user uprightly stands or the almost head rotation action under the very little situation of non-displacement or displacement of sitting state head portion and health, nodding action, backward wing, the yaw action.In the previous case, angle induction device (104) can be digital compass; Under latter event, angle induction device (104) can be three-axis gyroscope.Angle induction device (104) makes the spinning movement of head and deflection action by a small margin to be identified by immersive VR according to the present invention system, remedy action acquisition equipment (COM2) and be difficult to identify the spinning movement of head and the defective of deflection action by a small margin, can auxiliary data treating apparatus (COM3) play up corresponding video and graphic, make and to bring simulating reality effect more true to nature for the user according to immersive VR of the present invention system.
Signal processing apparatus (106) is the central control section of display (COM1), the data that receive and/or capture for the treatment of display (COM1).For example, as shown in Figure 3, signal processing apparatus (106) has image processing apparatus (1061), sound processing apparatus (1062), control device (1063) and memory storage (1064).Wherein, image processing apparatus (1061) converts the literal that receives, the digital signal of figure to simulating signal, and analog signal transmission to display device (101) is shown; For example, image processing apparatus (1061) can be video card.The analog signal conversion of the sound that sound processing apparatus (1062) will receive from microphone (102) becomes digital signal, and is transferred to data processing equipment (COM3); Also will convert simulating signal to from the audio digital signal of data processing equipment (COM3), and send to earphone (103) broadcast; For example sound processing apparatus (1062) can be sound card.Control device (1063) is used for all programs of control display (COM1) to be carried out, and coordinates the external unit operation; Control device (1063) can be central processing unit (CPU).Memory storage (1064) is used for program and the data of iatron (COM1); Memory storage (1064) can be internal memory and/or hard disk.Data transmission device (105) is for data transmission between display (COM1) and other devices (for example data processing equipment (COM3)), can adopt modes such as WHDI, WiFi, Ethernet Net, RF, USB.
Action acquisition equipment (COM2) is used for catching the movement locus at user's main node position, and it comprises at least one infrared transmitter (201) and at least one infrared pick-up head (202) at least.As shown in Figure 4, action acquisition equipment (COM2) can also comprise controller (203), output interface (205), and chase after burnt motor (204).During use, action acquisition equipment (COM2) makes user's scope of activities fall into the measurement space of action acquisition equipment (COM2) towards the user.Action acquisition equipment (COM2) can also comprise that colour imagery shot is used for color image shot.
Infrared transmitter (201) is used for evenly throwing infrared light at the measurement space at action acquisition equipment (COM2), and described measurement space is the scope that infrared transmitter (201) can cover.Infrared pick-up head (202) (claiming monochromatic cmos sensor or infrared cmos sensor again) is used for the speckle data of record measurement space, and gives controller (203) with the gained data transmission.Controller (202) is for the treatment of the data from infrared pick-up head (202), finish the action at human body main node position is caught, the data after the processing pass to other continuous devices (for example data processing equipment (COM3)) by output interface (205).Chase after the angle up and down that burnt motor (204) is used for adjusting picture pick-up device, can better must cover human object to guarantee measurement space.
When the even infrared light that infrared transmitter (201) is launched continuously to the space shines on the rough object (as human body surface or clothing surface), form reflected spot at random, i.e. speckle.Speckle has height random, also can along with the distance and changing patterns, the speckle at any two places all can be different pattern in the space, is equivalent to whole space has been added mark, so when any object enters this space and movement, can definitely be recorded to the position of this object.The speckle that infrared pick-up head (202) occurs in can record space, thus speckle information obtained, and be transferred to controller (203).
Controller (203) forms according to the data from infrared pick-up head (202) has the view data of the 3D degree of depth, and carries out processing such as for example organization of human body identification, the foundation of skeleton model.Machine learning techniques (machine learning) is adopted in organization of human body identification, sets up a large amount of image document storehouses.Each pixel of 3D degree of depth human body image is transferred in the image document storehouse, belong to the possibility of which body part respectively to distinguish pixel, thereby determine human body.In order to alleviate calculated amount, can take to cut apart the shade technology, be about to background object and reject, only transmit the part of cutting apart shade in treatment scheme, basic face data that simultaneously can be by the combination human body and skeleton data are identified in the measurement space everyone identity.Skeleton model is handled, and is to determine the turning joint point by the pixel of assessment output, then according to turning joint dot generation skeletal system, and assesses the actual present position of human body exactly.A plurality of common human postures that prestore in controller (203) simultaneously are not in order to can guess and match player's action comprehensively the time in the image data acquiring information that receives.
As shown in Figure 5, the human trajectory that action acquisition equipment (COM2) is caught, in general the movement locus that comprises human body main node position, the movement locus at human body main node position refers to that generally human body main node position is at the deformation trace in space, described main node position comprises head, hand, ancon, shoulder, waist, knee and/or foot at least, can also comprise hip, wrist and/or ankle.Action acquisition equipment (COM2) can be body sense video camera, and it can obtain by commercial system, for example the Kinect body sense video camera of Microsoft, perhaps the WAVI Xtion body sense video camera of HuaShuo Co., Ltd.
Yet though action acquisition equipment (COM2) can be caught the movement locus at user's main node position, it cannot or hardly identify the head rotation action.The action of described head rotation comprises the spinning movement of head and/or deflection action by a small margin; What the spinning movement of head referred to that head produces displacement hardly or produce very little displacement with respect to body is the spinning movement of axle with the cervical vertebra, the action of for example turning one's head comprises also certainly that the user uprightly stands or the almost action of turning one's head, the action of shaking the head under the very little situation of non-displacement or displacement of sitting state head portion and health; The deflection action by a small margin of head refers to that head produces the wobbling action of very little displacement with respect to body, the for example less yaw of amplitude action comprises also certainly that the user uprightly stands or almost nodding action, head backward wing or the yaw action under the very little situation of non-displacement or displacement of sitting state head portion and health.Yet these head rotation actions can make user's visual angle change a lot, thereby virtual reality system must identify these actions, and make the virtual environment of system simulation react to some extent above-mentioned action and change.Through the extensive and deep research of the inventor, according to the present invention, going up setting angle induction installation (104) at display (COM1) can address this problem, angle induction device (104) can be responded at least around the spinning movement of z axle, when user's head is made spinning movement, when for example turning one's head or shaking the head, angle induction device (104) can be sensed user's action and send related data, data processing equipment (COM3) receives related data and handles, data after handling are sent it back display (COM1), make the user can experience (for example see and/or hear) from display (COM1) and carry out along with the action of turning one's head and the virtual environment that produces changes, make the user that better impression on the spot in person be arranged.Better, angle induction device (104) can be responded to around the spinning movement of x, y and three axles of z, namely can respond to any head rotation action of user.When user's head is made spinning movement and/or deflection action by a small margin, even for example turn one's head and/or when the yaw that amplitude is less, nodding action, angle induction device (104) also can be sensed user's action and send related data, data processing equipment (COM3) receives related data and handles, data after handling are sent it back display (COM1), make the user can experience (for example see and/or hear) from display (COM1) and carry out along with action and the virtual environment that produces changes, make the user that better impression on the spot in person be arranged.
Data processing equipment (COM3) is for the treatment of from display (COM1) and the data obtained of action acquisition equipment (COM2), and the result after will handling is sent to display (COM1).As shown in Figure 6, data processing equipment (COM3) generally includes controller (301), arithmetical unit (302), storer (303), input media (304), and output unit (305).
Controller (301) is used for control program to be carried out, and it is according to the instruction sequence in the storer (program) work, and the control execution command, according to each parts coordinated manipulation of instruction control machine.Arithmetical unit (302) is used for carrying out data processing and handles for example various arithmetic, logical operation etc.Storer (303) is used for memory program and data, for example deposits program and data with the binary code form.The data that input media (304) is used for external device is sent are imported arithmetical unit (302).The data that output unit (305) is used for arithmetical unit (302) is handled export external device to.Described external device is display (COM1), action acquisition equipment (COM2), DEU data exchange unit (COM4), mouse, keyboard and/or touching display screen (for example capacitance plate or resistance screen) for example.
Input equipment (304) to arithmetical unit (302), is stored in storer (303) to the program of needs and data transmission by arithmetical unit (302) again.In the calculation process process, data are read in arithmetical unit (302) from storer (303) and are carried out computing, and operation result deposits storer (303) in, or is exported through output device (305) by arithmetical unit (302).Instruction also is stored in the storer (303) with data mode, instruction is sent into controller (301) by storer (303) during computing, produce control stream with the flow direction of control data stream and the work of controlling each parts by controller (301), data stream is processed processing.
Data processing equipment (COM3) can be for example computer, for example PC, notebook computer; Personal digital assistant (Personal Digital Assistant, PDA), the Dell AximX51V palm PC produced of Dell for example; Panel computer, for example TouchPad of the iPad of Apple's production and iTouch, Hewlett-Packard's production; Mobile phone, the iPhone that produces of Apple for example, the Nokia N85 audio frequency processing terminal that Nokia produces, for example MP3 and iPod; Video processing terminal, for example MP4; And/or game machine, for example PS2, PS3, PSP, NGC, NDSL, XBOX, Wii, GBA, 3DS and NDS.
The principle of work of data processing equipment (COM3) as shown in Figure 7.Data processing equipment (COM3) is responsible for that the data that receive are carried out data and is handled (701).The data that receive are that local user imports data (702) and/or long-range user imports data (703).Local user imports data (702) and refers to the data obtained from local user, for example comprises head rotation action data that microphone (102) audio data captured by local display (COM1), the angle induction device (104) by local display (COM1) obtain, by moving local user's human trajectory data that acquisition equipment (COM2) obtains etc. in this locality.Long-range user imports data (703) and refers to obtain and be sent to local data from long-range user, long-range user's human trajectory data of for example comprise head rotation action data that microphone (102 ') audio data captured by remote display (COM1 '), the angle induction device (104 ') by remote display (COM1 ') obtain, obtaining by long-range action acquisition equipment (COM2 ') etc.
Data are handled (701) and are generally included data acquisition module (7011), input data preparation module (7012), data processing module (7014), 3D rendering module (7016), audio frequency rendering module (7015) and output data preparation module (7017), and comprise local data output module (7018) and/or teledata output module (7013).
Data acquisition module (7011) obtains that local user imports data (702) and/or long-range user imports data (703), and with data transmission to importing data preparation module (7012).Input data preparation module (7012) is put the data that receive into different categories and is put the data layout that can use for data processing module in order, for example view data BITMAP, voice data MP3, video data MPEG4 etc., and reduced data is transferred to data processing module (7014) and/or teledata output module (7013).There is being under long-range user's the situation teledata output module (7013) that all local users are imported data, sending to the data processing equipment (COM3 ') of the immersive VR system at each long-range user place.Long-range user export data (705) be the local virtual reality system send to other virtual reality system data processing equipments (COM3 ', be the data processing equipment of the immersive VR system at long-range user place) data, local user's human trajectory data that it comprises head rotation action data that the angle induction device (104) of microphone (102) audio data captured, the display (COM1) of local virtual reality system display (COM1) obtains, obtain by local action acquisition equipment (COM2) etc.Data processing module (7014) is handled received reduced data, arrange virtual three-dimensional environment, long-range user and local user are placed on the position separately in the reality environment, and long-range user and local user's limbs posture and head angle are set, head angle according to local user, the 3D video camera (is formed concept for one in the virtual three-dimensional space that described 3D video camera is made up by the computer system computing, its effect is the virtual video camera concept of information such as assigned address direction of definition and the visual field, in order to render picture in the secondary three dimensions according to this virtual video camera.Usually the picture in the viewed 3D scene of user is the picture of the 3D shot by camera in this 3D scene.The not responsible rendered picture of 3D video camera has just determined the content that will take.) the position be arranged on local user's eye.Data after data processing module (7014) will be handled send 3D rendering module (7016) and/or audio frequency rendering module (7015).Audio frequency rendering module (7015) is according to the position in each local user place reality environment, plays up the audio frequency that environment audio frequency that this user hears and other users send; Preferred audio volume according to source of sound and this user the distance in reality environment adjust, for example then audio volume is bigger for close together, apart from more then audio volume is less.3D rendering module (7016) is played up the video pictures that each local user sees respectively according to the video camera that sets.Data after 3D rendering module (7016) and audio frequency rendering module (7015) will be played up are sent to output data preparation module (7017).The picture that output data preparation module (7017) will have been played up and voice data are put into different categories and are put the display of wearing with the local user of giving to be sent (COM1) in order; Having under a plurality of local users' the situation, the picture that output data preparation module (7017) will have been played up is put into different categories with voice data and is put the display of wearing with each corresponding local user of giving to be sent (COM1) in order.Output data preparation module (7017) sends to local data output module (7018) with the data of putting in order.Local data output module (7018) is with reduced data, and namely local user exports data (704) and is sent to the display (COM1) that corresponding local user wears.It is the data that transfer to local user after handling through data place (701) that local user exports data (704), and it comprises long-range user's microphone (102 ') audio data captured after the processing, other voice datas in the virtual environment, the video data in the virtual environment etc.
At least one display (COM1), at least one action acquisition equipment (COM2) and at least one data processing equipment (COM3) are formed a VRS.Virtual reality system according to the present invention with a display (COM1), an action acquisition equipment (COM2) and data treating apparatus (COM3) is called VRS-SM, and the virtual reality system according to the present invention with two above displays (COM1), an action acquisition equipment (COM2) and data treating apparatus (COM3) is called VRS-MM.Also can be constituted by a plurality of VRS according to virtual reality system of the present invention.
DEU data exchange unit (COM4) is for the treatment of a plurality of VRS (two above VRS-SM for example, perhaps two above VRS-MM, perhaps an above VRS-SM and an above VRS-MM) between data transmission (exchanges data of a plurality of user's remote interactions), and can be used for realizing being attached thereto the needed total virtual environment of a plurality of VRS that connect.DEU data exchange unit (COM4) is transmitted data mutual between the different VRS, even make these users in the long-range impression that also can obtain to be in Same Scene, carries out interactive exchange.These mutual data owners will comprise basic necessary datas such as user's limb action, voice.DEU data exchange unit (COM4) has contextual data and all users' data, could handle any one occurrence that all participants do in scene.DEU data exchange unit in the processing of a plurality of user's remote interactions (COM4) is a requisite critical component.Namely, for example DEU data exchange unit (COM4) receives the data that data processing equipment (COM3) spreads out of, if needed after received data (user's data) being handled and/or increased data message (for example scene information), be transferred to another data processing equipment (COM3 '); Simultaneously the data that also data processing equipment (COM3 ') spread out of after received data being handled and/or increased data message, are transferred to data processing equipment (COM3) if needed.And data processing equipment (COM3) will be transmitted to all users in the local VRS from the data of DEU data exchange unit (COM4), and according to the data from DEU data exchange unit (COM4), render image, make it to see that other VRS that DEU data exchange unit (COM4) connects go up user and the sound of hearing them.In the system of a plurality of VRS, rendering image is transmitted and be responsible for to data processing equipment (COM3) for DEU data exchange unit (COM4) provides data.In the system of single VRS, data processing equipment (COM3) can be finished at same device for the work of DEU data exchange unit (COM4).
As shown in Figure 8, DEU data exchange unit (COM4) generally includes controller (401), arithmetical unit (402), storer (403), input media (404), and output unit (405).
Controller (401) is used for control program to be carried out, and it is according to the instruction sequence in the storer (program) work, and the control execution command, according to each parts coordinated manipulation of instruction control machine.Arithmetical unit (402) is used for carrying out data processing and handles for example various arithmetic, logical operation etc.Storer (403) is used for memory program and data, for example deposits program and data with the binary code form.The data that input media (404) is used for external device is sent are imported arithmetical unit (402).The data that output unit (405) is used for arithmetical unit (402) is handled export external device to.Described external device is data processing equipment (COM3), mouse, keyboard and/or touching display screen (for example capacitance plate or resistance screen) for example.
Input equipment (404) to arithmetical unit (402), is stored in storer (403) to the program of needs and data transmission by arithmetical unit (402) again.In the calculation process process, data are read in arithmetical unit (402) from storer (403) and are carried out computing, and operation result deposits storer (403) in, or is exported through output device (405) by arithmetical unit (402).Instruction also is stored in the storer (403) with data mode, instruction is sent into controller (401) by storer (403) during computing, produce control stream with the flow direction of control data stream and the work of controlling each parts by controller (401), data stream is processed processing.
DEU data exchange unit (COM4) can be for example computer, for example PC, notebook computer; Personal digital assistant (Personal Digital Assistant, PDA), the Dell AximX51V palm PC produced of Dell for example; Panel computer, for example TouchPad of the iPad of Apple's production and iTouch, Hewlett-Packard's production; Mobile phone, the iPhone that produces of Apple for example, the Nokia N85 that Nokia produces; Audio frequency processing terminal, for example MP3 and iPod; Video processing terminal, for example MP4; And/or game machine, for example PS2, PS3, PSP, NGC, NDSL, XBOX, Wii, GBA, 3DS and NDS.
One according to virtual reality system of the present invention in, data processing equipment (COM3) and DEU data exchange unit (COM4) can be same devices.Preferably one according to virtual reality system of the present invention in, data processing equipment (COM3) and DEU data exchange unit (COM4) are not same devices.
According to virtual reality system of the present invention, data transmission between display (COM1), action acquisition equipment (COM2), data processing equipment (COM3) and the DEU data exchange unit (COM4) is undertaken by data transmission device (for example data transmission device (105), output interface (205), input media (304), output unit (305), input media (404) and/or output unit (405)), the data transfer mode that described data transmission device is taked can be wire transmission, also can be wireless transmission.Yet when adopting wire transmission mode, the connecting line on the data transmission device might disturb user's activity, influences user's experience, thereby, according to the present invention, data transmission device preferably adopts wireless transmission method, for example bluetooth, infrared, Wifi, WHDI etc.When adopting wireless transmission, topmost problem is transmission speed, system acquisition arrives the user in three-dimensional motion, then with the user in reality environment " should " vision and/or the auditory perception experienced send the user to, if because transmission speed produces time-delay, what the user experienced may be several seconds experience in the past, can weaken the function of virtual reality system so greatly.The time-delay that the user feels mainly comes from the transmission delay of local data transfer, i.e. transmission (transmission between display and motion capture device, display and data processing equipment and/or motion capture device and the described data processing equipment) and the time-delay that causes between each device in each VRS of virtual reality system.Transmission delay is divided into input time delay (for example display (COM1) and/or the time-delay of acquisition equipment (COM2) to the generation of data processing equipment (COM3) transmission information of moving) and output time-delay (for example data processing equipment (COM3) is to the time-delay of action acquisition equipment (COM1) transmission information generation).Input delay, because the input content mainly is data and the speech data that is produced by human action, data volume is less relatively, and is relatively low to the transmission medium bandwidth requirement.And output delay is row graph sequence and audio frequency owing to what transmit, and is so data volume is relatively large, higher relatively to the transmission medium bandwidth.Thereby according to the present invention, also the transmission speed of preference data transmitting device is not less than 18432000bps (bit per second, bits per second), more preferably be not less than 55296000bps, more preferably be not less than 55296000bps, also more preferably be not less than 147456000bps.More preferably data transmission device adopts WHDI (Wireless Home Digital Interface, i.e. wireless family digital interface) transmission mode.WHDI can realize the data rate up to 3Gbps, and working range can reach 30 meters, penetrable wall, and sending data delay can be less than 1 millisecond.Adopt the data transmission device of this technology to obtain in market, for example the MIPS-Based chip of AMIMON company or AMN2120/AMN2220 chip.
Can be used for many aspects such as recreation, scientific research, multimedia, amusement, emulation, teleconference according to immersive VR of the present invention system.
Now will more at large set forth the present invention in conjunction with the embodiments, the cited mode of embodiment only is typical example according to the present invention, only is used for explanation the present invention, and does not mean that the present invention is carried out any restriction.
Embodiment
Embodiment 1
One according to immersive VR of the present invention system (VRS 1, as shown in Figure 9), comprises a display (COM1), an action acquisition equipment (COM2), a data treating apparatus (COM3).Display (COM1) is that the eyes formula is worn video display, has a display device (Myvu company
Transmission-type liquid crystal micro display screen), a microphone (iron triangle PRO 51Q), an earphone (Sen Haisaier PX80), an angle induction device (digital compass, the Japan's AK8973 of Asahi Kasei Corporation digital compass chip), signal processing apparatus and a data transmission device (the STLC4420 type WiFi chip that ST Microelectronics produces), wherein, described signal processing apparatus comprises image processing apparatus, and (Microdisplay Controller (Custom) KCD-A210-BA that Kopin produces is integrated in Myvu company
On the transmission-type liquid crystal micro display screen), sound processing apparatus (based on the UDA1341 decoding chip of I2S bus), control device (the Samsung S3C2440 chip that has the I2C bus interface) and memory storage (two and the K4S511632B-CL75 that connects, the SDRAM chip of 512M bytes).Display (COM1) power unit adopts the low pressure reduction voltage stabilizing chip power supply of MCP1700.According to the opering characteristic of electric apparatus of said chip, the circuit diagram of design present embodiment display (COM1), (assembling forms display (COM1) for Printed circuit board, design PCB) and making to finish printed circuit board (PCB) according to circuit diagram.
Action acquisition equipment (COM2) is an individual sense video camera (Kinet of Microsoft camera), and data processing equipment (COM3) is a computer (man of association of the TL-WN321G type wireless Internet card that external TP-LINK company produces pleases the E2565 desktop computer).
After display (COM1) start, initialization Samsung S3C2440 chip, UDA1341 decoding chip, AK8973 digital compass chip and STLC4420 chip are carried out the master routine circulation then in control device SamsungS3C2440 chip.In the master routine circulation, control device obtain from angular transducer and microphone data, by data transmission device from the data of data processing equipment (COM3) and be stored in the memory storage, and will send to data processing equipment (COM3) from the data of angular transducer and microphone by data transmission device, the video-audio data of needs output is outputed to display device and earphone.
After action acquisition equipment (COM2) is caught user's action, generate the depth map data, be converted to the bone action data by the depth map data then, and the bone action data is sent to data processing equipment (COM3).Data processing equipment (COM3) is according to the data of the simulator program processing that writes realization virtual reality wherein in advance from display (COM1) and action acquisition equipment (COM2), obtain virtual scene information, comprise information such as position, limbs and headwork in scene of user for example, sound, and be converted into audio/video information and send to display (COM1).Display (COM1) has stored the data in the memory storage after receiving the audio/video information that generates from data processing equipment (COM3), and the voice data that will need to export exports earphone to, and the video data that needs are exported exports display device to.
Adopt WiFi to realize data transmission between display (COM1) and the data processing equipment (COM3), action acquisition equipment (COM2) carries out wired connection with data processing equipment (COM3) by specification.
Operation result shows according to immersive VR system of the present invention (VRS1) can normally carry out the virtual reality simulator program, does not have obviously to postpone.
Embodiment 2
One according to immersive VR of the present invention system (VRS2 as shown in Figure 10), comprises three displays (COM1), an action acquisition equipment (COM2), a data treating apparatus (COM3).Display (COM1) adopts three-axis gyroscope (three magnetoresistive transducers of the HMC5883L of Honeywell Inc.) except angle induction device, and signal transmitting apparatus adopts outside the CAT6023 chip, and is identical with display (COM1) among the embodiment 1.
Action acquisition equipment (COM2) is an individual sense video camera (Kinet of Microsoft camera), and data processing equipment (COM3) is a game machine (PlayStation of Sony 2 of the SEM-ST1O11 type WHDI STICK that external SEAMON company produces).
After display (COM1) start, initialization Samsung S3C2440 chip, UDA1341 decoding chip, three magnetoresistive transducers of HMC5883L and CAT6023 chip are carried out the master routine circulation then in control device SamsungS3C2440 chip.In the master routine circulation, each control device obtain from angular transducer and microphone separately data, by data transmission device separately from the data of data processing equipment (COM3) and be stored in separately the memory storage, and the data transmission device by separately will send to data processing equipment (COM3) from the data of angular transducer and microphone separately, and the video-audio data of needs output is outputed to separately display device and earphone.
After action acquisition equipment (COM2) is caught user's action, generate the depth map data, be converted to the bone action data by the depth map data then, and the bone action data is sent to data processing equipment (COM3).Data processing equipment (COM3) is according to the data of the simulator program processing that writes realization virtual reality wherein in advance from display (COM1) and action acquisition equipment (COM2), handle all users interactive relation, obtain virtual scene information, comprise position, limbs and headwork in scene of each user for example, sound, mutual information such as scene information, and be converted into audio/video information and send to corresponding display (COM1).After display (COM1) is received the audio/video information that generates from data processing equipment (COM3), stored the data in the memory storage separately, and the voice data that will need to export exports earphone separately to, exports the video data of needs output to separately display device.
Adopt WHDI to realize data transmission between display (COM1) and the data processing equipment (COM3), action acquisition equipment (COM2) carries out wired connection with data processing equipment (COM3) by specification.
Operation result shows according to immersive VR system of the present invention (VRS2) can normally carry out the virtual reality simulator program, does not have obviously to postpone.
Embodiment 3
One according to immersive VR system of the present invention (as shown in Figure 11), formed by two identical immersive VR systems (VRS3 and VRS3 '), VRS3 adopts three-axis gyroscope (three magnetoresistive transducers of the HMC5883L of Honeywell Inc.) except the angle induction device in the display (COM1), signal transmitting apparatus adopts the CAT6023 chip, data processing equipment (COM3) is that the man of association of the SEM-ST1011 type WHDI STICK that produces of external SEAMON company pleases outside the E2565 desktop computer, and is identical with VRS1 among the embodiment 1.Be connected and carry out data transmission by router (the TL-R478+ type router that TP-Link company produces) via data processing equipment (COM3 and COM3 ') between two VRS3.
After display (COM1, the COM1 ') start, initialization Samsung S3C2440 chip, UDA1341 decoding chip, three magnetoresistive transducers of HMC5883L and CAT6023 chip are carried out the master routine circulation then in control device Samsung S3C2440 chip.In the master routine circulation, the control device of display (COM1) obtain that data, its data transmission device from its angular transducer and microphone receive from the data of data processing equipment (COM3) and be stored in its memory storage, and will send to data processing equipment (COM3) from the data of its angular transducer and microphone by its data transmission device, the video-audio data of needs output is outputed to its display device and earphone.Display (COM1 ') operation is identical with display (COM1).
In VRS3, after action acquisition equipment (COM2) is caught user's action, generate the depth map data, be converted to the bone action data by the depth map data then, and the bone action data is sent to data processing equipment (COM3).Data processing equipment (COM3) is handled from display (COM1) according to the simulator program that writes realization virtual reality wherein in advance, the data of action acquisition equipment (COM2) and data processing equipment (COM3 '), handle all users interactive relation, obtain virtual scene information, comprise for example position of each user in scene, limbs and headwork, sound, mutual information such as scene information, corresponding VRS3 ' user's virtual scene information is sent to data processing equipment (COM3 '), corresponding VRS3 user's virtual scene information is converted into audio/video information sends to display (COM1).
In VRS3 ', after action acquisition equipment (COM2 ') is caught user's action, generate the depth map data, be converted to the bone action data by the depth map data then, and the bone action data is sent to data processing equipment (COM3 ').Data processing equipment (COM3 ') is handled data from display (COM1 '), action acquisition equipment (COM2 ') according to the simulator program that writes realization virtual reality wherein in advance, send to data processing equipment (COM3), also receive from data processing equipment (COM3) virtual scene information simultaneously, and be converted into audio/video information and send to display (COM1 ').
Display (COM1 and COM1 ') is received from behind the audio/video information that the data processing equipment of VRS (COM3 and COM3 ') generates separately, stored the data in the memory storage separately, and the voice data that will need to export exports earphone separately to, exports the video data of needs output to separately display device.
Move simulator program respectively simultaneously, operation result shows according to immersive VR of the present invention system can normally carry out the virtual reality simulator program, does not have obviously to postpone.
One according to immersive VR system of the present invention (as shown in Figure 12), is made up of two immersive VR systems (VRS3 and VRS4).VRS3 is identical with embodiment 3; VRS4 is except having three displays (COM1 '), and is identical with VRS3.Be connected by DEU data exchange unit (COM4) via data processing equipment (COM3 and COM3 ') between two VRS, data processing equipment (COM3 and COM3 ') is connected by router (the TL-R478+ type router that TP-Link company produces) with DEU data exchange unit (COM4).DEU data exchange unit (COM4) is day A8800t desktop computer of raising of Legend Company's production.
As embodiment 3, display (COM1 and COM1 ') opening initialization is also carried out the master routine circulation.In VRS3, after action acquisition equipment (COM2) is caught user's action, generate the depth map data, be converted to the bone action data by the depth map data then, and the bone action data is sent to data processing equipment (COM3).Data processing equipment (COM3) is handled from display (COM1), the data of acquisition equipment (COM2) of moving according to the simulator program that writes realization virtual reality wherein in advance, and sends to DEU data exchange unit (COM4).
After DEU data exchange unit (COM4) receives the data of the data processing equipment (COM3 and COM3 ') from each VRS, handle all users' interactive relation, obtain virtual scene information, comprise position, limbs and headwork in scene of each user for example, sound, mutual information such as scene information, corresponding VRS3 user's virtual scene information is sent to data processing equipment (COM3), corresponding VRS4 user's virtual scene information is sent to data processing equipment (COM3 ').
Data processing equipment (COM3) receives the virtual scene information from DEU data exchange unit (COM4), and is converted into audio/video information and sends to display (COM1).After display (COM1) is received the audio/video information that generates from data processing equipment (COM3), stored the data in the memory storage separately, and the voice data that will need to export exports earphone separately to, exports the video data of needs output to separately display device.
The display of VRS4 (COM1 '), action acquisition equipment (COM2 '), data processing equipment (COM3 ') are carried out display (COM1), action acquisition equipment (COM2), the essentially identical action of data processing equipment (COM3) with VRS3.
Move simulator program respectively simultaneously, operation result shows according to immersive VR of the present invention system can normally carry out the virtual reality simulator program, does not have obviously to postpone.
Embodiment 5
One according to immersive VR system of the present invention (as shown in Figure 13), by five immersive VR system (VRS1s identical with the VRS3 of embodiment 3, VRS1 ', VRS1 "; VRS1 " ' and VRS1 " ") form, between five VRS via data processing equipment (COM3, COM3 ', COM3 "; COM3 " ' with COM3 " ") be connected by DEU data exchange unit (COM4) and carry out data transmission, data processing equipment (COM3, COM3 ', COM3 "; COM3 " ' with COM3 " ") be connected by router (TP-Link company produce TL-R478+ type router) with DEU data exchange unit (COM4).DEU data exchange unit (COM4) is day A8800t desktop computer of raising of Legend Company's production.
As embodiment 3, display (COM1, COM1 ' etc.) opening initialization is also carried out the master routine circulation.In VRS3, after action acquisition equipment (COM2) is caught user's action, generate the depth map data, be converted to the bone action data by the depth map data then, and the bone action data is sent to data processing equipment (COM3).Data processing equipment (COM3) is handled from display (COM1), the data of acquisition equipment (COM2) of moving according to the simulator program that writes realization virtual reality wherein in advance, and sends to DEU data exchange unit (COM4).
DEU data exchange unit (COM4) receives the data processing equipment (COM3 from each VRS, COM3 ' etc.) after the data, handle all users' interactive relation, obtain virtual scene information, comprise for example position of each user in scene, limbs and headwork, sound, mutual information such as scene information, corresponding each VRS user's virtual scene information is sent to corresponding data processing equipment, for example the virtual scene information with corresponding VRS3 user sends to data processing equipment (COM3), and corresponding VRS3 ' user's virtual scene information is sent to data processing equipment (COM3 ').
Data processing equipment (COM3) receives the virtual scene information from DEU data exchange unit (COM4), and is converted into audio/video information and sends to display (COM1).After display (COM1) is received the audio/video information that generates from data processing equipment (COM3), stored the data in the memory storage separately, and the voice data that will need to export exports earphone separately to, exports the video data of needs output to separately display device.
The display of other VRS of present embodiment (COM1 ', COM1 " etc.), the action acquisition equipment (COM2 ', COM2 " etc.), data processing equipment (COM3 ', COM3 " etc.) carry out display (COM1), action acquisition equipment (COM2), the essentially identical action of data processing equipment (COM3) with VRS3.
Move simulator program respectively simultaneously, operation result shows according to immersive VR of the present invention system can normally carry out the virtual reality simulator program, does not have obviously to postpone.
Embodiment 6
One according to immersive VR system of the present invention (as shown in Figure 14), by three immersive VR system (VRS3s identical with the VRS3 of embodiment 3, VRS3 ' and VRS3 ") form with two immersive VR systems (VRS4 and VRS4 ') identical with the VRS4 of embodiment 4; between five VRS via data processing equipment (COM3; COM3 '; COM3 ", COM3 " ' with COM3 " ") be connected by DEU data exchange unit (COM4) and carry out data transmission, data processing equipment (COM3; COM3 '; COM3 ", COM3 " ' with COM3 " ") be connected by router (TP-Link company produce TL-R478+ type router) with DEU data exchange unit (COM4).DEU data exchange unit (COM4) is day A8800t desktop computer of raising of Legend Company's production.
Display (COM1, COM1 ' etc.) opening initialization is also carried out the master routine circulation.In VRS3, after action acquisition equipment (COM2) is caught user's action, generate the depth map data, be converted to the bone action data by the depth map data then, and the bone action data is sent to data processing equipment (COM3).Data processing equipment (COM3) is handled from display (COM1), the data of acquisition equipment (COM2) of moving according to the simulator program that writes realization virtual reality wherein in advance, and sends to DEU data exchange unit (COM4).
DEU data exchange unit (COM4) receives the data processing equipment (COM3 from each VRS, COM3 ' etc.) after the data, handle all users' interactive relation, obtain virtual scene information, comprise for example position of each user in scene, limbs and headwork, sound, mutual information such as scene information, each user's of corresponding each VRS virtual scene information is sent to corresponding data processing equipment, for example the virtual scene information with corresponding VRS3 user sends to data processing equipment (COM3), and corresponding VRS4 user's virtual scene information is sent to data processing equipment (COM3 " ').
Data processing equipment (COM3) receives the virtual scene information from DEU data exchange unit (COM4), and is converted into audio/video information and sends to display (COM1).After display (COM1) is received the audio/video information that generates from data processing equipment (COM3), stored the data in the memory storage separately, and the voice data that will need to export exports earphone separately to, exports the video data of needs output to separately display device.
The display of other VRS of present embodiment (COM1 ', COM1 " etc.), the action acquisition equipment (COM2 ', COM2 " etc.), data processing equipment (COM3 ', COM3 " etc.) carry out display (COM1), action acquisition equipment (COM2), the essentially identical action of data processing equipment (COM3) with VRS3.
Move simulator program respectively simultaneously, operation result shows according to immersive VR of the present invention system can normally carry out the virtual reality simulator program, does not have obviously to postpone.
Claims (24)
1. immersive VR system, described system comprises at least:
At least one can wear the display on user's health, at least one motion capture device, and
At least one data processing equipment;
Wherein,
Described display is used for showing image to the user, and the head rotation that is used for the seizure user is moved, and the head rotation action data is sent at least one described data processing equipment, and receive the data that at least one described data processing equipment sends, and transmit the video and/or audio of virtual environment to the user;
Described motion capture device catches the movement locus at user's main node position, and motion trace data is sent at least one described data processing equipment;
Described data processing equipment is for the treatment of the data of obtaining and transfer to described data processing equipment from least one described display and at least one described motion capture device, and the result after will handling is sent at least one display.
2. virtual reality system according to claim 1 is characterized in that described display comprises at least one angle induction device, and described angle induction device can be responded at least around the spinning movement of z axle.
3. virtual reality system according to claim 2 is characterized in that described angle induction device is digital compass.
4. virtual reality system according to claim 3 is characterized in that described angle induction device can respond to around the spinning movement of x, y and three axles of z.
5. virtual reality system according to claim 4 is characterized in that described angle induction device is three-axis gyroscope.
6. according to each described virtual reality system of claim 1-5, it is characterized in that described motion capture device comprises at least one infrared transmitter and at least one infrared pick-up head.
7. virtual reality system according to claim 6 is characterized in that described motion capture device is body sense video camera.
8. according to each described virtual reality system of claim 1-7, described data processing equipment is computer, personal digital assistant, panel computer, mobile phone, audio frequency processing terminal, Video processing terminal and/or game machine.
9. according to each described virtual reality system of claim 1-8, also comprise at least one DEU data exchange unit, for the treatment of the data transmission between a plurality of VRS.
10. virtual reality system according to claim 9, described DEU data exchange unit is computer, personal digital assistant, panel computer, mobile phone, audio frequency processing terminal, Video processing terminal and/or game machine.
11. according to each described virtual reality system of claim 1-10, the transmission speed between described display and described motion capture device, described display and described data processing equipment and/or described motion capture device and the described data processing equipment is not less than 18432000bps.
12. virtual reality system according to claim 11, the WHDI transmission mode is adopted in the transmission between described display and described motion capture device, described display and described data processing equipment and/or described motion capture device and the described data processing equipment.
13. a method that realizes immersive VR, described method comprise at least with the lower part:
Catch the movement locus at user's main node position by at least one motion capture device, and motion trace data be sent at least one data processing equipment,
The head rotation that catches the user by at least one display is moved, and the head rotation action data is sent at least one described data processing equipment,
By the data of at least one described data processing equipment processing from described motion capture device and described display, and the data after will handling are sent at least one described display;
Data after the processing that will receive from least one described data processing equipment by at least one described display are converted to video and/or audio, the reality environment that the user can be experienced follow oneself action to change.
14. the method for realization immersive VR according to claim 13 is characterized in that described display comprises at least one angle induction device, described angle induction device can be responded at least around the spinning movement of z axle.
15. the method for realization immersive VR according to claim 14 is characterized in that described angle induction device is digital compass.
16. the method for realization immersive VR according to claim 14 is characterized in that described angle induction device can respond to the spinning movement around x, y and three axles of z.
17. the method for realization immersive VR according to claim 16 is characterized in that described angle induction device is three-axis gyroscope.
18. according to the method for each described realization immersive VR of claim 13-17, it is characterized in that described motion capture device comprises at least one infrared transmitter and at least one infrared pick-up head.
19. the method for realization immersive VR according to claim 18 is characterized in that described motion capture device is body sense video camera.
20. according to the method for each described realization immersive VR of claim 13-19, described data processing equipment is computer, personal digital assistant, panel computer, mobile phone, audio frequency processing terminal, Video processing terminal and/or game machine.
21. according to each described virtual reality system of claim 13-20, also comprise by at least one DEU data exchange unit and handle data transmission between a plurality of VRS.
22. virtual reality system according to claim 21, described DEU data exchange unit are computer, personal digital assistant, panel computer, mobile phone, audio frequency processing terminal, Video processing terminal and/or game machine.
23. according to each described virtual reality system of claim 13-22, the transmission speed between described display and described motion capture device, described display and described data processing equipment and/or described motion capture device and the described data processing equipment is not less than 18432000bps.
24. virtual reality system according to claim 23, the WHDI transmission mode is adopted in the transmission between described display and described motion capture device, described display and described data processing equipment and/or described motion capture device and the described data processing equipment.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012100063389A CN103197757A (en) | 2012-01-09 | 2012-01-09 | Immersion type virtual reality system and implementation method thereof |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012100063389A CN103197757A (en) | 2012-01-09 | 2012-01-09 | Immersion type virtual reality system and implementation method thereof |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103197757A true CN103197757A (en) | 2013-07-10 |
Family
ID=48720413
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012100063389A Pending CN103197757A (en) | 2012-01-09 | 2012-01-09 | Immersion type virtual reality system and implementation method thereof |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103197757A (en) |
Cited By (42)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103605208A (en) * | 2013-08-30 | 2014-02-26 | 北京智谷睿拓技术服务有限公司 | Content projection system and method |
| CN103631381A (en) * | 2013-12-10 | 2014-03-12 | 国家电网公司 | Man-machine interaction system and man-machine interaction method for electric transmission line patrolling simulation system |
| CN104023005A (en) * | 2014-05-06 | 2014-09-03 | 山东歌尔谷崧精密工业有限公司 | Virtual conference system |
| CN104616544A (en) * | 2015-03-12 | 2015-05-13 | 马盼盼 | Intelligent teaching system |
| CN104702911A (en) * | 2014-11-24 | 2015-06-10 | 段然 | Wearable video device real-time wireless transmission method |
| CN104759095A (en) * | 2015-04-24 | 2015-07-08 | 吴展雄 | Virtual reality head wearing display system |
| CN105260016A (en) * | 2015-09-24 | 2016-01-20 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105404393A (en) * | 2015-06-30 | 2016-03-16 | 指点无限（美国）有限公司 | Low-latency virtual reality display system |
| CN105791977A (en) * | 2016-02-26 | 2016-07-20 | 北京视博云科技有限公司 | Virtual reality data processing method and system based on cloud service and devices |
| CN105872444A (en) * | 2016-04-22 | 2016-08-17 | 广东小天才科技有限公司 | Method, device and system for video call |
| CN105912110A (en) * | 2016-04-06 | 2016-08-31 | 北京锤子数码科技有限公司 | Method, device and system for performing target selection in virtual reality space |
| CN105912108A (en) * | 2016-04-05 | 2016-08-31 | 福建天晴数码有限公司 | Split-screen method and system of browser in IOS system |
| CN105912107A (en) * | 2016-04-05 | 2016-08-31 | 福建天晴数码有限公司 | Split-screen method and system for browser in Windows system |
| CN106020493A (en) * | 2016-03-13 | 2016-10-12 | 成都市微辣科技有限公司 | Product display device and method based on virtual reality |
| CN106110627A (en) * | 2016-06-20 | 2016-11-16 | 曲大方 | Physical culture and Wushu action correction equipment and method |
| CN106200983A (en) * | 2016-07-25 | 2016-12-07 | 圣域科技（深圳）有限公司 | A kind of combined with virtual reality and BIM realize the system of virtual reality scenario architectural design |
| CN106385348A (en) * | 2016-09-21 | 2017-02-08 | 平越 | Virtual reality entertainment system with step-by-step identification function and method of virtual reality entertainment system |
| CN106604200A (en) * | 2017-01-19 | 2017-04-26 | 浪潮（苏州）金融技术服务有限公司 | Audio data processing method and apparatus |
| CN106679933A (en) * | 2016-11-21 | 2017-05-17 | 福建省华渔教育科技有限公司 | Method and system for testing head-mounted display device performance |
| CN106873760A (en) * | 2015-12-14 | 2017-06-20 | 技嘉科技股份有限公司 | Portable virtual reality system |
| CN106909163A (en) * | 2015-12-22 | 2017-06-30 | 深圳信息职业技术学院 | Immersive VR unmanned plane control loop |
| CN107291999A (en) * | 2017-06-01 | 2017-10-24 | 中国农业大学 | A kind of farming analogy method and system based on virtual reality technology |
| CN107291244A (en) * | 2017-07-04 | 2017-10-24 | 宁波美象信息科技有限公司 | A kind of VR visits system |
| CN107430868A (en) * | 2015-03-06 | 2017-12-01 | 微软技术许可有限责任公司 | The Real-time Reconstruction of user speech in immersion visualization system |
| US9867756B2 (en) | 2013-08-22 | 2018-01-16 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Eyesight-protection imaging system and eyesight-protection imaging method |
| US9867532B2 (en) | 2013-07-31 | 2018-01-16 | Beijing Zhigu Rui Tuo Tech Co., Ltd | System for detecting optical parameter of eye, and method for detecting optical parameter of eye |
| US9870050B2 (en) | 2013-10-10 | 2018-01-16 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Interactive projection display |
| CN107979763A (en) * | 2016-10-21 | 2018-05-01 | 阿里巴巴集团控股有限公司 | A kind of virtual reality device generation video, playback method, apparatus and system |
| CN108108026A (en) * | 2018-01-18 | 2018-06-01 | 珠海金山网络游戏科技有限公司 | A kind of VR virtual realities motion capture system and motion capture method |
| CN108475439A (en) * | 2016-02-16 | 2018-08-31 | 乐天株式会社 | Threedimensional model generates system, threedimensional model generation method and program |
| CN108702490A (en) * | 2016-02-29 | 2018-10-23 | 微软技术许可有限责任公司 | Immersion interaction is long-range to be presented |
| US10191276B2 (en) | 2013-06-28 | 2019-01-29 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Imaging adjustment device and imaging adjustment method |
| CN109388142A (en) * | 2015-04-30 | 2019-02-26 | 广东虚拟现实科技有限公司 | A kind of method and system carrying out virtual reality travelling control based on inertial sensor |
| US10261345B2 (en) | 2013-06-28 | 2019-04-16 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Imaging adjustment device and imaging adjustment method |
| CN110099297A (en) * | 2018-01-31 | 2019-08-06 | 深圳市掌网科技股份有限公司 | A kind of radio transmitting method and system based on virtual reality device |
| US10395510B2 (en) | 2013-08-30 | 2019-08-27 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Reminding method and reminding device |
| CN110264846A (en) * | 2019-07-11 | 2019-09-20 | 广东电网有限责任公司 | A kind of electric network emergency skill training system based on CAVE |
| CN110291487A (en) * | 2017-02-01 | 2019-09-27 | 标致雪铁龙汽车股份有限公司 | For determining the analytical equipment of the time delay of immersed system of virtual reality |
| US10481396B2 (en) | 2013-06-28 | 2019-11-19 | Beijing Zhigu Rui Tuo Tech Co., Ltd. | Imaging device and imaging method |
| US10551638B2 (en) | 2013-07-31 | 2020-02-04 | Beijing Zhigu Rui Tuo Tech Co., Ltd. | Imaging apparatus and imaging method |
| US10583068B2 (en) | 2013-08-22 | 2020-03-10 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Eyesight-protection imaging apparatus and eyesight-protection imaging method |
| CN114339405A (en) * | 2022-01-04 | 2022-04-12 | 广州博冠信息科技有限公司 | AR video data stream remote manufacturing method and device, equipment and storage medium |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20030025652A1 (en) * | 2001-08-01 | 2003-02-06 | Susnfara Kenneth J. | Virtual reality viewing system and method |
| CN1168057C (en) * | 1996-08-14 | 2004-09-22 | 挪拉赫梅特·挪利斯拉莫维奇·拉都包夫 | Method for following and imaging a subject's three-dimensional position and orientation, method for presenting a virtual space to a subject,and systems for implementing said methods |
| US20100240988A1 (en) * | 2009-03-19 | 2010-09-23 | Kenneth Varga | Computer-aided system for 360 degree heads up display of safety/mission critical data |
| JP2011109462A (en) * | 2009-11-18 | 2011-06-02 | Konica Minolta Opto Inc | Image transmission device, imaging device, and remote work support system |
| US20110213664A1 (en) * | 2010-02-28 | 2011-09-01 | Osterhout Group, Inc. | Local advertising content on an interactive head-mounted eyepiece |
-
2012
- 2012-01-09 CN CN2012100063389A patent/CN103197757A/en active Pending
Patent Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1168057C (en) * | 1996-08-14 | 2004-09-22 | 挪拉赫梅特·挪利斯拉莫维奇·拉都包夫 | Method for following and imaging a subject's three-dimensional position and orientation, method for presenting a virtual space to a subject,and systems for implementing said methods |
| US20030025652A1 (en) * | 2001-08-01 | 2003-02-06 | Susnfara Kenneth J. | Virtual reality viewing system and method |
| US20100240988A1 (en) * | 2009-03-19 | 2010-09-23 | Kenneth Varga | Computer-aided system for 360 degree heads up display of safety/mission critical data |
| JP2011109462A (en) * | 2009-11-18 | 2011-06-02 | Konica Minolta Opto Inc | Image transmission device, imaging device, and remote work support system |
| US20110213664A1 (en) * | 2010-02-28 | 2011-09-01 | Osterhout Group, Inc. | Local advertising content on an interactive head-mounted eyepiece |
Cited By (57)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10481396B2 (en) | 2013-06-28 | 2019-11-19 | Beijing Zhigu Rui Tuo Tech Co., Ltd. | Imaging device and imaging method |
| US10191276B2 (en) | 2013-06-28 | 2019-01-29 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Imaging adjustment device and imaging adjustment method |
| US10261345B2 (en) | 2013-06-28 | 2019-04-16 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Imaging adjustment device and imaging adjustment method |
| US10551638B2 (en) | 2013-07-31 | 2020-02-04 | Beijing Zhigu Rui Tuo Tech Co., Ltd. | Imaging apparatus and imaging method |
| US9867532B2 (en) | 2013-07-31 | 2018-01-16 | Beijing Zhigu Rui Tuo Tech Co., Ltd | System for detecting optical parameter of eye, and method for detecting optical parameter of eye |
| US9867756B2 (en) | 2013-08-22 | 2018-01-16 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Eyesight-protection imaging system and eyesight-protection imaging method |
| US10583068B2 (en) | 2013-08-22 | 2020-03-10 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Eyesight-protection imaging apparatus and eyesight-protection imaging method |
| US10048750B2 (en) | 2013-08-30 | 2018-08-14 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Content projection system and content projection method |
| US10395510B2 (en) | 2013-08-30 | 2019-08-27 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Reminding method and reminding device |
| WO2015027599A1 (en) * | 2013-08-30 | 2015-03-05 | 北京智谷睿拓技术服务有限公司 | Content projection system and content projection method |
| CN103605208B (en) * | 2013-08-30 | 2016-09-28 | 北京智谷睿拓技术服务有限公司 | content projection system and method |
| CN103605208A (en) * | 2013-08-30 | 2014-02-26 | 北京智谷睿拓技术服务有限公司 | Content projection system and method |
| US9870050B2 (en) | 2013-10-10 | 2018-01-16 | Beijing Zhigu Rui Tuo Tech Co., Ltd | Interactive projection display |
| CN103631381A (en) * | 2013-12-10 | 2014-03-12 | 国家电网公司 | Man-machine interaction system and man-machine interaction method for electric transmission line patrolling simulation system |
| CN104023005A (en) * | 2014-05-06 | 2014-09-03 | 山东歌尔谷崧精密工业有限公司 | Virtual conference system |
| CN104702911A (en) * | 2014-11-24 | 2015-06-10 | 段然 | Wearable video device real-time wireless transmission method |
| CN107430868B (en) * | 2015-03-06 | 2021-02-09 | 微软技术许可有限责任公司 | Real-time reconstruction of user speech in immersive visualization system |
| CN107430868A (en) * | 2015-03-06 | 2017-12-01 | 微软技术许可有限责任公司 | The Real-time Reconstruction of user speech in immersion visualization system |
| CN104616544A (en) * | 2015-03-12 | 2015-05-13 | 马盼盼 | Intelligent teaching system |
| CN104759095A (en) * | 2015-04-24 | 2015-07-08 | 吴展雄 | Virtual reality head wearing display system |
| CN109388142A (en) * | 2015-04-30 | 2019-02-26 | 广东虚拟现实科技有限公司 | A kind of method and system carrying out virtual reality travelling control based on inertial sensor |
| CN105404393A (en) * | 2015-06-30 | 2016-03-16 | 指点无限（美国）有限公司 | Low-latency virtual reality display system |
| CN108762496A (en) * | 2015-09-24 | 2018-11-06 | 联想(北京)有限公司 | A kind of information processing method and electronic equipment |
| CN105260016A (en) * | 2015-09-24 | 2016-01-20 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN105260016B (en) * | 2015-09-24 | 2018-06-01 | 联想(北京)有限公司 | A kind of information processing method and electronic equipment |
| CN108762496B (en) * | 2015-09-24 | 2020-12-18 | 联想(北京)有限公司 | Information processing method and electronic equipment |
| CN106873760A (en) * | 2015-12-14 | 2017-06-20 | 技嘉科技股份有限公司 | Portable virtual reality system |
| CN106909163A (en) * | 2015-12-22 | 2017-06-30 | 深圳信息职业技术学院 | Immersive VR unmanned plane control loop |
| CN108475439B (en) * | 2016-02-16 | 2022-06-17 | 乐天集团股份有限公司 | Three-dimensional model generation system, three-dimensional model generation method, and recording medium |
| CN108475439A (en) * | 2016-02-16 | 2018-08-31 | 乐天株式会社 | Threedimensional model generates system, threedimensional model generation method and program |
| CN105791977A (en) * | 2016-02-26 | 2016-07-20 | 北京视博云科技有限公司 | Virtual reality data processing method and system based on cloud service and devices |
| CN105791977B (en) * | 2016-02-26 | 2019-05-07 | 北京视博云科技有限公司 | Virtual reality data processing method, equipment and system based on cloud service |
| CN108702490A (en) * | 2016-02-29 | 2018-10-23 | 微软技术许可有限责任公司 | Immersion interaction is long-range to be presented |
| CN106020493A (en) * | 2016-03-13 | 2016-10-12 | 成都市微辣科技有限公司 | Product display device and method based on virtual reality |
| CN105912108A (en) * | 2016-04-05 | 2016-08-31 | 福建天晴数码有限公司 | Split-screen method and system of browser in IOS system |
| CN105912107A (en) * | 2016-04-05 | 2016-08-31 | 福建天晴数码有限公司 | Split-screen method and system for browser in Windows system |
| CN105912108B (en) * | 2016-04-05 | 2018-10-12 | 福建天晴数码有限公司 | The method and system of IOS system browser split screens |
| CN105912107B (en) * | 2016-04-05 | 2019-03-26 | 福建天晴数码有限公司 | The method and system of Windows system browser split screen |
| CN105912110B (en) * | 2016-04-06 | 2019-09-06 | 北京锤子数码科技有限公司 | A kind of method, apparatus and system carrying out target selection in virtual reality space |
| CN105912110A (en) * | 2016-04-06 | 2016-08-31 | 北京锤子数码科技有限公司 | Method, device and system for performing target selection in virtual reality space |
| CN105872444B (en) * | 2016-04-22 | 2019-11-29 | 广东小天才科技有限公司 | A kind of video call method, apparatus and system |
| CN105872444A (en) * | 2016-04-22 | 2016-08-17 | 广东小天才科技有限公司 | Method, device and system for video call |
| CN106110627A (en) * | 2016-06-20 | 2016-11-16 | 曲大方 | Physical culture and Wushu action correction equipment and method |
| CN106200983B (en) * | 2016-07-25 | 2019-06-18 | 圣域科技（深圳）有限公司 | A kind of system of combination virtual reality and BIM realization virtual reality scenario architectural design |
| CN106200983A (en) * | 2016-07-25 | 2016-12-07 | 圣域科技（深圳）有限公司 | A kind of combined with virtual reality and BIM realize the system of virtual reality scenario architectural design |
| CN106385348A (en) * | 2016-09-21 | 2017-02-08 | 平越 | Virtual reality entertainment system with step-by-step identification function and method of virtual reality entertainment system |
| CN107979763A (en) * | 2016-10-21 | 2018-05-01 | 阿里巴巴集团控股有限公司 | A kind of virtual reality device generation video, playback method, apparatus and system |
| CN106679933A (en) * | 2016-11-21 | 2017-05-17 | 福建省华渔教育科技有限公司 | Method and system for testing head-mounted display device performance |
| CN106604200A (en) * | 2017-01-19 | 2017-04-26 | 浪潮（苏州）金融技术服务有限公司 | Audio data processing method and apparatus |
| CN110291487B (en) * | 2017-02-01 | 2023-03-24 | 标致雪铁龙汽车股份有限公司 | Analytical device for determining time delays of immersive virtual reality system |
| CN110291487A (en) * | 2017-02-01 | 2019-09-27 | 标致雪铁龙汽车股份有限公司 | For determining the analytical equipment of the time delay of immersed system of virtual reality |
| CN107291999A (en) * | 2017-06-01 | 2017-10-24 | 中国农业大学 | A kind of farming analogy method and system based on virtual reality technology |
| CN107291244A (en) * | 2017-07-04 | 2017-10-24 | 宁波美象信息科技有限公司 | A kind of VR visits system |
| CN108108026A (en) * | 2018-01-18 | 2018-06-01 | 珠海金山网络游戏科技有限公司 | A kind of VR virtual realities motion capture system and motion capture method |
| CN110099297A (en) * | 2018-01-31 | 2019-08-06 | 深圳市掌网科技股份有限公司 | A kind of radio transmitting method and system based on virtual reality device |
| CN110264846A (en) * | 2019-07-11 | 2019-09-20 | 广东电网有限责任公司 | A kind of electric network emergency skill training system based on CAVE |
| CN114339405A (en) * | 2022-01-04 | 2022-04-12 | 广州博冠信息科技有限公司 | AR video data stream remote manufacturing method and device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103197757A (en) | Immersion type virtual reality system and implementation method thereof | |
| JP7366196B2 (en) | Widespread simultaneous remote digital presentation world | |
| GB2556347B (en) | Virtual Reality | |
| TWI672168B (en) | System for processing content for a head mounted display(hmd), peripheral device for use in interfacing with a virtual reality scene generated by a computer for presentation on the hmd, and method of simulating a feeling of contact with a virtual object | |
| El Saddik et al. | Haptics technologies: Bringing touch to multimedia | |
| CN108815804B (en) | VR upper limb rehabilitation training platform and method based on MYO arm ring and mobile terminal | |
| US20150070274A1 (en) | Methods and systems for determining 6dof location and orientation of head-mounted display and associated user movements | |
| JP2022549853A (en) | Individual visibility in shared space | |
| US20170039986A1 (en) | Mixed Reality Social Interactions | |
| Romanus et al. | Mid-air haptic bio-holograms in mixed reality | |
| CN106575159A (en) | Glove interface object | |
| CN106233227A (en) | There is the game device of volume sensing | |
| CN112198959A (en) | Virtual reality interaction method, device and system | |
| Zhang et al. | The application of virtual reality technology in physical education teaching and training | |
| CN104470593A (en) | System and method for social dancing | |
| JP7160669B2 (en) | Program, Information Processing Apparatus, and Method | |
| CN107551554A (en) | Indoor sport scene simulation system and method are realized based on virtual reality | |
| CN105653020A (en) | Time traveling method and apparatus and glasses or helmet using same | |
| CN114708408A (en) | Experience system for virtual reality and meta-universe scene building in water | |
| WO2017061890A1 (en) | Wireless full body motion control sensor | |
| KR101618004B1 (en) | Interactive content providing apparatus based on the virtual reality and method thereof | |
| CN107632702B (en) | Holographic projection system adopting light-sensing data gloves and working method thereof | |
| CN211536496U (en) | Virtual reality system based on games | |
| CN207249607U (en) | A kind of three-dimension virtual reality system | |
| He | Virtual reality for budget smartphones |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| DD01 | Delivery of document by public notice |
Addressee: Water power (Beijing) Network Technology Co., Ltd. Document name: Notification of Publication of the Application for Invention |
|
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| DD01 | Delivery of document by public notice |
Addressee: Wei Bin Document name: Notification of Passing Examination on Formalities |
|
| DD01 | Delivery of document by public notice |
Addressee: Wei Bin Document name: the First Notification of an Office Action |
|
| DD01 | Delivery of document by public notice |
Addressee: Wei Bin Document name: Notification that Application Deemed not to be Proposed |
|
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20130710 |
|
| RJ01 | Rejection of invention patent application after publication |