WO2015017291A1 - Near-eye optic positioning in display devices - Google Patents
Near-eye optic positioning in display devices Download PDFInfo
- Publication number
- WO2015017291A1 WO2015017291A1 PCT/US2014/048337 US2014048337W WO2015017291A1 WO 2015017291 A1 WO2015017291 A1 WO 2015017291A1 US 2014048337 W US2014048337 W US 2014048337W WO 2015017291 A1 WO2015017291 A1 WO 2015017291A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- eye
- user
- positioning light
- light
- head
- Prior art date
Links
- 238000000034 method Methods 0.000 claims abstract description 25
- 230000000007 visual effect Effects 0.000 claims description 3
- 210000001747 pupil Anatomy 0.000 description 30
- 230000003287 optical effect Effects 0.000 description 29
- 238000001514 detection method Methods 0.000 description 17
- 238000005286 illumination Methods 0.000 description 17
- 238000004891 communication Methods 0.000 description 9
- 230000007246 mechanism Effects 0.000 description 6
- 230000006870 function Effects 0.000 description 5
- 239000000853 adhesive Substances 0.000 description 4
- 230000001070 adhesive effect Effects 0.000 description 4
- 230000008859 change Effects 0.000 description 4
- 239000003086 colorant Substances 0.000 description 4
- 230000008447 perception Effects 0.000 description 4
- 230000003190 augmentative effect Effects 0.000 description 3
- 239000008186 active pharmaceutical agent Substances 0.000 description 2
- 239000011248 coating agent Substances 0.000 description 2
- 238000000576 coating method Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 210000003128 head Anatomy 0.000 description 2
- 238000003384 imaging method Methods 0.000 description 2
- 230000002093 peripheral effect Effects 0.000 description 2
- 230000008569 process Effects 0.000 description 2
- 210000003786 sclera Anatomy 0.000 description 2
- 101000822695 Clostridium perfringens (strain 13 / Type A) Small, acid-soluble spore protein C1 Proteins 0.000 description 1
- 101000655262 Clostridium perfringens (strain 13 / Type A) Small, acid-soluble spore protein C2 Proteins 0.000 description 1
- 101000655256 Paraclostridium bifermentans Small, acid-soluble spore protein alpha Proteins 0.000 description 1
- 101000655264 Paraclostridium bifermentans Small, acid-soluble spore protein beta Proteins 0.000 description 1
- XUIMIQQOPSSXEZ-UHFFFAOYSA-N Silicon Chemical compound [Si] XUIMIQQOPSSXEZ-UHFFFAOYSA-N 0.000 description 1
- 238000004458 analytical method Methods 0.000 description 1
- 210000003484 anatomy Anatomy 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000003491 array Methods 0.000 description 1
- 230000007177 brain activity Effects 0.000 description 1
- 230000001419 dependent effect Effects 0.000 description 1
- 210000005069 ears Anatomy 0.000 description 1
- 230000000694 effects Effects 0.000 description 1
- 230000005684 electric field Effects 0.000 description 1
- 238000005516 engineering process Methods 0.000 description 1
- 239000004973 liquid crystal related substance Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 230000010287 polarization Effects 0.000 description 1
- 230000000644 propagated effect Effects 0.000 description 1
- 230000004044 response Effects 0.000 description 1
- 238000013515 script Methods 0.000 description 1
- 239000004065 semiconductor Substances 0.000 description 1
- 229910052710 silicon Inorganic materials 0.000 description 1
- 239000010703 silicon Substances 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
- 230000026683 transduction Effects 0.000 description 1
- 238000010361 transduction Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/016—Input arrangements with force or tactile feedback as computer generated output to the user
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
- G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
- G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
- G06V40/18—Eye characteristics, e.g. of the iris
- G06V40/19—Sensors therefor
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0123—Head-up displays characterised by optical features comprising devices increasing the field of view
- G02B2027/0125—Field-of-view increase by wavefront division
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0127—Head-up displays characterised by optical features comprising devices increasing the depth of field
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B2027/0178—Eyeglass type
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0185—Displaying image at variable distance
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
Definitions
- Near-eye display devices are configured to present images to a user via a display that is positioned close to the user's eyes.
- a see-through head-mounted augmented reality display device may be worn on a user's head to position a near-eye display directly in front of a user's eyes.
- An appearance of objects displayed on the near- eye display may be altered responsive to changes in distance between the user's eyes and the near-eye optics of the displays.
- Embodiments are disclosed herein that relate to aligning a near-eye display of a see-through head-mounted display device with an eye of a user.
- a method comprising directing positioning light to an eye of a user, and detecting the positioning light reflected from the eye of the user.
- the method further comprises determining a distance between the eye and a near-eye optic of the head-mounted display system based on attributes of the detected positioning light in order to determine whether the near-eye optic is misaligned with a target optic location.
- the method includes providing feedback for adjusting the distance between the eye and the near-eye optic in order to assist the user in positioning the near-eye optic in the target optic location.
- FIG. 1 shows a see-through head-mounted display device as worn by a user in accordance with an embodiment of the present disclosure.
- FIG. 2 schematically shows the see-through head-mounted display device of FIG. 1 in accordance with an embodiment of the present disclosure.
- FIG. 3 schematically shows a see-through display optic and optical paths for directing positioning light to an eye of a user in accordance with an embodiment of the present disclosure.
- FIG. 4 schematically shows a see-through display optic and optical paths for receiving positioning light at one or more cameras in accordance with an embodiment of the present disclosure.
- FIG. 5A schematically shows a see-through display and optical paths for directing display light and gaze-detection light to an eye of a user in accordance with an embodiment of the present disclosure.
- FIG. 5B schematically shows a see-through display and optical paths for receiving gaze-detection light at a camera in accordance with an embodiment of the present disclosure.
- FIG. 6 shows a flow diagram of a method of detecting eye location in a head- mounted display system in accordance with an embodiment of the present disclosure.
- FIG. 7 shows a block diagram of a computing system in accordance with an embodiment of the present disclosure.
- a see-through head-mounted display device including a near-eye display may use various optical systems to deliver an image to a user's eye, including but not limited to projection-based systems and waveguide-based systems.
- the optical systems of such near-eye displays may have relatively small exit pupils. Further, in some near-eye displays, optical performance may decay toward the edge of the exit pupil.
- a near-eye display device may include an adjustable fit system to allow a user to properly locate the exit pupil of the system. This may allow a user to adjust the system to avoid optical effects caused by misalignment.
- the proper adjustment of such a fit system may pose challenges for users. As a result, some users may perform sufficient fit adjustments to find a coarse fit that provides an acceptable level of performance, and then not perform additional adjustment to further optimize viewing. Thus, such viewers may not enjoy the full viewing experience offered by the device.
- embodiments are disclosed herein that relate to assisting users in adjusting a see-through head-mounted display device.
- the disclosed embodiments determine from image data a distance between the location of an eye of a user and a near- eye optic of a see-through head-mounted display device, and determine an adjustment to make to the see-through head-mounted display device that aligns the near-eye optic with a target optic location.
- the determined adjustment may be performed automatically and/or output as a recommendation for the user to perform manually. This may help to simplify adjusting the near-eye display system to more precisely align the see-through head- mounted display device with the user's eye or eyes.
- reference herein to a location of an eye may signify a location of the overall eye structure, the pupil of the eye, and/or any other anatomical feature of the eye.
- FIG. 1 shows an example embodiment of a near-eye display system in the form of a see-through head-mounted display device 100 worn by a user 102.
- the head-mounted display device 100 may be used, for example, to display augmented reality images via a see-through display in which displayed virtual objects are viewable along with physical objects in a real-world background scene. While described in the context of a head-mounted display device, it will be understood that the disclosed embodiments may be used with any other suitable near-eye display device. As discussed above, misalignment of the display optics of the head-mounted display device with the user's eye may result in vignetting of the field of view and other optical effects.
- the see-through head-mounted display device 100 may include a frame 104 to enable the user to position the device at the target viewing location.
- the target viewing location may correspond to a region in space inside of which an eye may properly perceive displayed images.
- FIG. 2 shows a schematic depiction of the see-through head-mounted display device 100 of FIG. 1.
- the depicted see-through head-mounted display device 100 includes a right see-through display 200a and a left see-through display 200b.
- Each see-through display may be configured to both display images to a user and allow a user to view the real-world environment.
- each see-through display may include a display device configured to emit display light through an optical configuration to an eye of a user. The optical configuration may also allow light from the real-world environment and/or positioning light to reach the eye of the user.
- the positioning light may enable the user and/or the see-through head-mounted display device to locate a position of the eye in order to provide feedback regarding adjustments that may be made to position the see-through displays at a target viewing location.
- FIG. 2 schematically illustrates speakers 202 that may be used to output acoustic information to a user, such as audio feedback relating to the adjustments and/or eye positioning described above.
- acoustic information may take any suitable form, including but not limited to a computer- generated voice output in an appropriate language (e.g. as selected by a user), tones or other sounds that are not specific to any language, and/or any other suitable sounds.
- other types of outputs may be provided by the see-through head- mounted display device 100, such as haptic/tactile outputs to assist the user in adjusting the see-through head-mounted display device.
- the see-through head-mounted display device 100 includes a right eye camera 204a and a left eye camera 204b, schematically illustrated in FIG. 2.
- the cameras 204a and 204b may be configured to capture images of each eye of a user for determining a gaze direction.
- the eye cameras 204a and 204b may receive gaze-detection light (e.g., from a glint light source) and/or positioning light (e.g., from one or more positioning light sources) as reflected off of the eyes of the user.
- gaze-detection light e.g., from a glint light source
- positioning light e.g., from one or more positioning light sources
- the eye cameras 204a and 204b may track a gaze of a user and/or determine a relative position of the eye to optics of the see-through head-mounted display device 100 based on the position and movement of the reflected gaze-detection and/or positioning light. If the relative position of the eye to optics of the see-through head-mounted display device 100 is determined to be misaligned with respect to the eye, the see-through head-mounted display device may provide feedback to the user to inform the user of the misalignment. In some embodiments, the see-through head-mounted display device may determine an adjustment that may be made to align the see-through head- mounted display device with the eye and perform the adjustment automatically or recommend the adjustment to the user.
- the head-mounted display device 100 may determine an adjustment to perform or recommend in any suitable manner.
- the head-mounted display system may determine an offset of the user's eye (or pupil or other anatomical feature of the user's eye) from the target viewing position for that eye, and may output a recommendation based upon a known or determined relationship between operation of an adjustment mechanism and a change in the location of the user's eye relative to the target viewing position as a function of the operation of the adjustment mechanism.
- Any suitable adjustment may be recommended and/or performed.
- some devices may offer multiple adjustment mechanisms (horizontal, vertical, angular, etc.). In such devices, multiple recommendations may be output, or multiple adjustments performed, in some situations, depending upon the adjustments to be made.
- the see-through displays 200a and 200b and the eye cameras 204a and 204b may be positioned at a viewing location relative to the eye via one or more securing mechanisms of the frame 104.
- the frame 104 may be supported by ears of a user via ear pieces 206 and by the nose of a user via nose bridge 208 to reduce slippage of the frame 104.
- supports such as the ear pieces 206, nose pieces, and nose bridge 208, illustrated in FIG. 2 are exemplary in nature, and the see-through displays of the head-mounted see-through display device may be positioned at the viewing location via any suitable mechanism.
- additional supports may be utilized, and/or one or more of the supports illustrated in FIG. 2 may be removed, replaced, and/or augmented to position the see-through displays at the viewing location.
- see-through displays may be positioned in a viewing location by mechanisms other than supports that physically touch a user without departing from the scope of this disclosure.
- FIG. 3 schematically shows an example of a see-through head-mounted display system 300 for assisting a user in adjusting a position of one or more components of the see-through head-mounted display system to a target location relative to an eye 302 of a user 304.
- the see-through head-mounted display system 300 may include one or more optical elements for directing display light and/or positioning light to the eye 302 of the user 304.
- the optical elements may include a focusing element 306, such as a freeform prism, and a beam splitter 308, such as a polarizing beam splitter and/or hot mirror. It will be understood that separate see-through display systems 300 may be used for left eye and right eye displays, as shown in FIG. 2.
- the see-through head-mounted display system 300 includes any suitable number of positioning light sources 310 for directing positioning light toward the eye 302 of the user 304.
- the see-through head-mounted display system 300 may include two or more spatially and/or angularly separated positioning light sources 310.
- the positioning light source(s) 310 may be positioned in any suitable location relative to other elements of the see-through head-mounted display system 300.
- positioning light source(s) 310 may be positioned substantially adjacent to or otherwise near a micro-display 312 and/or an eye-tracking camera 314.
- the positioning light source(s) 310 may emit positioning light L P (E) from the same or a substantially similar side of an optical element (e.g., beam splitter 308 and/or an associated prism) as the micro-display 312 and/or eye-tracking camera 314.
- Each of the positioning light source(s) 310 may be configured to emit visible and/or infrared light through a narrow exit pupil of the light sources to direct light to a particular location relative to the user's eye 302.
- one or more of the positioning light sources 310 may be configured to emit visible light to enable the user 304 to determine correct positioning based on a perceived attribute of the positioning light L P (E).
- a numerical aperture of the positioning light sources 310 may be controlled by adjusting an aperture stop near the positioning light sources or by using a light source with a well-defined cone angle (e.g., a laser light focused by a lens).
- the numerical aperture of the positioning light sources 310 may be controlled to enable light to reach a target location of a user's pupil relative to the display of the see-through head-mounted display system 300. Accordingly, the user 304 may determine that an adjustment is to be made to the display to position the display in the target viewing location if the positioning light or a portion of the positioning light is not perceived.
- each of the positioning light sources 310 may be configured to emit any suitable pattern of light (e.g., ring, a point, a line, etc.) detectable by the user.
- the positioning light emitted from one of the positioning light sources 310 may be a different color than positioning light emitted from another of the positioning light sources 310 such that a user may determine a relative location of perceived light from each source, indicating a type of adjustment to be made to the position of the see-through head- mounted display system.
- the emitted colors may also be configured to produce a different color when combined (e.g., when the user perceives the positioning light from the target pupil location) to indicate correct positioning of the see-through head-mounted display system to the eye.
- the positioning light sources 310 may continue emitting positioning light Lp(E) throughout adjustment in order to provide real-time feedback relating to the success of positional adjustments. For example, if a user detects no change or an increased divergence in the positioning light while moving the see-through display away from the eye, the user may determine that movement toward the eye will position the head-mounted display device at the target viewing location.
- Positioning light L P (E) may alternatively be directed outside of a target pupil location, such that perception of the positioning light indicates that adjustment of the head- mounted display device is recommended.
- Positioning light sources that are configured to direct positioning light outside of a target pupil location may additionally be used along with positioning light sources that are configured to direct positioning light to a target pupil location.
- the positioning light sources may be configured to emit positioning light having different colors dependent upon the location at which the light is directed. For example, positioning light directed outside of a target pupil location may have a first color, while positioning light directed to a target pupil location may have a second, different color. Accordingly, the perception of the first color by the user may indicate misalignment of the head-mounted display device.
- the positioning light sources 310 may be configured to emit a gradient of colors such that a saturation and/or hue of the perceived light may indicate an extent of misalignment. For example, positioning light directed to the target pupil location may have a yellow hue, positioning light directed just outside of the target pupil location may have an orange hue, and positioning light directed furthest away from the target pupil location may have a red hue. It is to be understood that any number and combination of colors and directed locations of positioning light may be utilized to inform the user of the presence and/or degree of misalignment of the head- mounted display device.
- Precision of the alignment system described above may be affected by the number of positioning light sources and the size of the user's pupil.
- the precision of the alignment system may increase with an increase in the number of positioning light sources and/or an increase in pupil size.
- a luminance of one or more of the positioning light sources and/or any other illumination light source may be increased.
- the positioning light source(s) 310 may be configured to emit visible light to enable a user to determine whether the head-mounted display device is misaligned based on perceived attributes of the visible positioning light.
- the positioning light source(s) 310 may emit visible and/or infrared light for detection at an eye -tracking camera 314 or other imaging device to determine a position of the user's eye 302 relative to a target position.
- the one or more positioning light sources 310 may be configured to illuminate the eye 302 such that the eye-tracking camera 314 may image the eye 302 and/or a pattern formed by positioning light L P (E) on the eye.
- the positioning light sources 310 may be configured to direct light onto the sclera as an addition or alternative to the pupil of the eye 302 in order to provide a more reflective surface for the positioning light L P .
- the eye-tracking camera 314 may image the reflected positioning light Lp(R) to determine attributes of the reflected positioning light L P (R) indicative of misalignment of the head-mounted display system 300 relative to the eye 302.
- a numerical aperture of two or more of the positioning light sources 310 may be controlled to direct two or more beams of positioning light and/or one or more positioning light patterns such that the positioning light converges onto a sclera of the user's eye 302. If the reflected beams and/or light patterns imaged by the eye-tracking camera 314 converge, the head-mounted display system 300 may be determined to be positioned in the target viewing location.
- the head-mounted display system 300 may be determined to be positioned outside of the target viewing area (e.g., too close or too far away from the user's eye 302).
- the head-mounted display system 300 may provide feedback for the user regarding an adjustment of the head-mounted display system.
- the micro-display 312 may be driven to display instructions to the user through the head-mounted display system 300 for moving the head-mounted display system in relation to the user's eye 302.
- the head-mounted display system 300 may provide feedback relating to misalignment and/or adjustment of the head-mounted display system for each eye and associated display (e.g., displays 200a and 200b of FIG. 2).
- feedback for each eye may be presented on a respective display (e.g., feedback for a left eye may be presented on a left display, while feedback for a right eye may be presented on a right display).
- Feedback via a display device may be provided simultaneously or sequentially for each eye.
- feedback may be provided via one or more speakers and/or haptic feedback devices of the head-mounted display system in addition to or as an alternative to feedback displayed on the display device(s).
- the feedback may provide any suitable level of detail regarding the misalignment, including whether or not the head-mounted display system is misaligned, a degree of misalignment of the head-mounted display system, and/or a manner in which the head-mounted display system may be adjusted to align the head-mounted display system with a target viewing location.
- FIG. 4 schematically shows an embodiment of an eye-positioning system of a see-through head-mounted display system 400 including a plurality of eye-tracking cameras 402a and 402b and a positioning light source 404.
- a positioning light source 404 may be included in the see-through head-mounted display system 400.
- the location of the eye 302 may be determined by evaluating the angles of incidence of light reflected from the eye 302 on each of the eye-tracking camera 402a and 402b (e.g., via triangulation). The relative position of the eye 302 with respect to the eye-tracking cameras 402a and 402b may be compared to a target position to determine whether the head-mounted display system 400 is misaligned.
- the see-through display 500 includes an optical configuration for directing display light, gaze-detection light, and/or positioning light to an eye 302 of a user 304.
- the see-through display 500 includes a display subsystem, shown schematically at 506, configured to produce an image for display to user 304.
- the display subsystem 506 may correspond to the micro-display 312 of FIGS. 3 and 4.
- the display subsystem 506 may comprise any suitable components for producing images for display, including but not limited to a micro-display and one or more light sources.
- Display light, LD, from display subsystem 506 travels along the display optical path (indicated by rays originating at the display subsystem 506) to reach the user's eye 302. It will be understood that separate see-through displays 500 may be used for left eye and right eye displays, as shown in FIG. 2.
- the display subsystem 506 may direct light through an illumination prism 508 including a beam splitter 510 (e.g., a polarizing beam splitter or hot mirror) configured to transmit the display light from the display device.
- a beam splitter 510 e.g., a polarizing beam splitter or hot mirror
- the beam splitter 510 may be configured to pass visible light while reflecting infrared light.
- the display subsystem 506 may include a reflective micro-display, such as a liquid crystal on silicon (LCoS) display.
- the display subsystem may include an emissive micro-display, such as organic light emitting diode (OLED) array display types, inorganic light emitting diode (iLED) array display types, and/or any other suitable micro- display.
- OLED organic light emitting diode
- iLED inorganic light emitting diode
- the beam splitter 510 may include a polarizing beam splitter, and an illumination light source 512 may be configured to emit illumination light, Li, into the illumination prism 508 (e.g., from an optical wedge 514 to the polarizing beam splitter).
- the illumination light source 512 may comprise one or more light sources, such as an RGB LED array, one or more white LEDs (e.g., with a color filter arrangement), and/or any suitable illumination light source configuration.
- the polarizing beam splitter splits the illumination light into beams of different polarization, the polarizing beam splitter may be configured to reflect a portion of the polarized illumination light toward the LCoS display for illuminating the display.
- the display may reflect the illumination light to generate the display light, LD, and the polarizing beam splitter may be configured to transmit a portion of the display light toward the eye 302 of the user 304.
- the see-through display 500 includes a freeform prism 516 for directing light from the illumination prism 508 to the eye 302 of the user 304.
- the freeform prism 516 may have positive optical power and comprise at least three surfaces, each surface being non-planar and non-spherical.
- the freeform prism 516 may include a user- facing surface 518 having a total internal reflective coating in order to direct the light into the user's eye 302.
- the freeform prism 516 may also include an outward-facing surface 520 opposite the user- facing total internal reflective surface, the outward-facing surface 520 having a coating that is highly reflective for infrared light and partially reflective for visible light.
- the outward-facing surface 520 may be more reflective for infrared light than for visible light.
- the outward-facing surface 520 may be configured to reflect substantially all infrared light and a portion of visible light.
- the see-through display 500 may include one or more compensators positioned around the freeform prism 516. As illustrated in FIG. 5 A, the see-through display 500 includes a user-facing compensator 522 positioned on an eye-side of the freeform prism and an outward-facing compensator 524 positioned on an environment-side of the freeform prism. Accordingly, the freeform prism 516 is positioned between the user- facing compensator 522 and the outward-facing compensator 524, such that light from the environment passes through the outward-facing compensator 524, then through the freeform prism 516, and then through the user-facing compensator 522 before reaching the eye 302.
- An eye-side surface 526 of the user-facing compensator 522 may be substantially parallel to an environment-side surface 528 of the outward-facing compensator 524 (e.g., the surface of the outward-facing compensator 524 that is furthest away from the eye 302).
- the compensators 522 and 524 are configured to have a substantially similar index of refraction as the freeform prism and to compensate for distortion of light from the real- world environment of the user 304 caused by the optical power of the freeform prism 516.
- the distortion compensation provided by the compensators 522 and 524 may be affected by the index of refraction of an adhesive used to couple the compensators to the freeform prism 516.
- adhesives having indices of refraction closer to that of the freeform prism 516 and/or compensators 522 and 524 may enable a greater amount of distortion compensation than adhesives having different indices of refraction. Similarly, thinner layers of adhesives may provide less distortion.
- the see-through display 500 also includes an eye tracking system comprising an imaging device, such as one or more eye tracking cameras 530, and one or more glint sources 532 (e.g. one or more infrared positioning light sources) configured to produce light for reflection from the user's eye.
- Emitted gaze-detection light, LG(E) from the glint sources 532 may travel along a gaze-detection optical path (indicated by rays originating at one of the glint sources 532) through at least a portion of the optics utilized for the display optical path.
- emitted gaze-detection light, LG(E), from the glint sources 532 may reflect from the user's eye 302, such that reflected gaze- detection light, LG(R), is returned to eye tracking camera 530 via a reverse gaze-detection optical path (e.g. along at least a portion of the gaze-detection optical path in a reverse direction).
- a reverse gaze-detection optical path e.g. along at least a portion of the gaze-detection optical path in a reverse direction.
- images from the user's eye are diverted away from the display subsystem 506 and toward the eye tracking camera 530 by the beam splitter 510 located immediately before the camera along the reverse display optical path.
- the eye tracking camera 530 images, or receives reflected gaze-detection light, LG(R), through a first side of the illumination prism 508 and the display subsystem 506 emits the display light, LD, through a second, different side of the illumination prism 508. Accordingly, the eye tracking camera 530 and the display subsystem 506 may interact with sides that are adjacent and/or substantially perpendicular to one another.
- the glint source 532 may emit the gaze-detection light, LG(E), through the first side of the illumination prism 508.
- the glint source may be positioned on a substantially similar side of the illumination prism as the eye tracking camera 530.
- the glint source 532 may emit the gaze-detection light, LG(E), through the second side of the illumination prism 508 and/or from any other suitable location.
- the optical paths may take any other suitable form.
- the eye tracking system may detect a location of the eye and/or anatomical structures thereof (e.g. a pupil of the eye 302).
- the eye tracking system may also detect a location of reflections from glint sources 532 in the image data acquired via eye tracking camera 530, and from this information determine a direction in which the eye is gazing.
- the eye tracking camera 530 may detect attributes of positioning light emitted from one or more positioning light sources 534. As depicted in FIGS. 5A and 5B, the positioning light sources 534 may occupy any of a plurality of optional locations relative to the display subsystem 506 and/or the eye tracking camera 530.
- positioning light sources 534 may additionally or alternatively occupy other locations within the see-through display 500.
- the positioning light sources 534 may direct positioning light through both the illumination prism 508 and the freeform prism 516 to the eye 302 of the user 304.
- FIG. 6 is a flow chart of a method 600 for detecting eye location in a head- mounted display system.
- elements of method 600 may be performed using the see-through head-mounted display system 300 of FIG. 3 and/or the see-through head- mounted display system 400 of FIG. 4.
- the method 600 includes directing positioning light to an eye of a user, as indicated at 602.
- the direction of light to the eye of the user may include controlling an aperture of a positioning light source to direct the positioning light to a selected location at 604.
- the selected location may optionally be a target pupil location.
- positioning light may be directed to a target pupil location so that the user may detect the positioning light when the user's eye is positioned in a correct location relative to the head-mounted display system.
- the positioning light may be additionally or alternatively be directed to a target pupil location to indicate a target location of a pupil for analysis by an eye-tracking camera.
- the selected location may optionally be outside of a target pupil location.
- positioning light may be directed outside of a target pupil location so that the user may detect the positioning light when the user's eye is positioned in an incorrect location (e.g., the head-mounted display system is positioned outside of a target viewing location).
- the positioning light may be directed outside of the target pupil location in order to target a more reflective surface than the pupil for assisting the eye-tracking camera with positioning an eye of the user.
- the positioning light may optionally be detected as reflected from the eye by an eye-tracking camera at 610.
- the method 600 includes determining a distance between the eye and a near-eye optic of the head-mounted display system based on attributes of the detected positioning light at 612.
- the attributes of the detected positioning light may be determined based on user perception and/or detection of reflected positioning light by one or more eye tracking cameras.
- the attributes may include determining a distance between light from two or more light sources, as indicated at 614.
- the distance between the eye and the near-eye optic may also be determined by determining a location of the eye using eye tracking camera, as indicated at 616 and described above with respect to FIG. 4.
- the method 600 includes providing feedback for adjusting the distance between the eye and the near-eye optic at 618.
- feedback may be provided to the user to perform the adjustment if the distance between the positioning light from a first light source and the positioning light from a second light source as reflected from the eye is above a threshold.
- the feedback may optionally include indicating a direction of movement of the near-eye optic relative to the eye to adjust the near-eye optic to a target optic location, as indicated at 620.
- one or more speakers, displays, and/or haptic feedback devices of the head-mounted display system may output instructions to assist the user in adjusting the distance between the eye and the near-eye optic to achieve the target positioning.
- a distance between an eye of a user and an associated display of a see-through head-mounted display system may change.
- an appearance of objects displayed on the displays may be altered responsive to such changes in distance between the eye and the displays.
- the methods and systems described herein assist a user in determining misalignment of the see-through head-mounted display system and correcting said misalignment in order to provide a comfortable user experience.
- FIG. 7 schematically shows a non-limiting embodiment of a computing system 700, for example a computing system of head-mounted display device 100 of FIG. 1.
- Computing system 700 is shown in simplified form.
- Computing system 700 may take the form of one or more see-through head-mounted display systems, personal computers, server computers, tablet computers, home-entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phone), and/or other computing devices.
- Computing system 700 includes a logic machine 702 and a storage machine 704.
- Computing system 700 may optionally include a display subsystem 706, input subsystem 708, communication subsystem 710, and/or other components not shown in FIG. 7.
- Logic machine 702 includes one or more physical devices configured to execute instructions.
- the logic machine may be configured to execute machine- readable instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs.
- Such instructions may be implemented to perform a task, implement a data type, transform the state of one or more components, achieve a technical effect, or otherwise arrive at a desired result.
- the logic machine may include one or more processors configured to execute software instructions. Additionally or alternatively, the logic machine may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. Processors of the logic machine may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and/or distributed processing. Individual components of the logic machine optionally may be distributed among two or more separate devices, which may be remotely located and/or configured for coordinated processing. Aspects of the logic machine may be virtualized and executed by remotely accessible, networked computing devices configured in a cloud-computing configuration.
- Storage machine 704 includes one or more physical devices configured to hold machine-readable instructions executable by the logic machine to implement the methods and processes described herein. When such methods and processes are implemented, the state of storage machine 704 may be transformed— e.g., to hold different data.
- Storage machine 704 may include removable and/or built-in devices.
- Storage machine 704 may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), among others.
- Storage machine 704 may include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content- addressable devices.
- storage machine 704 includes one or more physical devices.
- aspects of the instructions described herein alternatively may be propagated by a communication medium (e.g., an electromagnetic signal, an optical signal, etc.) that is not held by a physical device for a finite duration.
- a communication medium e.g., an electromagnetic signal, an optical signal, etc.
- logic machine 702 and storage machine 704 may be integrated together into one or more hardware-logic components.
- Such hardware-logic components may include field-programmable gate arrays (FPGAs), program- and application-specific integrated circuits (PASIC / ASICs), program- and application-specific standard products (PSSP / ASSPs), system-on-a-chip (SOC), and complex programmable logic devices (CPLDs), for example.
- FPGAs field-programmable gate arrays
- PASIC / ASICs program- and application-specific integrated circuits
- PSSP / ASSPs program- and application-specific standard products
- SOC system-on-a-chip
- CPLDs complex programmable logic devices
- modules, programs, and/or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, etc.
- the same module, program, and/or engine may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc.
- the terms “module”, “program”, and “engine” may encompass individual or groups of executable files, data files, libraries, drivers, scripts, database records, etc.
- a “service”, as used herein, is an application program executable across multiple user sessions.
- a service may be available to one or more system components, programs, and/or other services.
- a service may run on one or more server-computing devices.
- display subsystem 706 may be used to present a visual representation of data held by storage machine 704.
- This visual representation may take the form of a graphical user interface (GUI).
- GUI graphical user interface
- display subsystem 706 may include one or more display devices utilizing virtually any type of technology.
- display subsystem 706 may correspond to one or more elements of the see-through displays 200a and 200b of FIG. 2.
- Such display devices may be combined with logic machine 702 and/or storage machine 704 in a shared enclosure, or such display devices may be peripheral display devices.
- input subsystem 708 may comprise or interface with one or more user-input devices such as a keyboard, mouse, touch screen, or game controller.
- the input subsystem may comprise or interface with selected natural user input (NUI) componentry.
- NUI natural user input
- Such componentry may be integrated or peripheral, and the transduction and/or processing of input actions may be handled on- or off-board.
- NUI componentry may include a microphone for speech and/or voice recognition; an infrared, color, stereoscopic, and/or depth camera for machine vision and/or gesture recognition; a head tracker, eye tracker, accelerometer, and/or gyroscope for motion detection and/or intent recognition; as well as electric-field sensing componentry for assessing brain activity.
- communication subsystem 710 may be configured to communicatively couple computing system 700 with one or more other computing devices.
- Communication subsystem 710 may include wired and/or wireless communication devices compatible with one or more different communication protocols.
- the communication subsystem may be configured for communication via a wireless telephone network, or a wired or wireless local- or wide- area network.
- the communication subsystem may allow computing system 700 to send and/or receive messages to and/or from other devices via a network such as the Internet.
Abstract
Embodiments are disclosed for adjusting alignment of a near-eye optic (306) of a see-through head-mounted display system (300). In one embodiment, a method of detecting eye location for a head-mounted display system (300) includes directing positioning light to an eye (302) of a user (304) and detecting the positioning light reflected from the eye (302) of the user (304). The method further includes determining a distance between the eye (302) and a near-eye optic (306) of the head-mounted display system (300) based on attributes of the detected positioning light, and providing feedback for adjusting the distance between the eye (302) and the near-eye optic (306).
Description
NEAR-EYE OPTIC POSITIONING IN DISPLAY DEVICES
BACKGROUND
[0001] Near-eye display devices are configured to present images to a user via a display that is positioned close to the user's eyes. For example, a see-through head-mounted augmented reality display device may be worn on a user's head to position a near-eye display directly in front of a user's eyes. An appearance of objects displayed on the near- eye display may be altered responsive to changes in distance between the user's eyes and the near-eye optics of the displays.
SUMMARY
[0002] Embodiments are disclosed herein that relate to aligning a near-eye display of a see-through head-mounted display device with an eye of a user. For example, one disclosed embodiment provides, on a see-through head-mounted display device, a method comprising directing positioning light to an eye of a user, and detecting the positioning light reflected from the eye of the user. The method further comprises determining a distance between the eye and a near-eye optic of the head-mounted display system based on attributes of the detected positioning light in order to determine whether the near-eye optic is misaligned with a target optic location. The method includes providing feedback for adjusting the distance between the eye and the near-eye optic in order to assist the user in positioning the near-eye optic in the target optic location.
[0003] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used to limit the scope of the claimed subject matter. Furthermore, the claimed subject matter is not limited to implementations that solve any or all disadvantages noted in any part of this disclosure.
BRIEF DESCRIPTION OF THE DRAWINGS
[0004] FIG. 1 shows a see-through head-mounted display device as worn by a user in accordance with an embodiment of the present disclosure.
[0005] FIG. 2 schematically shows the see-through head-mounted display device of FIG. 1 in accordance with an embodiment of the present disclosure.
[0006] FIG. 3 schematically shows a see-through display optic and optical paths for directing positioning light to an eye of a user in accordance with an embodiment of the present disclosure.
[0007] FIG. 4 schematically shows a see-through display optic and optical paths for receiving positioning light at one or more cameras in accordance with an embodiment of the present disclosure.
[0008] FIG. 5A schematically shows a see-through display and optical paths for directing display light and gaze-detection light to an eye of a user in accordance with an embodiment of the present disclosure.
[0009] FIG. 5B schematically shows a see-through display and optical paths for receiving gaze-detection light at a camera in accordance with an embodiment of the present disclosure.
[0010] FIG. 6 shows a flow diagram of a method of detecting eye location in a head- mounted display system in accordance with an embodiment of the present disclosure.
[0011] FIG. 7 shows a block diagram of a computing system in accordance with an embodiment of the present disclosure.
DETAILED DESCRIPTION
[0012] A see-through head-mounted display device including a near-eye display may use various optical systems to deliver an image to a user's eye, including but not limited to projection-based systems and waveguide-based systems. However, the optical systems of such near-eye displays may have relatively small exit pupils. Further, in some near-eye displays, optical performance may decay toward the edge of the exit pupil.
[0013] As such, a near-eye display device may include an adjustable fit system to allow a user to properly locate the exit pupil of the system. This may allow a user to adjust the system to avoid optical effects caused by misalignment. However, the proper adjustment of such a fit system may pose challenges for users. As a result, some users may perform sufficient fit adjustments to find a coarse fit that provides an acceptable level of performance, and then not perform additional adjustment to further optimize viewing. Thus, such viewers may not enjoy the full viewing experience offered by the device.
[0014] Accordingly, embodiments are disclosed herein that relate to assisting users in adjusting a see-through head-mounted display device. Briefly, the disclosed embodiments determine from image data a distance between the location of an eye of a user and a near- eye optic of a see-through head-mounted display device, and determine an adjustment to make to the see-through head-mounted display device that aligns the near-eye optic with a target optic location. The determined adjustment may be performed automatically and/or output as a recommendation for the user to perform manually. This may help to simplify adjusting the near-eye display system to more precisely align the see-through head-
mounted display device with the user's eye or eyes. It will be understood that reference herein to a location of an eye may signify a location of the overall eye structure, the pupil of the eye, and/or any other anatomical feature of the eye.
[0015] FIG. 1 shows an example embodiment of a near-eye display system in the form of a see-through head-mounted display device 100 worn by a user 102. The head-mounted display device 100 may be used, for example, to display augmented reality images via a see-through display in which displayed virtual objects are viewable along with physical objects in a real-world background scene. While described in the context of a head- mounted display device, it will be understood that the disclosed embodiments may be used with any other suitable near-eye display device. As discussed above, misalignment of the display optics of the head-mounted display device with the user's eye may result in vignetting of the field of view and other optical effects. Thus, for proper viewing, a user may adjust the head-mounted display to position the display at a target viewing position relative to the user's eyes. The see-through head-mounted display device 100 may include a frame 104 to enable the user to position the device at the target viewing location. For example, the target viewing location may correspond to a region in space inside of which an eye may properly perceive displayed images.
[0016] FIG. 2 shows a schematic depiction of the see-through head-mounted display device 100 of FIG. 1. The depicted see-through head-mounted display device 100 includes a right see-through display 200a and a left see-through display 200b. Each see-through display may be configured to both display images to a user and allow a user to view the real-world environment. For example, each see-through display may include a display device configured to emit display light through an optical configuration to an eye of a user. The optical configuration may also allow light from the real-world environment and/or positioning light to reach the eye of the user. For example, the positioning light may enable the user and/or the see-through head-mounted display device to locate a position of the eye in order to provide feedback regarding adjustments that may be made to position the see-through displays at a target viewing location. FIG. 2 schematically illustrates speakers 202 that may be used to output acoustic information to a user, such as audio feedback relating to the adjustments and/or eye positioning described above. Such acoustic information may take any suitable form, including but not limited to a computer- generated voice output in an appropriate language (e.g. as selected by a user), tones or other sounds that are not specific to any language, and/or any other suitable sounds. In some embodiments, other types of outputs may be provided by the see-through head-
mounted display device 100, such as haptic/tactile outputs to assist the user in adjusting the see-through head-mounted display device.
[0017] The see-through head-mounted display device 100 includes a right eye camera 204a and a left eye camera 204b, schematically illustrated in FIG. 2. The cameras 204a and 204b may be configured to capture images of each eye of a user for determining a gaze direction. For example, the eye cameras 204a and 204b may receive gaze-detection light (e.g., from a glint light source) and/or positioning light (e.g., from one or more positioning light sources) as reflected off of the eyes of the user. Utilizing the light reflected off of the eye, the eye cameras 204a and 204b may track a gaze of a user and/or determine a relative position of the eye to optics of the see-through head-mounted display device 100 based on the position and movement of the reflected gaze-detection and/or positioning light. If the relative position of the eye to optics of the see-through head- mounted display device 100 is determined to be misaligned with respect to the eye, the see-through head-mounted display device may provide feedback to the user to inform the user of the misalignment. In some embodiments, the see-through head-mounted display device may determine an adjustment that may be made to align the see-through head- mounted display device with the eye and perform the adjustment automatically or recommend the adjustment to the user.
[0018] The head-mounted display device 100 may determine an adjustment to perform or recommend in any suitable manner. For example, the head-mounted display system may determine an offset of the user's eye (or pupil or other anatomical feature of the user's eye) from the target viewing position for that eye, and may output a recommendation based upon a known or determined relationship between operation of an adjustment mechanism and a change in the location of the user's eye relative to the target viewing position as a function of the operation of the adjustment mechanism. Any suitable adjustment may be recommended and/or performed. For example, some devices may offer multiple adjustment mechanisms (horizontal, vertical, angular, etc.). In such devices, multiple recommendations may be output, or multiple adjustments performed, in some situations, depending upon the adjustments to be made.
[0019] The see-through displays 200a and 200b and the eye cameras 204a and 204b may be positioned at a viewing location relative to the eye via one or more securing mechanisms of the frame 104. For example, as illustrated in FIG. 2, the frame 104 may be supported by ears of a user via ear pieces 206 and by the nose of a user via nose bridge 208 to reduce slippage of the frame 104. It is to be understood that supports, such as the
ear pieces 206, nose pieces, and nose bridge 208, illustrated in FIG. 2 are exemplary in nature, and the see-through displays of the head-mounted see-through display device may be positioned at the viewing location via any suitable mechanism. For example, additional supports may be utilized, and/or one or more of the supports illustrated in FIG. 2 may be removed, replaced, and/or augmented to position the see-through displays at the viewing location. Furthermore, see-through displays may be positioned in a viewing location by mechanisms other than supports that physically touch a user without departing from the scope of this disclosure.
[0020] FIG. 3 schematically shows an example of a see-through head-mounted display system 300 for assisting a user in adjusting a position of one or more components of the see-through head-mounted display system to a target location relative to an eye 302 of a user 304. As described in more detail with respect to FIGS. 5A and 5B, the see-through head-mounted display system 300 may include one or more optical elements for directing display light and/or positioning light to the eye 302 of the user 304. The optical elements may include a focusing element 306, such as a freeform prism, and a beam splitter 308, such as a polarizing beam splitter and/or hot mirror. It will be understood that separate see-through display systems 300 may be used for left eye and right eye displays, as shown in FIG. 2.
[0021] The see-through head-mounted display system 300 includes any suitable number of positioning light sources 310 for directing positioning light toward the eye 302 of the user 304. For example, the see-through head-mounted display system 300 may include two or more spatially and/or angularly separated positioning light sources 310. The positioning light source(s) 310 may be positioned in any suitable location relative to other elements of the see-through head-mounted display system 300. For example, as illustrated in FIG. 3, positioning light source(s) 310 may be positioned substantially adjacent to or otherwise near a micro-display 312 and/or an eye-tracking camera 314. Accordingly, the positioning light source(s) 310 may emit positioning light LP(E) from the same or a substantially similar side of an optical element (e.g., beam splitter 308 and/or an associated prism) as the micro-display 312 and/or eye-tracking camera 314. Each of the positioning light source(s) 310 may be configured to emit visible and/or infrared light through a narrow exit pupil of the light sources to direct light to a particular location relative to the user's eye 302.
[0022] In some embodiments, one or more of the positioning light sources 310 may be configured to emit visible light to enable the user 304 to determine correct positioning
based on a perceived attribute of the positioning light LP(E). For example, a numerical aperture of the positioning light sources 310 may be controlled by adjusting an aperture stop near the positioning light sources or by using a light source with a well-defined cone angle (e.g., a laser light focused by a lens). The numerical aperture of the positioning light sources 310 may be controlled to enable light to reach a target location of a user's pupil relative to the display of the see-through head-mounted display system 300. Accordingly, the user 304 may determine that an adjustment is to be made to the display to position the display in the target viewing location if the positioning light or a portion of the positioning light is not perceived.
[0023] In embodiments in which multiple positioning light sources are utilized, visible light may be emitted from two different angles configured to converge at a target pupil location. Accordingly, the user 304 may determine that an adjustment is to be made to the display if the positioning light is not perceived and/or if multiple points of light are perceived. In such examples, each of the positioning light sources 310 may be configured to emit any suitable pattern of light (e.g., ring, a point, a line, etc.) detectable by the user. The positioning light emitted from one of the positioning light sources 310 may be a different color than positioning light emitted from another of the positioning light sources 310 such that a user may determine a relative location of perceived light from each source, indicating a type of adjustment to be made to the position of the see-through head- mounted display system. The emitted colors may also be configured to produce a different color when combined (e.g., when the user perceives the positioning light from the target pupil location) to indicate correct positioning of the see-through head-mounted display system to the eye. The positioning light sources 310 may continue emitting positioning light Lp(E) throughout adjustment in order to provide real-time feedback relating to the success of positional adjustments. For example, if a user detects no change or an increased divergence in the positioning light while moving the see-through display away from the eye, the user may determine that movement toward the eye will position the head-mounted display device at the target viewing location.
[0024] Positioning light LP(E) may alternatively be directed outside of a target pupil location, such that perception of the positioning light indicates that adjustment of the head- mounted display device is recommended. Positioning light sources that are configured to direct positioning light outside of a target pupil location may additionally be used along with positioning light sources that are configured to direct positioning light to a target pupil location. In such embodiments, the positioning light sources may be configured to
emit positioning light having different colors dependent upon the location at which the light is directed. For example, positioning light directed outside of a target pupil location may have a first color, while positioning light directed to a target pupil location may have a second, different color. Accordingly, the perception of the first color by the user may indicate misalignment of the head-mounted display device. The positioning light sources 310 may be configured to emit a gradient of colors such that a saturation and/or hue of the perceived light may indicate an extent of misalignment. For example, positioning light directed to the target pupil location may have a yellow hue, positioning light directed just outside of the target pupil location may have an orange hue, and positioning light directed furthest away from the target pupil location may have a red hue. It is to be understood that any number and combination of colors and directed locations of positioning light may be utilized to inform the user of the presence and/or degree of misalignment of the head- mounted display device.
[0025] Precision of the alignment system described above may be affected by the number of positioning light sources and the size of the user's pupil. In particular, the precision of the alignment system may increase with an increase in the number of positioning light sources and/or an increase in pupil size. In order to increase the pupil size of the user, a luminance of one or more of the positioning light sources and/or any other illumination light source may be increased.
[0026] As discussed above, the positioning light source(s) 310 may be configured to emit visible light to enable a user to determine whether the head-mounted display device is misaligned based on perceived attributes of the visible positioning light. In additional or alternative embodiments, the positioning light source(s) 310 may emit visible and/or infrared light for detection at an eye -tracking camera 314 or other imaging device to determine a position of the user's eye 302 relative to a target position. For example, the one or more positioning light sources 310 may be configured to illuminate the eye 302 such that the eye-tracking camera 314 may image the eye 302 and/or a pattern formed by positioning light LP(E) on the eye. In some embodiments, the positioning light sources 310 may be configured to direct light onto the sclera as an addition or alternative to the pupil of the eye 302 in order to provide a more reflective surface for the positioning light LP.
[0027] Similarly to the embodiments discussed above regarding user perception of the positioning light, the eye-tracking camera 314 may image the reflected positioning light Lp(R) to determine attributes of the reflected positioning light LP(R) indicative of misalignment of the head-mounted display system 300 relative to the eye 302. For
example, a numerical aperture of two or more of the positioning light sources 310 may be controlled to direct two or more beams of positioning light and/or one or more positioning light patterns such that the positioning light converges onto a sclera of the user's eye 302. If the reflected beams and/or light patterns imaged by the eye-tracking camera 314 converge, the head-mounted display system 300 may be determined to be positioned in the target viewing location. Conversely, if multiple points of light and/or a non-converging pattern is imaged by the eye -tracking camera 314, the head-mounted display system 300 may be determined to be positioned outside of the target viewing area (e.g., too close or too far away from the user's eye 302).
[0028] In response to detected misalignment, the head-mounted display system 300 may provide feedback for the user regarding an adjustment of the head-mounted display system. For example, the micro-display 312 may be driven to display instructions to the user through the head-mounted display system 300 for moving the head-mounted display system in relation to the user's eye 302. As one eye may be closer to the head-mounted display than another, the head-mounted display system 300 may provide feedback relating to misalignment and/or adjustment of the head-mounted display system for each eye and associated display (e.g., displays 200a and 200b of FIG. 2). For example, feedback for each eye may be presented on a respective display (e.g., feedback for a left eye may be presented on a left display, while feedback for a right eye may be presented on a right display). Feedback via a display device may be provided simultaneously or sequentially for each eye.
[0029] As discussed above with respect to FIG. 2, feedback may be provided via one or more speakers and/or haptic feedback devices of the head-mounted display system in addition to or as an alternative to feedback displayed on the display device(s). The feedback may provide any suitable level of detail regarding the misalignment, including whether or not the head-mounted display system is misaligned, a degree of misalignment of the head-mounted display system, and/or a manner in which the head-mounted display system may be adjusted to align the head-mounted display system with a target viewing location.
[0030] FIG. 4 schematically shows an embodiment of an eye-positioning system of a see-through head-mounted display system 400 including a plurality of eye-tracking cameras 402a and 402b and a positioning light source 404. Although not illustrated in FIG. 4, it is to be understood that one or more additional positioning light sources may be included in the see-through head-mounted display system 400. As the distance between
eye-tracking cameras 402a and 402b is fixed, the location of the eye 302 may be determined by evaluating the angles of incidence of light reflected from the eye 302 on each of the eye-tracking camera 402a and 402b (e.g., via triangulation). The relative position of the eye 302 with respect to the eye-tracking cameras 402a and 402b may be compared to a target position to determine whether the head-mounted display system 400 is misaligned.
[0031] Turning now to FIG. 5A, an example embodiment of optical elements of a see- through display 500 is depicted. The see-through display 500 includes an optical configuration for directing display light, gaze-detection light, and/or positioning light to an eye 302 of a user 304. The see-through display 500 includes a display subsystem, shown schematically at 506, configured to produce an image for display to user 304. For example, the display subsystem 506 may correspond to the micro-display 312 of FIGS. 3 and 4. The display subsystem 506 may comprise any suitable components for producing images for display, including but not limited to a micro-display and one or more light sources. Display light, LD, from display subsystem 506 travels along the display optical path (indicated by rays originating at the display subsystem 506) to reach the user's eye 302. It will be understood that separate see-through displays 500 may be used for left eye and right eye displays, as shown in FIG. 2.
[0032] The display subsystem 506 may direct light through an illumination prism 508 including a beam splitter 510 (e.g., a polarizing beam splitter or hot mirror) configured to transmit the display light from the display device. For example, the beam splitter 510 may be configured to pass visible light while reflecting infrared light. In some embodiments, the display subsystem 506 may include a reflective micro-display, such as a liquid crystal on silicon (LCoS) display. In other embodiments, the display subsystem may include an emissive micro-display, such as organic light emitting diode (OLED) array display types, inorganic light emitting diode (iLED) array display types, and/or any other suitable micro- display. The beam splitter 510 may include a polarizing beam splitter, and an illumination light source 512 may be configured to emit illumination light, Li, into the illumination prism 508 (e.g., from an optical wedge 514 to the polarizing beam splitter). The illumination light source 512 may comprise one or more light sources, such as an RGB LED array, one or more white LEDs (e.g., with a color filter arrangement), and/or any suitable illumination light source configuration. As the polarizing beam splitter splits the illumination light into beams of different polarization, the polarizing beam splitter may be configured to reflect a portion of the polarized illumination light toward the LCoS display
for illuminating the display. The display may reflect the illumination light to generate the display light, LD, and the polarizing beam splitter may be configured to transmit a portion of the display light toward the eye 302 of the user 304.
[0033] The see-through display 500 includes a freeform prism 516 for directing light from the illumination prism 508 to the eye 302 of the user 304. The freeform prism 516 may have positive optical power and comprise at least three surfaces, each surface being non-planar and non-spherical. For example, the freeform prism 516 may include a user- facing surface 518 having a total internal reflective coating in order to direct the light into the user's eye 302. The freeform prism 516 may also include an outward-facing surface 520 opposite the user- facing total internal reflective surface, the outward-facing surface 520 having a coating that is highly reflective for infrared light and partially reflective for visible light. For example, the outward-facing surface 520 may be more reflective for infrared light than for visible light. In some embodiments, the outward-facing surface 520 may be configured to reflect substantially all infrared light and a portion of visible light.
[0034] The see-through display 500 may include one or more compensators positioned around the freeform prism 516. As illustrated in FIG. 5 A, the see-through display 500 includes a user-facing compensator 522 positioned on an eye-side of the freeform prism and an outward-facing compensator 524 positioned on an environment-side of the freeform prism. Accordingly, the freeform prism 516 is positioned between the user- facing compensator 522 and the outward-facing compensator 524, such that light from the environment passes through the outward-facing compensator 524, then through the freeform prism 516, and then through the user-facing compensator 522 before reaching the eye 302. An eye-side surface 526 of the user-facing compensator 522 (e.g., the surface of the user-facing compensator 522 that is closest to the eye 302) may be substantially parallel to an environment-side surface 528 of the outward-facing compensator 524 (e.g., the surface of the outward-facing compensator 524 that is furthest away from the eye 302). The compensators 522 and 524 are configured to have a substantially similar index of refraction as the freeform prism and to compensate for distortion of light from the real- world environment of the user 304 caused by the optical power of the freeform prism 516. The distortion compensation provided by the compensators 522 and 524 may be affected by the index of refraction of an adhesive used to couple the compensators to the freeform prism 516. Therefore, adhesives having indices of refraction closer to that of the freeform prism 516 and/or compensators 522 and 524 may enable a greater amount of distortion
compensation than adhesives having different indices of refraction. Similarly, thinner layers of adhesives may provide less distortion.
[0035] The see-through display 500 also includes an eye tracking system comprising an imaging device, such as one or more eye tracking cameras 530, and one or more glint sources 532 (e.g. one or more infrared positioning light sources) configured to produce light for reflection from the user's eye. Emitted gaze-detection light, LG(E), from the glint sources 532 may travel along a gaze-detection optical path (indicated by rays originating at one of the glint sources 532) through at least a portion of the optics utilized for the display optical path. Turning briefly to FIG. 5B, emitted gaze-detection light, LG(E), from the glint sources 532 may reflect from the user's eye 302, such that reflected gaze- detection light, LG(R), is returned to eye tracking camera 530 via a reverse gaze-detection optical path (e.g. along at least a portion of the gaze-detection optical path in a reverse direction). In the depicted example, images from the user's eye are diverted away from the display subsystem 506 and toward the eye tracking camera 530 by the beam splitter 510 located immediately before the camera along the reverse display optical path.
[0036] As shown in FIGS. 5A and 5B, the eye tracking camera 530 images, or receives reflected gaze-detection light, LG(R), through a first side of the illumination prism 508 and the display subsystem 506 emits the display light, LD, through a second, different side of the illumination prism 508. Accordingly, the eye tracking camera 530 and the display subsystem 506 may interact with sides that are adjacent and/or substantially perpendicular to one another. In some embodiments, the glint source 532 may emit the gaze-detection light, LG(E), through the first side of the illumination prism 508. Thus, the glint source may be positioned on a substantially similar side of the illumination prism as the eye tracking camera 530. In additional or alternative embodiments, the glint source 532 may emit the gaze-detection light, LG(E), through the second side of the illumination prism 508 and/or from any other suitable location. Although exemplary optical paths to the user's eye 302 and to the eye tracking camera 530 are depicted in FIGS. 5A and 5B, the optical paths may take any other suitable form.
[0037] The eye tracking system (e.g., the eye tracking camera 530 and/or the glint sources 532) may detect a location of the eye and/or anatomical structures thereof (e.g. a pupil of the eye 302). The eye tracking system may also detect a location of reflections from glint sources 532 in the image data acquired via eye tracking camera 530, and from this information determine a direction in which the eye is gazing. Furthermore, the eye tracking camera 530 may detect attributes of positioning light emitted from one or more
positioning light sources 534. As depicted in FIGS. 5A and 5B, the positioning light sources 534 may occupy any of a plurality of optional locations relative to the display subsystem 506 and/or the eye tracking camera 530. It is to be understood that the positioning light sources 534 may additionally or alternatively occupy other locations within the see-through display 500. The positioning light sources 534 may direct positioning light through both the illumination prism 508 and the freeform prism 516 to the eye 302 of the user 304.
[0038] FIG. 6 is a flow chart of a method 600 for detecting eye location in a head- mounted display system. For example, elements of method 600 may be performed using the see-through head-mounted display system 300 of FIG. 3 and/or the see-through head- mounted display system 400 of FIG. 4. The method 600 includes directing positioning light to an eye of a user, as indicated at 602. The direction of light to the eye of the user may include controlling an aperture of a positioning light source to direct the positioning light to a selected location at 604. As indicated at 606, the selected location may optionally be a target pupil location. For example, positioning light may be directed to a target pupil location so that the user may detect the positioning light when the user's eye is positioned in a correct location relative to the head-mounted display system. The positioning light may be additionally or alternatively be directed to a target pupil location to indicate a target location of a pupil for analysis by an eye-tracking camera. As indicated at 608, the selected location may optionally be outside of a target pupil location. For example, positioning light may be directed outside of a target pupil location so that the user may detect the positioning light when the user's eye is positioned in an incorrect location (e.g., the head-mounted display system is positioned outside of a target viewing location). Additionally or alternatively, the positioning light may be directed outside of the target pupil location in order to target a more reflective surface than the pupil for assisting the eye-tracking camera with positioning an eye of the user.
[0039] As discussed above, the positioning light may optionally be detected as reflected from the eye by an eye-tracking camera at 610. The method 600 includes determining a distance between the eye and a near-eye optic of the head-mounted display system based on attributes of the detected positioning light at 612. For example, the attributes of the detected positioning light may be determined based on user perception and/or detection of reflected positioning light by one or more eye tracking cameras. The attributes may include determining a distance between light from two or more light sources, as indicated at 614. The distance between the eye and the near-eye optic may also be determined by
determining a location of the eye using eye tracking camera, as indicated at 616 and described above with respect to FIG. 4.
[0040] Upon determining the distance, the method 600 includes providing feedback for adjusting the distance between the eye and the near-eye optic at 618. For example, feedback may be provided to the user to perform the adjustment if the distance between the positioning light from a first light source and the positioning light from a second light source as reflected from the eye is above a threshold. The feedback may optionally include indicating a direction of movement of the near-eye optic relative to the eye to adjust the near-eye optic to a target optic location, as indicated at 620. For example, one or more speakers, displays, and/or haptic feedback devices of the head-mounted display system may output instructions to assist the user in adjusting the distance between the eye and the near-eye optic to achieve the target positioning.
[0041] During use, a distance between an eye of a user and an associated display of a see-through head-mounted display system may change. As discussed above, an appearance of objects displayed on the displays may be altered responsive to such changes in distance between the eye and the displays. The methods and systems described herein assist a user in determining misalignment of the see-through head-mounted display system and correcting said misalignment in order to provide a comfortable user experience.
[0042] FIG. 7 schematically shows a non-limiting embodiment of a computing system 700, for example a computing system of head-mounted display device 100 of FIG. 1. Computing system 700 is shown in simplified form. Computing system 700 may take the form of one or more see-through head-mounted display systems, personal computers, server computers, tablet computers, home-entertainment computers, network computing devices, gaming devices, mobile computing devices, mobile communication devices (e.g., smart phone), and/or other computing devices.
[0043] Computing system 700 includes a logic machine 702 and a storage machine 704. Computing system 700 may optionally include a display subsystem 706, input subsystem 708, communication subsystem 710, and/or other components not shown in FIG. 7.
[0044] Logic machine 702 includes one or more physical devices configured to execute instructions. For example, the logic machine may be configured to execute machine- readable instructions that are part of one or more applications, services, programs, routines, libraries, objects, components, data structures, or other logical constructs. Such instructions may be implemented to perform a task, implement a data type, transform the
state of one or more components, achieve a technical effect, or otherwise arrive at a desired result.
[0045] The logic machine may include one or more processors configured to execute software instructions. Additionally or alternatively, the logic machine may include one or more hardware or firmware logic machines configured to execute hardware or firmware instructions. Processors of the logic machine may be single-core or multi-core, and the instructions executed thereon may be configured for sequential, parallel, and/or distributed processing. Individual components of the logic machine optionally may be distributed among two or more separate devices, which may be remotely located and/or configured for coordinated processing. Aspects of the logic machine may be virtualized and executed by remotely accessible, networked computing devices configured in a cloud-computing configuration.
[0046] Storage machine 704 includes one or more physical devices configured to hold machine-readable instructions executable by the logic machine to implement the methods and processes described herein. When such methods and processes are implemented, the state of storage machine 704 may be transformed— e.g., to hold different data.
[0047] Storage machine 704 may include removable and/or built-in devices. Storage machine 704 may include optical memory (e.g., CD, DVD, HD-DVD, Blu-Ray Disc, etc.), semiconductor memory (e.g., RAM, EPROM, EEPROM, etc.), and/or magnetic memory (e.g., hard-disk drive, floppy-disk drive, tape drive, MRAM, etc.), among others. Storage machine 704 may include volatile, nonvolatile, dynamic, static, read/write, read-only, random-access, sequential-access, location-addressable, file-addressable, and/or content- addressable devices.
[0048] It will be appreciated that storage machine 704 includes one or more physical devices. However, aspects of the instructions described herein alternatively may be propagated by a communication medium (e.g., an electromagnetic signal, an optical signal, etc.) that is not held by a physical device for a finite duration.
[0049] Aspects of logic machine 702 and storage machine 704 may be integrated together into one or more hardware-logic components. Such hardware-logic components may include field-programmable gate arrays (FPGAs), program- and application-specific integrated circuits (PASIC / ASICs), program- and application-specific standard products (PSSP / ASSPs), system-on-a-chip (SOC), and complex programmable logic devices (CPLDs), for example.
[0050] The terms "module", "program", and "engine" may be used to describe an aspect of computing system 700 implemented to perform a particular function. In some cases, a module, program, or engine may be instantiated via logic machine 702 executing instructions stored by storage machine 704. It will be understood that different modules, programs, and/or engines may be instantiated from the same application, service, code block, object, library, routine, API, function, etc. Likewise, the same module, program, and/or engine may be instantiated by different applications, services, code blocks, objects, routines, APIs, functions, etc. The terms "module", "program", and "engine" may encompass individual or groups of executable files, data files, libraries, drivers, scripts, database records, etc.
[0051] It will be appreciated that a "service", as used herein, is an application program executable across multiple user sessions. A service may be available to one or more system components, programs, and/or other services. In some implementations, a service may run on one or more server-computing devices.
[0052] When included, display subsystem 706 may be used to present a visual representation of data held by storage machine 704. This visual representation may take the form of a graphical user interface (GUI). As the herein described methods and processes change the data held by the storage machine, and thus transform the state of the storage machine, the state of display subsystem 706 may likewise be transformed to visually represent changes in the underlying data. Display subsystem 706 may include one or more display devices utilizing virtually any type of technology. For example, display subsystem 706 may correspond to one or more elements of the see-through displays 200a and 200b of FIG. 2. Such display devices may be combined with logic machine 702 and/or storage machine 704 in a shared enclosure, or such display devices may be peripheral display devices.
[0053] When included, input subsystem 708 may comprise or interface with one or more user-input devices such as a keyboard, mouse, touch screen, or game controller. In some embodiments, the input subsystem may comprise or interface with selected natural user input (NUI) componentry. Such componentry may be integrated or peripheral, and the transduction and/or processing of input actions may be handled on- or off-board. Example NUI componentry may include a microphone for speech and/or voice recognition; an infrared, color, stereoscopic, and/or depth camera for machine vision and/or gesture recognition; a head tracker, eye tracker, accelerometer, and/or gyroscope for motion
detection and/or intent recognition; as well as electric-field sensing componentry for assessing brain activity.
[0054] When included, communication subsystem 710 may be configured to communicatively couple computing system 700 with one or more other computing devices. Communication subsystem 710 may include wired and/or wireless communication devices compatible with one or more different communication protocols. As non-limiting examples, the communication subsystem may be configured for communication via a wireless telephone network, or a wired or wireless local- or wide- area network. In some embodiments, the communication subsystem may allow computing system 700 to send and/or receive messages to and/or from other devices via a network such as the Internet.
[0055] It will be understood that the configurations and/or approaches described herein are exemplary in nature, and that these specific embodiments or examples are not to be considered in a limiting sense, because numerous variations are possible. The specific routines or methods described herein may represent one or more of any number of processing strategies. As such, various acts illustrated and/or described may be performed in the sequence illustrated and/or described, in other sequences, in parallel, or omitted. Likewise, the order of the above-described processes may be changed.
[0056] The subject matter of the present disclosure includes all novel and non-obvious combinations and sub-combinations of the various processes, systems and configurations, and other features, functions, acts, and/or properties disclosed herein, as well as any and all equivalents thereof.
Claims
1. A method of detecting eye location for a head-mounted display system, the method comprising:
directing positioning light to an eye of a user;
detecting the positioning light reflected from the eye of the user;
determining a distance between the eye and a near-eye optic of the head-mounted display system based on attributes of the detected positioning light; and
providing feedback for adjusting the distance between the eye and the near-eye optic.
2. The method of claim 1, wherein the feedback for adjusting the distance between the eye and the near-eye optic includes visual feedback provided to the user via the near-eye optic.
3. The method of claim 1, wherein directing positioning light to the eye of the user includes directing positioning light from a first light source and a second light source to the eye.
4. The method of claim 3, wherein determining the distance between the eye and the near-eye optic of the head-mounted display system includes determining a distance between positioning light from the first light source and positioning light from the second light source as reflected from the eye.
5. The method of claim 4, further comprising providing feedback to the user to perform the adjustment if the distance between the positioning light from the first light source and the positioning light from the second light source as reflected from the eye is above a threshold.
6. The method of claim 1, wherein directing positioning light further comprises controlling a numerical aperture of a light source to direct light to a selected area of the eye of the user.
MS 338961.03
WO 2015/017291 PCT/US2014/048337
7. A head-mounted display system for detecting eye location, the head- mounted display system comprising:
a display device;
an eye-tracking camera;
a positioning light source;
a logic machine; and
a storage machine configured to store instructions executable by the logic machine to: direct positioning light to an eye of a user;
detect the positioning light reflected from the eye of the user;
determine a distance between the eye and a near-eye optic of the head-mounted display system based on attributes of the detected positioning light; and provide feedback for adjusting the distance between the eye and the near-eye optic.
8. The system of claim 7, wherein the positioning light source includes one or more infrared light sources.
9. The system of claim 7, wherein the positioning light source includes one or more visible light sources.
10. The system of claim 7, wherein the positioning light source includes two positioning light sources that are spatially and angularly separated from one another.
Priority Applications (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
CN201480042885.XA CN105531716B (en) | 2013-07-30 | 2014-07-28 | Near-to-eye optical positioning in display devices |
EP14750128.2A EP3028216B1 (en) | 2013-07-30 | 2014-07-28 | Near-eye optic positioning in display devices |
Applications Claiming Priority (4)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US201313954864A | 2013-07-30 | 2013-07-30 | |
US13/954,864 | 2013-07-30 | ||
US13/973,788 US10345903B2 (en) | 2013-07-30 | 2013-08-22 | Feedback for optic positioning in display devices |
US13/973,788 | 2013-08-22 |
Publications (1)
Publication Number | Publication Date |
---|---|
WO2015017291A1 true WO2015017291A1 (en) | 2015-02-05 |
Family
ID=52427201
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
PCT/US2014/048337 WO2015017291A1 (en) | 2013-07-30 | 2014-07-28 | Near-eye optic positioning in display devices |
Country Status (4)
Country | Link |
---|---|
US (1) | US10345903B2 (en) |
EP (1) | EP3028216B1 (en) |
CN (1) | CN105531716B (en) |
WO (1) | WO2015017291A1 (en) |
Cited By (42)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
KR20180007349A (en) * | 2015-03-03 | 2018-01-22 | 센소모토릭 인스트루멘츠 게젤샤프트 퓌어 이노바티브 센소릭 엠베하 | Head-mounted eye tracking device and method that provides drift-free eye tracking through a lens system |
US10089516B2 (en) | 2013-07-31 | 2018-10-02 | Digilens, Inc. | Method and apparatus for contact image sensing |
CN108885341A (en) * | 2015-11-25 | 2018-11-23 | 谷歌有限责任公司 | Eyes tracking based on prism |
US10145533B2 (en) | 2005-11-11 | 2018-12-04 | Digilens, Inc. | Compact holographic illumination device |
US10156681B2 (en) | 2015-02-12 | 2018-12-18 | Digilens Inc. | Waveguide grating device |
US10185154B2 (en) | 2011-04-07 | 2019-01-22 | Digilens, Inc. | Laser despeckler based on angular diversity |
US10209517B2 (en) | 2013-05-20 | 2019-02-19 | Digilens, Inc. | Holographic waveguide eye tracker |
US10216061B2 (en) | 2012-01-06 | 2019-02-26 | Digilens, Inc. | Contact image sensor using switchable bragg gratings |
US10234696B2 (en) | 2007-07-26 | 2019-03-19 | Digilens, Inc. | Optical apparatus for recording a holographic device and method of recording |
US10241330B2 (en) | 2014-09-19 | 2019-03-26 | Digilens, Inc. | Method and apparatus for generating input images for holographic waveguide displays |
US10330777B2 (en) | 2015-01-20 | 2019-06-25 | Digilens Inc. | Holographic waveguide lidar |
US10359736B2 (en) | 2014-08-08 | 2019-07-23 | Digilens Inc. | Method for holographic mastering and replication |
US10423222B2 (en) | 2014-09-26 | 2019-09-24 | Digilens Inc. | Holographic waveguide optical tracker |
US10437051B2 (en) | 2012-05-11 | 2019-10-08 | Digilens Inc. | Apparatus for eye tracking |
US10437064B2 (en) | 2015-01-12 | 2019-10-08 | Digilens Inc. | Environmentally isolated waveguide display |
US10459145B2 (en) | 2015-03-16 | 2019-10-29 | Digilens Inc. | Waveguide device incorporating a light pipe |
US10545346B2 (en) | 2017-01-05 | 2020-01-28 | Digilens Inc. | Wearable heads up displays |
US10591756B2 (en) | 2015-03-31 | 2020-03-17 | Digilens Inc. | Method and apparatus for contact image sensing |
US10642058B2 (en) | 2011-08-24 | 2020-05-05 | Digilens Inc. | Wearable data display |
US10670876B2 (en) | 2011-08-24 | 2020-06-02 | Digilens Inc. | Waveguide laser illuminator incorporating a despeckler |
US10678053B2 (en) | 2009-04-27 | 2020-06-09 | Digilens Inc. | Diffractive projection apparatus |
US10690916B2 (en) | 2015-10-05 | 2020-06-23 | Digilens Inc. | Apparatus for providing waveguide displays with two-dimensional pupil expansion |
US10690851B2 (en) | 2018-03-16 | 2020-06-23 | Digilens Inc. | Holographic waveguides incorporating birefringence control and methods for their fabrication |
US10732569B2 (en) | 2018-01-08 | 2020-08-04 | Digilens Inc. | Systems and methods for high-throughput recording of holographic gratings in waveguide cells |
US10859768B2 (en) | 2016-03-24 | 2020-12-08 | Digilens Inc. | Method and apparatus for providing a polarization selective holographic waveguide device |
US10890707B2 (en) | 2016-04-11 | 2021-01-12 | Digilens Inc. | Holographic waveguide apparatus for structured light projection |
US10914950B2 (en) | 2018-01-08 | 2021-02-09 | Digilens Inc. | Waveguide architectures and related methods of manufacturing |
US10942430B2 (en) | 2017-10-16 | 2021-03-09 | Digilens Inc. | Systems and methods for multiplying the image resolution of a pixelated display |
US10983340B2 (en) | 2016-02-04 | 2021-04-20 | Digilens Inc. | Holographic waveguide optical tracker |
CN112997106A (en) * | 2018-11-09 | 2021-06-18 | 脸谱科技有限责任公司 | Catadioptric and refractive optical structures for beam shaping |
US11307432B2 (en) | 2014-08-08 | 2022-04-19 | Digilens Inc. | Waveguide laser illuminator incorporating a Despeckler |
US11378732B2 (en) | 2019-03-12 | 2022-07-05 | DigLens Inc. | Holographic waveguide backlight and related methods of manufacturing |
US11402801B2 (en) | 2018-07-25 | 2022-08-02 | Digilens Inc. | Systems and methods for fabricating a multilayer optical structure |
US11442222B2 (en) | 2019-08-29 | 2022-09-13 | Digilens Inc. | Evacuated gratings and methods of manufacturing |
US11448937B2 (en) | 2012-11-16 | 2022-09-20 | Digilens Inc. | Transparent waveguide display for tiling a display having plural optical powers using overlapping and offset FOV tiles |
US11460621B2 (en) | 2012-04-25 | 2022-10-04 | Rockwell Collins, Inc. | Holographic wide angle display |
US11480788B2 (en) | 2015-01-12 | 2022-10-25 | Digilens Inc. | Light field displays incorporating holographic waveguides |
US11513350B2 (en) | 2016-12-02 | 2022-11-29 | Digilens Inc. | Waveguide device with uniform output illumination |
US11543594B2 (en) | 2019-02-15 | 2023-01-03 | Digilens Inc. | Methods and apparatuses for providing a holographic waveguide display using integrated gratings |
US11681143B2 (en) | 2019-07-29 | 2023-06-20 | Digilens Inc. | Methods and apparatus for multiplying the image resolution and field-of-view of a pixelated display |
US11726332B2 (en) | 2009-04-27 | 2023-08-15 | Digilens Inc. | Diffractive projection apparatus |
US11747568B2 (en) | 2019-06-07 | 2023-09-05 | Digilens Inc. | Waveguides incorporating transmissive and reflective gratings and related methods of manufacturing |
Families Citing this family (121)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9366867B2 (en) | 2014-07-08 | 2016-06-14 | Osterhout Group, Inc. | Optical systems for see-through displays |
US9298007B2 (en) | 2014-01-21 | 2016-03-29 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9952664B2 (en) | 2014-01-21 | 2018-04-24 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9229233B2 (en) | 2014-02-11 | 2016-01-05 | Osterhout Group, Inc. | Micro Doppler presentations in head worn computing |
US9965681B2 (en) | 2008-12-16 | 2018-05-08 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US10684687B2 (en) | 2014-12-03 | 2020-06-16 | Mentor Acquisition One, Llc | See-through computer display systems |
US11103122B2 (en) | 2014-07-15 | 2021-08-31 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US10649220B2 (en) | 2014-06-09 | 2020-05-12 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US9366868B2 (en) | 2014-09-26 | 2016-06-14 | Osterhout Group, Inc. | See-through computer display systems |
US10191279B2 (en) | 2014-03-17 | 2019-01-29 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9594246B2 (en) | 2014-01-21 | 2017-03-14 | Osterhout Group, Inc. | See-through computer display systems |
US9829707B2 (en) | 2014-08-12 | 2017-11-28 | Osterhout Group, Inc. | Measuring content brightness in head worn computing |
US9575321B2 (en) | 2014-06-09 | 2017-02-21 | Osterhout Group, Inc. | Content presentation in head worn computing |
US9939934B2 (en) | 2014-01-17 | 2018-04-10 | Osterhout Group, Inc. | External user interface for head worn computing |
US9299194B2 (en) | 2014-02-14 | 2016-03-29 | Osterhout Group, Inc. | Secure sharing in head worn computing |
US9746686B2 (en) | 2014-05-19 | 2017-08-29 | Osterhout Group, Inc. | Content position calibration in head worn computing |
US9810906B2 (en) | 2014-06-17 | 2017-11-07 | Osterhout Group, Inc. | External user interface for head worn computing |
US20160019715A1 (en) | 2014-07-15 | 2016-01-21 | Osterhout Group, Inc. | Content presentation in head worn computing |
US9841599B2 (en) | 2014-06-05 | 2017-12-12 | Osterhout Group, Inc. | Optical configurations for head-worn see-through displays |
US10254856B2 (en) | 2014-01-17 | 2019-04-09 | Osterhout Group, Inc. | External user interface for head worn computing |
US9651784B2 (en) | 2014-01-21 | 2017-05-16 | Osterhout Group, Inc. | See-through computer display systems |
US11737666B2 (en) | 2014-01-21 | 2023-08-29 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US9836122B2 (en) | 2014-01-21 | 2017-12-05 | Osterhout Group, Inc. | Eye glint imaging in see-through computer display systems |
US9811159B2 (en) * | 2014-01-21 | 2017-11-07 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US11669163B2 (en) | 2014-01-21 | 2023-06-06 | Mentor Acquisition One, Llc | Eye glint imaging in see-through computer display systems |
US11487110B2 (en) | 2014-01-21 | 2022-11-01 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US20150205135A1 (en) | 2014-01-21 | 2015-07-23 | Osterhout Group, Inc. | See-through computer display systems |
US9532714B2 (en) | 2014-01-21 | 2017-01-03 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US11892644B2 (en) | 2014-01-21 | 2024-02-06 | Mentor Acquisition One, Llc | See-through computer display systems |
US9494800B2 (en) | 2014-01-21 | 2016-11-15 | Osterhout Group, Inc. | See-through computer display systems |
US9766463B2 (en) | 2014-01-21 | 2017-09-19 | Osterhout Group, Inc. | See-through computer display systems |
US9753288B2 (en) | 2014-01-21 | 2017-09-05 | Osterhout Group, Inc. | See-through computer display systems |
US9651788B2 (en) | 2014-01-21 | 2017-05-16 | Osterhout Group, Inc. | See-through computer display systems |
US20150241963A1 (en) | 2014-02-11 | 2015-08-27 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9401540B2 (en) | 2014-02-11 | 2016-07-26 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US20160187651A1 (en) | 2014-03-28 | 2016-06-30 | Osterhout Group, Inc. | Safety for a vehicle operator with an hmd |
US10853589B2 (en) | 2014-04-25 | 2020-12-01 | Mentor Acquisition One, Llc | Language translation with head-worn computing |
US9651787B2 (en) | 2014-04-25 | 2017-05-16 | Osterhout Group, Inc. | Speaker assembly for headworn computer |
US9672210B2 (en) | 2014-04-25 | 2017-06-06 | Osterhout Group, Inc. | Language translation with head-worn computing |
US9690370B2 (en) * | 2014-05-05 | 2017-06-27 | Immersion Corporation | Systems and methods for viewport-based augmented reality haptic effects |
US10663740B2 (en) | 2014-06-09 | 2020-05-26 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US9684172B2 (en) | 2014-12-03 | 2017-06-20 | Osterhout Group, Inc. | Head worn computer display systems |
USD751552S1 (en) | 2014-12-31 | 2016-03-15 | Osterhout Group, Inc. | Computer glasses |
USD753114S1 (en) | 2015-01-05 | 2016-04-05 | Osterhout Group, Inc. | Air mouse |
US20160239985A1 (en) | 2015-02-17 | 2016-08-18 | Osterhout Group, Inc. | See-through computer display systems |
KR102634148B1 (en) | 2015-03-16 | 2024-02-05 | 매직 립, 인코포레이티드 | Methods and system for diagnosing and treating health ailments |
GB201510400D0 (en) * | 2015-06-15 | 2015-07-29 | The Technology Partnership Plc | Optical system |
US9599825B1 (en) * | 2015-09-30 | 2017-03-21 | Daqri, Llc | Visual indicator for transparent display alignment |
US10338677B2 (en) | 2015-10-28 | 2019-07-02 | Microsoft Technology Licensing, Llc | Adjusting image frames based on tracking motion of eyes |
US10241569B2 (en) | 2015-12-08 | 2019-03-26 | Facebook Technologies, Llc | Focus adjustment method for a virtual reality headset |
US10445860B2 (en) | 2015-12-08 | 2019-10-15 | Facebook Technologies, Llc | Autofocus virtual reality headset |
WO2017113245A1 (en) * | 2015-12-30 | 2017-07-06 | 深圳市柔宇科技有限公司 | Automatically-adjustable head-mounted display device and adjustment method therefor |
US10303246B2 (en) * | 2016-01-20 | 2019-05-28 | North Inc. | Systems, devices, and methods for proximity-based eye tracking |
US11106276B2 (en) | 2016-03-11 | 2021-08-31 | Facebook Technologies, Llc | Focus adjusting headset |
TWI614530B (en) * | 2016-03-23 | 2018-02-11 | 宏達國際電子股份有限公司 | Head-mounted display |
US10379356B2 (en) | 2016-04-07 | 2019-08-13 | Facebook Technologies, Llc | Accommodation based optical correction |
US9910284B1 (en) | 2016-09-08 | 2018-03-06 | Osterhout Group, Inc. | Optical systems for head-worn computers |
JP6923552B2 (en) | 2016-04-08 | 2021-08-18 | マジック リープ, インコーポレイテッドMagic Leap,Inc. | Augmented reality systems and methods with varifocal lens elements |
US10684478B2 (en) | 2016-05-09 | 2020-06-16 | Mentor Acquisition One, Llc | User interface systems for head-worn computers |
US10824253B2 (en) | 2016-05-09 | 2020-11-03 | Mentor Acquisition One, Llc | User interface systems for head-worn computers |
US10466491B2 (en) | 2016-06-01 | 2019-11-05 | Mentor Acquisition One, Llc | Modular systems for head-worn computers |
US10178378B2 (en) * | 2016-04-12 | 2019-01-08 | Microsoft Technology Licensing, Llc | Binocular image alignment for near-eye display |
US10156723B2 (en) * | 2016-05-12 | 2018-12-18 | Google Llc | Display pre-distortion methods and apparatus for head-mounted displays |
US10430988B2 (en) | 2016-06-03 | 2019-10-01 | Facebook Technologies, Llc | Facial animation using facial sensors within a head-mounted display |
US9959678B2 (en) * | 2016-06-03 | 2018-05-01 | Oculus Vr, Llc | Face and eye tracking using facial sensors within a head-mounted display |
US10437325B2 (en) * | 2016-08-19 | 2019-10-08 | North Inc. | Retinal see through display power level determination method and apparatus |
US10324291B2 (en) | 2016-09-12 | 2019-06-18 | Microsoft Technology Licensing, Llc | Display active alignment system for waveguide displays |
US10216263B2 (en) | 2016-09-12 | 2019-02-26 | Microsoft Technology Licensing, Llc | Display active alignment systems utilizing test patterns for calibrating signals in waveguide displays |
JP7090601B2 (en) | 2016-10-05 | 2022-06-24 | マジック リープ, インコーポレイテッド | Peripheral test for mixed reality calibration |
US10877556B2 (en) | 2016-10-21 | 2020-12-29 | Apple Inc. | Eye tracking system |
US20180157320A1 (en) * | 2016-12-01 | 2018-06-07 | Oculus Vr, Llc | Air spaced optical assembly with integrated eye tracking |
WO2018103072A1 (en) * | 2016-12-09 | 2018-06-14 | 深圳市柔宇科技有限公司 | Adjustment method and adjustment system for user interface, and head-mounted display device |
CN115145023B (en) | 2016-12-31 | 2024-02-09 | 鲁姆斯有限公司 | Device for deriving gaze direction of human eyes |
CN106963547A (en) * | 2017-01-12 | 2017-07-21 | 乔丽萍 | The intelligent eye-cover for weak sight of monitoring in real time |
US10310598B2 (en) * | 2017-01-17 | 2019-06-04 | Facebook Technologies, Llc | Varifocal head-mounted display including modular air spaced optical assembly |
EP3574360A4 (en) * | 2017-01-28 | 2020-11-11 | Lumus Ltd. | Augmented reality imaging system |
US10520742B1 (en) * | 2017-02-13 | 2019-12-31 | Facebook Technologies, Llc | Beamsplitter assembly for eye tracking in head-mounted displays |
KR102601052B1 (en) | 2017-02-23 | 2023-11-09 | 매직 립, 인코포레이티드 | Display system with variable power reflector |
US10679052B2 (en) * | 2017-03-13 | 2020-06-09 | Htc Corporation | Head mounted display device, object tracking apparatus and method for tracking object thereof |
TWI667495B (en) * | 2017-03-13 | 2019-08-01 | 宏達國際電子股份有限公司 | Head mounted display device, object tracking apparatus and method for tracking object thereof |
KR102365726B1 (en) * | 2017-06-13 | 2022-02-22 | 한국전자통신연구원 | Method for providing composite image based on optical see-through and apparatus using the same |
US10422995B2 (en) | 2017-07-24 | 2019-09-24 | Mentor Acquisition One, Llc | See-through computer display systems with stray light management |
US10578869B2 (en) | 2017-07-24 | 2020-03-03 | Mentor Acquisition One, Llc | See-through computer display systems with adjustable zoom cameras |
US11409105B2 (en) | 2017-07-24 | 2022-08-09 | Mentor Acquisition One, Llc | See-through computer display systems |
US10969584B2 (en) | 2017-08-04 | 2021-04-06 | Mentor Acquisition One, Llc | Image expansion optic for head-worn computer |
US11567318B1 (en) * | 2017-09-25 | 2023-01-31 | Meta Platforms Technologies, Llc | Determining features of a user's eye from depth mapping of the user's eye via indirect time of flight |
JP2019074582A (en) | 2017-10-13 | 2019-05-16 | ソニー株式会社 | Information processing apparatus, information processing method, and program |
CN109725418B (en) * | 2017-10-30 | 2020-10-16 | 华为技术有限公司 | Display device, method and device for adjusting image presentation of display device |
CN109839742A (en) * | 2017-11-29 | 2019-06-04 | 深圳市掌网科技股份有限公司 | A kind of augmented reality device based on Eye-controlling focus |
WO2019143844A1 (en) | 2018-01-17 | 2019-07-25 | Magic Leap, Inc. | Eye center of rotation determination, depth plane selection, and render camera positioning in display systems |
EP4339692A2 (en) * | 2018-01-17 | 2024-03-20 | Magic Leap, Inc. | Display systems and methods for determining registration between a display and a user's eyes |
WO2019160698A2 (en) * | 2018-02-16 | 2019-08-22 | Valve Corporation | Using detected pupil location to align optical components of a head-mounted display |
US10572731B1 (en) * | 2018-03-13 | 2020-02-25 | Facebook Technologies, Llc | Infrared transparent backlight device for eye tracking applications |
DE102018110644B4 (en) * | 2018-05-03 | 2024-02-15 | Carl Zeiss Meditec Ag | Digital microscope and digital microscopy method |
KR20190132590A (en) * | 2018-05-17 | 2019-11-28 | 삼성디스플레이 주식회사 | Head-up display for an automobile |
IL259518B2 (en) | 2018-05-22 | 2023-04-01 | Lumus Ltd | Optical system and method for improvement of light field uniformity |
US10509467B1 (en) * | 2018-06-01 | 2019-12-17 | Facebook Technologies, Llc | Determining fixation of a user's eyes from images of portions of the user's face enclosed by a head mounted display |
EP3827426A4 (en) | 2018-07-24 | 2022-07-27 | Magic Leap, Inc. | Display systems and methods for determining registration between a display and eyes of a user |
US11914141B1 (en) * | 2018-08-23 | 2024-02-27 | Apple Inc. | Electronic device with protected light sources |
TW202026685A (en) | 2018-11-08 | 2020-07-16 | 以色列商魯姆斯有限公司 | Light-guide display with reflector |
US10634907B1 (en) * | 2018-12-19 | 2020-04-28 | Facebook Technologies, Llc | Eye tracking based on polarization volume grating |
KR20210103541A (en) * | 2018-12-20 | 2021-08-23 | 스냅 인코포레이티드 | Flexible eyewear device with dual cameras for generating stereoscopic images |
CN111381367A (en) * | 2018-12-27 | 2020-07-07 | 宏星技术股份有限公司 | Eyeball tracker and display device |
CN111487035B (en) * | 2019-01-25 | 2022-02-01 | 舜宇光学(浙江)研究院有限公司 | Alignment method and system for near-eye detection system |
US11650403B2 (en) | 2019-02-08 | 2023-05-16 | Meta Platforms Technologies, Llc | Optical elements for beam-shaping and illumination |
US11237389B1 (en) * | 2019-02-11 | 2022-02-01 | Facebook Technologies, Llc | Wedge combiner for eye-tracking |
SE543332C2 (en) | 2019-02-19 | 2020-12-01 | Tobii Ab | Method, computer program and head mountable arrangement for assisting a subject to acquire spatial information about an environment |
CN110088665A (en) * | 2019-03-13 | 2019-08-02 | 香港应用科技研究院有限公司 | Compact optical structure for big visual field optical perspective type head-mounted display designs |
US11822081B2 (en) | 2019-08-29 | 2023-11-21 | Apple Inc. | Optical module for head-mounted device |
US11885965B1 (en) | 2019-09-23 | 2024-01-30 | Apple Inc. | Head-mounted display and display modules thereof |
US10965931B1 (en) | 2019-12-06 | 2021-03-30 | Snap Inc. | Sensor misalignment compensation |
WO2021118596A1 (en) * | 2019-12-13 | 2021-06-17 | Google Llc | A compact rim-mounted curved optical see-through lightguide based eyewear as mobile augmented reality display |
US20210302726A1 (en) * | 2020-03-31 | 2021-09-30 | Htc Corporation | Head mounted display |
US11327564B2 (en) * | 2020-08-19 | 2022-05-10 | Htc Corporation | Head mounted display apparatus and eye-tracking apparatus thereof |
TWI738473B (en) * | 2020-08-19 | 2021-09-01 | 宏達國際電子股份有限公司 | Head mounted display apparatus and eye-tracking apparatus thereof |
KR20230148324A (en) | 2021-03-01 | 2023-10-24 | 루머스 리미티드 | Optical system with compact coupling from projector to waveguide |
KR20230133841A (en) | 2021-05-19 | 2023-09-19 | 루머스 리미티드 | active optical engine |
WO2022170287A2 (en) * | 2021-06-07 | 2022-08-11 | Panamorph, Inc. | Near-eye display system |
WO2023026266A1 (en) | 2021-08-23 | 2023-03-02 | Lumus Ltd. | Methods of fabrication of compound light-guide optical elements having embedded coupling-in reflectors |
CN114355615B (en) * | 2022-03-15 | 2023-05-05 | 荣耀终端有限公司 | Head-mounted display device and control method thereof |
DE102022202622A1 (en) * | 2022-03-17 | 2023-09-21 | Robert Bosch Gesellschaft mit beschränkter Haftung | Method for determining an eye distance in data glasses and data glasses |
Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2013037021A (en) * | 2011-08-03 | 2013-02-21 | Canon Inc | Display and head-mounted display |
US20130050833A1 (en) * | 2011-08-30 | 2013-02-28 | John R. Lewis | Adjustment of a mixed reality display for inter-pupillary distance alignment |
US20130114850A1 (en) * | 2011-11-07 | 2013-05-09 | Eye-Com Corporation | Systems and methods for high-resolution gaze tracking |
Family Cites Families (68)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPS55101241A (en) * | 1979-01-30 | 1980-08-01 | Tokyo Optical | Eyeground camera equipped with operation distance detector |
US4711512A (en) | 1985-07-12 | 1987-12-08 | Environmental Research Institute Of Michigan | Compact head-up display |
US5151722A (en) * | 1990-11-05 | 1992-09-29 | The Johns Hopkins University | Video display on spectacle-like frame |
US7310072B2 (en) * | 1993-10-22 | 2007-12-18 | Kopin Corporation | Portable communication display device |
US5815126A (en) * | 1993-10-22 | 1998-09-29 | Kopin Corporation | Monocular portable communication and display system |
US6147805A (en) * | 1994-08-24 | 2000-11-14 | Fergason; James L. | Head mounted display and viewing system using a remote retro-reflector and method of displaying and viewing an image |
US5684497A (en) * | 1994-12-21 | 1997-11-04 | Siliscape, Inc. | Twice folded compound magnified virtual image electronic display |
US5856842A (en) | 1996-08-26 | 1999-01-05 | Kaiser Optical Systems Corporation | Apparatus facilitating eye-contact video communications |
CA2326767C (en) | 1998-04-02 | 2009-06-23 | Yeda Research And Development Co., Ltd. | Holographic optical devices |
US6487021B1 (en) | 1999-06-22 | 2002-11-26 | Koninklijke Philips Electronics N.V. | Head-mounted display |
US6323970B1 (en) | 1999-09-29 | 2001-11-27 | Digilents, Inc. | Method of producing switchable holograms |
US6419671B1 (en) * | 1999-12-23 | 2002-07-16 | Visx, Incorporated | Optical feedback system for vision correction |
US6753828B2 (en) | 2000-09-25 | 2004-06-22 | Siemens Corporated Research, Inc. | System and method for calibrating a stereo optical see-through head-mounted display system for augmented reality |
US6669340B2 (en) | 2001-11-06 | 2003-12-30 | Reichert, Inc. | Alignment system for an ophthalmic instrument |
IL148804A (en) | 2002-03-21 | 2007-02-11 | Yaacov Amitai | Optical device |
FI114945B (en) | 2002-09-19 | 2005-01-31 | Nokia Corp | Electrically adjustable diffractive gate element |
US6943754B2 (en) | 2002-09-27 | 2005-09-13 | The Boeing Company | Gaze tracking system, eye-tracking assembly and an associated method of calibration |
US6867753B2 (en) * | 2002-10-28 | 2005-03-15 | University Of Washington | Virtual image registration in augmented display field |
US7380938B2 (en) * | 2003-03-25 | 2008-06-03 | Sarnoff Corporation | Apparatus to detect and measure saccade and pupilary changes |
WO2004102226A2 (en) | 2003-05-09 | 2004-11-25 | Sbg Labs, Inc. | Switchable viewfinder display |
US7963652B2 (en) * | 2003-11-14 | 2011-06-21 | Queen's University At Kingston | Method and apparatus for calibration-free eye tracking |
JP3972934B2 (en) * | 2004-01-30 | 2007-09-05 | セイコーエプソン株式会社 | LIGHTING DEVICE, DISPLAY DEVICE, AND PROJECTION TYPE DISPLAY DEVICE |
US20060025832A1 (en) * | 2004-02-25 | 2006-02-02 | O'keeffe Declan | System and method for stimulating peripheral nerves to treat pain |
US7530948B2 (en) * | 2005-02-28 | 2009-05-12 | University Of Washington | Tethered capsule endoscope for Barrett's Esophagus screening |
US20070291277A1 (en) * | 2006-06-20 | 2007-12-20 | Everett Matthew J | Spectral domain optical coherence tomography system |
JP2010511486A (en) * | 2006-12-04 | 2010-04-15 | ファテ,シナ | System, method and apparatus for correction of amblyopia and eyeball deviation |
WO2008081070A1 (en) | 2006-12-28 | 2008-07-10 | Nokia Corporation | Device for expanding an exit pupil in two dimensions |
US8446341B2 (en) * | 2007-03-07 | 2013-05-21 | University Of Washington | Contact lens with integrated light-emitting component |
CN101051349B (en) | 2007-05-18 | 2010-10-13 | 北京中科虹霸科技有限公司 | Multiple iris collecting device using active vision feedback |
US8403490B2 (en) * | 2007-09-26 | 2013-03-26 | Panasonic Corporation | Beam scanning-type display device, method, program and integrated circuit |
JP4466749B2 (en) * | 2008-03-06 | 2010-05-26 | 株式会社デンソー | Map display device |
DE102008027832A1 (en) * | 2008-06-11 | 2009-12-17 | Vrmagic Gmbh | Ophthalmoscope simulator |
US20100079865A1 (en) | 2008-09-26 | 2010-04-01 | Nokia Corporation | Near-to-eye scanning display with exit-pupil expansion |
US9495589B2 (en) * | 2009-01-26 | 2016-11-15 | Tobii Ab | Detection of gaze point assisted by optical reference signal |
EP2419780B1 (en) | 2009-04-14 | 2017-09-20 | BAE Systems PLC | Optical waveguide and display device |
US8233204B1 (en) | 2009-09-30 | 2012-07-31 | Rockwell Collins, Inc. | Optical displays |
US11320571B2 (en) | 2012-11-16 | 2022-05-03 | Rockwell Collins, Inc. | Transparent waveguide display providing upper and lower fields of view with uniform light extraction |
US20120212484A1 (en) * | 2010-02-28 | 2012-08-23 | Osterhout Group, Inc. | System and method for display content placement using distance and location information |
US8355208B1 (en) * | 2010-06-15 | 2013-01-15 | Rockwell Collins, Inc. | Quick adjustment mechanism for head or helmet mounted displays |
US20120019891A1 (en) * | 2010-07-22 | 2012-01-26 | Dewell Douglas A | Window Having A Selective Light Modulation System |
US8531394B2 (en) * | 2010-07-23 | 2013-09-10 | Gregory A. Maltz | Unitized, vision-controlled, wireless eyeglasses transceiver |
US9304319B2 (en) * | 2010-11-18 | 2016-04-05 | Microsoft Technology Licensing, Llc | Automatic focus improvement for augmented reality displays |
US9185352B1 (en) * | 2010-12-22 | 2015-11-10 | Thomas Jacques | Mobile eye tracking system |
DE102012204259A1 (en) * | 2011-03-31 | 2012-10-04 | Canon Kabushiki Kaisha | Control device, ophthalmic device, system, control method and program |
US8872766B2 (en) | 2011-05-10 | 2014-10-28 | Raytheon Company | System and method for operating a helmet mounted display |
US20150077312A1 (en) * | 2011-05-13 | 2015-03-19 | Google Inc. | Near-to-eye display having adaptive optics |
WO2012156778A1 (en) * | 2011-05-13 | 2012-11-22 | Sony Ericsson Mobile Communications Ab | Adjusting parallax barriers |
WO2012172295A1 (en) | 2011-06-16 | 2012-12-20 | Milan Momcilo Popovich | Holographic beam deflector for autostereoscopic displays |
JP6057567B2 (en) * | 2011-07-14 | 2017-01-11 | キヤノン株式会社 | Imaging control apparatus, ophthalmic imaging apparatus, imaging control method, and program |
US8754831B2 (en) * | 2011-08-02 | 2014-06-17 | Microsoft Corporation | Changing between display device viewing modes |
US8629815B2 (en) * | 2011-08-09 | 2014-01-14 | Google Inc. | Laser alignment of binocular head mounted display |
EP2748670B1 (en) | 2011-08-24 | 2015-11-18 | Rockwell Collins, Inc. | Wearable data display |
WO2013033195A2 (en) | 2011-08-30 | 2013-03-07 | Microsoft Corporation | Head mounted display with iris scan profiling |
US9213163B2 (en) | 2011-08-30 | 2015-12-15 | Microsoft Technology Licensing, Llc | Aligning inter-pupillary distance in a near-eye display system |
WO2013036621A1 (en) * | 2011-09-09 | 2013-03-14 | Thales Avionics, Inc. | Controlling vehicle entertainment systems responsive to sensed passenger gestures |
US8998414B2 (en) | 2011-09-26 | 2015-04-07 | Microsoft Technology Licensing, Llc | Integrated eye tracking and display system |
GB2496184A (en) * | 2011-11-05 | 2013-05-08 | Optovate Ltd | Illumination system |
KR101692323B1 (en) * | 2011-12-14 | 2017-01-04 | 인텔 코포레이션 | Gaze activated content transfer system |
KR102264765B1 (en) * | 2012-01-24 | 2021-06-11 | 더 아리조나 보드 오브 리전츠 온 비핼프 오브 더 유니버시티 오브 아리조나 | Compact eye-tracked head-mounted display |
WO2013163347A1 (en) | 2012-04-25 | 2013-10-31 | Rockwell Collins, Inc. | Holographic wide angle display |
JP6248931B2 (en) * | 2012-07-24 | 2017-12-20 | ソニー株式会社 | Image display device and image display method |
KR20140025812A (en) * | 2012-08-22 | 2014-03-05 | 삼성전기주식회사 | Apparatus and method for sensing drowsy driving |
US9933684B2 (en) | 2012-11-16 | 2018-04-03 | Rockwell Collins, Inc. | Transparent waveguide display providing upper and lower fields of view having a specific light output aperture configuration |
TWI559895B (en) * | 2013-01-08 | 2016-12-01 | Altek Biotechnology Corp | Camera device and photographing method |
US20140247208A1 (en) * | 2013-03-01 | 2014-09-04 | Tobii Technology Ab | Invoking and waking a computing device from stand-by mode based on gaze detection |
US9239460B2 (en) * | 2013-05-10 | 2016-01-19 | Microsoft Technology Licensing, Llc | Calibration of eye location |
US20140375540A1 (en) | 2013-06-24 | 2014-12-25 | Nathan Ackerman | System for optimal eye fit of headset display device |
US20140375541A1 (en) * | 2013-06-25 | 2014-12-25 | David Nister | Eye tracking via depth camera |
-
2013
- 2013-08-22 US US13/973,788 patent/US10345903B2/en active Active
-
2014
- 2014-07-28 WO PCT/US2014/048337 patent/WO2015017291A1/en active Application Filing
- 2014-07-28 EP EP14750128.2A patent/EP3028216B1/en active Active
- 2014-07-28 CN CN201480042885.XA patent/CN105531716B/en active Active
Patent Citations (3)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP2013037021A (en) * | 2011-08-03 | 2013-02-21 | Canon Inc | Display and head-mounted display |
US20130050833A1 (en) * | 2011-08-30 | 2013-02-28 | John R. Lewis | Adjustment of a mixed reality display for inter-pupillary distance alignment |
US20130114850A1 (en) * | 2011-11-07 | 2013-05-09 | Eye-Com Corporation | Systems and methods for high-resolution gaze tracking |
Non-Patent Citations (1)
Title |
---|
HANSEN D W ET AL: "In the Eye of the Beholder: A Survey of Models for Eyes and Gaze", IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE, IEEE COMPUTER SOCIETY, USA, vol. 30, no. 3, March 2010 (2010-03-01), pages 478 - 500, XP011293073, ISSN: 0162-8828 * |
Cited By (67)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US10145533B2 (en) | 2005-11-11 | 2018-12-04 | Digilens, Inc. | Compact holographic illumination device |
US10234696B2 (en) | 2007-07-26 | 2019-03-19 | Digilens, Inc. | Optical apparatus for recording a holographic device and method of recording |
US10725312B2 (en) | 2007-07-26 | 2020-07-28 | Digilens Inc. | Laser illumination device |
US10678053B2 (en) | 2009-04-27 | 2020-06-09 | Digilens Inc. | Diffractive projection apparatus |
US11726332B2 (en) | 2009-04-27 | 2023-08-15 | Digilens Inc. | Diffractive projection apparatus |
US11175512B2 (en) | 2009-04-27 | 2021-11-16 | Digilens Inc. | Diffractive projection apparatus |
US10185154B2 (en) | 2011-04-07 | 2019-01-22 | Digilens, Inc. | Laser despeckler based on angular diversity |
US11487131B2 (en) | 2011-04-07 | 2022-11-01 | Digilens Inc. | Laser despeckler based on angular diversity |
US10642058B2 (en) | 2011-08-24 | 2020-05-05 | Digilens Inc. | Wearable data display |
US10670876B2 (en) | 2011-08-24 | 2020-06-02 | Digilens Inc. | Waveguide laser illuminator incorporating a despeckler |
US11287666B2 (en) | 2011-08-24 | 2022-03-29 | Digilens, Inc. | Wearable data display |
US10459311B2 (en) | 2012-01-06 | 2019-10-29 | Digilens Inc. | Contact image sensor using switchable Bragg gratings |
US10216061B2 (en) | 2012-01-06 | 2019-02-26 | Digilens, Inc. | Contact image sensor using switchable bragg gratings |
US11460621B2 (en) | 2012-04-25 | 2022-10-04 | Rockwell Collins, Inc. | Holographic wide angle display |
US10437051B2 (en) | 2012-05-11 | 2019-10-08 | Digilens Inc. | Apparatus for eye tracking |
US11448937B2 (en) | 2012-11-16 | 2022-09-20 | Digilens Inc. | Transparent waveguide display for tiling a display having plural optical powers using overlapping and offset FOV tiles |
US11662590B2 (en) | 2013-05-20 | 2023-05-30 | Digilens Inc. | Holographic waveguide eye tracker |
US10209517B2 (en) | 2013-05-20 | 2019-02-19 | Digilens, Inc. | Holographic waveguide eye tracker |
US10423813B2 (en) | 2013-07-31 | 2019-09-24 | Digilens Inc. | Method and apparatus for contact image sensing |
US10089516B2 (en) | 2013-07-31 | 2018-10-02 | Digilens, Inc. | Method and apparatus for contact image sensing |
US10359736B2 (en) | 2014-08-08 | 2019-07-23 | Digilens Inc. | Method for holographic mastering and replication |
US11709373B2 (en) | 2014-08-08 | 2023-07-25 | Digilens Inc. | Waveguide laser illuminator incorporating a despeckler |
US11307432B2 (en) | 2014-08-08 | 2022-04-19 | Digilens Inc. | Waveguide laser illuminator incorporating a Despeckler |
US11726323B2 (en) | 2014-09-19 | 2023-08-15 | Digilens Inc. | Method and apparatus for generating input images for holographic waveguide displays |
US10241330B2 (en) | 2014-09-19 | 2019-03-26 | Digilens, Inc. | Method and apparatus for generating input images for holographic waveguide displays |
US10423222B2 (en) | 2014-09-26 | 2019-09-24 | Digilens Inc. | Holographic waveguide optical tracker |
US11480788B2 (en) | 2015-01-12 | 2022-10-25 | Digilens Inc. | Light field displays incorporating holographic waveguides |
US11740472B2 (en) | 2015-01-12 | 2023-08-29 | Digilens Inc. | Environmentally isolated waveguide display |
US11726329B2 (en) | 2015-01-12 | 2023-08-15 | Digilens Inc. | Environmentally isolated waveguide display |
US10437064B2 (en) | 2015-01-12 | 2019-10-08 | Digilens Inc. | Environmentally isolated waveguide display |
US10330777B2 (en) | 2015-01-20 | 2019-06-25 | Digilens Inc. | Holographic waveguide lidar |
US10527797B2 (en) | 2015-02-12 | 2020-01-07 | Digilens Inc. | Waveguide grating device |
US10156681B2 (en) | 2015-02-12 | 2018-12-18 | Digilens Inc. | Waveguide grating device |
US11703645B2 (en) | 2015-02-12 | 2023-07-18 | Digilens Inc. | Waveguide grating device |
KR20180007349A (en) * | 2015-03-03 | 2018-01-22 | 센소모토릭 인스트루멘츠 게젤샤프트 퓌어 이노바티브 센소릭 엠베하 | Head-mounted eye tracking device and method that provides drift-free eye tracking through a lens system |
KR102073460B1 (en) * | 2015-03-03 | 2020-02-04 | 애플 인크. | Head-mounted eye tracking device and method that provides drift-free eye tracking through lens system |
US10459145B2 (en) | 2015-03-16 | 2019-10-29 | Digilens Inc. | Waveguide device incorporating a light pipe |
US10591756B2 (en) | 2015-03-31 | 2020-03-17 | Digilens Inc. | Method and apparatus for contact image sensing |
US11281013B2 (en) | 2015-10-05 | 2022-03-22 | Digilens Inc. | Apparatus for providing waveguide displays with two-dimensional pupil expansion |
US10690916B2 (en) | 2015-10-05 | 2020-06-23 | Digilens Inc. | Apparatus for providing waveguide displays with two-dimensional pupil expansion |
US11754842B2 (en) | 2015-10-05 | 2023-09-12 | Digilens Inc. | Apparatus for providing waveguide displays with two-dimensional pupil expansion |
CN108885341A (en) * | 2015-11-25 | 2018-11-23 | 谷歌有限责任公司 | Eyes tracking based on prism |
US10983340B2 (en) | 2016-02-04 | 2021-04-20 | Digilens Inc. | Holographic waveguide optical tracker |
US10859768B2 (en) | 2016-03-24 | 2020-12-08 | Digilens Inc. | Method and apparatus for providing a polarization selective holographic waveguide device |
US11604314B2 (en) | 2016-03-24 | 2023-03-14 | Digilens Inc. | Method and apparatus for providing a polarization selective holographic waveguide device |
US10890707B2 (en) | 2016-04-11 | 2021-01-12 | Digilens Inc. | Holographic waveguide apparatus for structured light projection |
US11513350B2 (en) | 2016-12-02 | 2022-11-29 | Digilens Inc. | Waveguide device with uniform output illumination |
US10545346B2 (en) | 2017-01-05 | 2020-01-28 | Digilens Inc. | Wearable heads up displays |
US11586046B2 (en) | 2017-01-05 | 2023-02-21 | Digilens Inc. | Wearable heads up displays |
US11194162B2 (en) | 2017-01-05 | 2021-12-07 | Digilens Inc. | Wearable heads up displays |
US10942430B2 (en) | 2017-10-16 | 2021-03-09 | Digilens Inc. | Systems and methods for multiplying the image resolution of a pixelated display |
US10732569B2 (en) | 2018-01-08 | 2020-08-04 | Digilens Inc. | Systems and methods for high-throughput recording of holographic gratings in waveguide cells |
US10914950B2 (en) | 2018-01-08 | 2021-02-09 | Digilens Inc. | Waveguide architectures and related methods of manufacturing |
US10690851B2 (en) | 2018-03-16 | 2020-06-23 | Digilens Inc. | Holographic waveguides incorporating birefringence control and methods for their fabrication |
US11726261B2 (en) | 2018-03-16 | 2023-08-15 | Digilens Inc. | Holographic waveguides incorporating birefringence control and methods for their fabrication |
US11150408B2 (en) | 2018-03-16 | 2021-10-19 | Digilens Inc. | Holographic waveguides incorporating birefringence control and methods for their fabrication |
US11402801B2 (en) | 2018-07-25 | 2022-08-02 | Digilens Inc. | Systems and methods for fabricating a multilayer optical structure |
CN112997106A (en) * | 2018-11-09 | 2021-06-18 | 脸谱科技有限责任公司 | Catadioptric and refractive optical structures for beam shaping |
CN112997106B (en) * | 2018-11-09 | 2023-08-01 | 元平台技术有限公司 | Catadioptric and refractive optical structures for beam shaping |
US11500185B2 (en) | 2018-11-09 | 2022-11-15 | Meta Platforms Technologies, Llc | Catadioptric and refractive optical structures for beam shaping |
US11543594B2 (en) | 2019-02-15 | 2023-01-03 | Digilens Inc. | Methods and apparatuses for providing a holographic waveguide display using integrated gratings |
US11378732B2 (en) | 2019-03-12 | 2022-07-05 | DigLens Inc. | Holographic waveguide backlight and related methods of manufacturing |
US11747568B2 (en) | 2019-06-07 | 2023-09-05 | Digilens Inc. | Waveguides incorporating transmissive and reflective gratings and related methods of manufacturing |
US11681143B2 (en) | 2019-07-29 | 2023-06-20 | Digilens Inc. | Methods and apparatus for multiplying the image resolution and field-of-view of a pixelated display |
US11442222B2 (en) | 2019-08-29 | 2022-09-13 | Digilens Inc. | Evacuated gratings and methods of manufacturing |
US11592614B2 (en) | 2019-08-29 | 2023-02-28 | Digilens Inc. | Evacuated gratings and methods of manufacturing |
US11899238B2 (en) | 2019-08-29 | 2024-02-13 | Digilens Inc. | Evacuated gratings and methods of manufacturing |
Also Published As
Publication number | Publication date |
---|---|
US10345903B2 (en) | 2019-07-09 |
US20150035744A1 (en) | 2015-02-05 |
CN105531716A (en) | 2016-04-27 |
EP3028216B1 (en) | 2021-03-17 |
EP3028216A1 (en) | 2016-06-08 |
CN105531716B (en) | 2020-01-21 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
EP3028216B1 (en) | Near-eye optic positioning in display devices | |
US10228561B2 (en) | Eye-tracking system using a freeform prism and gaze-detection light | |
US9625723B2 (en) | Eye-tracking system using a freeform prism | |
KR102460874B1 (en) | Near eye display with a spherical mirror and a decoupled aspherical element | |
US20140375542A1 (en) | Adjusting a near-eye display device | |
US20170295362A1 (en) | Binocular image alignment for near-eye display | |
US10482676B2 (en) | Systems and methods to provide an interactive environment over an expanded field-of-view | |
US10282912B1 (en) | Systems and methods to provide an interactive space over an expanded field-of-view with focal distance tuning | |
US20200117006A1 (en) | Polarization-based dynamic focuser | |
US11536969B2 (en) | Scene camera | |
WO2022203827A1 (en) | Eye tracker illumination through a waveguide | |
US11397465B1 (en) | Glint-based eye tracker illumination using dual-sided and dual-layered architectures | |
US11940628B2 (en) | Display device having common light path region | |
US11876952B2 (en) | Calibration of stereoscopic display using waveguide combiner | |
US20230314804A1 (en) | Polarization-recycling waveguided illumination system for microdisplay | |
US20240069347A1 (en) | System and method using eye tracking illumination |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
WWE | Wipo information: entry into national phase |
Ref document number: 201480042885.X Country of ref document: CN |
|
121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 14750128 Country of ref document: EP Kind code of ref document: A1 |
|
DPE1 | Request for preliminary examination filed after expiration of 19th month from priority date (pct application filed from 20040101) | ||
WWE | Wipo information: entry into national phase |
Ref document number: 2014750128 Country of ref document: EP |
|
NENP | Non-entry into the national phase |
Ref country code: DE |