US20150277113A1 - Sensor dependent content position in head worn computing - Google Patents
Sensor dependent content position in head worn computing Download PDFInfo
- Publication number
- US20150277113A1 US20150277113A1 US14/516,013 US201414516013A US2015277113A1 US 20150277113 A1 US20150277113 A1 US 20150277113A1 US 201414516013 A US201414516013 A US 201414516013A US 2015277113 A1 US2015277113 A1 US 2015277113A1
- Authority
- US
- United States
- Prior art keywords
- light
- eye
- hwc
- user
- wearer
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
- 230000001419 dependent effect Effects 0.000 title description 22
- 238000000034 method Methods 0.000 claims abstract description 63
- 238000003384 imaging method Methods 0.000 claims description 55
- 230000003139 buffering effect Effects 0.000 claims 1
- 230000001960 triggered effect Effects 0.000 claims 1
- 230000003287 optical effect Effects 0.000 description 144
- KRUDZWOELKQDJW-UHFFFAOYSA-N 3-amino-5-bromo-1-methylpyridin-2-one Chemical compound CN1C=C(Br)C=C(N)C1=O KRUDZWOELKQDJW-UHFFFAOYSA-N 0.000 description 126
- 230000033001 locomotion Effects 0.000 description 97
- 230000008859 change Effects 0.000 description 51
- 230000010287 polarization Effects 0.000 description 46
- 238000005286 illumination Methods 0.000 description 44
- 239000013598 vector Substances 0.000 description 38
- 230000000694 effects Effects 0.000 description 34
- PUFGCEQWYLJYNJ-UHFFFAOYSA-N didodecyl benzene-1,2-dicarboxylate Chemical compound CCCCCCCCCCCCOC(=O)C1=CC=CC=C1C(=O)OCCCCCCCCCCCC PUFGCEQWYLJYNJ-UHFFFAOYSA-N 0.000 description 33
- 210000003128 head Anatomy 0.000 description 31
- 230000008569 process Effects 0.000 description 24
- 239000007787 solid Substances 0.000 description 22
- 239000011521 glass Substances 0.000 description 20
- 238000005516 engineering process Methods 0.000 description 18
- 230000006854 communication Effects 0.000 description 15
- 238000004891 communication Methods 0.000 description 14
- 239000002096 quantum dot Substances 0.000 description 14
- 238000012545 processing Methods 0.000 description 12
- 230000005540 biological transmission Effects 0.000 description 11
- 230000007613 environmental effect Effects 0.000 description 11
- 238000005259 measurement Methods 0.000 description 11
- 230000009471 action Effects 0.000 description 10
- 238000000576 coating method Methods 0.000 description 10
- 238000012544 monitoring process Methods 0.000 description 10
- 230000003190 augmentative effect Effects 0.000 description 9
- 239000011248 coating agent Substances 0.000 description 9
- 230000004044 response Effects 0.000 description 9
- 238000004458 analytical method Methods 0.000 description 8
- 230000006870 function Effects 0.000 description 8
- 230000001629 suppression Effects 0.000 description 8
- 230000008901 benefit Effects 0.000 description 7
- 239000000463 material Substances 0.000 description 6
- 230000007935 neutral effect Effects 0.000 description 6
- 239000011358 absorbing material Substances 0.000 description 5
- 239000003086 colorant Substances 0.000 description 5
- 238000013461 design Methods 0.000 description 5
- 230000004424 eye movement Effects 0.000 description 5
- 230000004297 night vision Effects 0.000 description 5
- 238000002310 reflectometry Methods 0.000 description 5
- 238000010521 absorption reaction Methods 0.000 description 4
- 230000002093 peripheral effect Effects 0.000 description 4
- 125000006850 spacer group Chemical group 0.000 description 4
- 239000000758 substrate Substances 0.000 description 4
- 230000000007 visual effect Effects 0.000 description 4
- OKTJSMMVPCPJKN-UHFFFAOYSA-N Carbon Chemical compound [C] OKTJSMMVPCPJKN-UHFFFAOYSA-N 0.000 description 3
- 238000006243 chemical reaction Methods 0.000 description 3
- 239000006059 cover glass Substances 0.000 description 3
- 238000000151 deposition Methods 0.000 description 3
- 229910002804 graphite Inorganic materials 0.000 description 3
- 239000010439 graphite Substances 0.000 description 3
- 230000004886 head movement Effects 0.000 description 3
- 210000001747 pupil Anatomy 0.000 description 3
- 230000001360 synchronised effect Effects 0.000 description 3
- 238000012795 verification Methods 0.000 description 3
- UGFAIRIUMAVXCW-UHFFFAOYSA-N Carbon monoxide Chemical compound [O+]#[C-] UGFAIRIUMAVXCW-UHFFFAOYSA-N 0.000 description 2
- NIXOWILDQLNWCW-UHFFFAOYSA-N acrylic acid group Chemical group C(C=C)(=O)O NIXOWILDQLNWCW-UHFFFAOYSA-N 0.000 description 2
- 239000000853 adhesive Substances 0.000 description 2
- 230000001070 adhesive effect Effects 0.000 description 2
- 230000004397 blinking Effects 0.000 description 2
- 238000009530 blood pressure measurement Methods 0.000 description 2
- 229910002091 carbon monoxide Inorganic materials 0.000 description 2
- 238000012790 confirmation Methods 0.000 description 2
- 230000008021 deposition Effects 0.000 description 2
- 230000004438 eyesight Effects 0.000 description 2
- 230000003760 hair shine Effects 0.000 description 2
- 230000036541 health Effects 0.000 description 2
- 230000009191 jumping Effects 0.000 description 2
- 210000004072 lung Anatomy 0.000 description 2
- 238000012423 maintenance Methods 0.000 description 2
- 238000007726 management method Methods 0.000 description 2
- 230000007246 mechanism Effects 0.000 description 2
- 230000000116 mitigating effect Effects 0.000 description 2
- 230000004461 rapid eye movement Effects 0.000 description 2
- 230000003595 spectral effect Effects 0.000 description 2
- 239000000126 substance Substances 0.000 description 2
- 238000001429 visible spectrum Methods 0.000 description 2
- 238000007794 visualization technique Methods 0.000 description 2
- 239000002699 waste material Substances 0.000 description 2
- -1 IMU Chemical compound 0.000 description 1
- 238000002835 absorbance Methods 0.000 description 1
- 230000001133 acceleration Effects 0.000 description 1
- 239000008186 active pharmaceutical agent Substances 0.000 description 1
- 229910052782 aluminium Inorganic materials 0.000 description 1
- XAGFODPZIPBFFR-UHFFFAOYSA-N aluminium Chemical compound [Al] XAGFODPZIPBFFR-UHFFFAOYSA-N 0.000 description 1
- 238000013459 approach Methods 0.000 description 1
- 238000000149 argon plasma sintering Methods 0.000 description 1
- 230000007175 bidirectional communication Effects 0.000 description 1
- 239000008280 blood Substances 0.000 description 1
- 210000004369 blood Anatomy 0.000 description 1
- 208000029028 brain injury Diseases 0.000 description 1
- 238000004040 coloring Methods 0.000 description 1
- 230000009514 concussion Effects 0.000 description 1
- 238000010276 construction Methods 0.000 description 1
- 238000013016 damping Methods 0.000 description 1
- 230000007423 decrease Effects 0.000 description 1
- 230000003247 decreasing effect Effects 0.000 description 1
- 230000000593 degrading effect Effects 0.000 description 1
- 230000001934 delay Effects 0.000 description 1
- 230000003111 delayed effect Effects 0.000 description 1
- 238000001514 detection method Methods 0.000 description 1
- 239000010432 diamond Substances 0.000 description 1
- 229910003460 diamond Inorganic materials 0.000 description 1
- 238000009826 distribution Methods 0.000 description 1
- 238000012854 evaluation process Methods 0.000 description 1
- 210000000887 face Anatomy 0.000 description 1
- 230000001815 facial effect Effects 0.000 description 1
- 238000010304 firing Methods 0.000 description 1
- 230000004313 glare Effects 0.000 description 1
- 230000003993 interaction Effects 0.000 description 1
- 238000004519 manufacturing process Methods 0.000 description 1
- 239000003550 marker Substances 0.000 description 1
- 229910052751 metal Inorganic materials 0.000 description 1
- 239000002184 metal Substances 0.000 description 1
- 238000010295 mobile communication Methods 0.000 description 1
- 239000003973 paint Substances 0.000 description 1
- 238000010422 painting Methods 0.000 description 1
- 230000000737 periodic effect Effects 0.000 description 1
- 230000002085 persistent effect Effects 0.000 description 1
- 238000000103 photoluminescence spectrum Methods 0.000 description 1
- 239000004033 plastic Substances 0.000 description 1
- 239000004417 polycarbonate Substances 0.000 description 1
- 229920000515 polycarbonate Polymers 0.000 description 1
- 238000003825 pressing Methods 0.000 description 1
- 230000008439 repair process Effects 0.000 description 1
- 230000003252 repetitive effect Effects 0.000 description 1
- 210000001525 retina Anatomy 0.000 description 1
- 239000005336 safety glass Substances 0.000 description 1
- 239000010980 sapphire Substances 0.000 description 1
- 229910052594 sapphire Inorganic materials 0.000 description 1
- 230000002459 sustained effect Effects 0.000 description 1
- 238000002834 transmittance Methods 0.000 description 1
- 238000002211 ultraviolet spectrum Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
- XLYOFNOQVPJJNP-UHFFFAOYSA-N water Substances O XLYOFNOQVPJJNP-UHFFFAOYSA-N 0.000 description 1
- 210000000707 wrist Anatomy 0.000 description 1
Images
Classifications
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/0093—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00 with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/1613—Constructional details or arrangements for portable computers
- G06F1/163—Wearable computers, e.g. on a belt
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T1/00—General purpose image data processing
- G06T1/60—Memory management
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T19/00—Manipulating 3D models or images for computer graphics
- G06T19/006—Mixed reality
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/332—Displays for viewing with the aid of special glasses or head-mounted displays [HMD]
- H04N13/344—Displays for viewing with the aid of special glasses or head-mounted displays [HMD] with head-mounted left-right displays
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N13/366—Image reproducers using viewer tracking
- H04N13/383—Image reproducers using viewer tracking for tracking with gaze detection, i.e. detecting the lines of sight of the viewer's eyes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0118—Head-up displays characterised by optical features comprising devices for improving the contrast of the display / brillance control visibility
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0123—Head-up displays characterised by optical features comprising devices increasing the field of view
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/0138—Head-up displays characterised by optical features comprising image capture systems, e.g. camera
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B27/0172—Head mounted characterised by optical features
- G02B2027/0174—Head mounted characterised by optical features holographic
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
- G02B2027/0178—Eyeglass type
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2215/00—Indexing scheme for image rendering
- G06T2215/16—Using real world measurements to influence rendering
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N13/00—Stereoscopic video systems; Multi-view video systems; Details thereof
- H04N13/30—Image reproducers
- H04N2013/40—Privacy aspects, i.e. devices showing different images to different viewers, the images not being viewpoints of the same scene
- H04N2013/405—Privacy aspects, i.e. devices showing different images to different viewers, the images not being viewpoints of the same scene the images being stereoscopic or three dimensional
Abstract
Description
- This application claims the benefit of priority to and is a continuation of the following U.S. patent applications, each of which is hereby incorporated by reference in its entirety:
- U.S. non-provisional application Ser. No. 14/228,526, entitled SENSOR DEPENDENT CONTENT POSITION IN HEAD WORN COMPUTING, filed Mar. 28, 2014.
- 1. Field of the Invention
- This invention relates to head worn computing. More particularly, this invention positioning content within an FOV of a head worn computing depending on sensor feedback.
- 2. Description of Related Art
- Wearable computing systems have been developed and are beginning to be commercialized. Many problems persist in the wearable computing field that need to be resolved to make them meet the demands of the market.
- Aspects of the present invention relate to methods and systems for positioning content within an FOV of a head worn computing depending on sensor feedback.
- These and other systems, methods, objects, features, and advantages of the present invention will be apparent to those skilled in the art from the following detailed description of the preferred embodiment and the drawings. All documents mentioned herein are hereby incorporated in their entirety by reference.
- Embodiments are described with reference to the following Figures. The same numbers may be used throughout to reference like features and components that are shown in the Figures:
-
FIG. 1 illustrates a head worn computing system in accordance with the principles of the present invention. -
FIG. 2 illustrates a head worn computing system with optical system in accordance with the principles of the present invention. -
FIG. 3 a illustrates a large prior art optical arrangement. -
FIG. 3 b illustrates an upper optical module in accordance with the principles of the present invention. -
FIG. 4 illustrates an upper optical module in accordance with the principles of the present invention. -
FIG. 4 a illustrates an upper optical module in accordance with the principles of the present invention. -
FIG. 4 b illustrates an upper optical module in accordance with the principles of the present invention. -
FIG. 5 illustrates an upper optical module in accordance with the principles of the present invention. -
FIG. 5 a illustrates an upper optical module in accordance with the principles of the present invention. -
FIG. 5 b illustrates an upper optical module and dark light trap according to the principles of the present invention. -
FIG. 5 c illustrates an upper optical module and dark light trap according to the principles of the present invention. -
FIG. 5 d illustrates an upper optical module and dark light trap according to the principles of the present invention. -
FIG. 5 e illustrates an upper optical module and dark light trap according to the principles of the present invention. -
FIG. 6 illustrates upper and lower optical modules in accordance with the principles of the present invention. -
FIG. 7 illustrates angles of combiner elements in accordance with the principles of the present invention. -
FIG. 8 illustrates upper and lower optical modules in accordance with the principles of the present invention. -
FIG. 8 a illustrates upper and lower optical modules in accordance with the principles of the present invention. -
FIG. 8 b illustrates upper and lower optical modules in accordance with the principles of the present invention. -
FIG. 8 c illustrates upper and lower optical modules in accordance with the principles of the present invention. -
FIG. 9 illustrates an eye imaging system in accordance with the principles of the present invention. -
FIG. 10 illustrates a light source in accordance with the principles of the present invention. -
FIG. 10 a illustrates a back lighting system in accordance with the principles of the present invention. -
FIG. 10 b illustrates a back lighting system in accordance with the principles of the present invention. -
FIGS. 11 a to 11 d illustrate light source and filters in accordance with the principles of the present invention. -
FIGS. 12 a to 12 c illustrate light source and quantum dot systems in accordance with the principles of the present invention. -
FIGS. 13 a to 13 c illustrate peripheral lighting systems in accordance with the principles of the present invention. -
FIGS. 14 a to 14 c illustrate a light suppression systems in accordance with the principles of the present invention. -
FIG. 15 illustrates an external user interface in accordance with the principles of the present invention. -
FIGS. 16 a to 16 c illustrate distance control systems in accordance with the principles of the present invention. -
FIGS. 17 a to 17 c illustrate force interpretation systems in accordance with the principles of the present invention. -
FIGS. 18 a to 18 c illustrate user interface mode selection systems in accordance with the principles of the present invention. -
FIG. 19 illustrates interaction systems in accordance with the principles of the present invention. -
FIG. 20 illustrates external user interfaces in accordance with the principles of the present invention. -
FIG. 21 illustrates mD trace representations presented in accordance with the principles of the present invention. -
FIG. 22 illustrates mD trace representations presented in accordance with the principles of the present invention. -
FIG. 23 illustrates an mD scanned environment in accordance with the principles of the present invention. -
FIG. 23 a illustrates mD trace representations presented in accordance with the principles of the present invention. -
FIG. 24 illustrates a stray light suppression technology in accordance with the principles of the present invention. -
FIG. 25 illustrates a stray light suppression technology in accordance with the principles of the present invention. -
FIG. 26 illustrates a stray light suppression technology in accordance with the principles of the present invention. -
FIG. 27 illustrates a stray light suppression technology in accordance with the principles of the present invention. -
FIGS. 28 a to 28 c illustrate DLP mirror angles. -
FIGS. 29 to 33 illustrate eye imaging systems according to the principles of the present invention. -
FIGS. 34 and 34 a illustrate structured eye lighting systems according to the principles of the present invention. -
FIG. 35 illustrates eye glint in the prediction of eye direction analysis in accordance with the principles of the present invention. -
FIG. 36 a illustrates eye characteristics that may be used in personal identification through analysis of a system according to the principles of the present invention. -
FIG. 36 b illustrates a digital content presentation reflection off of the wearer's eye that may be analyzed in accordance with the principles of the present invention. -
FIG. 37 illustrates eye imaging along various virtual target lines and various focal planes in accordance with the principles of the present invention. -
FIG. 38 illustrates content control with respect to eye movement based on eye imaging in accordance with the principles of the present invention. -
FIG. 39 illustrates eye imaging and eye convergence in accordance with the principles of the present invention. -
FIG. 40 illustrates content position dependent on sensor feedback in accordance with the principles of the present invention. -
FIG. 41 illustrates content position dependent on sensor feedback in accordance with the principles of the present invention. -
FIG. 42 illustrates content position dependent on sensor feedback in accordance with the principles of the present invention. -
FIG. 43 illustrates content position dependent on sensor feedback in accordance with the principles of the present invention. -
FIG. 44 illustrates content position dependent on sensor feedback in accordance with the principles of the present invention. -
FIG. 45 illustrates various headings over time in an example. -
FIG. 46 illustrates content position dependent on sensor feedback in accordance with the principles of the present invention. -
FIG. 47 illustrates content position dependent on sensor feedback in accordance with the principles of the present invention. -
FIG. 48 illustrates content position dependent on sensor feedback in accordance with the principles of the present invention. -
FIG. 49 illustrates content position dependent on sensor feedback in accordance with the principles of the present invention. - While the invention has been described in connection with certain preferred embodiments, other embodiments would be understood by one of ordinary skill in the art and are encompassed herein.
- Aspects of the present invention relate to head-worn computing (“HWC”) systems. HWC involves, in some instances, a system that mimics the appearance of head-worn glasses or sunglasses. The glasses may be a fully developed computing platform, such as including computer displays presented in each of the lenses of the glasses to the eyes of the user. In embodiments, the lenses and displays may be configured to allow a person wearing the glasses to see the environment through the lenses while also seeing, simultaneously, digital imagery, which forms an overlaid image that is perceived by the person as a digitally augmented image of the environment, or augmented reality (“AR”).
- HWC involves more than just placing a computing system on a person's head. The system may need to be designed as a lightweight, compact and fully functional computer display, such as wherein the computer display includes a high resolution digital display that provides a high level of emersion comprised of the displayed digital content and the see-through view of the environmental surroundings. User interfaces and control systems suited to the HWC device may be required that are unlike those used for a more conventional computer such as a laptop. For the HWC and associated systems to be most effective, the glasses may be equipped with sensors to determine environmental conditions, geographic location, relative positioning to other points of interest, objects identified by imaging and movement by the user or other users in a connected group, and the like. The HWC may then change the mode of operation to match the conditions, location, positioning, movements, and the like, in a method generally referred to as a contextually aware HWC. The glasses also may need to be connected, wirelessly or otherwise, to other systems either locally or through a network. Controlling the glasses may be achieved through the use of an external device, automatically through contextually gathered information, through user gestures captured by the glasses sensors, and the like. Each technique may be further refined depending on the software application being used in the glasses. The glasses may further be used to control or coordinate with external devices that are associated with the glasses.
- Referring to
FIG. 1 , an overview of theHWC system 100 is presented. As shown, theHWC system 100 comprises aHWC 102, which in this instance is configured as glasses to be worn on the head with sensors such that theHWC 102 is aware of the objects and conditions in theenvironment 114. In this instance, theHWC 102 also receives and interprets control inputs such as gestures andmovements 116. TheHWC 102 may communicate withexternal user interfaces 104. Theexternal user interfaces 104 may provide a physical user interface to take control instructions from a user of theHWC 102 and theexternal user interfaces 104 and theHWC 102 may communicate bi-directionally to affect the user's command and provide feedback to theexternal device 108. TheHWC 102 may also communicate bi-directionally with externally controlled or coordinatedlocal devices 108. For example, anexternal user interface 104 may be used in connection with theHWC 102 to control an externally controlled or coordinatedlocal device 108. The externally controlled or coordinatedlocal device 108 may provide feedback to theHWC 102 and a customized GUI may be presented in theHWC 102 based on the type of device or specifically identifieddevice 108. TheHWC 102 may also interact with remote devices andinformation sources 112 through anetwork connection 110. Again, theexternal user interface 104 may be used in connection with theHWC 102 to control or otherwise interact with any of theremote devices 108 andinformation sources 112 in a similar way as when theexternal user interfaces 104 are used to control or otherwise interact with the externally controlled or coordinatedlocal devices 108. Similarly,HWC 102 may interpret gestures 116 (e.g captured from forward, downward, upward, rearward facing sensors such as camera(s), range finders, IR sensors, etc.) or environmental conditions sensed in theenvironment 114 to control either local orremote devices - We will now describe each of the main elements depicted on
FIG. 1 in more detail; however, these descriptions are intended to provide general guidance and should not be construed as limiting. Additional description of each element may also be further described herein. - The
HWC 102 is a computing platform intended to be worn on a person's head. TheHWC 102 may take many different forms to fit many different functional requirements. In some situations, theHWC 102 will be designed in the form of conventional glasses. The glasses may or may not have active computer graphics displays. In situations where theHWC 102 has integrated computer displays the displays may be configured as see-through displays such that the digital imagery can be overlaid with respect to the user's view of theenvironment 114. There are a number of see-through optical designs that may be used, including ones that have a reflective display (e.g. LCoS, DLP), emissive displays (e.g. OLED, LED), hologram, TIR waveguides, and the like. In embodiments, lighting systems used in connection with the display optics may be solid state lighting systems, such as LED, OLED, quantum dot, quantum dot LED, etc. In addition, the optical configuration may be monocular or binocular. It may also include vision corrective optical components. In embodiments, the optics may be packaged as contact lenses. In other embodiments, theHWC 102 may be in the form of a helmet with a see-through shield, sunglasses, safety glasses, goggles, a mask, fire helmet with see-through shield, police helmet with see through shield, military helmet with see-through shield, utility form customized to a certain work task (e.g. inventory control, logistics, repair, maintenance, etc.), and the like. - The
HWC 102 may also have a number of integrated computing facilities, such as an integrated processor, integrated power management, communication structures (e.g. cell net, WiFi, Bluetooth, local area connections, mesh connections, remote connections (e.g. client server, etc.)), and the like. TheHWC 102 may also have a number of positional awareness sensors, such as GPS, electronic compass, altimeter, tilt sensor, IMU, and the like. It may also have other sensors such as a camera, rangefinder, hyper-spectral camera, Geiger counter, microphone, spectral illumination detector, temperature sensor, chemical sensor, biologic sensor, moisture sensor, ultrasonic sensor, and the like. - The
HWC 102 may also have integrated control technologies. The integrated control technologies may be contextual based control, passive control, active control, user control, and the like. For example, theHWC 102 may have an integrated sensor (e.g. camera) that captures user hand or body gestures 116 such that the integrated processing system can interpret the gestures and generate control commands for theHWC 102. In another example, theHWC 102 may have sensors that detect movement (e.g. a nod, head shake, and the like) including accelerometers, gyros and other inertial measurements, where the integrated processor may interpret the movement and generate a control command in response. TheHWC 102 may also automatically control itself based on measured or perceived environmental conditions. For example, if it is bright in the environment theHWC 102 may increase the brightness or contrast of the displayed image. In embodiments, the integrated control technologies may be mounted on theHWC 102 such that a user can interact with it directly. For example, theHWC 102 may have a button(s), touch capacitive interface, and the like. - As described herein, the
HWC 102 may be in communication withexternal user interfaces 104. The external user interfaces may come in many different forms. For example, a cell phone screen may be adapted to take user input for control of an aspect of theHWC 102. The external user interface may be a dedicated UI, such as a keyboard, touch surface, button(s), joy stick, and the like. In embodiments, the external controller may be integrated into another device such as a ring, watch, bike, car, and the like. In each case, theexternal user interface 104 may include sensors (e.g. IMU, accelerometers, compass, altimeter, and the like) to provide additional input for controlling theHWD 104. - As described herein, the
HWC 102 may control or coordinate with otherlocal devices 108. Theexternal devices 108 may be an audio device, visual device, vehicle, cell phone, computer, and the like. For instance, the localexternal device 108 may be anotherHWC 102, where information may then be exchanged between theseparate HWCs 108. - Similar to the way the
HWC 102 may control or coordinate with local devices 106, theHWC 102 may control or coordinate withremote devices 112, such as theHWC 102 communicating with theremote devices 112 through anetwork 110. Again, the form of theremote device 112 may have many forms. Included in these forms is anotherHWC 102. For example, eachHWC 102 may communicate its GPS position such that all theHWCs 102 know where all ofHWC 102 are located. -
FIG. 2 illustrates aHWC 102 with an optical system that includes an upperoptical module 202 and a loweroptical module 204. While the upper and loweroptical modules upper module 202 includes a computer controlled display (e.g. LCoS, DLP, OLED, etc.) and image light delivery optics. In embodiments, the lower module includes eye delivery optics that are configured to receive the upper module's image light and deliver the image light to the eye of a wearer of the HWC. InFIG. 2 , it should be noted that while the upper and loweroptical modules -
FIG. 3 b illustrates an upperoptical module 202 in accordance with the principles of the present invention. In this embodiment, the upperoptical module 202 includes a DLP (also known as DMD or digital micromirror device) computer operateddisplay 304 which includes pixels comprised of rotatable mirrors (such as, for example, the DLP3000 available from Texas Instruments), polarizedlight source 302, ¼wave retarder film 308,reflective polarizer 310 and afield lens 312. The polarizedlight source 302 provides substantially uniform polarized light that is generally directed towards thereflective polarizer 310. The reflective polarizer reflects light of one polarization state (e.g. S polarized light) and transmits light of the other polarization state (e.g. P polarized light). The polarizedlight source 302 and thereflective polarizer 310 are oriented so that the polarized light from the polarizedlight source 302 is reflected generally towards theDLP 304. The light then passes through the ¼wave film 308 once before illuminating the pixels of theDLP 304 and then again after being reflected by the pixels of theDLP 304. In passing through the ¼wave film 308 twice, the light is converted from one polarization state to the other polarization state (e.g. the light is converted from S to P polarized light). The light then passes through thereflective polarizer 310. In the event that the DLP pixel(s) are in the “on” state (i.e. the mirrors are positioned to reflect light towards thefield lens 312, the “on” pixels reflect the light generally along the optical axis and into thefield lens 312. This light that is reflected by “on” pixels and which is directed generally along the optical axis of thefield lens 312 will be referred to asimage light 316. Theimage light 316 then passes through the field lens to be used by a loweroptical module 204. - The light that is provided by the polarized
light source 302, which is subsequently reflected by thereflective polarizer 310 before it reflects from theDLP 304, will generally be referred to as illumination light. The light that is reflected by the “off” pixels of theDLP 304 is reflected at a different angle than the light reflected by the ‘on” pixels, so that the light from the “off” pixels is generally directed away from the optical axis of thefield lens 312 and toward the side of the upperoptical module 202 as shown inFIG. 3 . The light that is reflected by the “off” pixels of theDLP 304 will be referred to asdark state light 314. - The
DLP 304 operates as a computer controlled display and is generally thought of as a MEMs device. The DLP pixels are comprised of small mirrors that can be directed. The mirrors generally flip from one angle to another angle. The two angles are generally referred to as states. When light is used to illuminate the DLP the mirrors will reflect the light in a direction depending on the state. In embodiments herein, we generally refer to the two states as “on” and “off,” which is intended to depict the condition of a display pixel. “On” pixels will be seen by a viewer of the display as emitting light because the light is directed along the optical axis and into the field lens and the associated remainder of the display system. “Off” pixels will be seen by a viewer of the display as not emitting light because the light from these pixels is directed to the side of the optical housing and into a light trap or light dump where the light is absorbed. The pattern of “on” and “off” pixels produces image light that is perceived by a viewer of the display as a computer generated image. Full color images can be presented to a user by sequentially providing illumination light with complimentary colors such as red, green and blue. Where the sequence is presented in a recurring cycle that is faster than the user can perceive as separate images and as a result the user perceives a full color image comprised of the sum of the sequential images. Bright pixels in the image are provided by pixels that remain in the “on” state for the entire time of the cycle, while dimmer pixels in the image are provided by pixels that switch between the “on” state and “off” state within the time of the cycle, or frame time when in a video sequence of images. -
FIG. 3 a shows an illustration of a system for aDLP 304 in which the unpolarizedlight source 350 is pointed directly at theDLP 304. In this case, the angle required for the illumination light is such that thefield lens 352 must be positioned substantially distant from theDLP 304 to avoid the illumination light from being clipped by thefield lens 352. The large distance between thefield lens 352 and theDLP 304 along with the straight path of the dark state light 354, means that the light trap for the dark state light 354 is also located at a substantial distance from the DLP. For these reasons, this configuration is larger in size compared to theupper optics module 202 of the preferred embodiments. - The configuration illustrated in
FIG. 3 b can be lightweight and compact such that it fits into a small portion of a HWC. For example, theupper modules 202 illustrated herein can be physically adapted to mount in an upper frame of a HWC such that the image light can be directed into a loweroptical module 204 for presentation of digital content to a wearer's eye. The package of components that combine to generate the image light (i.e. the polarizedlight source 302,DLP 304,reflective polarizer 310 and ¼ wave film 308) is very light and is compact. The height of the system, excluding the field lens, may be less than 8 mm. The width (i.e. from front to back) may be less than 8 mm. The weight may be less than 2 grams. The compactness of this upperoptical module 202 allows for a compact mechanical design of the HWC and the light weight nature of these embodiments help make the HWC lightweight to provide for a HWC that is comfortable for a wearer of the HWC. - The configuration illustrated in
FIG. 3 b can produce sharp contrast, high brightness and deep blacks, especially when compared to LCD or LCoS displays used in HWC. The “on” and “off” states of the DLP provide for a strong differentiator in the light reflection path representing an “on” pixel and an “off” pixel. As will be discussed in more detail below, the dark state light from the “off” pixel reflections can be managed to reduce stray light in the display system to produce images with high contrast. -
FIG. 4 illustrates another embodiment of an upperoptical module 202 in accordance with the principles of the present invention. This embodiment includes alight source 404, but in this case, the light source can provide unpolarized illumination light. The illumination light from thelight source 404 is directed into aTIR wedge 418 such that the illumination light is incident on an internal surface of the TIR wedge 418 (shown as the angled lower surface of theTRI wedge 418 inFIG. 4 ) at an angle that is beyond the critical angle as defined byEqn 1. -
Critical angle=arc−sin(1/n)Eqn 1 - Where the critical angle is the angle beyond which the illumination light is reflected from the internal surface when the internal surface comprises an interface from a solid with a higher refractive index (n) to air with a refractive index of 1 (e.g. for an interface of acrylic, with a refractive index of n=1.5, to air, the critical angle is 41.8 degrees; for an interface of polycarbonate, with a refractive index of n=1.59, to air the critical angle is 38.9 degrees). Consequently, the
TIR wedge 418 is associated with athin air gap 408 along the internal surface to create an interface between a solid with a higher refractive index and air. By choosing the angle of thelight source 404 relative to theDLP 402 in correspondence to the angle of the internal surface of theTIR wedge 418, illumination light is turned toward theDLP 402 at an angle suitable for providing image light 414 as reflected from “on” pixels. Wherein, the illumination light is provided to theDLP 402 at approximately twice the angle of the pixel mirrors in theDLP 402 that are in the “on” state, such that after reflecting from the pixel mirrors, theimage light 414 is directed generally along the optical axis of the field lens. Depending on the state of the DLP pixels, the illumination light from “on” pixels may be reflected as image light 414 which is directed towards a field lens and a loweroptical module 204, while illumination light reflected from “off” pixels (generally referred to herein as “dark” state light, “off” pixel light or “off” state light) 410 is directed in a separate direction, which may be trapped and not used for the image that is ultimately presented to the wearer's eye. - The light trap for the dark state light 410 may be located along the optical axis defined by the direction of the dark state light 410 and in the side of the housing, with the function of absorbing the dark state light. To this end, the light trap may be comprised of an area outside of the cone of image light 414 from the “on” pixels. The light trap is typically made up of materials that absorb light including coatings of black paints or other light absorbing materials to prevent light scattering from the dark state light degrading the image perceived by the user. In addition, the light trap may be recessed into the wall of the housing or include masks or guards to block scattered light and prevent the light trap from being viewed adjacent to the displayed image.
- The embodiment of
FIG. 4 also includes acorrective wedge 420 to correct the effect of refraction of theimage light 414 as it exits theTIR wedge 418. By including thecorrective wedge 420 and providing a thin air gap 408 (e.g. 25 micron), the image light from the “on” pixels can be maintained generally in a direction along the optical axis of the field lens (i.e. the same direction as that defined by the image light 414) so it passes into the field lens and the loweroptical module 204. As shown inFIG. 4 , the image light 414 from the “on” pixels exits thecorrective wedge 420 generally perpendicular to the surface of thecorrective wedge 420 while the dark state light exits at an oblique angle. As a result, the direction of the image light 414 from the “on” pixels is largely unaffected by refraction as it exits from the surface of thecorrective wedge 420. In contrast, the dark state light 410 is substantially changed in direction by refraction when the dark state light 410 exits thecorrective wedge 420. - The embodiment illustrated in
FIG. 4 has the similar advantages of those discussed in connection with the embodiment ofFIG. 3 b. The dimensions and weight of theupper module 202 depicted inFIG. 4 may be approximately 8×8 mm with a weight of less than 3 grams. A difference in overall performance between the configuration illustrated inFIG. 3 b and the configuration illustrated inFIG. 4 is that the embodiment ofFIG. 4 doesn't require the use of polarized light as supplied by thelight source 404. This can be an advantage in some situations as will be discussed in more detail below (e.g. increased see-through transparency of the HWC optics from the user's perspective). Polarized light may be used in connection with the embodiment depicted inFIG. 4 , in embodiments. An additional advantage of the embodiment ofFIG. 4 compared to the embodiment shown inFIG. 3 b is that the dark state light (shown as DLP off light 410) is directed at a steeper angle away from the optical axis of theimage light 414 due to the added refraction encountered when the dark state light 410 exits thecorrective wedge 420. This steeper angle of the dark state light 410 allows for the light trap to be positioned closer to theDLP 402 so that the overall size of theupper module 202 can be reduced. The light trap can also be made larger since the light trap doesn't interfere with the field lens, thereby the efficiency of the light trap can be increased and as a result, stray light can be reduced and the contrast of the image perceived by the user can be increased.FIG. 4 a illustrates the embodiment described in connection withFIG. 4 with an example set of corresponding angles at the various surfaces with the reflected angles of a ray of light passing through the upperoptical module 202. In this example, the DLP mirrors are provided at 17 degrees to the surface of the DLP device. The angles of the TIR wedge are selected in correspondence to one another to provide TIR reflected illumination light at the correct angle for the DLP mirrors while allowing the image light and dark state light to pass through the thin air gap, various combinations of angles are possible to achieve this. -
FIG. 5 illustrates yet another embodiment of an upperoptical module 202 in accordance with the principles of the present invention. As with the embodiment shown inFIG. 4 , the embodiment shown inFIG. 5 does not require the use of polarized light. Polarized light may be used in connection with this embodiment, but it is not required. Theoptical module 202 depicted inFIG. 5 is similar to that presented in connection withFIG. 4 ; however, the embodiment ofFIG. 5 includes an offlight redirection wedge 502. As can be seen from the illustration, the offlight redirection wedge 502 allows theimage light 414 to continue generally along the optical axis toward the field lens and into the lower optical module 204 (as illustrated). However, theoff light 504 is redirected substantially toward the side of thecorrective wedge 420 where it passes into the light trap. This configuration may allow further height compactness in the HWC because the light trap (not illustrated) that is intended to absorb theoff light 504 can be positioned laterally adjacent the upperoptical module 202 as opposed to below it. In the embodiment depicted inFIG. 5 there is a thin air gap between theTIR wedge 418 and the corrective wedge 420 (similar to the embodiment ofFIG. 4 ). There is also a thin air gap between thecorrective wedge 420 and the offlight redirection wedge 502. There may be HWC mechanical configurations that warrant the positioning of a light trap for the dark state light elsewhere and the illustration depicted inFIG. 5 should be considered illustrative of the concept that the off light can be redirected to create compactness of the overall HWC.FIG. 5 a illustrates an example of the embodiment described in connection withFIG. 5 with the addition of more details on the relative angles at the various surfaces and a light ray trace for image light and a light ray trace for dark light are shown as it passes through the upperoptical module 202. Again, various combinations of angles are possible. -
FIG. 4 b shows an illustration of a further embodiment in which a solid transparent matched set ofwedges 456 is provided with areflective polarizer 450 at the interface between the wedges. Wherein the interface between the wedges in the wedge set 456 is provided at an angle so that illumination light 452 from the polarizedlight source 458 is reflected at the proper angle (e.g. 34 degrees for a 17 degree DLP mirror) for the DLP mirror “on” state so that the reflectedimage light 414 is provided along the optical axis of the field lens. The general geometry of the wedges in the wedge set 456 is similar to that shown inFIGS. 4 and 4 a. Aquarter wave film 454 is provided on theDLP 402 surface so that theillumination light 452 is one polarization state (e.g. S polarization state) while in passing through thequarter wave film 454, reflecting from the DLP mirror and passing back through thequarter wave film 454, theimage light 414 is converted to the other polarization state (e.g. P polarization state). The reflective polarizer is oriented such that theillumination light 452 with it's polarization state is reflected and theimage light 414 with it's other polarization state is transmitted. Since the dark state light from the “offpixels 410 also passes through thequarter wave film 454 twice, it is also the other polarization state (e.g. P polarization state) so that it is transmitted by thereflective polarizer 450. - The angles of the faces of the wedge set 450 correspond to the needed angles to provide
illumination light 452 at the angle needed by the DLP mirrors when in the “on” state so that the reflectedimage light 414 is reflected from the DLP along the optical axis of the field lens. The wedge set 456 provides an interior interface where a reflective polarizer film can be located to redirect theillumination light 452 toward the mirrors of theDLP 402. The wedge set also provides a matched wedge on the opposite side of thereflective polarizer 450 so that the image light 414 from the “on” pixels exits the wedge set 450 substantially perpendicular to the exit surface, while the dark state light from the ‘off’pixels 410 exits at an oblique angle to the exit surface. As a result, theimage light 414 is substantially unrefracted upon exiting the wedge set 456, while the dark state light from the “off”pixels 410 is substantially refracted upon exiting the wedge set 456 as shown inFIG. 4 b. - By providing a solid transparent matched wedge set, the flatness of the interface is reduced, because variations in the flatness have a negligible effect as long as they are within the cone angle of the illuminating
light 452. Which can be f#2.2 with a 26 degree cone angle. In a preferred embodiment, the reflective polarizer is bonded between the matched internal surfaces of the wedge set 456 using an optical adhesive so that Fresnel reflections at the interfaces on either side of thereflective polarizer 450 are reduced. The optical adhesive can be matched in refractive index to the material of the wedge set 456 and the pieces of the wedge set 456 can be all made from the same material such as BK7 glass or cast acrylic. Wherein the wedge material can be selected to have low birefringence as well to reduce non-uniformities in brightness. The wedge set 456 and thequarter wave film 454 can also be bonded to theDLP 402 to further reduce Fresnel reflections at the DLP interface losses. In addition, since theimage light 414 is substantially normal to the exit surface of the wedge set 456, the flatness of the surface is not critical to maintain the wavefront of theimage light 414 so that high image quality can be obtained in the displayed image without requiring very tightly toleranced flatness on the exit surface. - A yet further embodiment of the invention that is not illustrated, combines the embodiments illustrated in
FIG. 4 b andFIG. 5 . In this embodiment, the wedge set 456 is comprised of three wedges with the general geometry of the wedges in the wedge set corresponding to that shown inFIGS. 5 and 5 a. A reflective polarizer is bonded between the first and second wedges similar to that shown inFIG. 4 b, however, a third wedge is provided similar to the embodiment ofFIG. 5 . Wherein there is an angled thin air gap between the second and third wedges so that the dark state light is reflected by TIR toward the side of the second wedge where it is absorbed in a light trap. This embodiment, like the embodiment shown inFIG. 4 b, uses a polarized light source as has been previously described. The difference in this embodiment is that the image light is transmitted through the reflective polarizer and is transmitted through the angled thin air gap so that it exits normal to the exit surface of the third wedge. -
FIG. 5 b illustrates an upperoptical module 202 with a darklight trap 514 a. As described in connection withFIGS. 4 and 4 a, image light can be generated from a DLP when using a TIR and corrective lens configuration. The upper module may be mounted in aHWC housing 510 and thehousing 510 may include a darklight trap 514 a. The darklight trap 514 a is generally positioned/constructed/formed in a position that is optically aligned with the dark lightoptical axis 512. As illustrated, the dark light trap may have depth such that the trap internally reflects dark light in an attempt to further absorb the light and prevent the dark light from combining with the image light that passes through the field lens. The dark light trap may be of a shape and depth such that it absorbs the dark light. In addition, the darklight trap 514 b, in embodiments, may be made of light absorbing materials or coated with light absorbing materials. In embodiments, the recessedlight trap 514 a may include baffles to block a view of the dark state light. This may be combined with black surfaces and textured or fiberous surfaces to help absorb the light. The baffles can be part of the light trap, associated with the housing, or field lens, etc. -
FIG. 5 c illustrates another embodiment with alight trap 514 b. As can be seen in the illustration, the shape of the trap is configured to enhance internal reflections within thelight trap 514 b to increase the absorption of thedark light 512.FIG. 5 d illustrates another embodiment with alight trap 514 c. As can be seen in the illustration, the shape of thetrap 514 c is configured to enhance internal reflections to increase the absorption of thedark light 512. -
FIG. 5 e illustrates another embodiment of an upperoptical module 202 with a darklight trap 514 d. This embodiment ofupper module 202 includes an offlight reflection wedge 502, as illustrated and described in connection with the embodiment ofFIGS. 5 and 5 a. As can be seen inFIG. 5 e, thelight trap 514 d is positioned along the optical path of thedark light 512. The darklight trap 514 d may be configured as described in other embodiments herein. The embodiment of thelight trap 514 d illustrated inFIG. 5 e includes a black area on the side wall of the wedge, wherein the side wall is located substantially away from the optical axis of theimage light 414. In addition, baffles 5252 may be added to one or more edges of thefield lens 312 to block the view of thelight trap 514 d adjacent to the displayed image seen by the user. -
FIG. 6 illustrates a combination of an upperoptical module 202 with a loweroptical module 204. In this embodiment, the image light projected from the upperoptical module 202 may or may not be polarized. The image light is reflected off aflat combiner element 602 such that it is directed towards the user's eye. Wherein, thecombiner element 602 is a partial mirror that reflects image light while transmitting a substantial portion of light from the environment so the user can look through the combiner element and see the environment surrounding the HWC. - The
combiner 602 may include a holographic pattern, to form a holographic mirror. If a monochrome image is desired, there may be a single wavelength reflection design for the holographic pattern on the surface of thecombiner 602. If the intention is to have multiple colors reflected from the surface of thecombiner 602, a multiple wavelength holographic mirror maybe included on the combiner surface. For example, in a three-color embodiment, where red, green and blue pixels are generated in the image light, the holographic mirror may be reflective to wavelengths substantially matching the wavelengths of the red, green and blue light provided by the light source. This configuration can be used as a wavelength specific mirror where pre-determined wavelengths of light from the image light are reflected to the user's eye. This configuration may also be made such that substantially all other wavelengths in the visible pass through thecombiner element 602 so the user has a substantially clear view of the surroundings when looking through thecombiner element 602. The transparency between the user's eye and the surrounding may be approximately 80% when using a combiner that is a holographic mirror. Wherein holographic mirrors can be made using lasers to produce interference patterns in the holographic material of the combiner where the wavelengths of the lasers correspond to the wavelengths of light that are subsequently reflected by the holographic mirror. - In another embodiment, the
combiner element 602 may include a notch mirror comprised of a multilayer coated substrate wherein the coating is designed to substantially reflect the wavelengths of light provided by the light source and substantially transmit the remaining wavelengths in the visible spectrum. For example, in the case where red, green and blue light is provided by the light source to enable full color images to be provided to the user, the notch mirror is a tristimulus notch mirror wherein the multilayer coating is designed to reflect narrow bands of red, green and blue light that are matched to the what is provided by the light source and the remaining visible wavelengths are transmitted through the coating to enable a view of the environment through the combiner. In another example where monochrome images are provided to the user, the notch mirror is designed to reflect a single narrow band of light that is matched to the wavelength range of the light provided by the light source while transmitting the remaining visible wavelengths to enable a see-thru view of the environment. Thecombiner 602 with the notch mirror would operate, from the user's perspective, in a manner similar to the combiner that includes a holographic pattern on thecombiner element 602. The combiner, with the tristimulus notch mirror, would reflect the “on” pixels to the eye because of the match between the reflective wavelengths of the notch mirror and the color of the image light, and the wearer would be able to see with high clarity the surroundings. The transparency between the user's eye and the surrounding may be approximately 80% when using the tristimulus notch mirror. In addition, the image provided by the upperoptical module 202 with the notch mirror combiner can provide higher contrast images than the holographic mirror combiner due to less scattering of the imaging light by the combiner. - Light can escape through the
combiner 602 and may produce face glow as the light is generally directed downward onto the cheek of the user. When using a holographic mirror combiner or a tristimulus notch mirror combiner, the escaping light can be trapped to avoid face glow. In embodiments, if the image light is polarized before the combiner, a linear polarizer can be laminated, or otherwise associated, to the combiner, with the transmission axis of the polarizer oriented relative to the polarized image light so that any escaping image light is absorbed by the polarizer. In embodiments, the image light would be polarized to provide S polarized light to the combiner for better reflection. As a result, the linear polarizer on the combiner would be oriented to absorb S polarized light and pass P polarized light. This provides the preferred orientation of polarized sunglasses as well. - If the image light is unpolarized, a microlouvered film such as a privacy filter can be used to absorb the escaping image light while providing the user with a see-thru view of the environment. In this case, the absorbance or transmittance of the microlouvered film is dependent on the angle of the light. Where steep angle light is absorbed and light at less of an angle is transmitted. For this reason, in an embodiment, the combiner with the microlouver film is angled at greater than 45 degrees to the optical axis of the image light (e.g. the combiner can be oriented at 50 degrees so the image light from the file lens is incident on the combiner at an oblique angle.
-
FIG. 7 illustrates an embodiment of acombiner element 602 at various angles when thecombiner element 602 includes a holographic mirror. Normally, a mirrored surface reflects light at an angle equal to the angle that the light is incident to the mirrored surface. Typically, this necessitates that the combiner element be at 45 degrees, 602 a, if the light is presented vertically to the combiner so the light can be reflected horizontally towards the wearer's eye. In embodiments, the incident light can be presented at angles other than vertical to enable the mirror surface to be oriented at other than 45 degrees, but in all cases wherein a mirrored surface is employed (including the tristimulus notch mirror described previously), the incident angle equals the reflected angle. As a result, increasing the angle of thecombiner 602 a requires that the incident image light be presented to thecombiner 602 a at a different angle which positions the upperoptical module 202 to the left of the combiner as shown inFIG. 7 . In contrast, a holographic mirror combiner, included in embodiments, can be made such that light is reflected at a different angle from the angle that the light is incident onto the holographic mirrored surface. This allows freedom to select the angle of thecombiner element 602 b independent of the angle of the incident image light and the angle of the light reflected into the wearer's eye. In embodiments, the angle of thecombiner element 602 b is greater than 45 degrees (shown inFIG. 7 ) as this allows a more laterally compact HWC design. The increased angle of thecombiner element 602 b decreases the front to back width of the loweroptical module 204 and may allow for a thinner HWC display (i.e. the furthest element from the wearer's eye can be closer to the wearer's face). -
FIG. 8 illustrates another embodiment of a loweroptical module 204. In this embodiment, polarized image light provided by the upperoptical module 202, is directed into the loweroptical module 204. The image light reflects off apolarized mirror 804 and is directed to a focusing partiallyreflective mirror 802, which is adapted to reflect the polarized light. An optical element such as a ¼ wave film located between thepolarized mirror 804 and the partiallyreflective mirror 802, is used to change the polarization state of the image light such that the light reflected by the partiallyreflective mirror 802 is transmitted by thepolarized mirror 804 to present image light to the eye of the wearer. The user can also see through thepolarized mirror 804 and the partiallyreflective mirror 802 to see the surrounding environment. As a result, the user perceives a combined image comprised of the displayed image light overlaid onto the see-thru view of the environment. - While many of the embodiments of the present invention have been referred to as upper and lower modules containing certain optical components, it should be understood that the image light and dark light production and management functions described in connection with the upper module may be arranged to direct light in other directions (e.g. upward, sideward, etc.). In embodiments, it may be preferred to mount the
upper module 202 above the wearer's eye, in which case the image light would be directed downward. In other embodiments it may be preferred to produce light from the side of the wearer's eye, or from below the wearer's eye. In addition, the lower optical module is generally configured to deliver the image light to the wearer's eye and allow the wearer to see through the lower optical module, which may be accomplished through a variety of optical components. -
FIG. 8 a illustrates an embodiment of the present invention where the upperoptical module 202 is arranged to direct image light into aTIR waveguide 810. In this embodiment, the upperoptical module 202 is positioned above the wearer'seye 812 and the light is directed horizontally into theTIR waveguide 810. The TIR waveguide is designed to internally reflect the image light in a series of downward TIR reflections until it reaches the portion in front of the wearer's eye, where the light passes out of theTIR waveguide 812 into the wearer's eye. In this embodiment, anouter shield 814 is positioned in front of theTIR waveguide 810. -
FIG. 8 b illustrates an embodiment of the present invention where the upperoptical module 202 is arranged to direct image light into aTIR waveguide 818. In this embodiment, the upperoptical module 202 is arranged on the side of theTIR waveguide 818. For example, the upper optical module may be positioned in the arm or near the arm of the HWC when configured as a pair of head worn glasses. TheTIR waveguide 818 is designed to internally reflect the image light in a series of TIR reflections until it reaches the portion in front of the wearer's eye, where the light passes out of theTIR waveguide 812 into the wearer's eye. -
FIG. 8 c illustrates yet further embodiments of the present invention where an upperoptical module 202 is directing polarized image light into anoptical guide 828 where the image light passes through apolarized reflector 824, changes polarization state upon reflection of theoptical element 822 which includes a ¼ wave film for example and then is reflected by thepolarized reflector 824 towards the wearer's eye, due to the change in polarization of the image light. The upperoptical module 202 may be positioned to direct light to amirror 820, to position the upperoptical module 202 laterally, in other embodiments, the upperoptical module 202 may direct the image light directly towards thepolarized reflector 824. It should be understood that the present invention comprises other optical arrangements intended to direct image light into the wearer's eye. - Another aspect of the present invention relates to eye imaging. In embodiments, a camera is used in connection with an upper
optical module 202 such that the wearer's eye can be imaged using pixels in the “off” state on the DLP.FIG. 9 illustrates a system where theeye imaging camera 802 is mounted and angled such that the field of view of theeye imaging camera 802 is redirected toward the wearer's eye by the mirror pixels of theDLP 402 that are in the “off” state. In this way, theeye imaging camera 802 can be used to image the wearer's eye along the same optical axis as the displayed image that is presented to the wearer. Wherein, image light that is presented to the wearer's eye illuminates the wearer's eye so that the eye can be imaged by theeye imaging camera 802. In the process, the light reflected by the eye passes back though the optical train of the loweroptical module 204 and a portion of the upper optical module to where the light is reflected by the “off” pixels of theDLP 402 toward theeye imaging camera 802. - In embodiments, the eye imaging camera may image the wearer's eye at a moment in time where there are enough “off” pixels to achieve the required eye image resolution. In another embodiment, the eye imaging camera collects eye image information from “off” pixels over time and forms a time lapsed image. In another embodiment, a modified image is presented to the user wherein enough “off” state pixels are included that the camera can obtain the desired resolution and brightness for imaging the wearer's eye and the eye image capture is synchronized with the presentation of the modified image.
- The eye imaging system may be used for security systems. The HWC may not allow access to the HWC or other system if the eye is not recognized (e.g. through eye characteristics including retina or iris characteristics, etc.). The HWC may be used to provide constant security access in some embodiments. For example, the eye security confirmation may be a continuous, near-continuous, real-time, quasi real-time, periodic, etc. process so the wearer is effectively constantly being verified as known. In embodiments, the HWC may be worn and eye security tracked for access to other computer systems.
- The eye imaging system may be used for control of the HWC. For example, a blink, wink, or particular eye movement may be used as a control mechanism for a software application operating on the HWC or associated device.
- The eye imaging system may be used in a process that determines how or when the
HWC 102 delivers digitally displayed content to the wearer. For example, the eye imaging system may determine that the user is looking in a direction and then HWC may change the resolution in an area of the display or provide some content that is associated with something in the environment that the user may be looking at. Alternatively, the eye imaging system may identify different user's and change the displayed content or enabled features provided to the user. User's may be identified from a database of users eye characteristics either located on theHWC 102 or remotely located on thenetwork 110 or on aserver 112. In addition, the HWC may identify a primary user or a group of primary users from eye characteristics wherein the primary user(s) are provided with an enhanced set of features and all other user's are provided with a different set of features. Thus in this use case, theHWC 102 uses identified eye characteristics to either enable features or not and eye characteristics need only be analyzed in comparison to a relatively small database of individual eye characteristics. -
FIG. 10 illustrates a light source that may be used in association with the upper optics module 202 (e.g. polarized light source if the light from the solid state light source is polarized such as polarizedlight source 302 and 458), andlight source 404. In embodiments, to provide a uniform surface of light 1008 to be directed into the upperoptical module 202 and towards the DLP of the upper optical module, either directly or indirectly, the solidstate light source 1002 may be projected into a backlightingoptical system 1004. The solidstate light source 1002 may be one or more LEDs, laser diodes, OLEDs. In embodiments, the backlightingoptical system 1004 includes an extended section with a length/distance ratio of greater than 3, wherein the light undergoes multiple reflections from the sidewalls to mix of homogenize the light as supplied by the solidstate light source 1002. The backlightingoptical system 1004 can also include structures on the surface opposite (on the left side as shown inFIG. 10 ) to where the uniform light 1008 exits thebacklight 1004 to change the direction of the light toward theDLP 302 and thereflective polarizer 310 or theDLP 402 and theTIR wedge 418. The backlightingoptical system 1004 may also include structures to collimate the uniform light 1008 to provide light to the DLP with a smaller angular distribution or narrower cone angle. Diffusers or polarizers can be used on the entrance or exit surface of the backlighting optical system. Diffusers can be used to spread or uniformize the exiting light from the backlight to improve the uniformity or increase the angular spread of theuniform light 1008. Elliptical diffusers that diffuse the light more in some directions and less in others can be used to improve the uniformity or spread of the uniform light 1008 in directions orthogonal to the optical axis of theuniform light 1008. Linear polarizers can be used to convert unpolarized light as supplied by the solidstate light source 1002 to polarized light so theuniform light 1008 is polarized with a desired polarization state. A reflective polarizer can be used on the exit surface of thebacklight 1004 to polarize the uniform light 1008 to the desired polarization state, while reflecting the other polarization state back into the backlight where it is recycled by multiple reflections within thebacklight 1004 and at the solidstate light source 1002. Therefore by including a reflective polarizer at the exit surface of thebacklight 1004, the efficiency of the polarized light source is improved. -
FIGS. 10 a and 10 b show illustrations of structures in backlightoptical systems 1004 that can be used to change the direction of the light provided to theentrance face 1045 by the light source and then collimates the light in a direction lateral to the optical axis of the exitinguniform light 1008.Structure 1060 includes an angled sawtooth pattern in a transparent waveguide wherein the left edge of each sawtooth clips the steep angle rays of light thereby limiting the angle of the light being redirected. The steep surface at the right (as shown) of each sawtooth then redirects the light so that it reflects off the left angled surface of each sawtooth and is directed toward theexit surface 1040. The sawtooth surfaces shown on the lower surface inFIGS. 10 a and 10 b, can be smooth and coated (e.g. with an aluminum coating or a dielectric mirror coating) to provide a high level of reflectivity without scattering.Structure 1050 includes a curved face on the left side (as shown) to focus the rays after they pass through theexit surface 1040, thereby providing a mechanism for collimating theuniform light 1008. In a further embodiment, a diffuser can be provided between the solidstate light source 1002 and theentrance face 1045 to homogenize the light provided by the solidstate light source 1002. In yet a further embodiment, a polarizer can be used between the diffuser and theentrance face 1045 of thebacklight 1004 to provide a polarized light source. Because the sawtooth pattern provides smooth reflective surfaces, the polarization state of the light can be preserved from theentrance face 1045 to theexit face 1040. In this embodiment, the light entering the backlight from the solidstate light source 1002 passes through the polarizer so that it is polarized with the desired polarization state. If the polarizer is an absorptive linear polarizer, the light of the desired polarization state is transmitted while the light of the other polarization state is absorbed. If the polarizer is a reflective polarizer, the light of the desired polarization state is transmitted into thebacklight 1004 while the light of the other polarization state is reflected back into the solidstate light source 1002 where it can be recycled as previously described, to increase the efficiency of the polarized light source. -
FIG. 11 a illustrates alight source 1100 that may be used in association with theupper optics module 202. In embodiments, thelight source 1100 may provide light to a backlightingoptical system 1004 as described above in connection withFIG. 10 . In embodiments, thelight source 1100 includes atristimulus notch filter 1102. Thetristimulus notch filter 1102 has narrow band pass filters for three wavelengths, as indicated inFIG. 11 c in atransmission graph 1108. The graph shown inFIG. 11 b, as 1104 illustrates an output of three different colored LEDs. One can see that the bandwidths of emission are narrow, but they have long tails. Thetristimulus notch filter 1102 can be used in connection with such LEDs to provide alight source 1100 that emits narrow filtered wavelengths of light as shown inFIG. 11 d as the transmission graph 1110. Wherein the clipping effects of thetristimulus notch filter 1102 can be seen to have cut the tails from theLED emission graph 1104 to provide narrower wavelength bands of light to the upperoptical module 202. Thelight source 1100 can be used in connection with acombiner 602 with a holographic mirror or tristimulus notch mirror to provide narrow bands of light that are reflected toward the wearer's eye with less waste light that does not get reflected by the combiner, thereby improving efficiency and reducing escaping light that can cause faceglow. -
FIG. 12 a illustrates anotherlight source 1200 that may be used in association with theupper optics module 202. In embodiments, thelight source 1200 may provide light to a backlightingoptical system 1004 as described above in connection withFIG. 10 . In embodiments, thelight source 1200 includes a quantumdot cover glass 1202. Where the quantum dots absorb light of a shorter wavelength and emit light of a longer wavelength (FIG. 12 b shows an example wherein aUV spectrum 1202 applied to a quantum dot results in the quantum dot emitting a narrow band shown as a PL spectrum 1204) that is dependent on the material makeup and size of the quantum dot. As a result, quantum dots in the quantumdot cover glass 1202 can be tailored to provide one or more bands of narrow bandwidth light (e.g. red, green and blue emissions dependent on the different quantum dots included as illustrated in the graph shown inFIG. 12 c where three different quantum dots are used. In embodiments, the LED driver light emits UV light, deep blue or blue light. For sequential illumination of different colors, multiplelight sources 1200 would be used where eachlight source 1200 would include a quantumdot cover glass 1202 with a quantum dot selected to emit at one of the desired colors. Thelight source 1100 can be used in connection with acombiner 602 with a holographic mirror or tristimulus notch mirror to provide narrow transmission bands of light that are reflected toward the wearer's eye with less waste light that does not get reflected. - Another aspect of the present invention relates to the generation of peripheral image lighting effects for a person wearing a HWC. In embodiments, a solid state lighting system (e.g. LED, OLED, etc), or other lighting system, may be included inside the optical elements of an lower
optical module 204. The solid state lighting system may be arranged such that lighting effects outside of a field of view (FOV) of the presented digital content is presented to create an emersive effect for the person wearing the HWC. To this end, the lighting effects may be presented to any portion of the HWC that is visible to the wearer. The solid state lighting system may be digitally controlled by an integrated processor on the HWC. In embodiments, the integrated processor will control the lighting effects in coordination with digital content that is presented within the FOV of the HWC. For example, a movie, picture, game, or other content, may be displayed or playing within the FOV of the HWC. The content may show a bomb blast on the right side of the FOV and at the same moment, the solid state lighting system inside of the upper module optics may flash quickly in concert with the FOV image effect. The effect may not be fast, it may be more persistent to indicate, for example, a general glow or color on one side of the user. The solid state lighting system may be color controlled, with red, green and blue LEDs, for example, such that color control can be coordinated with the digitally presented content within the field of view. -
FIG. 13 a illustrates optical components of a loweroptical module 204 together with anouter lens 1302.FIG. 13 a also shows an embodiment including effects LED's 1308 a and 1308 b.FIG. 13 a illustrates image light 1312, as described herein elsewhere, directed into the upper optical module where it will reflect off of thecombiner element 1304, as described herein elsewhere. Thecombiner element 1304 in this embodiment is angled towards the wearer's eye at the top of the module and away from the wearer's eye at the bottom of the module, as also illustrated and described in connection withFIG. 8 (e.g. at a 45 degree angle). The image light 1312 provided by an upper optical module 202 (not shown inFIG. 13 a) reflects off of thecombiner element 1304 towards thecollimating mirror 1310, away from the wearer's eye, as described herein elsewhere. The image light 1312 then reflects and focuses off of thecollimating mirror 1304, passes back through thecombiner element 1304, and is directed into the wearer's eye. The wearer can also view the surrounding environment through the transparency of thecombiner element 1304,collimating mirror 1310, and outer lens 1302 (if it is included). As described herein elsewhere, various surfaces are polarized to create the optical path for the image light and to provide transparency of the elements such that the wearer can view the surrounding environment. The wearer will generally perceive that the image light forms an image in theFOV 1305. In embodiments, theouter lens 1302 may be included. Theouter lens 1302 is an outer lens that may or may not be corrective and it may be designed to conceal the lower optical module components in an effort to make the HWC appear to be in a form similar to standard glasses or sunglasses. - In the embodiment illustrated in
FIG. 13 a, theeffects LEDs combiner element 1304 and theouter lens 1302 and/or thecollimating mirror 1310. In embodiments, theeffects LEDs 1308 a are positioned within the confines defined by thecombiner element 1304 and theouter lens 1302 and/or the collimating mirror. Theeffects LEDs FOV 1305. In this arrangement, theeffects LEDs FOV 1305. In embodiments the light emitted from theeffects LEDs combiner element 1304 toward the wearer's eye and does not pass through theouter lens 1302 and/or thecollimating mirror 1310. This arrangement provides peripheral lighting effects to the wearer in a more private setting by not transmitting the lighting effects through the front of the HWC into the surrounding environment. However, in other embodiments, theeffects LEDs -
FIG. 13 b illustrates a cross section of the embodiment described in connection withFIG. 13 a. As illustrated, the effects LED 1308 a is located in the upper-front area inside of the optical components of the lower optical module. It should be understood that the effects LED 1308 a position in the described embodiments is only illustrative and alternate placements are encompassed by the present invention. Additionally, in embodiments, there may be one ormore effects LEDs 1308 a in each of the two sides of HWC to provide peripheral lighting effects near one or both eyes of the wearer. -
FIG. 13 c illustrates an embodiment where thecombiner element 1304 is angled away from the eye at the top and towards the eye at the bottom (e.g. in accordance with the holographic or notch filter embodiments described herein). In this embodiment, the effects LED 1308 a is located on theouter lens 1302 side of thecombiner element 1304 to provide a concealed appearance of the lighting effects. As with other embodiments, the effects LED 1308 a ofFIG. 13 c may include a polarizer such that the emitted light can pass through a polarized element associated with thecombiner element 1304 and be blocked by a polarized element associated with theouter lens 1302. - Another aspect of the present invention relates to the mitigation of light escaping from the space between the wearer's face and the HWC itself. Another aspect of the present invention relates to maintaining a controlled lighting environment in proximity to the wearer's eyes. In embodiments, both the maintenance of the lighting environment and the mitigation of light escape are accomplished by including a removable and replaceable flexible shield for the HWC. Wherein the removable and replaceable shield can be provided for one eye or both eyes in correspondence to the use of the displays for each eye. For example, in a night vision application, the display to only one eye could be used for night vision while the display to the other eye is turned off to provide good see-thru when moving between areas where visible light is available and dark areas where night vision enhancement is needed.
-
FIG. 14 a illustrates a removable and replaceableflexible eye cover 1402 with anopening 1408 that can be attached and removed quickly from theHWC 102 through the use of magnets. Other attachment methods may be used, but for illustration of the present invention we will focus on a magnet implementation. In embodiments, magnets may be included in theeye cover 1402 and magnets of an opposite polarity may be included (e.g. embedded) in the frame of theHWC 102. The magnets of the two elements would attract quite strongly with the opposite polarity configuration. In another embodiment, one of the elements may have a magnet and the other side may have metal for the attraction. In embodiments, theeye cover 1402 is a flexible elastomeric shield. In embodiments, theeye cover 1402 may be an elastomeric bellows design to accommodate flexibility and more closely align with the wearer's face.FIG. 14 b illustrates a removable and replaceableflexible eye cover 1404 that is adapted as a single eye cover. In embodiments, a single eye cover may be used for each side of the HWC to cover both eyes of the wearer. In embodiments, the single eye cover may be used in connection with a HWC that includes only one computer display for one eye. These configurations prevent light that is generated and directed generally towards the wearer's face by covering the space between the wearer's face and the HWC. Theopening 1408 allows the wearer to look through theopening 1408 to view the displayed content and the surrounding environment through the front of the HWC. The image light in the loweroptical module 204 can be prevented from emitting from the front of the HWC through internal optics polarization schemes, as described herein, for example. -
FIG. 14 c illustrates another embodiment of a light suppression system. In this embodiment, theeye cover 1410 may be similar to theeye cover 1402, buteye cover 1410 includes afront light shield 1412. Thefront light shield 1412 may be opaque to prevent light from escaping the front lens of the HWC. In other embodiments, thefront light shield 1412 is polarized to prevent light from escaping the front lens. In a polarized arrangement, in embodiments, the internal optical elements of the HWC (e.g. of the lower optical module 204) may polarize light transmitted towards the front of the HWC and thefront light shield 1412 may be polarized to prevent the light from transmitting through thefront light shield 1412. - In embodiments, an opaque front
light shield 1412 may be included and the digital content may include images of the surrounding environment such that the wearer can visualize the surrounding environment. One eye may be presented with night vision environmental imagery and this eye's surrounding environment optical path may be covered using an opaque frontlight shield 1412. In other embodiments, this arrangement may be associated with both eyes. - Another aspect of the present invention relates to automatically configuring the lighting system(s) used in the
HWC 102. In embodiments, the display lighting and/or effects lighting, as described herein, may be controlled in a manner suitable for when aneye cover 1408 is attached or removed from theHWC 102. For example, at night, when the light in the environment is low, the lighting system(s) in the HWC may go into a low light mode to further control any amounts of stray light escaping from the HWC and the areas around the HWC. Covert operations at night, while using night vision or standard vision, may require a solution which prevents as much escaping light as possible so a user may clip on the eye cover(s) 1408 and then the HWC may go into a low light mode. The low light mode may, in some embodiments, only go into a low light mode when theeye cover 1408 is attached if the HWC identifies that the environment is in low light conditions (e.g. through environment light level sensor detection). In embodiments, the low light level may be determined to be at an intermediate point between full and low light dependent on environmental conditions. - Another aspect of the present invention relates to automatically controlling the type of content displayed in the HWC when eye covers 1408 are attached or removed from the HWC. In embodiments, when the eye cover(s) 1408 is attached to the HWC, the displayed content may be restricted in amount or in color amounts. For example, the display(s) may go into a simple content delivery mode to restrict the amount of information displayed. This may be done to reduce the amount of light produced by the display(s). In an embodiment, the display(s) may change from color displays to monochrome displays to reduce the amount of light produced. In an embodiment, the monochrome lighting may be red to limit the impact on the wearer's eyes to maintain an ability to see better in the dark.
- Referring to
FIG. 15 , we now turn to describe a particularexternal user interface 104, referred to generally as apen 1500. Thepen 1500 is a specially designedexternal user interface 104 and can operate as a user interface, such as to many different styles ofHWC 102. Thepen 1500 generally follows the form of a conventional pen, which is a familiar user handled device and creates an intuitive physical interface for many of the operations to be carried out in theHWC system 100. Thepen 1500 may be one ofseveral user interfaces 104 used in connection with controlling operations within theHWC system 100. For example, theHWC 102 may watch for and interprethand gestures 116 as control signals, where thepen 1500 may also be used as a user interface with thesame HWC 102. Similarly, a remote keyboard may be used as anexternal user interface 104 in concert with thepen 1500. The combination of user interfaces or the use of just one control system generally depends on the operation(s) being executed in the HWC'ssystem 100. - While the
pen 1500 may follow the general form of a conventional pen, it contains numerous technologies that enable it to function as anexternal user interface 104.FIG. 15 illustrates technologies comprised in thepen 1500. As can be seen, thepen 1500 may include acamera 1508, which is arranged to view throughlens 1502. The camera may then be focused, such as throughlens 1502, to image a surface upon which a user is writing or making other movements to interact with theHWC 102. There are situations where thepen 1500 will also have an ink, graphite, or other system such that what is being written can be seen on the writing surface. There are other situations where thepen 1500 does not have such a physical writing system so there is no deposit on the writing surface, where the pen would only be communicating data or commands to theHWC 102. The lens configuration is described in greater detail herein. The function of the camera is to capture information from an unstructured writing surface such that pen strokes can be interpreted as intended by the user. To assist in the predication of the intended stroke path, thepen 1500 may include a sensor, such as anIMU 1512. Of course, the IMU could be included in thepen 1500 in its separate parts (e.g. gyro, accelerometer, etc.) or an IMU could be included as a single unit. In this instance, theIMU 1512 is used to measure and predict the motion of thepen 1500. In turn, the integrated microprocessor 1510 would take the IMU information and camera information as inputs and process the information to form a prediction of the pen tip movement. - The
pen 1500 may also include apressure monitoring system 1504, such as to measure the pressure exerted on thelens 1502. As will be described in greater detail herein, the pressure measurement can be used to predict the user's intention for changing the weight of a line, type of a line, type of brush, click, double click, and the like. In embodiments, the pressure sensor may be constructed using any force or pressure measurement sensor located behind thelens 1502, including for example, a resistive sensor, a current sensor, a capacitive sensor, a voltage sensor such as a piezoelectric sensor, and the like. - The
pen 1500 may also include acommunications module 1518, such as for bi-directional communication with theHWC 102. In embodiments, thecommunications module 1518 may be a short distance communication module (e.g. Bluetooth). Thecommunications module 1518 may be security matched to theHWC 102. Thecommunications module 1518 may be arranged to communicate data and commands to and from the microprocessor 1510 of thepen 1500. The microprocessor 1510 may be programmed to interpret data generated from thecamera 1508,IMU 1512, andpressure sensor 1504, and the like, and then pass a command onto theHWC 102 through thecommunications module 1518, for example. In another embodiment, the data collected from any of the input sources (e.g. camera 1508,IMU 1512, pressure sensor 1504) by the microprocessor may be communicated by thecommunication module 1518 to theHWC 102, and theHWC 102 may perform data processing and prediction of the user's intention when using thepen 1500. In yet another embodiment, the data may be further passed on through anetwork 110 to aremote device 112, such as a server, for the data processing and prediction. The commands may then be communicated back to theHWC 102 for execution (e.g. display writing in the glasses display, make a selection within the UI of the glasses display, control a remoteexternal device 112, control a local external device 108), and the like. The pen may also includememory 1514 for long or short term uses. - The
pen 1500 may also include a number of physical user interfaces, such asquick launch buttons 1522, atouch sensor 1520, and the like. Thequick launch buttons 1522 may be adapted to provide the user with a fast way of jumping to a software application in theHWC system 100. For example, the user may be a frequent user of communication software packages (e.g. email, text, Twitter, Instagram, Facebook, Google+, and the like), and the user may program aquick launch button 1522 to command theHWC 102 to launch an application. Thepen 1500 may be provided with severalquick launch buttons 1522, which may be user programmable or factory programmable. Thequick launch button 1522 may be programmed to perform an operation. For example, one of the buttons may be programmed to clear the digital display of theHWC 102. This would create a fast way for the user to clear the screens on theHWC 102 for any reason, such as for example to better view the environment. The quick launch button functionality will be discussed in further detail below. Thetouch sensor 1520 may be used to take gesture style input from the user. For example, the user may be able to take a single finger and run it across thetouch sensor 1520 to affect a page scroll. - The
pen 1500 may also include alaser pointer 1524. Thelaser pointer 1524 may be coordinated with theIMU 1512 to coordinate gestures and laser pointing. For example, a user may use thelaser 1524 in a presentation to help with guiding the audience with the interpretation of graphics and theIMU 1512 may, either simultaneously or when thelaser 1524 is off, interpret the user's gestures as commands or data input. -
FIGS. 16A-C illustrate several embodiments of lens andcamera arrangements 1600 for thepen 1500. One aspect relates to maintaining a constant distance between the camera and the writing surface to enable the writing surface to be kept in focus for better tracking of movements of thepen 1500 over the writing surface. Another aspect relates to maintaining an angled surface following the circumference of the writing tip of thepen 1500 such that thepen 1500 can be rolled or partially rolled in the user's hand to create the feel and freedom of a conventional writing instrument. -
FIG. 16A illustrates an embodiment of the writing lens end of thepen 1500. The configuration includes aball lens 1604, a camera orimage capture surface 1602, and adomed cover lens 1608. In this arrangement, the camera views the writing surface through theball lens 1604 anddome cover lens 1608. Theball lens 1604 causes the camera to focus such that the camera views the writing surface when thepen 1500 is held in the hand in a natural writing position, such as with thepen 1500 in contact with a writing surface. In embodiments, theball lens 1604 should be separated from the writing surface to obtain the highest resolution of the writing surface at thecamera 1602. In embodiments, theball lens 1604 is separated by approximately 1 to 3 mm. In this configuration, thedomed cover lens 1608 provides a surface that can keep theball lens 1604 separated from the writing surface at a constant distance, such as substantially independent of the angle used to write on the writing surface. For instance, in embodiments the field of view of the camera in this arrangement would be approximately 60 degrees. - The domed cover lens, or
other lens 1608 used to physically interact with the writing surface, will be transparent or transmissive within the active bandwidth of thecamera 1602. In embodiments, thedomed cover lens 1608 may be spherical or other shape and comprised of glass, plastic, sapphire, diamond, and the like. In other embodiments where low resolution imaging of the surface is acceptable. Thepen 1500 can omit thedomed cover lens 1608 and theball lens 1604 can be in direct contact with the surface. -
FIG. 16B illustrates another structure where the construction is somewhat similar to that described in connection withFIG. 16A ; however this embodiment does not use adome cover lens 1608, but instead uses aspacer 1610 to maintain a predictable distance between theball lens 1604 and the writing surface, wherein the spacer may be spherical, cylindrical, tubular or other shape that provides spacing while allowing for an image to be obtained by thecamera 1602 through thelens 1604. In a preferred embodiment, thespacer 1610 is transparent. In addition, while thespacer 1610 is shown as spherical, other shapes such as an oval, doughnut shape, half sphere, cone, cylinder or other form may be used. -
FIG. 16C illustrates yet another embodiment, where the structure includes apost 1614, such as running through the center of the lensed end of thepen 1500. Thepost 1614 may be an ink deposition system (e.g. ink cartridge), graphite deposition system (e.g. graphite holder), or a dummy post whose purpose is mainly only that of alignment. The selection of the post type is dependent on the pen's use. For instance, in the event the user wants to use thepen 1500 as a conventional ink depositing pen as well as a fully functionalexternal user interface 104, the ink system post would be the best selection. If there is no need for the ‘writing’ to be visible on the writing surface, the selection would be the dummy post. The embodiment ofFIG. 16C includes camera(s) 1602 and an associatedlens 1612, where thecamera 1602 andlens 1612 are positioned to capture the writing surface without substantial interference from thepost 1614. In embodiments, thepen 1500 may includemultiple cameras 1602 andlenses 1612 such that more or all of the circumference of thetip 1614 can be used as an input system. In an embodiment, thepen 1500 includes a contoured grip that keeps the pen aligned in the user's hand so that thecamera 1602 andlens 1612 remains pointed at the surface. - Another aspect of the
pen 1500 relates to sensing the force applied by the user to the writing surface with thepen 1500. The force measurement may be used in a number of ways. For example, the force measurement may be used as a discrete value, or discontinuous event tracking, and compared against a threshold in a process to determine a user's intent. The user may want the force interpreted as a ‘click’ in the selection of an object, for instance. The user may intend multiple force exertions interpreted as multiple clicks. There may be times when the user holds thepen 1500 in a certain position or holds a certain portion of the pen 1500 (e.g. a button or touch pad) while clicking to affect a certain operation (e.g. a ‘right click’). In embodiments, the force measurement may be used to track force and force trends. The force trends may be tracked and compared to threshold limits, for example. There may be one such threshold limit, multiple limits, groups of related limits, and the like. For example, when the force measurement indicates a fairly constant force that generally falls within a range of related threshold values, the microprocessor 1510 may interpret the force trend as an indication that the user desires to maintain the current writing style, writing tip type, line weight, brush type, and the like. In the event that the force trend appears to have gone outside of a set of threshold values intentionally, the microprocessor may interpret the action as an indication that the user wants to change the current writing style, writing tip type, line weight, brush type, and the like. Once the microprocessor has made a determination of the user's intent, a change in the current writing style, writing tip type, line weight, brush type, and the like may be executed. In embodiments, the change may be noted to the user (e.g. in a display of the HWC 102), and the user may be presented with an opportunity to accept the change. -
FIG. 17A illustrates an embodiment of a force sensing surface tip 1700 of apen 1500. The force sensing surface tip 1700 comprises a surface connection tip 1702 (e.g. a lens as described herein elsewhere) in connection with a force orpressure monitoring system 1504. As a user uses thepen 1500 to write on a surface or simulate writing on a surface theforce monitoring system 1504 measures the force or pressure the user applies to the writing surface and the force monitoring system communicates data to the microprocessor 1510 for processing. In this configuration, the microprocessor 1510 receives force data from theforce monitoring system 1504 and processes the data to make predictions of the user's intent in applying the particular force that is currently being applied. In embodiments, the processing may be provided at a location other than on the pen (e.g. at a server in theHWC system 100, on the HWC 102). For clarity, when reference is made herein to processing information on the microprocessor 1510, the processing of information contemplates processing the information at a location other than on the pen. The microprocessor 1510 may be programmed with force threshold(s), force signature(s), force signature library and/or other characteristics intended to guide an inference program in determining the user's intentions based on the measured force or pressure. The microprocessor 1510 may be further programmed to make inferences from the force measurements as to whether the user has attempted to initiate a discrete action (e.g. a user interface selection ‘click’) or is performing a constant action (e.g. writing within a particular writing style). The inferencing process is important as it causes thepen 1500 to act as an intuitiveexternal user interface 104. -
FIG. 17B illustrates a force 1708 versustime 1710 trend chart with asingle threshold 1718. Thethreshold 1718 may be set at a level that indicates a discrete force exertion indicative of a user's desire to cause an action (e.g. select an object in a GUI).Event 1712, for example, may be interpreted as a click or selection command because the force quickly increased from below thethreshold 1718 to above thethreshold 1718. Theevent 1714 may be interpreted as a double click because the force quickly increased above thethreshold 1718, decreased below thethreshold 1718 and then essentially repeated quickly. The user may also cause the force to go above thethreshold 1718 and hold for a period indicating that the user is intending to select an object in the GUI (e.g. a GUI presented in the display of the HWC 102) and ‘hold’ for a further operation (e.g. moving the object). - While a threshold value may be used to assist in the interpretation of the user's intention, a signature force event trend may also be used. The threshold and signature may be used in combination or either method may be used alone. For example, a single-click signature may be represented by a certain force trend signature or set of signatures. The single-click signature(s) may require that the trend meet a criteria of a rise time between x any y values, a hold time of between a and b values and a fall time of between c and d values, for example. Signatures may be stored for a variety of functions such as click, double click, right click, hold, move, etc. The microprocessor 1510 may compare the real-time force or pressure tracking against the signatures from a signature library to make a decision and issue a command to the software application executing in the GUI.
-
FIG. 17C illustrates a force 1708 versustime 1710 trend chart withmultiple thresholds 1718. By way of example, the force trend is plotted on the chart with several pen force or pressure events. As noted, there are both presumablyintentional events 1720 and presumablynon-intentional events 1722. The twothresholds 1718 ofFIG. 4C create three zones of force: a lower, middle and higher range. The beginning of the trend indicates that the user is placing a lower zone amount of force. This may mean that the user is writing with a given line weight and does not intend to change the weight, the user is writing. Then the trend shows asignificant increase 1720 in force into the middle force range. This force change appears, from the trend to have been sudden and thereafter it is sustained. The microprocessor 1510 may interpret this as an intentional change and as a result change the operation in accordance with preset rules (e.g. change line width, increase line weight, etc.). The trend then continues with a second apparentlyintentional event 1720 into the higher-force range. During the performance in the higher-force range, the force dips below theupper threshold 1718. This may indicate an unintentional force change and the microprocessor may detect the change in range however not affect a change in the operations being coordinated by thepen 1500. As indicated above, the trend analysis may be done with thresholds and/or signatures. - Generally, in the present disclosure, instrument stroke parameter changes may be referred to as a change in line type, line weight, tip type, brush type, brush width, brush pressure, color, and other forms of writing, coloring, painting, and the like.
- Another aspect of the
pen 1500 relates to selecting an operating mode for thepen 1500 dependent on contextual information and/or selection interface(s). Thepen 1500 may have several operating modes. For instance, thepen 1500 may have a writing mode where the user interface(s) of the pen 1500 (e.g. the writing surface end,quick launch buttons 1522,touch sensor 1520, motion based gesture, and the like) is optimized or selected for tasks associated with writing. As another example, thepen 1500 may have a wand mode where the user interface(s) of the pen is optimized or selected for tasks associated with software or device control (e.g. theHWC 102, external local device,remote device 112, and the like). Thepen 1500, by way of another example, may have a presentation mode where the user interface(s) is optimized or selected to assist a user with giving a presentation (e.g. pointing with thelaser pointer 1524 while using the button(s) 1522 and/or gestures to control the presentation or applications relating to the presentation). The pen may, for example, have a mode that is optimized or selected for a particular device that a user is attempting to control. Thepen 1500 may have a number of other modes and an aspect of the present invention relates to selecting such modes. -
FIG. 18A illustrates an automatic user interface(s) mode selection based on contextual information. The microprocessor 1510 may be programmed withIMU thresholds thresholds angle pen 1500 for certain expected positions during certain predicted modes. When the microprocessor 1510 determines that thepen 1500 is being held or otherwise positioned withinangles 1802 corresponding to writingthresholds 1814, for example, the microprocessor 1510 may then institute a writing mode for the pen's user interfaces. Similarly, if the microprocessor 1510 determines (e.g. through the IMU 1512) that the pen is being held at anangle 1804 that falls between thepredetermined wand thresholds 1812, the microprocessor may institute a wand mode for the pen's user interface. Both of these examples may be referred to as context based user interface mode selection as the mode selection is based on contextual information (e.g. position) collected automatically and then used through an automatic evaluation process to automatically select the pen's user interface(s) mode. - As with other examples presented herein, the microprocessor 1510 may monitor the contextual trend (e.g. the angle of the pen over time) in an effort to decide whether to stay in a mode or change modes. For example, through signatures, thresholds, trend analysis, and the like, the microprocessor may determine that a change is an unintentional change and therefore no user interface mode change is desired.
-
FIG. 18B illustrates an automatic user interface(s) mode selection based on contextual information. In this example, thepen 1500 is monitoring (e.g. through its microprocessor) whether or not the camera at the writingsurface end 1508 is imaging a writing surface in close proximity to the writing surface end of thepen 1500. If thepen 1500 determines that a writing surface is within a predetermined relatively short distance, thepen 1500 may decide that a writing surface is present 1820 and the pen may go into a writing mode user inteface(s) mode. In the event that thepen 1500 does not detect a relativelyclose writing surface 1822, the pen may predict that the pen is not currently being used to as a writing instrument and the pen may go into a non-writing user interface(s) mode. -
FIG. 18C illustrates a manual user interface(s) mode selection. The user interface(s) mode may be selected based on a twist of asection 1824 of thepen 1500 housing, clicking an end button 1828, pressing aquick launch button 1522, interacting withtouch sensor 1520, detecting a predetermined action at the pressure monitoring system (e.g. a click), detecting a gesture (e.g. detected by the IMU), etc. The manual mode selection may involve selecting an item in a GUI associated with the pen 1500 (e.g. an image presented in the display of HWC 102). - In embodiments, a confirmation selection may be presented to the user in the event a mode is going to change. The presentation may be physical (e.g. a vibration in the pen 1500), through a GUI, through a light indicator, etc.
-
FIG. 19 illustrates a couple pen use-scenarios FIG. 19 as a way of illustrating use scenarios to further the understanding of the reader. As such, the use-scenarios should be considered illustrative and non-limiting. - Use
scenario 1900 is a writing scenario where thepen 1500 is used as a writing instrument. In this example, quick launch button 122A is pressed to launch anote application 1910 in theGUI 1908 of theHWC 102display 1904. Once the quick launch button 122A is pressed, theHWC 102 launches thenote program 1910 and puts the pen into a writing mode. The user uses thepen 1500 toscribe symbols 1902 on a writing surface, the pen records the scribing and transmits the scribing to theHWC 102 where symbols representing the scribing are displayed 1912 within thenote application 1910. - Use
scenario 1901 is a gesture scenario where thepen 1500 is used as a gesture capture and command device. In this example, the quick launch button 122B is activated and thepen 1500 activates a wand mode such that an application launched on theHWC 102 can be controlled. Here, the user sees anapplication chooser 1918 in the display(s) of theHWC 102 where different software applications can be chosen by the user. The user gestures (e.g. swipes, spins, turns, etc.) with the pen to cause theapplication chooser 1918 to move from application to application. Once the correct application is identified (e.g. highlighted) in thechooser 1918, the user may gesture or click or otherwise interact with thepen 1500 such that the identified application is selected and launched. Once an application is launched, the wand mode may be used to scroll, rotate, change applications, select items, initiate processes, and the like, for example. - In an embodiment, the quick launch button 122A may be activated and the
HWC 102 may launch an application chooser presenting to the user a set of applications. For example, the quick launch button may launch a chooser to show all communication programs (e.g. SMS, Twitter, Instagram, Facebook, email, etc.) available for selection such that the user can select the program the user wants and then go into a writing mode. By way of further example, the launcher may bring up selections for various other groups that are related or categorized as generally being selected at a given time (e.g. Microsoft Office products, communication products, productivity products, note products, organizational products, and the like) -
FIG. 20 illustrates yet another embodiment of the present invention.FIG. 2000 illustrates a watchband clip oncontroller 2000. The watchband clip on controller may be a controller used to control theHWC 102 or devices in theHWC system 100. The watchband clip oncontroller 2000 has a fastener 2018 (e.g. rotatable clip) that is mechanically adapted to attach to a watchband, as illustrated at 2004. - The
watchband controller 2000 may have quick launch interfaces 2008 (e.g. to launch applications and choosers as described herein), a touch pad 2014 (e.g. to be used as a touch style mouse for GUI control in aHWC 102 display) and adisplay 2012. Theclip 2018 may be adapted to fit a wide range of watchbands so it can be used in connection with a watch that is independently selected for its function. The clip, in embodiments, is rotatable such that a user can position it in a desirable manner. In embodiments the clip may be a flexible strap. In embodiments, the flexible strap may be adapted to be stretched to attach to a hand, wrist, finger, device, weapon, and the like. - In embodiments, the watchband controller may be configured as a removable and replacable watchband. For example, the controller may be incorporated into a band with a certain width, segment spacing's, etc. such that the watchband, with its incorporated controller, can be attached to a watch body. The attachment, in embodiments, may be mechanically adapted to attach with a pin upon which the watchband rotates. In embodiments, the watchband controller may be electrically connected to the watch and/or watch body such that the watch, watch body and/or the watchband controller can communicate data between them.
- The watchband controller may have 3-axis motion monitoring (e.g. through an IMU, accelerometers, magnetometers, gyroscopes, etc.) to capture user motion. The user motion may then be interpreted for gesture control.
- In embodiments, the watchband controller may comprise fitness sensors and a fitness computer. The sensors may track heart rate, calories burned, strides, distance covered, and the like. The data may then be compared against performance goals and/or standards for user feedback.
- Another aspect of the present invention relates to visual display techniques relating to micro Doppler (“mD”) target tracking signatures (“mD signatures”). mD is a radar technique that uses a series of angle dependent electromagnetic pulses that are broadcast into an environment and return pulses are captured. Changes between the broadcast pulse and return pulse are indicative of changes in the shape, distance and angular location of objects or targets in the environment. These changes provide signals that can be used to track a target and identify the target through the mD signature. Each target or target type has a unique mD signature. Shifts in the radar pattern can be analyzed in the time domain and frequency domain based on mD techniques to derive information about the types of targets present (e.g. whether people are present), the motion of the targets and the relative angular location of the targets and the distance to the targets. By selecting a frequency used for the mD pulse relative to known objects in the environment, the pulse can penetrate the known objects to enable information about targets to be gathered even when the targets are visually blocked by the known objects. For example, pulse frequencies can be used that will penetrate concrete buildings to enable people to be identified inside the building. Multiple pulse frequencies can be used as well in the mD radar to enable different types of information to be gathered about the objects in the environment. In addition, the mD radar information can be combined with other information such as distance measurements or images captured of the environment that are analyzed jointly to provide improved object identification and improved target identification and tracking. In embodiments, the analysis can be performed on the HWC or the information can be transmitted to a remote network for analysis and results transmitted back to the HWC. Distance measurements can be provided by laser range finding, structured lighting, stereoscopic depth maps or sonar measurements. Images of the environment can be captured using one or more cameras capable of capturing images from visible, ultraviolet or infrared light. The mD radar can be attached to the HWC, located adjacently (e.g. in a vehicle) and associated wirelessly with the HWC or located remotely. Maps or other previously determined information about the environment can also be used in the analysis of the mD radar information. Embodiments of the present invention relate to visualizing the mD signatures in useful ways.
-
FIG. 21 illustrates aFOV 2102 of aHWC 102 from a wearer's perspective. The wearer, as described herein elsewhere, has a see-throughFOV 2102 wherein the wearer views adjacent surroundings, such as the buildings illustrated inFIG. 21 . The wearer, as described herein elsewhere, can also see displayed digital content presented within a portion of theFOV 2102. The embodiment illustrated inFIG. 21 is indicating that the wearer can see the buildings and other surrounding elements in the environment and digital content representing traces, or travel paths, of bullets being fired by different people in the area. The surroundings are viewed through the transparency of theFOV 2102. The traces are presented via the digital computer display, as described herein elsewhere. In embodiments, the trace presented is based on a mD signature that is collected and communicated to the HWC in real time. The mD radar itself may be on or near the wearer of theHWC 102 or it may be located remote from the wearer. In embodiments, the mD radar scans the area, tracks and identifies targets, such as bullets, and communicates traces, based on locations, to theHWC 102. - There are
several traces FIG. 21 . The traces communicated from the mD radar may be associated with GPS locations and the GPS locations may be associated with objects in the environment, such as people, buildings, vehicles, etc, both in latitude and longitude perspective and an elevation perspective. The locations may be used as markers for the HWC such that the traces, as presented in the FOV, can be associated, or fixed in space relative to the markers. For example, if thefriendly fire trace 2108 is determined, by the mD radar, to have originated from the upper right window of the building on the left, as illustrated inFIG. 21 , then a virtual marker may be set on or near the window. When the HWC views, through it's camera or other sensor, for example, the building's window, the trace may then virtually anchor with the virtual marker on the window. Similarly, a marker may be set near the termination position or other flight position of thefriendly fire trace 2108, such as the upper left window of the center building on the right, as illustrated inFIG. 21 . This technique fixes in space the trace such that the trace appears fixed to the environmental positions independent of where the wearer is looking So, for example, as the wearer's head turns, the trace appears fixed to the marked locations. - In embodiments, certain user positions may be known and thus identified in the FOV. For example, the shooter of the
friendly fire trace 2108 may be from a known friendly combatant and as such his location may be known. The position may be known based on his GPS location based on a mobile communication system on him, such as anotherHWC 102. In other embodiments, the friendly combatant may be marked by another friendly. For example, if the friendly position in the environment is known through visual contact or communicated information, a wearer of theHWC 102 may use a gesture orexternal user interface 104 to mark the location. If a friendly combatant location is known the originating position of thefriendly fire trace 2108 may be color coded or otherwise distinguished from unidentified traces on the displayed digital content. Similarly, enemy fire traces 2104 may be color coded or otherwise distinguished on the displayed digital content. In embodiments, there may be an additional distinguished appearance on the displayed digital content for unknown traces. - In addition to situationally associated trace appearance, the trace colors or appearance may be different from the originating position to the terminating position. This path appearance change may be based on the mD signature. The mD signature may indicate that the bullet, for example, is slowing as it propagates and this slowing pattern may be reflected in the
FOV 2102 as a color or pattern change. This can create an intuitive understanding of wear the shooter is located. For example, the originating color may be red, indicative of high speed, and it may change over the course of the trace to yellow, indicative of a slowing trace. This pattern changing may also be different for a friendly, enemy and unknown combatant. The enemy may go blue to green for a friendly trace, for example. -
FIG. 21 illustrates an embodiment where the user sees the environment through the FOV and may also see color coded traces, which are dependent on bullet speed and combatant type, where the traces are fixed in environmental positions independent on the wearer's perspective. Other information, such as distance, range, range rings, time of day, date, engagement type (e.g. hold, stop firing, back away, etc.) may also be displayed in the FOV. - Another aspect of the present invention relates to mD radar techniques that trace and identify targets through other objects, such as walls (referred to generally as through wall mD), and visualization techniques related therewith.
FIG. 22 illustrates a through wall mD visualization technique according to the principles of the present invention. As described herein elsewhere, the mD radar scanning the environment may be local or remote from the wearer of aHWC 102. The mD radar may identify a target (e.g. a person) that is visible 2204 and then track the target as he goes behind awall 2208. The tracking may then be presented to the wearer of aHWC 102 such that digital content reflective of the target and the target's movement, even behind the wall, is presented in the FOV 2202 of theHWC 102. In embodiments, the target, when out of visible sight, may be represented by an avatar in the FOV to provide the wearer with imagery representing the target. - mD target recognition methods can identify the identity of a target based on the vibrations and other small movements of the target. This can provide a personal signature for the target. In the case of humans, this may result in a personal identification of a target that has been previously characterized. The cardio, heart beat, lung expansion and other small movements within the body may be unique to a person and if those attributes are pre-identified they may be matched in real time to provide a personal identification of a person in the FOV 2202. The person's mD signatures may be determined based on the position of the person. For example, the database of personal mD signature attributes may include mD signatures for a person standing, sitting, laying down, running, walking, jumping, etc. This may improve the accuracy of the personal data match when a target is tracked through mD signature techniques in the field. In the event a person is personally identified, a specific indication of the person's identity may be presented in the FOV 2202. The indication may be a color, shape, shade, name, indication of the type of person (e.g. enemy, friendly, etc.), etc. to provide the wearer with intuitive real time information about the person being tracked. This may be very useful in a situation where there is more than one person in an area of the person being tracked. If just one person in the area is personally identified, that person or the avatar of that person can be presented differently than other people in the area.
-
FIG. 23 illustrates an mD scannedenvironment 2300. An mD radar may scan an environment in an attempt to identify objects in the environment. In this embodiment, the mD scanned environment reveals twovehicles enemy combatant 2309, twofriendly combatants shot trace 2318. Each of these objects may be personally identified or type identified. For example, thevehicles enemy combatant 2309 may be identified as a type (e.g. enemy combatant) or more personally (e.g. by name). The friendly combatants may be identified as a type (e.g. friendly combatant) or more personally (e.g. by name). Theshot trace 2318 may be characterized by type of projectile or weapon type for the projectile, for example. -
FIG. 23 a illustrates twoseparate HWC 102 FOV display techniques according to the principles of the present invention.FOV 2312 illustrates a map view 2310 where the mD scanned environment is presented. Here, the wearer has a perspective on the mapped area so he can understand all tracked targets in the area. This allows the wearer to traverse the area with knowledge of the targets.FOV 2312 illustrates a heads-up view to provide the wearer with an augmented reality style view of the environment that is in proximity of the wearer. - An aspect of the present invention relates to suppression of extraneous or stray light. As discussed herein elsewhere, eyeglow and faceglow are two such artifacts that develop from such light. Eyeglow and faceglow can be caused by image light escaping from the optics module. The escaping light is then visible, particularly in dark environments when the user is viewing bright displayed images with the HWC. Light that escapes through the front of the HWC is visible as eyeglow as it that light that is visible in the region of the user's eyes. Eyeglow can appear in the form of a small version of the displayed image that the user is viewing. Light that escapes from the bottom of the HWC shines onto the user's face, cheek or chest so that these portions of the user appear to glow. Eyeglow and faceglow can both increase the visibility of the user and highlight the use of the HWC, which may be viewed negatively by the user. As such, reducing eyeglow and faceglow is advantageous. In combat situations (e.g. the mD trace presentation scenerios described herein) and certain gaming situations, the suppression of extraneous or stray light is very important.
- The disclosure relating to
FIG. 6 shows an example where a portion of the image light passes through thecombiner 602 such that the light shines onto the user's face, thereby illuminating a portion of the user's face in what is generally referred to herein as faceglow. Faceglow be caused by any portion of light from the HWC that illuminates the user's face. - An example of the source for the faceglow light can come from wide cone angle light associated with the image light incident onto the
combiner 602. Where the combiner can include a holographic mirror or a notch mirror in which the narrow bands of high reflectivity are matched to wavelengths of light by the light source. The wide cone angle associated with the image light corresponds with the field of view provided by the HWC. Typically the reflectivity of holographic mirrors and notch mirrors is reduced as the cone angle of the incident light is increased above 8 degrees. As a result, for a field of view of 30 degrees, substantial image light can pass through the combiner and cause faceglow. -
FIG. 24 shows an illustration of alight trap 2410 for the faceglow light. In this embodiment, an extension of the outer shield lens of the HWC is coated with a light absorbing material in the region where the converging light responsible for faceglow is absorbed in alight trap 2410. The light absorbing material can be black or it can be a filter designed to absorb only the specific wavelengths of light provided by the light source(s) in the HWC. In addition, the surface of thelight trap 2410 may be textured or fibrous to further improve the absorption. -
FIG. 25 illustrates an optical system for a HWC that includes an outerabsorptive polarizer 2520 to block the faceglow light. In this embodiment, the image light is polarized and as a result the light responsible for faceglow is similarly polarized. The absorptive polarizer is oriented with a transmission axis such that the faceglow light is absorbed and not transmitted. In this case, the rest of the imaging system in the HWC may not require polarized image light and the image light may be polarized at any point before the combiner. In embodiments, the transmission axis of theabsorptive polarizer 2520 is oriented vertically so that external glare from water (S polarized light) is absorbed and correspondingly, the polarization of the image light is selected to be horizontal (S polarization). Consequently, image light that passes through thecombiner 602 and is then incident onto theabsorptive polarizer 2520, is absorbed. InFIG. 25 theabsorptive polarizer 2520 is shown outside the shield lens, alternatively theabsorptive polarizer 2520 can be located inside the shield lens. -
FIG. 26 illustrates an optical system for a HWC that includes a film with anabsorptive notch filter 2620. In this case, the absorptive notch filter absorbs narrow bands of light that are selected to match the light provided by the optical system's light source. As a result, the absorptive notch filter is opaque with respect to the faceglow light and is transparent to the remainder of the wavelengths included in the visible spectrum so that the user has a clear view of the surrounding environment. A triple notch filter suitable for this approach is available from Iridian Spectral Technologies, Ottawa, ON: http://www.ilphotonics.com/cdv2/Iridian-Interference%20Filters/New%20filters/Triple%20Notch%20Filter.pdf - In embodiments, the
combiner 602 may include a notch mirror coating to reflect the wavelengths of light in the image light and anotch filter 2620 can be selected in correspondence to the wavelengths of light provided by the light source and the narrow bands of high reflectivity provided by the notch mirror. In this way, image light that is not reflected by the notch mirror is absorbed by thenotch filter 2620. In embodiments of the invention the light source can provide one narrow band of light for a monochrome imaging or three narrow bands of light for full color imaging. The notch mirror and associated notch filter would then each provide one narrow band or three narrow bands of high reflectivity and absorption respectively. -
FIG. 27 includes amicrolouver film 2750 to block the faceglow light. Microlouver film is sold by 3M as ALCF-P, for example and is typically used as a privacy filter for computer. See http://multimedia.3 m.com/mws/mediawebserver?mwsId=SSSSSuH8gc7nZxtUoY_xlY_e evUqe17zHvTSevTSeSSSSSS--&fn=ALCF-P_ABR2 Control Film DS .pdf The microlouver film transmits light within a somewhat narrow angle (e.g. 30 degrees of normal and absorbs light beyond 30 degrees of normal). InFIG. 27 , themicrolouver film 2750 is positioned such that thefaceglow light 2758 is incident beyond 30 degrees from normal while the see-through light 2755 is incident within 30 degrees of normal to themicrolouver film 2750. As such, thefaceglow light 2758 is absorbed by the microlouver film and the see-through light 2755 is transmitted so that the user has a bright see-thru view of the surrounding environment. - We now turn back to a description of eye imaging technologies. Aspects of the present invention relate to various methods of imaging the eye of a person wearing the
HWC 102. In embodiments, technologies for imaging the eye using an optical path involving the “off” state and “no power” state, which is described in detail below, are described. In embodiments, technologies for imaging the eye with optical configurations that do not involve reflecting the eye image off of DLP mirrors is described. In embodiments, unstructured light, structured light, or controlled lighting conditions, are used to predict the eye's position based on the light reflected off of the front of the wearer's eye. In embodiments, a reflection of a presented digital content image is captured as it reflects off of the wearer's eye and the reflected image may be processed to determine the quality (e.g. sharpness) of the image presented. In embodiments, the image may then be adjusted (e.g. focused differently) to increase the quality of the image presented based on the image reflection. -
FIGS. 28 a, 28 b and 28 c show illustrations of the various positions of the DLP mirrors.FIG. 28 a shows the DLP mirrors in the “on”state 2815. With the mirror in the “on”state 2815,illumination light 2810 is reflected along anoptical axis 2820 that extends into the loweroptical module 204.FIG. 28 b shows the DLP mirrors in the “off”state 2825. With the mirror in the “off”state 2825,illumination light 2810 is reflected along anoptical axis 2830 that is substantially to the side ofoptical axis 2820 so that the “off” state light is directed toward a dark light trap as has been described herein elsewhere.FIG. 28 c shows the DLP mirrors in a third position, which occurs when no power is applied to the DLP. This “no power” state differs from the “on” and “off” states in that the mirror edges are not in contact with the substrate and as such are less accurately positioned.FIG. 28 c shows all of the DLP mirrors in the “no power”state 2835. The “no power” state is achieved by simultaneously setting the voltage to zero for the “on” contact and “off” contact for a DLP mirror, as a result, the mirror returns to a no stress position where the DLP mirror is in the plane of the DLP platform as shown inFIG. 28 c. Although not normally done, it is also possible to apply the “no power” state to individual DLP mirrors. When the DLP mirrors are in the “no power” state they do not contribute image content. Instead, as shown inFIG. 28 c, when the DLP mirrors are in the “no power” state, theillumination light 2810 is reflected along anoptical axis 2840 that is between theoptical axes optical module 204 and as a result the displayed image contrast can be reduced or artifacts can be created in the image that detract from the image content. Consequently, it is generally desirable, in embodiments, to limit the time associated with the “no power” state to times when images are not displayed or to reduce the time associated with having DLP mirrors in the “no power” state so that the affect of the scattered light is reduced. -
FIG. 29 shows an embodiment of the invention that can be used for displaying digital content images to a wearer of theHWC 102 and capturing images of the wearer's eye. In this embodiment, light from theeye 2971 passes back through the optics in thelower module 204, the solidcorrective wedge 2966, at least a portion of the light passes through the partiallyreflective layer 2960, thesolid illumination wedge 2964 and is reflected by a plurality of DLP mirrors on theDLP 2955 that are in the “no power” state. The reflected light then passes back through theillumination wedge 2964 and at least a portion of the light is reflected by the partiallyreflective layer 2960 and the light is captured by thecamera 2980. - For comparison, illuminating
light rays 2973 from thelight source 2958 are also shown being reflected by the partiallyreflective layer 2960. Where the angle of the illuminating light 2973 is such that the DLP mirrors, when in the “on” state, reflect the illuminating light 2973 to form image light 2969 that substantially shares the same optical axis as the light from the wearer'seye 2971. In this way, images of the wearer's eye are captured in a field of view that overlaps the field of view for the displayed image content. In contrast, light reflected by DLP mirrors in the “off” state form dark light 2975 which is directed substantially to the side of theimage light 2969 and the light fromeye 2971. Dark light 2975 is directed toward alight trap 2962 that absorbs the dark light to improve the contrast of the displayed image as has been described above in this specification. - In an embodiment, partially
reflective layer 2960 is a reflective polarizer. The light that is reflected from theeye 2971 can then be polarized prior to entering the corrective wedge 2966 (e.g with an absorptive polarizer between theupper module 202 and the lower module 204), with a polarization orientation relative to the reflective polarizer that enables the light reflected from theeye 2971 to substantially be transmitted by the reflective polarizer. A quarterwave retarder layer 2957 is then included adjacent to the DLP 2955 (as previously disclosed inFIG. 3 b) so that the light reflected from theeye 2971 passes through the quarterwave retarder layer 2957 once before being reflected by the plurality of DLP mirrors in the “no power” state and then passes through a second time after being reflected. By passing through the quarterwave retarder layer 2957 twice, the polarization state of the light from theeye 2971 is reversed, such that when it is incident upon the reflective polarizer, the light from theeye 2971 is then substantially reflected toward thecamera 2980. By using a partiallyreflective layer 2960 that is a reflective polarizer and polarizing the light from theeye 2971 prior to entering thecorrective wedge 2964, losses attributed to the partiallyreflective layer 2960 are reduced. -
FIG. 28 c shows the case wherein the DLP mirrors are simultaneously in the “no power” state, this mode of operation can be particularly useful when theHWC 102 is first put onto the head of the wearer. When theHWC 102 is first put onto the head of the wearer, it is not necessary to display an image yet. As a result, the DLP can be in a “no power” state for all the DLP mirrors and an image of the wearer's eyes can be captured. The captured image of the wearer's eye can then be compared to a database, using iris identification techniques, or other eye pattern identification techniques to determine, for example, the identity of the wearer. - In a further embodiment illustrated by
FIG. 29 all of the DLP mirrors are put into the “no power” state for a portion of a frame time (e.g. 50% of a frame time for the displayed digital content image) and the capture of the eye image is synchronized to occur at the same time and for the same duration. By reducing the time that the DLP mirrors are in the “no power” state, the time where light is scattered by the DLP mirrors being in the “no power” state is reduced such that the wearer doesn't perceive a change in the displayed image quality. This is possible because the DLP mirrors have a response time on the order of microseconds while typical frame times for a displayed image are on the order of 0.016 seconds. This method of capturing images of the wearer's eye can be used periodically to capture repetitive images of the wearer's eye. For example, eye images could be captured for 50% of the frame time of every 10th frame displayed to the wearer. In another example, eye images could be captured for 10% of the frame time of every frame displayed to the wearer. - Alternately, the “no power” state can be applied to a subset of the DLP mirrors (e.g. 10% of the DLP mirrors) within while another subset is in busy generating image light for content to be displayed. This enables the capture of an eye image(s) during the display of digital content to the wearer. The DLP mirrors used for eye imaging can, for example, be distributed randomly across the area of the DLP to minimize the impact on the quality of the digital content being displayed to the wearer. To improve the displayed image perceived by the wearer, the individual DLP mirrors put into the “no power” state for capturing each eye image, can be varied over time such as in a random pattern, for example. In yet a further embodiment, the DLP mirrors put into the “no power” state for eye imaging may be coordinated with the digital content in such a way that the “no power” mirrors are taken from a portion of the image that requires less resolution.
- In the embodiments of the invention as illustrated in
FIGS. 9 and 29 , in both cases the reflective surfaces provided by the DLP mirrors do not preserve the wavefront of the light from the wearer's eye so that the image quality of captured image of the eye is somewhat limited. It may still be useful in certain embodiments, but it is somewhat limited. This is due to the DLP mirrors not being constrained to be on the same plane. In the embodiment illustrated inFIG. 9 , the DLP mirrors are tilted so that they form rows of DLP mirrors that share common planes. In the embodiment illustrated inFIG. 29 , the individual DLP mirrors are not accurately positioned to be in the same plane since they are not in contact with the substrate. Examples of advantages of the embodiments associated withFIG. 29 are: first, thecamera 2980 can be located between theDLP 2955 and theillumination light source 2958 to provide a more compactupper module 202. Second, the polarization state of the light reflected from theeye 2971 can be the same as that of the image light 2969 so that the optical path of the light reflected from the eye and the image light can be the same in thelower module 204. -
FIG. 30 shows an illustration of an embodiment for displaying images to the wearer and simultaneously capturing images of the wearer's eye, wherein light from theeye 2971 is reflected towards acamera 3080 by the partiallyreflective layer 2960. The partiallyreflective layer 2960 can be an optically flat layer such that the wavefront of the light from theeye 2971 is preserved and as a result, higher quality images of the wearer's eye can be captured. In addition, since theDLP 2955 is not included in the optical path for the light from theeye 2971, and the eye imaging process shown inFIG. 30 does not interfere with the displayed image, images of the wearer's eye can be captured independently (e.g. with independent of timing, impact on resolution, or pixel count used in the image light) from the displayed images. - In the embodiment illustrated in
FIG. 30 , the partiallyreflective layer 2960 is a reflective polarizer, the illuminating light 2973 is polarized, the light from theeye 2971 is polarized and thecamera 3080 is located behind apolarizer 3085. The polarization axis of the illuminating light 2973 and the polarization axis of the light from the eye are oriented perpendicular to the transmission axis of the reflective polarizer so that they are both substantially reflected by the reflective polarizer. The illumination light 2973 passes through aquarter wave layer 2957 before being reflected by the DLP mirrors in theDLP 2955. The reflected light passes back through thequarter wave layer 2957 so that the polarization states of theimage light 2969 and dark light 2975 are reversed in comparison to theillumination light 2973. As such, theimage light 2969 and dark light 2975 are substantially transmitted by the reflective polarizer. Where the DLP mirrors in the “on” state provide theimage light 2969 along an optical axis that extends into the loweroptical module 204 to display an image to the wearer. At the same time, DLP mirrors in the “off” state provide thedark light 2975 along an optical axis that extends to the side of theupper optics module 202. In the region of thecorrective wedge 2966 where the dark light 2975 is incident on the side of theupper optics module 202, anabsorptive polarizer 3085 is positioned with it's transmission axis perpendicular to the polarization axis of the dark light and parallel to the polarization axis of the light from the eye so that the dark light 2975 is absorbed and the light from theeye 2971 is transmitted to thecamera 3080. -
FIG. 31 shows an illustration of another embodiment of a system for displaying images and simultaneously capturing image of the wearer's eye that is similar to the one shown inFIG. 30 . The difference in the system shown inFIG. 31 is that the light from theeye 2971 is subjected to multiple reflections before being captured by thecamera 3180. To enable the multiple reflections, amirror 3187 is provided behind theabsorptive polarizer 3185. Therefore, the light from theeye 2971 is polarized prior to entering thecorrective wedge 2966 with a polarization axis that is perpendicular to the transmission axis of the reflective polarizer that comprises the partiallyreflective layer 2960. In this way, the light from theeye 2971 is reflected first by the reflective polarizer, reflected second by themirror 3187 and reflected third by the reflective polarizer before being captured by thecamera 3180. While the light from theeye 2971 passes through theabsorptive polarizer 3185 twice, since the polarization axis of the light from theeye 2971 is oriented parallel to the polarization axis of the light from theeye 2971, it is substantially transmitted by theabsorptive polarizer 3185. As with the system described in connection withFIG. 30 , the system shown inFIG. 31 includes an optically flat partiallyreflective layer 2960 that preserves the wavefront of the light from theeye 2971 so that higher quality images of the wearer's eye can be captured. Also, since theDLP 2955 is not included in the optical path for the light reflected from theeye 2971 and the eye imaging process shown inFIG. 31 does not interfere with the displayed image, images of the wearer's eye can be captured independently from the displayed images. -
FIG. 32 shows an illustration of a system for displaying images and simultaneously capturing images of the wearer's eye that includes abeam splitter plate 3212 comprised of a reflective polarizer, which is held in air between thelight source 2958, theDLP 2955 and thecamera 3280. Theillumination light 2973 and the light from theeye 2971 are both polarized with polarization axes that are perpendicular to the transmission axis of the reflective polarizer. As a result, both theillumination light 2973 and the light from theeye 2971 are substantially reflected by the reflective polarizer. The illumination light 2873 is reflected toward theDLP 2955 by the reflective polarizer and split into image light 2969 and dark light 3275 depending on whether the individual DLP mirrors are respectively in the “on” state or the “off” state. By passing through thequarter wave layer 2957 twice, the polarization state of theillumination light 2973 is reversed in comparison to the polarization state of theimage light 2969 and thedark light 3275. As a result, theimage light 2969 and the dark light 3275 are then substantially transmitted by the reflective polarizer. Theabsorptive polarizer 3285 at the side of thebeam splitter plate 3212 has a transmission axis that is perpendicular to the polarization axis of thedark light 3275 and parallel to the polarization axis of the light from theeye 2971 so that the dark light 3275 is absorbed and the light from theeye 2971 is transmitted to thecamera 3280. As in the system shown inFIG. 30 , the system shown inFIG. 31 includes an optically flatbeam splitter plate 3212 that preserves the wavefront of the light from theeye 2971 so that higher quality images of the wearer's eye can be captured. Also, since theDLP 2955 is not included in the optical path for the light from theeye 2971 and the eye imaging process shown inFIG. 31 does not interfere with the displayed image, images of the wearer's eye can be captured independently from the displayed images. - Eye imaging systems where the polarization state of the light from the
eye 2971 needs to be opposite to that of the image light 2969 (as shown inFIGS. 30 , 31 and 32), need to be used withlower modules 204 that include combiners that will reflect both polarization states. As such, theseupper modules 202 are best suited for use with thelower modules 204 that include combiners that are reflective regardless of polarization state, examples of these lower modules are shown inFIGS. 6 , 8 a, 8 b, 8 c and 24-27. - In a further embodiment shown in
FIG. 33 , the partiallyreflective layer 3360 is comprised of a reflective polarizer on the side facing theillumination light 2973 and a short pass dichroic mirror on the side facing the light from theeye 3371 and thecamera 3080. Where the short pass dichroic mirror is a dielectric mirror coating that transmits visible light and reflects infrared light. The partiallyreflective layer 3360 can be comprised of a reflective polarizer bonded to the inner surface of theillumination wedge 2964 and a short pass dielectric mirror coating on the opposing inner surface of thecorrective wedge 2966, wherein theillumination wedge 2964 and thecorrective wedge 2966 are then optically bonded together. Alternatively, the partiallyreflective layer 3360 can be comprised of a thin substrate that has a reflective polarizer bonded to one side and a short pass dichroic mirror coating on the other side, where the partiallyreflective layer 3360 is then bonded between theillumination wedge 2964 and thecorrective wedge 2966. In this embodiment, an infrared light is included to illuminate the eye so that the light from the eye and the images captured of the eye are substantially comprised of infrared light. The wavelength of the infrared light is then matched to the reflecting wavelength of the shortpass dichroic mirror and the wavelength that the camera can capture images, for example an 800 nm wavelength can be used. In this way, the short pass dichroic mirror transmits the image light and reflects the light from the eye. Thecamera 3080 is then positioned at the side of thecorrective wedge 2966 in the area of the absorbinglight trap 3382, which is provided to absorb thedark light 2975. By positioning thecamera 3080 in a depression in the absorbinglight trap 3382, scattering of the dark light 2975 by thecamera 3080 can be reduced so that higher contrast images can be displayed to the wearer. An advantage of this embodiment is that the light from the eye need not be polarized, which can simplify the optical system and increase efficiency for the eye imaging system. - In yet another embodiment shown in
FIG. 32 a abeam splitter plate 3222 is comprised of a reflective polarizer on the side facing theillumination light 2973 and a short pass dichroic mirror on the side facing the light from theeye 3271 and thecamera 3280. An absorbingsurface 3295 is provided to trap thedark light 3275 and thecamera 3280 is positioned in an opening in the absorbingsurface 3295. In this way the system ofFIG. 32 can be made to function with unpolarized light from theeye 3271. - In embodiments directed to capturing images of the wearer's eye, light to illuminate the wearer's eye can be provided by several different sources including: light from the displayed image (i.e. image light); light from the environment that passes through the combiner or other optics; light provided by a dedicated eye light, etc.
FIGS. 34 and 34 a show illustrations of dedicated eye illumination lights 3420.FIG. 34 shows an illustration from a side view in which the dedicated illumination eye light 3420 is positioned at a corner of thecombiner 3410 so that it doesn't interfere with theimage light 3415. The dedicatedeye illumination light 3420 is pointed so that theeye illumination light 3425 illuminates theeyebox 3427 where theeye 3430 is located when the wearer is viewing displayed images provided by theimage light 3415.FIG. 34 a shows an illustration from the perspective of the eye of the wearer to show how the dedicatedeye illumination light 3420 is positioned at the corner of thecombiner 3410. While the dedicatedeye illumination light 3420 is shown at the upper left corner of thecombiner 3410, other positions along one of the edges of thecombiner 3410, or other optical or mechanical components, are possible as well. In other embodiments, more than one dedicated eye light 3420 with different positions can be used. In an embodiment, the dedicated eye light 3420 is an infrared light that is not visible by the wearer (e.g. 800 nm) so that theeye illumination light 3425 doesn't interfere with the displayed image perceived by the wearer. -
FIG. 35 shows a series of illustrations of captured eye images that show the eye glint (i.e. light that reflects off the front of the eye) produced by a dedicated eye light. In this embodiment of the invention, captured images of the wearer's eye are analyzed to determine the relative positions of theiris 3550, pupil, or other portion of the eye, and theeye glint 3560. The eye glint is a reflected image of the dedicated eye light 3420 when the dedicated light is used.FIG. 35 illustrates the relative positions of theiris 3550 and theeye glint 3560 for a variety of eye positions. By providing a dedicated eye light 3420 in a fixed position, combined with the fact that the human eye is essentially spherical, or at least a reliably repeatable shape, the eye glint provides a fixed reference point against which the determined position of the iris can be compared to determine where the wearer is looking, either within the displayed image or within the see-through view of the surrounding environment. By positioning the dedicated eye light 3420 at a corner of thecombiner 3410, theeye glint 3560 is formed away from theiris 3550 in the captured images. As a result, the positions of the iris and the eye glint can be determined more easily and more accurately during the analysis of the captured images, since they do not interfere with one another. In a further embodiment, the combiner includes an associated cut filter that prevents infrared light from the environment from entering the HWC and the camera is an infrared camera, so that the eye glint is only provided by light from the dedicated eye light. For example, the combiner can include a low pass filter that passes visible light while absorbing infrared light and the camera can include a high pass filter that absorbs visible light while passing infrared light. - In an embodiment of the eye imaging system, the lens for the camera is designed to take into account the optics associated with the
upper module 202 and thelower module 204. This is accomplished by designing the camera to include the optics in theupper module 202 and optics in thelower module 204, so that a high MTF image is produced, at the image sensor in the camera, of the wearer's eye. In yet a further embodiment, the camera lens is provided with a large depth of field to eliminate the need for focusing the camera to enable sharp image of the eye to be captured. Where a large depth of field is typically provided by a high f/# lens (e.g. f/# >5). In this case, the reduced light gathering associated with high f/# lenses is compensated by the inclusion of a dedicated eye light to enable a bright image of the eye to be captured. Further, the brightness of the dedicated eye light can be modulated and synchronized with the capture of eye images so that the dedicated eye light has a reduced duty cycle and the brightness of infrared light on the wearer's eye is reduced. - In a further embodiment,
FIG. 36 a shows an illustration of an eye image that is used to identify the wearer of the HWC. In this case, an image of the wearer'seye 3611 is captured and analyzed for patterns ofidentifiable features 3612. The patterns are then compared to a database of eye images to determine the identity of the wearer. After the identity of the wearer has been verified, the operating mode of the HWC and the types of images, applications, and information to be displayed, can be adjusted and controlled in correspondence to the determined identity of the wearer. Examples of adjustments to the operating mode depending on who the wearer is determined to be or not be include: making different operating modes or feature sets available, shutting down or sending a message to an external network, allowing guest features and applications to run, etc. -
FIG. 36 b is an illustration of another embodiment using eye imaging, in which the sharpness of the displayed image is determined based on the eye glint produced by the reflection of the displayed image from the wearer's eye surface. By capturing images of the wearer'seye 3611, aneye glint 3622, which is a small version of the displayed image can be captured and analyzed for sharpness. If the displayed image is determined to not be sharp, then an automated adjustment to the focus of the HWC optics can be performed to improve the sharpness. This ability to perform a measurement of the sharpness of a displayed image at the surface of the wearer's eye can provide a very accurate measurement of image quality. Having the ability to measure and automatically adjust the focus of displayed images can be very useful in augmented reality imaging where the focus distance of the displayed image can be varied in response to changes in the environment or changes in the method of use by the wearer. - An aspect of the present invention relates to controlling the
HWC 102 through interpretations of eye imagery. In embodiments, eye-imaging technologies, such as those described herein, are used to capture an eye image or series of eye images for processing. The image(s) may be process to determine a user intended action, an HWC predetermined reaction, or other action. For example, the imagery may be interpreted as an affirmative user control action for an application on theHWC 102. Or, the imagery may cause, for example, theHWC 102 to react in a pre-determined way such that theHWC 102 is operating safely, intuitively, etc. -
FIG. 37 illustrates a eye imagery process that involves imaging theHWC 102 wearer's eye(s) and processing the images (e.g. through eye imaging technologies described herein) to determine in what position 3702 the eye is relative to it's neutral or forward looking position and/or theFOV 3708. The process may involve a calibration step where the user is instructed, through guidance provided in the FOV of theHWC 102, to look in certain directions such that a more accurate prediction of the eye position relative to areas of the FOV can be made. In the event the wearer's eye is determined to be looking towards the right side of the FOV 3708 (as illustrated inFIG. 37 , the eye is looking out of the page) a virtual target line may be established to project what in the environment the wearer may be looking towards or at. The virtual target line may be used in connection with an image captured by camera on theHWC 102 that images the surrounding environment in front of the wearer. In embodiments, the field of view of the camera capturing the surrounding environment matches, or can be matched (e.g. digitally), to theFOV 3708 such that making the comparison is made more clear. For example, with the camera capturing the image of the surroundings in an angle that matches theFOV 3708 the virtual line can be processed (e.g. in 2d or 3d, depending on the camera images capabilities and/or the processing of the images) by projecting what surrounding environment objects align with the virtual target line. In the event there are multiple objects along the virtual target line, focal planes may be established corresponding to each of the objects such that digital content may be placed in an area in theFOV 3708 that aligns with the virtual target line and falls at a focal plane of an intersecting object. The user then may see the digital content when he focuses on the object in the environment, which is at the same focal plane. In embodiments, objects in line with the virtual target line may be established by comparison to mapped information of the surroundings. - In embodiments, the digital content that is in line with the virtual target line may not be displayed in the FOV until the eye position is in the right position.
- This may be a predetermined process. For example, the system may be set up such that a particular piece of digital content (e.g. an advertisement, guidance information, object information, etc.) will appear in the event that the wearer looks at a certain object(s) in the environment. A virtual target line(s) may be developed that virtually connects the wearer's eye with an object(s) in the environment (e.g. a building, portion of a building, mark on a building, gps location, etc.) and the virtual target line may be continually updated depending on the position and viewing direction of the wearer (e.g. as determined through GPS, e-compass, IMU, etc.) and the position of the object. When the virtual target line suggests that the wearer's pupil is substantially aligned with the virtual target line or about to be aligned with the virtual target line, the digital content may be displayed in the
FOV 3704. - In embodiments, the time spent looking along the virtual target line and/or a particular portion of the
FOV 3708 may indicate that the wearer is interested in an object in the environment and/or digital content being displayed. In the event there is no digital content being displayed at the time a predetermined period of time is spent looking at a direction, digital content may be presented in the area of theFOV 3708. The time spent looking at an object may be interpreted as a command to display information about the object, for example. In other embodiments, the content may not relate to the object and may be presented because of the indication that the person is relatively inactive. In embodiments, the digital content may be positioned in proximity to the virtual target line, but not in-line with it such that the wearer's view of the surroundings are not obstructed but information can augment the wearer's view of the surroundings. In embodiments, the time spent looking along a target line in the direction of displayed digital content may be an indication of interest in the digital content. This may be used as a conversion event in advertising. For example, an advertiser may pay more for an add placement if the wearer of theHWC 102 looks at a displayed advertisement for a certain period of time. As such, in embodiments, the time spent looking at the advertisement, as assessed by comparing eye position with the content placement, target line or other appropriate position may be used to determine a rate of conversion or other compensation amount due for the presentation. - An aspect of the invention relates to removing content from the FOV of the
HWC 102 when the wearer of theHWC 102 apparently wants to view the surrounding environments clearly.FIG. 38 illustrates a situation where eye imagery suggests that the eye has or is moving quickly so thedigital content 3804 in the FOV 3808 is removed from the FOV 3808. In this example, the wearer may be looking quickly to the side indicating that there is something on the side in the environment that has grabbed the wearer's attention. Thiseye movement 3802 may be captured through eye imaging techniques (e.g. as described herein) and if the movement matches a predetermined movement (e.g. speed, rate, pattern, etc.) the content may be removed from view. In embodiments, the eye movement is used as one input and HWC movements indicated by other sensors (e.g. IMU in the HWC) may be used as another indication. These various sensor movements may be used together to project an event that should cause a change in the content being displayed in the FOV. - In embodiments, to remove the entire displayed image from the FOV and thereby provide an unencumbered view of the surrounding environment, power to the light source is turned OFF while power to the electronics in the HWC is left ON. As a result, the entire displayed image disappears instantly but the electronics associated with displaying images continues to run so that when eye movement or other triggering event occurs, a displayed image can be instantly provided without having to wait for the electronics to reboot. By turning the light source OFF and ON, to make the displayed image respectively disappear and then reappear, rapid changes between a display mode and a see-through mode can be provided. In an example, a rapid eye movement to the side as captured by eye imaging can cause the light source to turn OFF thereby providing an unencumbered view of the environment. Later when the eye returns to a central position, the light source may be turned ON so the displayed image returns. In another example, a rapid eye movement or rapid head nod by the wearer can be used in an operating mode to cause the light source to be turned OFF to obtain an unencumbered view of the environment for a predetermined period of time. After the predetermined period of time, the light source is turned ON and the displayed image is returned.
- Another aspect of the present invention relates to determining a focal plane based on the wearer's eye convergence. Eyes are generally converged slightly and converge more when the person focuses on something very close. This is generally referred to as convergence. In embodiments, convergence is calibrated for the wearer. That is, the wearer may be guided through certain focal plane exercises to determine how much the wearer's eyes converge at various focal planes and at various viewing angles. The convergence information may then be stored in a database for later reference. In embodiments, a general table may be used in the event there is no calibration step or the person skips the calibration step. The two eyes may then be imaged periodically to determine the convergence in an attempt to understand what focal plane the wearer is focused on. In embodiments, the eyes may be imaged to determine a virtual target line and then the eye's convergence may be determined to establish the wearer's focus, and the digital content may be displayed or altered based thereon.
-
FIG. 39 illustrates a situation where digital content is moved 3902 within one or both of theFOVs 3908 and 3910 to align with the convergence of the eyes as determined by thepupil movement 3904. By moving the digital content to maintain alignment, in embodiments, the overlapping nature of the content is maintained so the object appears properly to the wearer. This can be important in situations where 3D content is displayed. - An aspect of the present invention relates to controlling the
HWC 102 based on events detected through eye imaging. A wearer winking, blinking, moving his eyes in a certain pattern, etc. may, for example, control an application of theHWC 102. Eye imaging (e.g. as described herein) may be used to monitor the eye(s) of the wearer and once a pre-determined pattern is detected an application control command may be initiated. - An aspect of the invention relates to monitoring the health of a person wearing a
HWC 102 by monitoring the wearer's eye(s). Calibrations may be made such that the normal performance, under various conditions (e.g. lighting conditions, image light conditions, etc.) of a wearer's eyes may be documented. The wearer's eyes may then be monitored through eye imaging (e.g. as described herein) for changes in their performance. Changes in performance may be indicative of a health concern (e.g. concussion, brain injury, stroke, loss of blood, etc.). If detected the data indicative of the change or event may be communicated from theHWC 102. - Aspects of the present invention relate to security and access of computer assets (e.g. the HWC itself and related computer systems) as determined through eye image verification. As discussed herein elsewhere, eye imagery may be compared to known person eye imagery to confirm a person's identity. Eye imagery may also be used to confirm the identity of people wearing the
HWCs 102 before allowing them to link together or share files, streams, information, etc. - An aspect of the present invention relates to securely linking
HWC 102 and securely sharing files, streams, etc. (referred to generally as file sharing) with other HWC's 102 and/or other computers. Eye imaging, position, and tracking described herein elsewhere may be used in connection with the secure linking and file sharing. For example, afirst HWC 102 may only be permitted to securely link with anotherHWC 102 if the wearer of theother HWC 102 is verified as a known person of a certain security level. In embodiments, the security level may be a government determined security level, a known person, a known friend, etc. Eye imaging may be used to identify theother HWCs 102 that may be allowed for sharing. For example, GPS or other location technologies may be used to identify other HWC's 102 in the proximity of afirst HWC 102 and those proximal HWC's may be sorted into ones verified as secure HWC's, as verified by eye imaging, for example, and ones not verified. The sorted information or portion thereof may be presented in thefirst HWC 102 such that the wearer of thefirst HWC 102 can select secure sharing partners. Other sensor information may be used in connection with the secure sharing process. For example, identifying that two HWC's 102 are looking at one another (e.g. through e-compass readings, orHWC 102 forward facing camera image capture processing, etc.) may indicate that the two would like to share files or otherwise link communications and this information may be used in connection with the eye imaging verification for the secure sharing process. Similarly, facial recognition, biometric information (e.g. heart rate signature, lung signature, etc.), near field communication identification, gestures by and between people wearing the HWC's, external user interfaces, voice commands, voice signatures, light emission signatures/signals, wireless communication signals (e.g. encrypted signals/commands), etc. may be used in combination with eye imaging and each other as a way of identifying potential secure share partners and sharing. In the event secure share partner(s) are identified, linking may be done automatically or through a user interface action, for example. In the event a user action is required, the display in theHWC 102 may display indications of who is a potential secure share partner by presented the indications on a map, through AR, or otherwise. The AR interface may involve presenting the indication in at positions in the field of view of theHWC 102 that are indicative of a GPS location of the potential secure sharing partner (as described herein elsewhere). For further information on secure linking and file sharing see U.S. patent application Ser. No. 14/181,473, entitled Secure Sharing in Head Worn Computing, filed Feb. 14, 2014, which is hereby incorporated by reference herein in its entirety. - Another aspect of the present invention relates to presentation techniques relating to the presentation of a known location or person or object at the known location. In embodiments, a spatially three-dimensional target line may be established between a
first HWC 102 and a known geo-spatial location (e.g. as described by a gps location or other triangulation technology). Digital content may then be presented in a position in the field of view of theHWC 102 that corresponds, or is in-line, with the target line. The digital content may further be presented such that it comes into the wearer's view when the wearer focuses on a particular focal plane in the distance. The focal plane could then be aligned with the geo-spatial location such that when the wearer looks at the geo-spatial location, which is in-line with the target line, the digital content will come into view for the wearer and indicate the location with a proper visual perspective. In embodiments, the perceived environmental position indicative of the geo-spatial location may be verified by comparing the distance and angle defined by the virtual target line with a distance and angle measured by a range finder. This may be done to confirm that there are no obstacles in between the wearer of theHWC 102 and the geo-spatial location, for example. In the event that the verification indicates that there is an obstacle and that the geo-spatial location cannot be directly viewed, the digital content indicative of the location may be altered to indicate that the location is obstructed. The digital content may indicate if the location is in close proximity to the obstruction, far from the obstruction, etc. For further information on geo-spatial position visualizations see U.S. patent application Ser. No. 14/205,313 entitled Spatial Location Presentation in Head Worn Computing, filed Mar. 11, 2014, which is hereby incorporated by reference herein in its entirety. - An aspect of the present invention relates to presenting digital content in a field of view of a see-through head worn optical system at a position that is dependent on feedback from a sensor. In embodiments, the sensor provides information relating to a speed of forward motion of a person wearing the see-through head worn optical system. In embodiments, the content may be shifted to a portion of the FOV that provides for greater see-through visibility for the wearer. For example, if the wearer is walking, the content may be shifted towards the edge of the FOV so the person can focus on the surrounding environment while walking but still be able to view the content at the edges of the FOV, In other embodiments, the person may be moving very fast, driving a car, for example, and the content may be shifted even further towards the edge of the FOV, or even past the FOV such that the person can more intently focus on the environment. In embodiments, when the content is shifted beyond the FOV, the content may be removed from display entirely and it may re-appear when an indication is presented that the wearer wants to view it or that the motion has slowed. For example, the wearer may use an external UI, gesture, eye control command, etc. to cause the content to come hack into the FOV. In embodiments, the wearer may be able to affirmatively cause the content to be moved to a location in the FOV dependent on his needs and this may modify or over-ride the content position shift based on the person's movement speed. For example, if the wearer is driving a car and the car is moving quickly, the wearer may choose to have the content displayed closer to the middle of the FOV and over-ride the auto-shift content position function.
- When a user is moving through an environment while wearing a head worn computer, the method in which information is displayed may differ depending on a variety of factors that describe the way in which the user is moving. Factors such as speed, gaze direction, rate of change of gaze direction, rate of change of movement direction and the type of information being displayed may all affect how the user wants the information to be displayed.
- In describing the method of display when moving, the pertinent information includes the heading that the user is looking along (generally referred to herein as “sight heading” or “sight vector”), the heading that the user is moving along (generally referred to herein as “movement heading” or “movement vector”) and the heading where the displayed information is visible by the user (generally referred to herein as “display heading” or “display vector”). Where headings are described in term of degrees as measured, for example, with a compass. Sight heading can be determined, for example, with an electronic compass or magnetometer in the HWC. Movement heading can be determined by multiple GPS readings, IMU readings, etc.
- A method of presenting content is to provide the displayed information directly in front of the user all the time. In this case, the sight heading is the same as the display heading. This method can be less than ideal when the user is moving because the displayed information interferes with the view of the surrounding environment. With a see-through HWC, there can be times when the user would prefer to be provided with an unencumbered view of the environment. As such, there will be times when the user would prefer to not have displayed information in the see-through field of view. In embodiments, the process adjusts the display heading in response to the sight heading, movement heading and speed of movement. In further embodiments the method of the invention adjusts the display heading based on the type of information being displayed and in response to the sight heading, movement heading and speed of movement. In yet other embodiments, the method adjusts the display heading in response to indications of how the user is moving. In further embodiments, the brightness of the displayed image is reduced in response to the sight heading, movement heading and speed of movement.
- Another example of when a user may not want to have displayed information presented in the see-through field of view is when some type of stimulus occurs in the adjacent vicinity such as a loud noise to the side of the user. Typically people respond to loud noises by turning their head to look toward the loud noise. At that point in time, the user would prefer to have an unencumbered view of the environment and, in embodiments, the content is removed based on the sound received at the HWC.
- In another example, the user moves through an environment at various speeds. When a person is stationary, the person can look in any direction so the sight heading can change, but the movement heading doesn't change because there is no movement. If a person is walking, the person can look back and forth while moving, as such the sight heading can vary on either side of the movement heading and the sight heading can vary slowly. As the speed increases, it becomes more difficult for the person to look to the side for long periods of time so the sight heading can change more rapidly as the person glances to the side and in general the sight heading is more likely to match the movement heading. At high speed, such as running, the person needs to look directly ahead most of the time, so the sight heading matches the movement heading most of the time. When in a vehicle traveling at even greater speed, the person can only look briefly to the side when driving and most of the time must look ahead at the road where the sight heading matches the movement heading with only brief departures. Of course when the person is riding in a vehicle but not driving the vehicle, the person can look sideways for extended periods of time, as a result the sight heading can vary substantially from the movement heading.
-
FIG. 45 shows an example set of data for a person moving through an environment over a path that starts with a movement heading of 0 degrees and ends with a movement heading of 114 degrees during which time the speed of movement varies from 0 m/sec to 20 m/sec. The sight heading can be seen to vary on either side of the movement heading while moving as the person looks from side to side. Large changes in sight heading occur when the movement speed is 0 m/sec when the person is standing still, followed by step changes in movement heading. - Embodiments provide a process for determining the display heading that takes into account the way a user moves through an environment and provides a display heading that makes it easy for the user to find the displayed information while also providing unencumbered see-through views of the environment in response to different movements, speed of movement or different types of information being displayed.
-
FIG. 46 illustrates a see-through view as may be seen when using a HWC wherein information is overlaid onto a see-through view of the environment. The tree and the building are actually in the environment and the text is displayed in the see-through display such that it appears overlaid on the environment. In addition to text information such as, for example, instructions and weather information, some augmented reality information is shown that relates to nearby objects in the environment. - In an embodiment, the display heading is determined based on speed of movement. At low speeds, the display heading may be substantially the same as the sight heading while at high speed the display heading may be substantially the same as the movement heading. In embodiments, as long as the user remains stationary, the displayed information is presented directly in front of the user and HWC. However, as the movement speed increases (e.g. above a threshold or continually, etc.) the display heading becomes substantially the same as the movement heading regardless of the direction the user is looking, so that when the user looks in the direction of movement, the displayed information is directly in front of the user and HMD and when the user looks to the side the displayed information is not visible.
- Rapid changes in sight heading can be followed by a slower change in the display heading to provide a damped response to head rotation. Alternatively, the display heading can be substantially the time averaged sight heading so that the displayed information is presented at a heading that is in the middle of a series of sight headings over a period of time. In this embodiment, if the user stops moving their head, the display heading gradually becomes the same as the sight heading and the displayed information moves into the display field of view in front of the user and HMD. In embodiments, when there is a high rate of sight heading change, the process delays the effect of the time averaged sight heading on the display heading. In this way, the effect of rapid head movements on display heading is reduced and the positioning of the displayed information within the display field of view is stabilized laterally.
- In another embodiment, display heading is determined based on speed of movement where at high-speed, the display heading is substantially the same as the movement heading. At mid-speed the display heading is substantially the same as a time averaged sight heading so that rapid head rotations are damped out and the display heading is in the middle of back and forth head movements.
- In yet another embodiment, the type of information being displayed is included in determining how the information should be displayed. Augmented reality information that is connected to objects in the environment is given a display heading that substantially matches the sight heading. In this way, as the user rotates their head, augmented reality information comes into view that is related to objects that are in the see-through view of the environment. At the same time, information that is not connected to objects in the environment is given a display heading that is determined based on the type of movements and speed of movements as previously described in this specification.
- In yet a further embodiment, when the speed of movement is determined to be above a threshold, the information displayed is moved downward in the display field of view so that the upper portion of the display field of view has less information or no information displayed to provide the user with an unencumbered see-through view of the environment.
-
FIGS. 47 and 48 show illustrations of a see-through view including overlaid displayed information.FIG. 47 shows the see-through view immediately after a rapid change in sight heading from the sight heading associated with the see-through view shown inFIG. 46 wherein the change in sight heading comes from a head rotation. In this case, the display heading is delayed.FIG. 48 shows how at a later time, the display heading catches up to the sight heading. The augmented reality information remains in positions within the display field of view where the association with objects in the environment can be readily made by the user. -
FIG. 49 shows an illustration of a see-through view example including overlaid displayed information that has been shifted downward in the display field of view to provide an unencumbered see-through view in the upper portion of the see-through view. At the same time, augmented reality labels have been maintained in locations within the display field of view so they can be readily associated with objects in the environment. - In embodiments, the content may be positioned with respect to other types of sensors. For example, a sensor may detect an environmental condition that may be of interest and this may effect the position of the content in the display. For example, the sensor may be a chemical sensor, fire sensor, mechanical sensor, electrical sensor, audio sensor, biologic sensor, etc. and the sensor may detect the presence of danger or other condition that causes the wearer to want to view the surroundings in a clear view so the content may then shift in or out of the field of view, as described herein. In embodiments, the sensor(s) may be mounted on/in the
HWC 102, local to theHWC 102, remote from theHWC 102, or otherwise located. In embodiments, the sensor input may cause content to be presented in the field of view and then the sensor input, from the same or a different sensor or combination of sensors, may cause the content to be re-positioned. -
FIG. 40 illustrates an embodiment in which digital content presented in a see-through FOV is positioned based on the speed in which the wearer is moving. When the person is not moving, as measured by sensor(s) in the HWC 102 (e.g. IMU, GPS based tracking, etc.), digital content may be presented at the stationaryperson content position 4004. Thecontent position 4004 is indicated as being in the middle of the see-throughFOV 4002; however, this is meant to illustrate that the digital content is positioned within the see-through FOV at a place that is generally desirable knowing that the wearer is not moving and as such the wearer's surrounding see through view can be somewhat obstructed. So, the stationary person content position, or neutral position, may not be centered in the see-through FOV; it may be positioned somewhere in the see-through FOV deemed desirable and the sensor feedback may shift the digital content from the neutral position. The movement of the digital content for a quickly moving person is also shown inFIG. 40 wherein as the person turns their head to the side, the digital content moves out of the see-through FOV tocontent position 4008 and then moves back as the person turns their head back. For a slowly moving person, the head movement can be more complex and as such the movement of the digital content in an out of the see-through FOV can follow a path such as that shown bycontent position 4010. - In embodiments, the sensor that assesses the wearer's movements may be a GPS sensor, IMU, accelerometer, etc. The content position may be shifted from a neutral position to a position towards a side edge of the field of view as the forward motion increases. The content position may be shifted from a neutral position to a position towards a top or bottom edge of the field of view as the forward motion increases. The content position may shift based on a threshold speed of the assessed motion. The content position may shift linearly based on the speed of the forward motion. The content position may shift non-linearly based on the speed of the forward motion. The content position may shift outside of the field of view. In embodiments, the content is no longer displayed if the speed of movement exceeds a predetermined threshold and will be displayed again once the forward motion slows.
- In embodiments, the content position may generally be referred to as shifting; it should be understood that the term shifting encompasses a process where the movement from one position to another within the see-through FOV or out of the FOV is visible to the wearer (e.g. the content appears to slowly or quickly move and the user perceives the movement itself) or the movement from one position to another may not be visible to the wearer (e.g. the content appears to jump in a discontinuous fashion or the content disappears and then reappears in the new position).
- Another aspect of the present invention relates to removing the content from the field of view or shifting it to a position within the field of view that increases the wearer's view of the surrounding environment when a sensor causes an alert command to be issued. In embodiments, the alert may be due to a sensor or combination of sensors that sense a condition above a threshold value. For example, if an audio sensor detects a loud sound of a certain pitch, content in the field of view may be removed or shifted to provide a clear view of the surrounding environment for the wearer. In addition to the shifting of the content, in embodiments, an indication of why the content was shifted may be presented in the field of view or provided through audio feedback to the wearer. For instance, if a carbon monoxide sensor detects a high concentration in the area, content in the field of view may be shifted to the side of the field of view or removed from the field of view and an indication may be provided to the wearer that there is a high concentration of carbon monoxide in the area. This new information, when presented in the field of view, may similarly be shifted within or outside of the field of view depending on the movement speed of the wearer.
-
FIG. 41 illustrates how content may be shifted from aneutral position 4104 to analert position 4108. In this embodiment, the content is shifted outside of the see-throughFOV 4102. In other embodiments, the content may be shifted as described herein. - Another aspect of the present invention relates to identification of various vectors or headings related to the
HWC 102, along with sensor inputs, to determine how to position content in the field of view. In embodiments, the speed of movement of the wearer is detected and used as an input for position of the content and, depending on the speed, the content may be positioned with respect to a movement vector or heading (i.e. the direction of the movement), or a sight vector or heading (i.e. the direction of the wearer's sight direction). For example, if the wearer is moving very fast the content may be positioned within the field of view with respect to the movement vector because the wearer is only going to be looking towards the sides of himself periodically and for short periods of time. As another example, if the wearer is moving slowly, the content may be positioned with respect to the sight heading because the user may more freely be shifting his view from side to side. -
FIG. 42 illustrates two examples where the movement vector may effect content positioning.Movement vector A 4202 is shorter thanmovement vector B 4210 indicating that the forward speed and/or acceleration of movement of the person associated withmovement vector A 4202 is lower than the person associated withmovement vector B 4210. Each person is also indicated as having a sight vector or heading 4208 and 4212. The sight vectors A 4208 andB 4210 are the same from a relative perspective. The white area inside of the black triangle in front of each person is indicative of how much time each person likely spends looking at a direction that is not in line with the movement vector. The time spent looking offangle A 4204 is indicated as being more than that of the time spent looking offangle B 4214. This may be because the movement vector speed A is lower than movement vector speed B. The faster the person moves forward the more the person tends to look in the forward direction, typically. TheFOVs A 4218 andB 4222 illustrate how content may be aligned depending on themovement vectors sight vectors sight vector 4220. This may be due to the lower speed of themovement vector A 4202. This may also be due to the prediction of a larger amount of time spent looking offangle A 4204.FOV B 4222 is illustrated as presenting content in line with themovement vector 4224. This may be due to the higher speed ofmovement vector B 4210. This may also be due to the prediction of a shorter amount of time spent looking offangle B 4214. - Another aspect of the present invention relates to damping a rate of content position change within the field of view. As illustrated in
FIG. 43 , the sight vector may undergo arapid change 4304. This rapid change may be an isolated event or it may be made at or near a time when other sight vector changes are occurring. The wearer's head may be turning back and forth for some reason. In embodiments, the rapid successive changes in sight vector may cause a damped rate ofcontent position change 4308 within the FOV 4302. For example, the content may be positioned with respect to the sight vector, as described herein, and the rapid change in sight vector may normally cause a rapid content position change; however, since the sight vector is successively changing, the rate of position change with respect to the sight vector may be damped, slowed, or stopped. The position rate change may be altered based on the rate of change of the sight vector, average of the sight vector changes, or otherwise altered. - Another aspect of the present invention relates to simultaneously presenting more than one content in the field of view of a see-through optical system of a
HWC 102 and positioning one content with the sight heading and one content with the movement heading.FIG. 44 illustrates two FOV'sA 4414 andB 4420, which correspond respectively to the two identified sight vectors A 4402 andB 4404.FIG. 44 also illustrates an object in theenvironment 4408 at a position relative to the sight vectors A 4402 andB 4404. When the person is looking alongsight vector A 4402, theenvironment object 4408 can be seen through the field ofview A 4414 atposition 4412. As illustrated, sight heading aligned content is presented as TEXT in proximity with theenvironment object 4412. At the same time,other content 4418 is presented in the field ofview A 4414 at a position aligned in correspondence with the movement vector. As the movement speed increases, thecontent 4418 may shift as described herein. When the sight vector of the person issight vector B 4404 theenvironmental object 4408 is not seen in the field ofview B 4420. As a result, the sight alignedcontent 4410 is not presented in field ofview B 4420; however, the movement alignedcontent 4418 is presented and is still dependent on the speed of the motion. - In a further embodiment, in an operating mode such as when the user is moving in an environment, digital content is presented at the side of the user's see-through FOV so that the user can only view the digital content by turning their head. In this case, when the user is looking straight ahead, such as when the movement heading matches the sight heading, the see-through view FOV does not include digital content. The user then accesses the digital content by turning their head to the side whereupon the digital content moves laterally into the user's see-through FOV. In another embodiment, the digital content is ready for presentation and will be presented if an indication for it's presentation is received. For example, the information may be ready for presentation and if the sight heading or predetermined position of the
HWC 102 is achieved the content may then be presented. The wearer may look to the side and the content may be presented. In another embodiment, the user may cause the content to move into an area in the field of view by looking in a direction for a predetermined period of time, blinking, winking, or displaying some other pattern that can be captured through eye imaging technologies (e.g. as described herein elsewhere). - In yet another embodiment, an operating mode is provided wherein the user can define sight headings wherein the associated see-through FOV includes digital content or does not include digital content. In an example, this operating mode can be used in an office environment where when the user is looking at a wall digital content is provided within the FOV, whereas when the user is looking toward a hallway, the FOV is unencumbered by digital content. In another example, when the user is looking horizontally digital content is provided within the FOV, but when the user looks down (e.g. to look at a desktop or a cellphone) the digital content is removed from the FOV.
- Although embodiments of HWC have been described in language specific to features, systems, computer processes and/or methods, the appended claims are not necessarily limited to the specific features, systems, computer processes and/or methods described. Rather, the specific features, systems, computer processes and/or and methods are disclosed as non-limited example implementations of HWC. All documents referenced herein are hereby incorporated by reference.
Claims (9)
Priority Applications (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/516,013 US20150277113A1 (en) | 2014-03-28 | 2014-10-16 | Sensor dependent content position in head worn computing |
Applications Claiming Priority (2)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US14/228,526 US20150277118A1 (en) | 2014-03-28 | 2014-03-28 | Sensor dependent content position in head worn computing |
US14/516,013 US20150277113A1 (en) | 2014-03-28 | 2014-10-16 | Sensor dependent content position in head worn computing |
Related Parent Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/228,526 Continuation US20150277118A1 (en) | 2014-01-17 | 2014-03-28 | Sensor dependent content position in head worn computing |
Publications (1)
Publication Number | Publication Date |
---|---|
US20150277113A1 true US20150277113A1 (en) | 2015-10-01 |
Family
ID=54190086
Family Applications (7)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/228,526 Abandoned US20150277118A1 (en) | 2014-01-17 | 2014-03-28 | Sensor dependent content position in head worn computing |
US14/290,023 Abandoned US20150279104A1 (en) | 2014-03-28 | 2014-05-29 | Sensor dependent content position in head worn computing |
US14/515,315 Abandoned US20150277122A1 (en) | 2014-03-28 | 2014-10-15 | Sensor dependent content position in head worn computing |
US14/516,009 Abandoned US20150277549A1 (en) | 2014-03-28 | 2014-10-16 | Sensor dependent content position in head worn computing |
US14/516,013 Abandoned US20150277113A1 (en) | 2014-03-28 | 2014-10-16 | Sensor dependent content position in head worn computing |
US14/516,017 Abandoned US20150279107A1 (en) | 2014-03-28 | 2014-10-16 | Sensor dependent content position in head worn computing |
US14/547,134 Active US9423612B2 (en) | 2014-03-28 | 2014-11-19 | Sensor dependent content position in head worn computing |
Family Applications Before (4)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/228,526 Abandoned US20150277118A1 (en) | 2014-01-17 | 2014-03-28 | Sensor dependent content position in head worn computing |
US14/290,023 Abandoned US20150279104A1 (en) | 2014-03-28 | 2014-05-29 | Sensor dependent content position in head worn computing |
US14/515,315 Abandoned US20150277122A1 (en) | 2014-03-28 | 2014-10-15 | Sensor dependent content position in head worn computing |
US14/516,009 Abandoned US20150277549A1 (en) | 2014-03-28 | 2014-10-16 | Sensor dependent content position in head worn computing |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US14/516,017 Abandoned US20150279107A1 (en) | 2014-03-28 | 2014-10-16 | Sensor dependent content position in head worn computing |
US14/547,134 Active US9423612B2 (en) | 2014-03-28 | 2014-11-19 | Sensor dependent content position in head worn computing |
Country Status (1)
Country | Link |
---|---|
US (7) | US20150277118A1 (en) |
Cited By (61)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9286728B2 (en) | 2014-02-11 | 2016-03-15 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US9377625B2 (en) | 2014-01-21 | 2016-06-28 | Osterhout Group, Inc. | Optical configurations for head worn computing |
US9401540B2 (en) | 2014-02-11 | 2016-07-26 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US9423842B2 (en) | 2014-09-18 | 2016-08-23 | Osterhout Group, Inc. | Thermal management for head-worn computer |
US9423612B2 (en) | 2014-03-28 | 2016-08-23 | Osterhout Group, Inc. | Sensor dependent content position in head worn computing |
US9436006B2 (en) | 2014-01-21 | 2016-09-06 | Osterhout Group, Inc. | See-through computer display systems |
US9448409B2 (en) | 2014-11-26 | 2016-09-20 | Osterhout Group, Inc. | See-through computer display systems |
US9494800B2 (en) | 2014-01-21 | 2016-11-15 | Osterhout Group, Inc. | See-through computer display systems |
US9523856B2 (en) | 2014-01-21 | 2016-12-20 | Osterhout Group, Inc. | See-through computer display systems |
US9529192B2 (en) | 2014-01-21 | 2016-12-27 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9529195B2 (en) | 2014-01-21 | 2016-12-27 | Osterhout Group, Inc. | See-through computer display systems |
US9532715B2 (en) | 2014-01-21 | 2017-01-03 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9547465B2 (en) | 2014-02-14 | 2017-01-17 | Osterhout Group, Inc. | Object shadowing in head worn computing |
US9575321B2 (en) | 2014-06-09 | 2017-02-21 | Osterhout Group, Inc. | Content presentation in head worn computing |
US9651784B2 (en) | 2014-01-21 | 2017-05-16 | Osterhout Group, Inc. | See-through computer display systems |
US9651787B2 (en) | 2014-04-25 | 2017-05-16 | Osterhout Group, Inc. | Speaker assembly for headworn computer |
US9671613B2 (en) | 2014-09-26 | 2017-06-06 | Osterhout Group, Inc. | See-through computer display systems |
US9672210B2 (en) | 2014-04-25 | 2017-06-06 | Osterhout Group, Inc. | Language translation with head-worn computing |
US9684172B2 (en) | 2014-12-03 | 2017-06-20 | Osterhout Group, Inc. | Head worn computer display systems |
USD792400S1 (en) | 2014-12-31 | 2017-07-18 | Osterhout Group, Inc. | Computer glasses |
US9715112B2 (en) | 2014-01-21 | 2017-07-25 | Osterhout Group, Inc. | Suppression of stray light in head worn computing |
US9720234B2 (en) | 2014-01-21 | 2017-08-01 | Osterhout Group, Inc. | See-through computer display systems |
USD794637S1 (en) | 2015-01-05 | 2017-08-15 | Osterhout Group, Inc. | Air mouse |
US9740280B2 (en) | 2014-01-21 | 2017-08-22 | Osterhout Group, Inc. | Eye imaging in head worn computing |
WO2017143128A1 (en) * | 2016-02-18 | 2017-08-24 | Osterhout Group, Inc. | Haptic systems for head-worn computers |
US9746686B2 (en) | 2014-05-19 | 2017-08-29 | Osterhout Group, Inc. | Content position calibration in head worn computing |
US9753288B2 (en) | 2014-01-21 | 2017-09-05 | Osterhout Group, Inc. | See-through computer display systems |
US9766463B2 (en) | 2014-01-21 | 2017-09-19 | Osterhout Group, Inc. | See-through computer display systems |
US9784973B2 (en) | 2014-02-11 | 2017-10-10 | Osterhout Group, Inc. | Micro doppler presentations in head worn computing |
US9810906B2 (en) | 2014-06-17 | 2017-11-07 | Osterhout Group, Inc. | External user interface for head worn computing |
US9811152B2 (en) | 2014-01-21 | 2017-11-07 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9829707B2 (en) | 2014-08-12 | 2017-11-28 | Osterhout Group, Inc. | Measuring content brightness in head worn computing |
US9836122B2 (en) | 2014-01-21 | 2017-12-05 | Osterhout Group, Inc. | Eye glint imaging in see-through computer display systems |
US9841599B2 (en) | 2014-06-05 | 2017-12-12 | Osterhout Group, Inc. | Optical configurations for head-worn see-through displays |
US9852545B2 (en) | 2014-02-11 | 2017-12-26 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US9939934B2 (en) | 2014-01-17 | 2018-04-10 | Osterhout Group, Inc. | External user interface for head worn computing |
US9939646B2 (en) | 2014-01-24 | 2018-04-10 | Osterhout Group, Inc. | Stray light suppression for head worn computing |
US9952664B2 (en) | 2014-01-21 | 2018-04-24 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9965681B2 (en) | 2008-12-16 | 2018-05-08 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US10062182B2 (en) | 2015-02-17 | 2018-08-28 | Osterhout Group, Inc. | See-through computer display systems |
US10191279B2 (en) | 2014-03-17 | 2019-01-29 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US10254856B2 (en) | 2014-01-17 | 2019-04-09 | Osterhout Group, Inc. | External user interface for head worn computing |
US10558050B2 (en) | 2014-01-24 | 2020-02-11 | Mentor Acquisition One, Llc | Haptic systems for head-worn computers |
US10591728B2 (en) | 2016-03-02 | 2020-03-17 | Mentor Acquisition One, Llc | Optical systems for head-worn computers |
US10649220B2 (en) | 2014-06-09 | 2020-05-12 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US10663740B2 (en) | 2014-06-09 | 2020-05-26 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US10667981B2 (en) | 2016-02-29 | 2020-06-02 | Mentor Acquisition One, Llc | Reading assistance system for visually impaired |
US10684687B2 (en) | 2014-12-03 | 2020-06-16 | Mentor Acquisition One, Llc | See-through computer display systems |
US10853589B2 (en) | 2014-04-25 | 2020-12-01 | Mentor Acquisition One, Llc | Language translation with head-worn computing |
US10878775B2 (en) | 2015-02-17 | 2020-12-29 | Mentor Acquisition One, Llc | See-through computer display systems |
US11024091B2 (en) | 2016-12-21 | 2021-06-01 | Pcms Holdings, Inc. | System and method for placement of augmented reality information for users based on their activity |
US11104272B2 (en) | 2014-03-28 | 2021-08-31 | Mentor Acquisition One, Llc | System for assisted operator safety using an HMD |
US11103122B2 (en) | 2014-07-15 | 2021-08-31 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US11227294B2 (en) | 2014-04-03 | 2022-01-18 | Mentor Acquisition One, Llc | Sight information collection in head worn computing |
US11232530B2 (en) * | 2017-02-28 | 2022-01-25 | Nec Corporation | Inspection assistance device, inspection assistance method, and recording medium |
US11269182B2 (en) | 2014-07-15 | 2022-03-08 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US11487110B2 (en) | 2014-01-21 | 2022-11-01 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US11615724B2 (en) | 2020-10-05 | 2023-03-28 | Volkswagen Aktiengesellschaft | Horizontal position determination for displaying virtual objects in a head-up display |
US11669163B2 (en) | 2014-01-21 | 2023-06-06 | Mentor Acquisition One, Llc | Eye glint imaging in see-through computer display systems |
US11737666B2 (en) | 2014-01-21 | 2023-08-29 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US11892644B2 (en) | 2014-01-21 | 2024-02-06 | Mentor Acquisition One, Llc | See-through computer display systems |
Families Citing this family (10)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JP6376807B2 (en) * | 2014-04-02 | 2018-08-22 | キヤノン株式会社 | Display device, display control method, and program |
US20170092002A1 (en) * | 2015-09-30 | 2017-03-30 | Daqri, Llc | User interface for augmented reality system |
US10482661B2 (en) | 2016-03-01 | 2019-11-19 | International Business Machines Corporation | Displaying of augmented reality objects |
US10387719B2 (en) * | 2016-05-20 | 2019-08-20 | Daqri, Llc | Biometric based false input detection for a wearable computing device |
JP6450890B2 (en) * | 2016-07-06 | 2019-01-09 | 株式会社オプティム | Image providing system, image providing method, and program |
US11422764B1 (en) | 2018-06-03 | 2022-08-23 | Epic Optix, Inc. | Multi-platform integrated display |
US10616565B2 (en) * | 2018-08-21 | 2020-04-07 | The Boeing Company | System and method for foveated simulation |
US11126850B1 (en) | 2020-04-09 | 2021-09-21 | Facebook Technologies, Llc | Systems and methods for detecting objects within the boundary of a defined space while in artificial reality |
US11232644B1 (en) * | 2020-12-31 | 2022-01-25 | Facebook Technologies, Llc | Systems and methods for providing spatial awareness in virtual reality |
CN114451887B (en) * | 2022-01-20 | 2023-10-24 | 西安理工大学 | Contact type overhead dimension measuring method and measuring device |
Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110170061A1 (en) * | 2010-01-08 | 2011-07-14 | Gordon Gary B | Gaze Point Tracking Using Polarized Light |
US20120212398A1 (en) * | 2010-02-28 | 2012-08-23 | Osterhout Group, Inc. | See-through near-eye display glasses including a partially reflective, partially transmitting optical element |
US20130083009A1 (en) * | 2011-09-30 | 2013-04-04 | Kevin A. Geisner | Exercising applications for personal audio/visual system |
US20130154918A1 (en) * | 2011-12-20 | 2013-06-20 | Benjamin Isaac Vaught | Enhanced user eye gaze estimation |
US20140247286A1 (en) * | 2012-02-20 | 2014-09-04 | Google Inc. | Active Stabilization for Heads-Up Displays |
US8947323B1 (en) * | 2012-03-20 | 2015-02-03 | Hayes Solos Raffle | Content display methods |
US20150205494A1 (en) * | 2014-01-23 | 2015-07-23 | Jason Scott | Gaze swipe selection |
Family Cites Families (458)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US3305294A (en) | 1964-12-03 | 1967-02-21 | Optical Res & Dev Corp | Two-element variable-power spherical lens |
GB1540992A (en) | 1975-04-22 | 1979-02-21 | Smiths Industries Ltd | Display or other systems and equipment for use in such systems |
US7030925B1 (en) | 1978-04-23 | 2006-04-18 | Canon, Inc. | Camera system having converting means, recording means, reproduction means, plate-shaped display and protection means |
SE454250B (en) | 1984-09-24 | 1988-04-18 | Asea Ab | INDUSTRIAL ROBOT WITH LINEAR DRIVE DEVICES |
US4842389A (en) | 1987-06-12 | 1989-06-27 | Flight Dynamics, Inc. | Vehicle display system using a holographic windshield prepared to withstand lamination process |
US4852988A (en) | 1988-09-12 | 1989-08-01 | Applied Science Laboratories | Visor and camera providing a parallax-free field-of-view image for a head-mounted eye movement measurement system |
US4928301A (en) | 1988-12-30 | 1990-05-22 | Bell Communications Research, Inc. | Teleconferencing terminal with camera behind display screen |
USD327674S (en) | 1990-02-21 | 1992-07-07 | Primax Electronics Ltd. | Video display control or similar article |
US5151722A (en) | 1990-11-05 | 1992-09-29 | The Johns Hopkins University | Video display on spectacle-like frame |
US8730129B2 (en) | 1990-12-07 | 2014-05-20 | Dennis J Solomon | Advanced immersive visual display system |
US5257094A (en) | 1991-07-30 | 1993-10-26 | Larussa Joseph | Helmet mounted display system |
US5303085A (en) | 1992-02-07 | 1994-04-12 | Rallison Richard D | Optically corrected helmet mounted display |
US5621424A (en) | 1992-08-24 | 1997-04-15 | Olympus Optical Co., Ltd. | Head mount display apparatus allowing easy switching operation from electronic image to external field image |
US5490647A (en) | 1993-08-09 | 1996-02-13 | Rice; Gregory H. | Palm rest for use with computer data entry devices |
DE69434843T2 (en) | 1993-08-12 | 2007-02-15 | Seiko Epson Corp. | Head-mounted image display device and data processing system incorporating the same |
JPH07110735A (en) | 1993-10-14 | 1995-04-25 | Nippon Telegr & Teleph Corp <Ntt> | Fitting type pen input device |
USD352930S (en) | 1993-10-29 | 1994-11-29 | Hunter Fan Company | Remote control for a ceiling fan |
US5717422A (en) | 1994-01-25 | 1998-02-10 | Fergason; James L. | Variable intensity high contrast passive display |
US5606458A (en) | 1994-08-24 | 1997-02-25 | Fergason; James L. | Head mounted display and viewing system using a remote retro-reflector and method of displaying and viewing an image |
JP3390289B2 (en) | 1995-06-16 | 2003-03-24 | 富士重工業株式会社 | Alarm device |
US5767841A (en) | 1995-11-03 | 1998-06-16 | Hartman; William M. | Two-sided trackball |
USD375748S (en) | 1995-11-03 | 1996-11-19 | Hartman William M | Hand held remote |
USD376790S (en) | 1995-12-21 | 1996-12-24 | Goulet Matthew G | Computer hand controller |
JPH09219832A (en) | 1996-02-13 | 1997-08-19 | Olympus Optical Co Ltd | Image display |
US5729242A (en) | 1996-05-08 | 1998-03-17 | Hughes Electronics | Dual PDLC-projection head-up display |
US6046712A (en) | 1996-07-23 | 2000-04-04 | Telxon Corporation | Head mounted communication system for providing interactive visual communications with a remote system |
US6310733B1 (en) | 1996-08-16 | 2001-10-30 | Eugene Dolgoff | Optical elements and methods for their manufacture |
US6847336B1 (en) | 1996-10-02 | 2005-01-25 | Jerome H. Lemelson | Selectively controllable heads-up display system |
US6204974B1 (en) | 1996-10-08 | 2001-03-20 | The Microoptical Corporation | Compact image display system for eyeglasses or other head-borne frames |
US5914818A (en) | 1996-11-29 | 1999-06-22 | Texas Instruments Incorporated | Offset projection lens for use with reflective spatial light modulators |
US6160552A (en) | 1997-01-09 | 2000-12-12 | Sun Microsystems, Inc. | Method and apparatus for managing multiple hierarchical lists within a browser |
USD392959S (en) | 1997-02-26 | 1998-03-31 | Kensington Microware Limited | Computer pointing device |
US6028608A (en) | 1997-05-09 | 2000-02-22 | Jenkins; Barry | System and method of perception-based image generation and encoding |
US6034653A (en) | 1997-08-01 | 2000-03-07 | Colorado Microdisplay, Inc. | Head-set display device |
US5991084A (en) | 1998-02-04 | 1999-11-23 | Inviso | Compact compound magnified virtual image display with a reflective/transmissive optic |
USD410638S (en) | 1998-05-08 | 1999-06-08 | Logitech Incorporated | Optical trackball |
US6610917B2 (en) | 1998-05-15 | 2003-08-26 | Lester F. Ludwig | Activity indication, external source, and processing loop provisions for driven vibrating-element environments |
US6734838B1 (en) | 1998-05-18 | 2004-05-11 | Dimension Technologies Inc. | Enhanced resolution for image generation |
JP2000102036A (en) | 1998-09-22 | 2000-04-07 | Mr System Kenkyusho:Kk | Composite actual feeling presentation system, composite actual feeling presentation method, man-machine interface device and man-machine interface method |
JP2000194726A (en) | 1998-10-19 | 2000-07-14 | Sony Corp | Device, method and system for processing information and providing medium |
JP2000199883A (en) | 1998-10-29 | 2000-07-18 | Fujitsu Ltd | Reflection type projector device |
US6535182B2 (en) | 1998-12-07 | 2003-03-18 | Koninklijke Philips Electronics N.V. | Head-mounted projection display system |
US6563626B1 (en) | 1999-02-25 | 2003-05-13 | Brother Kogyo Kabushiki Kaisha | Display device |
US6222677B1 (en) | 1999-04-12 | 2001-04-24 | International Business Machines Corporation | Compact optical system for use in virtual display applications |
CN1322370C (en) | 1999-06-29 | 2007-06-20 | 3M创新有限公司 | Optical system for projection displays |
US6456438B1 (en) | 1999-08-12 | 2002-09-24 | Honeywell Inc. | Variable immersion vignetting display |
US6480174B1 (en) | 1999-10-09 | 2002-11-12 | Optimize Incorporated | Eyeglass-mount display having personalized fit module |
US20020149545A1 (en) | 1999-11-15 | 2002-10-17 | Ryotaro Hanayama | Head mounted display system |
USD451892S1 (en) | 1999-11-19 | 2001-12-11 | Tefal S.A. | Switch with a light |
US6717348B2 (en) | 1999-12-09 | 2004-04-06 | Fuji Photo Film Co., Ltd. | Display apparatus |
JP3957468B2 (en) | 2000-03-31 | 2007-08-15 | 日立造船株式会社 | Mixed reality realization system |
US6642945B1 (en) | 2000-05-04 | 2003-11-04 | Microsoft Corporation | Method and system for optimizing a visual display for handheld computer systems |
US6995753B2 (en) | 2000-06-06 | 2006-02-07 | Semiconductor Energy Laboratory Co., Ltd. | Display device and method of manufacturing the same |
US6417970B1 (en) | 2000-06-08 | 2002-07-09 | Interactive Imaging Systems | Two stage optical system for head mounted display |
JP4626019B2 (en) | 2000-07-05 | 2011-02-02 | 株式会社ニコン | Glasses frame |
US6747611B1 (en) | 2000-07-27 | 2004-06-08 | International Business Machines Corporation | Compact optical system and packaging for head mounted display |
US7003308B1 (en) | 2000-09-12 | 2006-02-21 | At&T Corp. | Method and system for handwritten electronic messaging |
US6563648B2 (en) | 2000-10-20 | 2003-05-13 | Three-Five Systems, Inc. | Compact wide field of view imaging system |
JP4560958B2 (en) | 2000-12-21 | 2010-10-13 | 日本テキサス・インスツルメンツ株式会社 | Micro electro mechanical system |
KR100408518B1 (en) | 2001-04-12 | 2003-12-06 | 삼성전자주식회사 | Pen input device and Measuring method of coordinate |
US6957089B2 (en) | 2001-05-31 | 2005-10-18 | Coby Electronics Corporation | Compact hands-free adapter for use with a cellular telephone |
US7452098B2 (en) | 2001-06-15 | 2008-11-18 | Apple Inc. | Active enclosure for computing device |
US7088234B2 (en) | 2001-11-27 | 2006-08-08 | Matsushita Electric Industrial Co., Ltd. | Wearing information notifying unit |
US6959990B2 (en) | 2001-12-31 | 2005-11-01 | Texas Instruments Incorporated | Prism for high contrast projection |
IL148804A (en) | 2002-03-21 | 2007-02-11 | Yaacov Amitai | Optical device |
JP2003279881A (en) | 2002-03-27 | 2003-10-02 | Hitachi Ltd | Portable information device |
USD470144S1 (en) | 2002-04-18 | 2003-02-11 | Huixia Li | Computer mouse |
US6870303B2 (en) | 2002-05-08 | 2005-03-22 | Pohang University Of Science And Technology Foundation | Multi-mode vibration damping device and method using negative capacitance shunt circuits |
JP2003337963A (en) | 2002-05-17 | 2003-11-28 | Seiko Epson Corp | Device and method for image processing, and image processing program and recording medium therefor |
USD473871S1 (en) | 2002-07-08 | 2003-04-29 | Mar Santos | Desktop/hand-held trackball |
USD478052S1 (en) | 2002-07-12 | 2003-08-05 | Hunter Fan Company | Ceiling fan remote control |
US6896655B2 (en) | 2002-08-05 | 2005-05-24 | Eastman Kodak Company | System and method for conditioning the psychological state of a subject using an adaptive autostereoscopic display |
US20040032392A1 (en) | 2002-08-19 | 2004-02-19 | Topseed Technology Corp. | Mouse pen device having remote-control function |
US6906836B2 (en) | 2002-10-04 | 2005-06-14 | William Parker | Full color holographic image combiner system |
US20040130522A1 (en) | 2003-01-08 | 2004-07-08 | Wen-Po Lin | System and method for presenting real handwriting trace |
US7685538B2 (en) | 2003-01-31 | 2010-03-23 | Wacom Co., Ltd. | Method of triggering functions in a computer application using a digitizer having a stylus and a digitizer system |
US8106911B2 (en) | 2003-03-13 | 2012-01-31 | Sony Corporation | Mobile motion capture cameras |
US7333113B2 (en) | 2003-03-13 | 2008-02-19 | Sony Corporation | Mobile motion capture cameras |
JP2005138755A (en) | 2003-11-07 | 2005-06-02 | Denso Corp | Device and program for displaying virtual images |
EP2148504B1 (en) | 2003-12-03 | 2012-01-25 | Nikon Corporation | Information Display Device |
US20050156915A1 (en) | 2004-01-16 | 2005-07-21 | Fisher Edward N. | Handwritten character recording and recognition device |
USD513233S1 (en) | 2004-02-28 | 2005-12-27 | Hunter Fan Company | Ceiling fan remote |
USD514525S1 (en) | 2004-02-28 | 2006-02-07 | Hunter Fan Company | Ceiling fan wall controller |
USD512027S1 (en) | 2004-03-15 | 2005-11-29 | Nokia Corporation | Remote control |
JP2005274656A (en) | 2004-03-23 | 2005-10-06 | Fuji Photo Film Co Ltd | Display device and display method |
EP1755441B1 (en) | 2004-04-01 | 2015-11-04 | Eyefluence, Inc. | Biosensors, communicators, and controllers monitoring eye movement and methods for using them |
US9460346B2 (en) | 2004-04-19 | 2016-10-04 | Google Inc. | Handheld device for capturing text from both a document printed on paper and a document displayed on a dynamic display device |
JP4373286B2 (en) | 2004-05-06 | 2009-11-25 | オリンパス株式会社 | Head-mounted display device |
WO2005116939A1 (en) | 2004-05-27 | 2005-12-08 | Canon Kabushiki Kaisha | Information processing method, information processing apparatus, and image sensing apparatus |
IL162572A (en) | 2004-06-17 | 2013-02-28 | Lumus Ltd | High brightness optical device |
US6987787B1 (en) | 2004-06-28 | 2006-01-17 | Rockwell Collins | LED brightness control system for a wide-range of luminance control |
US7307793B2 (en) | 2004-07-02 | 2007-12-11 | Insight Technology, Inc. | Fusion night vision system |
US7295904B2 (en) | 2004-08-31 | 2007-11-13 | International Business Machines Corporation | Touch gesture based interface for motor vehicle |
US7545571B2 (en) | 2004-09-08 | 2009-06-09 | Concurrent Technologies Corporation | Wearable display system |
JP4560368B2 (en) | 2004-10-08 | 2010-10-13 | キヤノン株式会社 | Eye detection device and image display device |
JP4533087B2 (en) | 2004-10-28 | 2010-08-25 | キヤノン株式会社 | Image processing method and image processing apparatus |
US7350919B2 (en) | 2004-12-03 | 2008-04-01 | Searete Llc | Vision modification with reflected image |
US7053866B1 (en) | 2004-12-18 | 2006-05-30 | Emile Mimran | Portable adaptor and software for use with a heads-up display unit |
US7619616B2 (en) | 2004-12-21 | 2009-11-17 | Microsoft Corporation | Pressure sensitive controls |
USD541226S1 (en) | 2004-12-21 | 2007-04-24 | Kabushiki Kaisha Toshiba | Controller for forceps for medical robot |
USD521493S1 (en) | 2005-01-21 | 2006-05-23 | Koninklikjke Philips Electronics, N.V. | Gaming headphone |
EP1686554A3 (en) | 2005-01-31 | 2008-06-18 | Canon Kabushiki Kaisha | Virtual space generating system, image processing apparatus and information processing method |
US7457434B2 (en) | 2005-04-04 | 2008-11-25 | Massachusetts Eye & Ear Infirmary | Adaptively focusing extra-ocular vision prostheses |
US20060288233A1 (en) | 2005-04-25 | 2006-12-21 | Douglas Kozlay | Attachable biometric authentication apparatus for watchbands and other personal items |
US20060250322A1 (en) | 2005-05-09 | 2006-11-09 | Optics 1, Inc. | Dynamic vergence and focus control for head-mounted displays |
USD529467S1 (en) | 2005-06-01 | 2006-10-03 | Research In Motion Limited | Handset |
US20090183929A1 (en) | 2005-06-08 | 2009-07-23 | Guanglie Zhang | Writing system with camera |
US20070003168A1 (en) | 2005-06-29 | 2007-01-04 | Microsoft Corporation | Computer input device |
US20070004451A1 (en) | 2005-06-30 | 2007-01-04 | C Anderson Eric | Controlling functions of a handheld multifunction device |
US20070025273A1 (en) | 2005-07-29 | 2007-02-01 | Chung Yau W | Methods and systems for detecting video signals and sources |
US20070024823A1 (en) | 2005-07-29 | 2007-02-01 | Optoma Technology, Inc. | Methods and systems for improving operation of a video projector |
US20070024764A1 (en) | 2005-07-29 | 2007-02-01 | Optoma Technology, Inc. | Methods and systems that compensate for distortion introduced by anamorphic lenses in a video projector |
US7529029B2 (en) | 2005-07-29 | 2009-05-05 | 3M Innovative Properties Company | Polarizing beam splitter |
US8089567B2 (en) | 2005-07-29 | 2012-01-03 | Optoma Technology, Inc. | Methods and systems for displaying video on an adjustable screen |
US7701518B2 (en) | 2005-07-29 | 2010-04-20 | Optoma Technology, Inc. | Methods and systems for displaying video in multiple aspect ratios |
US7434937B2 (en) | 2005-07-29 | 2008-10-14 | Optoma Technology, Inc. | Methods and systems for calibrating rear projection video |
US20070035563A1 (en) | 2005-08-12 | 2007-02-15 | The Board Of Trustees Of Michigan State University | Augmented reality spatial interaction and navigational system |
JP2007079943A (en) | 2005-09-14 | 2007-03-29 | Toshiba Corp | Character reading program, character reading method and character reader |
US7707035B2 (en) | 2005-10-13 | 2010-04-27 | Integrated Wave Technologies, Inc. | Autonomous integrated headset and sound processing system for tactical applications |
US8018579B1 (en) | 2005-10-21 | 2011-09-13 | Apple Inc. | Three-dimensional imaging and display system |
US7543943B1 (en) | 2005-10-28 | 2009-06-09 | Hewlett-Packard Development Company, L.P. | Color permuting light projector |
US7777819B2 (en) | 2005-11-10 | 2010-08-17 | Bae Systems Plc | Display source |
JP4341661B2 (en) | 2005-11-22 | 2009-10-07 | ソニー株式会社 | Input device, input method, and input device manufacturing method |
US7522344B1 (en) | 2005-12-14 | 2009-04-21 | University Of Central Florida Research Foundation, Inc. | Projection-based head-mounted display with eye-tracking capabilities |
US8092007B2 (en) | 2006-01-13 | 2012-01-10 | Switch Vision, Llc | Eyewear frames with magnetic lens attachments |
CA2636895C (en) | 2006-01-13 | 2014-03-11 | Liberty Sport, Inc. | Eyewear frames with magnetic lens attachments |
US20070178950A1 (en) | 2006-01-19 | 2007-08-02 | International Business Machines Corporation | Wearable multimodal computing device with hands-free push to talk |
USD631881S1 (en) | 2006-03-28 | 2011-02-01 | Quinn Bryan C | Computer mouse |
US7734414B2 (en) | 2006-04-04 | 2010-06-08 | Yariv Gershony | Device, system and method for displaying a cell phone control signal in front of a driver |
WO2007128034A1 (en) | 2006-05-01 | 2007-11-15 | Eye Diagnostics Pty Ltd | Portable eye monitoring device and methods for using the same |
TWI292052B (en) | 2006-05-09 | 2008-01-01 | Young Optics Inc | Optical projection and image detection apparatus |
US20080121441A1 (en) | 2006-05-12 | 2008-05-29 | Velosum, Inc. | Systems and methods for mutually exclusive options on a paper form for use with a digital pen |
SE0601216L (en) | 2006-05-31 | 2007-12-01 | Abb Technology Ltd | Virtual workplace |
USD571816S1 (en) | 2006-06-19 | 2008-06-24 | Logitech Europe S.A. | Computer mouse topcase |
US7928926B2 (en) | 2006-06-27 | 2011-04-19 | Panasonic Corporation | Display apparatus and method for hands free operation that selects a function when window is within field of view |
JP3125129U (en) | 2006-06-28 | 2006-09-07 | 敏貴 並木 | mouse |
US7542210B2 (en) | 2006-06-29 | 2009-06-02 | Chirieleison Sr Anthony | Eye tracking head mounted display |
US7855743B2 (en) | 2006-09-08 | 2010-12-21 | Sony Corporation | Image capturing and displaying apparatus and image capturing and displaying method |
JP4375377B2 (en) | 2006-09-19 | 2009-12-02 | 富士ゼロックス株式会社 | WRITING INFORMATION PROCESSING SYSTEM, WRITING INFORMATION GENERATION DEVICE, AND PROGRAM |
JP5017989B2 (en) | 2006-09-27 | 2012-09-05 | ソニー株式会社 | Imaging apparatus and imaging method |
US8212859B2 (en) | 2006-10-13 | 2012-07-03 | Apple Inc. | Peripheral treatment for head-mounted displays |
USD559793S1 (en) | 2006-10-25 | 2008-01-15 | Hannspree, Inc. | Remote control |
SG142292A1 (en) | 2006-11-07 | 2008-05-28 | Agency Science Tech & Res | Device and method to realize a light processor |
JP2010512693A (en) | 2006-12-07 | 2010-04-22 | アダックス,インク. | System and method for data addition, recording and communication |
DE112008000168T5 (en) | 2007-01-12 | 2009-12-03 | Kopin Corporation, Taunton | Head mounted monocular display |
US20080191965A1 (en) | 2007-02-09 | 2008-08-14 | Raffaele Martini Pandozy | Apparatus and method for eye exercises |
US20080219025A1 (en) | 2007-03-07 | 2008-09-11 | Spitzer Mark B | Bi-directional backlight assembly |
US8515728B2 (en) | 2007-03-29 | 2013-08-20 | Microsoft Corporation | Language translation of visual and audio input |
JP5009361B2 (en) | 2007-03-29 | 2012-08-22 | 京セラ株式会社 | Portable radio |
US7777690B2 (en) | 2007-03-30 | 2010-08-17 | Itt Manufacturing Enterprises, Inc. | Radio frequency lens and method of suppressing side-lobes |
US8832557B2 (en) | 2007-05-04 | 2014-09-09 | Apple Inc. | Adjusting media display in a personal display system based on perspective |
US7934291B2 (en) | 2007-06-07 | 2011-05-03 | Apple Inc. | Multi-position magnetic detents |
EP3435373B1 (en) | 2007-07-31 | 2024-04-24 | Kopin Corporation | Mobile wireless display providing speech to speech translation and avatar simulating human attributes |
US20090040296A1 (en) | 2007-08-06 | 2009-02-12 | Moscato Jonathan D | Head mounted display assembly |
US7954047B2 (en) | 2007-08-06 | 2011-05-31 | Apple Inc. | Cutting and copying discontiguous selections of cells |
US7904485B2 (en) | 2007-09-06 | 2011-03-08 | Apple Inc. | Graphical representation of assets stored on a portable media device |
US7777960B2 (en) | 2007-09-10 | 2010-08-17 | Microvision, Inc. | Wide field of view head-up display system |
US8286734B2 (en) | 2007-10-23 | 2012-10-16 | Weatherford/Lamb, Inc. | Low profile rotating control device |
JP4956375B2 (en) | 2007-10-30 | 2012-06-20 | キヤノン株式会社 | Image processing apparatus and image processing method |
US7800360B2 (en) | 2007-10-31 | 2010-09-21 | Sony Ericsson Mobile Communications Ab | Connector system with magnetic audio volume control and method |
CN101589329B (en) | 2007-11-21 | 2011-10-12 | 松下电器产业株式会社 | Display |
US9158116B1 (en) | 2014-04-25 | 2015-10-13 | Osterhout Group, Inc. | Temple and ear horn assembly for headworn computer |
FR2926373B1 (en) | 2008-01-11 | 2010-07-30 | Essilor Int | TRANSPARENT COMPONENT WITH SWITCHABLE REFLECTING ELEMENTS, AND DEVICES COMPRISING SUCH A COMPONENT |
US8166421B2 (en) | 2008-01-14 | 2012-04-24 | Primesense Ltd. | Three-dimensional user interface |
JP2009171505A (en) | 2008-01-21 | 2009-07-30 | Nikon Corp | Head-mounted display |
US8384997B2 (en) | 2008-01-21 | 2013-02-26 | Primesense Ltd | Optical pattern projection |
US8786675B2 (en) | 2008-01-23 | 2014-07-22 | Michael F. Deering | Systems using eye mounted displays |
US20090251441A1 (en) | 2008-04-03 | 2009-10-08 | Livescribe, Inc. | Multi-Modal Controller |
US20100149073A1 (en) | 2008-11-02 | 2010-06-17 | David Chaum | Near to Eye Display System and Appliance |
WO2009133698A1 (en) | 2008-04-30 | 2009-11-05 | パナソニック株式会社 | Scanning image display device, eyeglasses-style head-mount display, and automobile |
US8423288B2 (en) | 2009-11-30 | 2013-04-16 | Apple Inc. | Dynamic alerts for calendar events |
US7926951B2 (en) | 2008-07-11 | 2011-04-19 | Eastman Kodak Company | Laser illuminated micro-mirror projector |
USD680112S1 (en) | 2008-07-25 | 2013-04-16 | Michael J. Monahan | Movement filtered mouse |
US7690799B2 (en) | 2008-08-26 | 2010-04-06 | Microvision, Inc. | Optical relay for compact head up display |
US7850306B2 (en) | 2008-08-28 | 2010-12-14 | Nokia Corporation | Visual cognition aware display and visual data transmission architecture |
US20100060713A1 (en) | 2008-09-10 | 2010-03-11 | Eastman Kodak Company | System and Method for Enhancing Noverbal Aspects of Communication |
US20100079508A1 (en) | 2008-09-30 | 2010-04-01 | Andrew Hodge | Electronic devices with gaze detection capabilities |
US8482545B2 (en) | 2008-10-02 | 2013-07-09 | Wacom Co., Ltd. | Combination touch and transducer input system and method |
US8585609B2 (en) | 2008-10-09 | 2013-11-19 | Neuro Kinetics, Inc. | Quantitative, non-invasive, clinical diagnosis of traumatic brain injury using simulated distance visual stimulus device for neurologic testing |
US9480919B2 (en) | 2008-10-24 | 2016-11-01 | Excalibur Ip, Llc | Reconfiguring reality using a reality overlay device |
WO2010062481A1 (en) | 2008-11-02 | 2010-06-03 | David Chaum | Near to eye display system and appliance |
US9715112B2 (en) | 2014-01-21 | 2017-07-25 | Osterhout Group, Inc. | Suppression of stray light in head worn computing |
US9400390B2 (en) | 2014-01-24 | 2016-07-26 | Osterhout Group, Inc. | Peripheral lighting for head worn computing |
US9229233B2 (en) | 2014-02-11 | 2016-01-05 | Osterhout Group, Inc. | Micro Doppler presentations in head worn computing |
US9952664B2 (en) | 2014-01-21 | 2018-04-24 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9298007B2 (en) | 2014-01-21 | 2016-03-29 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9366867B2 (en) | 2014-07-08 | 2016-06-14 | Osterhout Group, Inc. | Optical systems for see-through displays |
US20150277120A1 (en) | 2014-01-21 | 2015-10-01 | Osterhout Group, Inc. | Optical configurations for head worn computing |
US20150205111A1 (en) | 2014-01-21 | 2015-07-23 | Osterhout Group, Inc. | Optical configurations for head worn computing |
US9965681B2 (en) | 2008-12-16 | 2018-05-08 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US8594467B2 (en) | 2008-12-19 | 2013-11-26 | Microsoft Corporation | Interactive virtual display system for ubiquitous devices |
GB2466497B (en) | 2008-12-24 | 2011-09-14 | Light Blue Optics Ltd | Touch sensitive holographic displays |
CN101774179B (en) | 2009-01-10 | 2012-09-19 | 鸿富锦精密工业(深圳)有限公司 | Robot connecting shaft |
US8482520B2 (en) | 2009-01-30 | 2013-07-09 | Research In Motion Limited | Method for tap detection and for interacting with and a handheld electronic device, and a handheld electronic device configured therefor |
US8494215B2 (en) | 2009-03-05 | 2013-07-23 | Microsoft Corporation | Augmenting a field of view in connection with vision-tracking |
US20100240988A1 (en) | 2009-03-19 | 2010-09-23 | Kenneth Varga | Computer-aided system for 360 degree heads up display of safety/mission critical data |
US8159751B2 (en) | 2009-04-05 | 2012-04-17 | Miguel Marques Martins | Apparatus for head mounted image display |
US8570656B1 (en) | 2009-04-06 | 2013-10-29 | Paul Weissman | See-through optical system |
KR20110004027A (en) | 2009-07-07 | 2011-01-13 | 삼성전자주식회사 | Apparatus of pen-type inputting device and inputting method thereof |
US9728006B2 (en) | 2009-07-20 | 2017-08-08 | Real Time Companies, LLC | Computer-aided system for 360° heads up display of safety/mission critical data |
US20130009907A1 (en) | 2009-07-31 | 2013-01-10 | Rosenberg Ilya D | Magnetic Stylus |
DE102009049849B4 (en) | 2009-10-19 | 2020-09-24 | Apple Inc. | Method for determining the pose of a camera, method for recognizing an object in a real environment and method for creating a data model |
US20110102234A1 (en) | 2009-11-03 | 2011-05-05 | Vawd Applied Science And Technology Corporation | Standoff range sense through obstruction radar system |
EP2502410B1 (en) | 2009-11-19 | 2019-05-01 | eSight Corporation | A method for augmenting sight |
WO2011062591A1 (en) | 2009-11-21 | 2011-05-26 | Douglas Peter Magyari | Head mounted display device |
KR20120128609A (en) | 2009-12-18 | 2012-11-27 | 티피 비전 홀딩 비.브이. | Ambience lighting system using global content characteristics |
JP5146845B2 (en) | 2009-12-24 | 2013-02-20 | ブラザー工業株式会社 | Head mounted display |
US8905547B2 (en) | 2010-01-04 | 2014-12-09 | Elbit Systems Of America, Llc | System and method for efficiently delivering rays from a light source to create an image |
US8400548B2 (en) | 2010-01-05 | 2013-03-19 | Apple Inc. | Synchronized, interactive augmented reality displays for multifunction devices |
US8890771B2 (en) | 2010-01-06 | 2014-11-18 | Apple Inc. | Transparent electronic device |
US8922530B2 (en) | 2010-01-06 | 2014-12-30 | Apple Inc. | Communicating stylus |
USD631882S1 (en) | 2010-01-31 | 2011-02-01 | Swiftpoint Limited | Computer interface device |
US8463543B2 (en) | 2010-02-05 | 2013-06-11 | Apple Inc. | Schematic maps |
US8489326B1 (en) | 2010-02-09 | 2013-07-16 | Google Inc. | Placemarked based navigation and ad auction based on placemarks |
US8353729B2 (en) | 2010-02-18 | 2013-01-15 | Apple Inc. | Low profile connector system |
US20120120103A1 (en) | 2010-02-28 | 2012-05-17 | Osterhout Group, Inc. | Alignment control in an augmented reality headpiece |
WO2011106798A1 (en) | 2010-02-28 | 2011-09-01 | Osterhout Group, Inc. | Local advertising content on an interactive head-mounted eyepiece |
US20120194553A1 (en) | 2010-02-28 | 2012-08-02 | Osterhout Group, Inc. | Ar glasses with sensor and user action based control of external devices with feedback |
US20120194550A1 (en) | 2010-02-28 | 2012-08-02 | Osterhout Group, Inc. | Sensor-based command and control of external devices with feedback from the external device to the ar glasses |
US20110213664A1 (en) | 2010-02-28 | 2011-09-01 | Osterhout Group, Inc. | Local advertising content on an interactive head-mounted eyepiece |
US20120212499A1 (en) | 2010-02-28 | 2012-08-23 | Osterhout Group, Inc. | System and method for display content control during glasses movement |
US9091851B2 (en) | 2010-02-28 | 2015-07-28 | Microsoft Technology Licensing, Llc | Light control in head mounted displays |
US8964298B2 (en) | 2010-02-28 | 2015-02-24 | Microsoft Corporation | Video display modification based on sensor input for a see-through near-to-eye display |
US9128281B2 (en) | 2010-09-14 | 2015-09-08 | Microsoft Technology Licensing, Llc | Eyepiece with uniformly illuminated reflective display |
US8467133B2 (en) | 2010-02-28 | 2013-06-18 | Osterhout Group, Inc. | See-through display with an optical assembly including a wedge-shaped illumination system |
US9129295B2 (en) | 2010-02-28 | 2015-09-08 | Microsoft Technology Licensing, Llc | See-through near-eye display glasses with a fast response photochromic film system for quick transition from dark to clear |
US20140063054A1 (en) | 2010-02-28 | 2014-03-06 | Osterhout Group, Inc. | Ar glasses specific control interface based on a connected external device type |
KR20110101944A (en) | 2010-03-10 | 2011-09-16 | 삼성전자주식회사 | 3-dimension glasses, method for driving 3-dimension glass and system for providing 3d image |
US20110234631A1 (en) | 2010-03-25 | 2011-09-29 | Bizmodeline Co., Ltd. | Augmented reality systems |
US8690750B2 (en) | 2010-05-14 | 2014-04-08 | Wesley W. O. Krueger | System and method for measuring and minimizing the effects of vertigo, motion sickness, motion intolerance, and/or spatial disorientation |
US8570273B1 (en) | 2010-05-20 | 2013-10-29 | Lockheed Martin Corporation | Input device configured to control a computing device |
US8791900B2 (en) | 2010-05-21 | 2014-07-29 | Microsoft Corporation | Computing device notes |
US8594425B2 (en) | 2010-05-31 | 2013-11-26 | Primesense Ltd. | Analysis of three-dimensional scenes |
US9046999B1 (en) | 2010-06-08 | 2015-06-02 | Google Inc. | Dynamic input at a touch-based interface based on pressure |
US8531394B2 (en) | 2010-07-23 | 2013-09-10 | Gregory A. Maltz | Unitized, vision-controlled, wireless eyeglasses transceiver |
US9760123B2 (en) | 2010-08-06 | 2017-09-12 | Dynavox Systems Llc | Speech generation device with a projected display and optical inputs |
US8957948B2 (en) | 2010-08-24 | 2015-02-17 | Siemens Corporation | Geometric calibration of head-worn multi-camera eye tracking system |
US20120050140A1 (en) | 2010-08-25 | 2012-03-01 | Border John N | Head-mounted display control |
JP5459150B2 (en) | 2010-09-03 | 2014-04-02 | セイコーエプソン株式会社 | Light guide plate and virtual image display device including the same |
US8619005B2 (en) | 2010-09-09 | 2013-12-31 | Eastman Kodak Company | Switchable head-mounted display transition |
US8649099B2 (en) | 2010-09-13 | 2014-02-11 | Vuzix Corporation | Prismatic multiple waveguide for near-eye display |
US8773464B2 (en) | 2010-09-15 | 2014-07-08 | Sharp Laboratories Of America, Inc. | Methods and systems for collaborative-writing-surface image formation |
JP5974008B2 (en) | 2010-09-20 | 2016-08-23 | コピン コーポレーション | Wireless interface such as Bluetooth (registered trademark) with power management function for head mounted display |
US8941559B2 (en) | 2010-09-21 | 2015-01-27 | Microsoft Corporation | Opacity filter for display device |
US8376548B2 (en) | 2010-09-22 | 2013-02-19 | Vuzix Corporation | Near-eye display with on-axis symmetry |
US20120078628A1 (en) | 2010-09-28 | 2012-03-29 | Ghulman Mahmoud M | Head-mounted text display system and method for the hearing impaired |
WO2012042974A1 (en) | 2010-09-30 | 2012-04-05 | 富士フイルム株式会社 | Information presentation device, digital camera, head mount display, projector, information presentation method, and information presentation program |
WO2012054231A2 (en) | 2010-10-04 | 2012-04-26 | Gerard Dirk Smits | System and method for 3-d projection and enhancements for interactivity |
US8837880B2 (en) | 2010-10-08 | 2014-09-16 | Seiko Epson Corporation | Virtual image display device |
US20120092329A1 (en) | 2010-10-13 | 2012-04-19 | Qualcomm Incorporated | Text-based 3d augmented reality |
US8884984B2 (en) | 2010-10-15 | 2014-11-11 | Microsoft Corporation | Fusing virtual content into real content |
US20140043682A1 (en) | 2010-10-21 | 2014-02-13 | Patrick Hussey | Flip Up Interchangeable System |
US8692845B2 (en) | 2010-10-28 | 2014-04-08 | Eastman Kodak Company | Head-mounted display control with image-content analysis |
US20120113514A1 (en) | 2010-11-08 | 2012-05-10 | Polycom, Inc. | Picoprojector with Image Stabilization [Image-Stabilized Projector] |
US9292973B2 (en) | 2010-11-08 | 2016-03-22 | Microsoft Technology Licensing, Llc | Automatic variable virtual focus for augmented reality displays |
US8576276B2 (en) | 2010-11-18 | 2013-11-05 | Microsoft Corporation | Head-mounted display device which provides surround video |
US20130154913A1 (en) | 2010-12-16 | 2013-06-20 | Siemens Corporation | Systems and methods for a gaze and gesture interface |
US9690099B2 (en) | 2010-12-17 | 2017-06-27 | Microsoft Technology Licensing, Llc | Optimized focal area for augmented reality displays |
WO2012088454A1 (en) | 2010-12-22 | 2012-06-28 | Energy Focus, Inc. | An elongated led lighting arrangement |
US8531773B2 (en) | 2011-01-10 | 2013-09-10 | Microvision, Inc. | Substrate guided relay having a homogenizing layer |
US20120188245A1 (en) | 2011-01-20 | 2012-07-26 | Apple Inc. | Display resolution increase with mechanical actuation |
US8787006B2 (en) | 2011-01-31 | 2014-07-22 | Apple Inc. | Wrist-worn electronic device and methods therefor |
JP5760465B2 (en) | 2011-02-04 | 2015-08-12 | セイコーエプソン株式会社 | Virtual image display device |
JP5633406B2 (en) | 2011-02-04 | 2014-12-03 | セイコーエプソン株式会社 | Virtual image display device |
JP2012163656A (en) | 2011-02-04 | 2012-08-30 | Seiko Epson Corp | Virtual image display device |
JP5742263B2 (en) | 2011-02-04 | 2015-07-01 | セイコーエプソン株式会社 | Virtual image display device |
WO2012107892A2 (en) | 2011-02-09 | 2012-08-16 | Primesense Ltd. | Gaze detection in a 3d mapping environment |
US20120212593A1 (en) | 2011-02-17 | 2012-08-23 | Orcam Technologies Ltd. | User wearable visual assistance system |
DE112012001022T5 (en) | 2011-02-28 | 2013-12-19 | Osterhout Group, Inc. | Alignment control in a head-worn augmented reality device |
US20120223885A1 (en) | 2011-03-02 | 2012-09-06 | Microsoft Corporation | Immersive display experience |
US9020832B2 (en) | 2011-03-07 | 2015-04-28 | KBA2 Inc. | Systems and methods for analytic data gathering from image providers at an event or geographic location |
US8670183B2 (en) | 2011-03-07 | 2014-03-11 | Microsoft Corporation | Augmented view of advertisements |
EP2499960B1 (en) | 2011-03-18 | 2015-04-22 | SensoMotoric Instruments Gesellschaft für innovative Sensorik mbH | Method for determining at least one parameter of two eyes by setting data rates and optical measuring device |
US9033502B2 (en) | 2011-03-18 | 2015-05-19 | Sensomotoric Instruments Gesellschaft Fur Innovative Sensorik Mbh | Optical measuring device and method for capturing at least one parameter of at least one eye wherein an illumination characteristic is adjustable |
US9895058B2 (en) | 2011-03-25 | 2018-02-20 | Carl Zeiss Meditec Ag | Heads-up vision analyzer |
JP5960796B2 (en) | 2011-03-29 | 2016-08-02 | クアルコム,インコーポレイテッド | Modular mobile connected pico projector for local multi-user collaboration |
JP2012212990A (en) | 2011-03-30 | 2012-11-01 | Brother Ind Ltd | Head-mounted display |
US8953242B2 (en) | 2011-03-31 | 2015-02-10 | Honeywell International Inc. | Varible focus stereoscopic display system and method |
US20120264510A1 (en) | 2011-04-12 | 2012-10-18 | Microsoft Corporation | Integrated virtual environment |
US8885877B2 (en) | 2011-05-20 | 2014-11-11 | Eyefluence, Inc. | Systems and methods for identifying gaze tracking scene reference locations |
US9330499B2 (en) | 2011-05-20 | 2016-05-03 | Microsoft Technology Licensing, Llc | Event augmentation with real-time information |
US20120306850A1 (en) | 2011-06-02 | 2012-12-06 | Microsoft Corporation | Distributed asynchronous localization and mapping for augmented reality |
US20120326948A1 (en) | 2011-06-22 | 2012-12-27 | Microsoft Corporation | Environmental-light filter for see-through head-mounted display device |
US20120327040A1 (en) | 2011-06-22 | 2012-12-27 | Simon David I | Identifiable stylus |
US20120327116A1 (en) | 2011-06-23 | 2012-12-27 | Microsoft Corporation | Total field of view classification for head-mounted display |
US8558759B1 (en) | 2011-07-08 | 2013-10-15 | Google Inc. | Hand gestures to signify what is important |
US8228315B1 (en) | 2011-07-12 | 2012-07-24 | Google Inc. | Methods and systems for a virtual input device |
US8593795B1 (en) | 2011-08-09 | 2013-11-26 | Google Inc. | Weight distribution for wearable computing device |
CA2750287C (en) | 2011-08-29 | 2012-07-03 | Microsoft Corporation | Gaze detection in a see-through, near-eye, mixed reality display |
JP6127359B2 (en) | 2011-09-15 | 2017-05-17 | セイコーエプソン株式会社 | Virtual image display device and method of manufacturing virtual image display device |
US8786686B1 (en) | 2011-09-16 | 2014-07-22 | Google Inc. | Head mounted display eyepiece with integrated depth sensing |
US8941560B2 (en) | 2011-09-21 | 2015-01-27 | Google Inc. | Wearable computer with superimposed controls and instructions for external device |
US8767306B1 (en) | 2011-09-22 | 2014-07-01 | Google Inc. | Display system |
CN103946732B (en) | 2011-09-26 | 2019-06-14 | 微软技术许可有限责任公司 | Video based on the sensor input to perspective, near-eye display shows modification |
US8998414B2 (en) * | 2011-09-26 | 2015-04-07 | Microsoft Technology Licensing, Llc | Integrated eye tracking and display system |
JP5834705B2 (en) | 2011-09-28 | 2015-12-24 | セイコーエプソン株式会社 | Electro-optical device and electronic apparatus |
JP5786601B2 (en) | 2011-09-28 | 2015-09-30 | セイコーエプソン株式会社 | Electro-optical device and electronic apparatus |
US9121724B2 (en) | 2011-09-30 | 2015-09-01 | Apple Inc. | 3D position tracking for panoramic imagery navigation |
JP2013080040A (en) | 2011-10-03 | 2013-05-02 | Seiko Epson Corp | Electrooptical device, method for manufacturing electrooptical device, and electronic equipment |
US20130088413A1 (en) | 2011-10-05 | 2013-04-11 | Google Inc. | Method to Autofocus on Near-Eye Display |
US20130088507A1 (en) | 2011-10-06 | 2013-04-11 | Nokia Corporation | Method and apparatus for controlling the visual representation of information upon a see-through display |
US9081177B2 (en) * | 2011-10-07 | 2015-07-14 | Google Inc. | Wearable computer with nearby object response |
US8813109B2 (en) | 2011-10-21 | 2014-08-19 | The Nielsen Company (Us), Llc | Methods and apparatus to identify exposure to 3D media presentations |
USD666237S1 (en) | 2011-10-24 | 2012-08-28 | Google Inc. | Wearable display device |
US8970452B2 (en) | 2011-11-02 | 2015-03-03 | Google Inc. | Imaging method |
US8752963B2 (en) | 2011-11-04 | 2014-06-17 | Microsoft Corporation | See-through display brightness control |
US9311883B2 (en) | 2011-11-11 | 2016-04-12 | Microsoft Technology Licensing, Llc | Recalibration of a flexible mixed reality device |
US8553910B1 (en) | 2011-11-17 | 2013-10-08 | Jianchun Dong | Wearable computing device with behind-ear bone-conduction speaker |
US20150143297A1 (en) * | 2011-11-22 | 2015-05-21 | Google Inc. | Input detection for a head mounted device |
US8611015B2 (en) | 2011-11-22 | 2013-12-17 | Google Inc. | User interface |
US8235529B1 (en) * | 2011-11-30 | 2012-08-07 | Google Inc. | Unlocking a screen using eye tracking information |
US8872853B2 (en) | 2011-12-01 | 2014-10-28 | Microsoft Corporation | Virtual light in augmented reality |
US8824779B1 (en) | 2011-12-20 | 2014-09-02 | Christopher Charles Smyth | Apparatus and method for determining eye gaze from stereo-optic views |
US8982471B1 (en) | 2012-01-04 | 2015-03-17 | Google Inc. | HMD image source as dual-purpose projector/near-eye display |
US20130176626A1 (en) | 2012-01-05 | 2013-07-11 | Google Inc. | Wearable device assembly with input and output structures |
US8878749B1 (en) | 2012-01-06 | 2014-11-04 | Google Inc. | Systems and methods for position estimation |
US8955973B2 (en) | 2012-01-06 | 2015-02-17 | Google Inc. | Method and system for input detection using structured light projection |
US8638989B2 (en) | 2012-01-17 | 2014-01-28 | Leap Motion, Inc. | Systems and methods for capturing motion in three-dimensional space |
US8971023B2 (en) | 2012-03-21 | 2015-03-03 | Google Inc. | Wearable computing device frame |
CN104204904B (en) | 2012-01-24 | 2018-05-18 | 亚利桑那大学评议会 | Close-coupled eyes track head-mounted display |
US9414746B2 (en) | 2012-01-26 | 2016-08-16 | Nokia Technologies Oy | Eye tracking |
US8894484B2 (en) | 2012-01-30 | 2014-11-25 | Microsoft Corporation | Multiplayer game invitation system |
US20130194389A1 (en) | 2012-01-31 | 2013-08-01 | Ben Vaught | Head-mounted display device to measure attentiveness |
US8854433B1 (en) | 2012-02-03 | 2014-10-07 | Aquifi, Inc. | Method and system enabling natural user interface gestures with an electronic system |
US9076368B2 (en) | 2012-02-06 | 2015-07-07 | Battelle Memorial Institute | Image generation systems and image generation methods |
US8982014B2 (en) | 2012-02-06 | 2015-03-17 | Battelle Memorial Institute | Image generation systems and image generation methods |
KR101709844B1 (en) | 2012-02-15 | 2017-02-23 | 애플 인크. | Apparatus and method for mapping |
US8745058B1 (en) | 2012-02-21 | 2014-06-03 | Google Inc. | Dynamic data item searching |
JP5970872B2 (en) | 2012-03-07 | 2016-08-17 | セイコーエプソン株式会社 | Head-mounted display device and method for controlling head-mounted display device |
US20130241805A1 (en) | 2012-03-15 | 2013-09-19 | Google Inc. | Using Convergence Angle to Select Among Different UI Elements |
JP5884576B2 (en) | 2012-03-16 | 2016-03-15 | セイコーエプソン株式会社 | Head-mounted display device and method for controlling head-mounted display device |
US8760765B2 (en) | 2012-03-19 | 2014-06-24 | Google Inc. | Optical beam tilt for offset head mounted display |
US8985803B2 (en) | 2012-03-21 | 2015-03-24 | Microsoft Technology Licensing, Llc | Freeform-prism eyepiece with illumination waveguide |
US9274338B2 (en) | 2012-03-21 | 2016-03-01 | Microsoft Technology Licensing, Llc | Increasing field of view of reflective waveguide |
JP5987387B2 (en) | 2012-03-22 | 2016-09-07 | ソニー株式会社 | Head mounted display and surgical system |
USD718305S1 (en) | 2012-03-22 | 2014-11-25 | Google Inc. | Wearable display device |
JP5938977B2 (en) | 2012-03-23 | 2016-06-22 | ソニー株式会社 | Head mounted display and surgical system |
US20130248691A1 (en) | 2012-03-23 | 2013-09-26 | Google Inc. | Methods and Systems for Sensing Ambient Light |
US9207468B2 (en) | 2012-03-30 | 2015-12-08 | Honeywell International Inc. | Personal protection equipment verification |
NZ725322A (en) | 2012-04-05 | 2017-12-22 | Magic Leap Inc | Wide-field of view (fov) imaging devices with active foveation capability |
US8937591B2 (en) | 2012-04-06 | 2015-01-20 | Apple Inc. | Systems and methods for counteracting a perceptual fading of a movable indicator |
US9417660B2 (en) | 2012-04-25 | 2016-08-16 | Kopin Corporation | Collapsible head set computer |
US20130293530A1 (en) | 2012-05-04 | 2013-11-07 | Kathryn Stone Perez | Product augmentation and advertising in see through displays |
US20130300634A1 (en) | 2012-05-09 | 2013-11-14 | Nokia Corporation | Method and apparatus for determining representations of displayed information based on focus distance |
CN104285438B (en) | 2012-05-21 | 2018-09-07 | 奥林巴斯株式会社 | Glasses type wearable device, the front part of glasses type wearable device and temple portion |
US9747306B2 (en) | 2012-05-25 | 2017-08-29 | Atheer, Inc. | Method and apparatus for identifying input features for later recognition |
USD690684S1 (en) | 2012-05-30 | 2013-10-01 | Samsung Electronics Co., Ltd. | Remote controller |
TWI474069B (en) | 2012-06-05 | 2015-02-21 | Largan Precision Co Ltd | Image capturing optical lens assembly |
JP2013257492A (en) | 2012-06-14 | 2013-12-26 | Sony Corp | Control device, display device, control method, illumination control method, and program |
US9219901B2 (en) * | 2012-06-19 | 2015-12-22 | Qualcomm Incorporated | Reactive user interface for head-mounted display |
US9874936B2 (en) | 2012-06-22 | 2018-01-23 | Cape Evolution Limited | Wearable electronic device |
US9696547B2 (en) | 2012-06-25 | 2017-07-04 | Microsoft Technology Licensing, Llc | Mixed reality system learned input and functions |
KR101861380B1 (en) | 2012-07-16 | 2018-05-28 | 마이크로소프트 테크놀로지 라이센싱, 엘엘씨 | A Method of Providing Contents Using Head Mounted Display and a Head Mounted Display Thereof |
CN103576315B (en) | 2012-07-30 | 2017-03-01 | 联想(北京)有限公司 | Display device |
USD689862S1 (en) | 2012-08-15 | 2013-09-17 | Kye Systems Corp. | Ring mouse |
KR101958778B1 (en) | 2012-08-31 | 2019-03-15 | 엘지전자 주식회사 | A Head Mounted Display and a Method for Controlling a Digital Device Using the Same |
ITTO20120756A1 (en) | 2012-08-31 | 2014-03-01 | St Microelectronics Srl | PICO-PROJECTOR DEVICE STABILIZED AND RELATIVE TO IMAGE STABILIZATION METHOD |
US8836768B1 (en) | 2012-09-04 | 2014-09-16 | Aquifi, Inc. | Method and system enabling natural user interface gestures with user wearable glasses |
JP6036065B2 (en) | 2012-09-14 | 2016-11-30 | 富士通株式会社 | Gaze position detection device and gaze position detection method |
US8482527B1 (en) | 2012-09-14 | 2013-07-09 | Lg Electronics Inc. | Apparatus and method of providing user interface on head mounted display and head mounted display thereof |
US9063563B1 (en) | 2012-09-25 | 2015-06-23 | Amazon Technologies, Inc. | Gesture actions for interface elements |
USD710928S1 (en) | 2012-09-25 | 2014-08-12 | Google Inc. | Wearable display device |
USD711456S1 (en) | 2012-09-25 | 2014-08-19 | Google Inc. | Wearable display device |
US10573037B2 (en) | 2012-12-20 | 2020-02-25 | Sri International | Method and apparatus for mentoring via an augmented reality assistant |
US10620902B2 (en) | 2012-09-28 | 2020-04-14 | Nokia Technologies Oy | Method and apparatus for providing an indication regarding content presented to another user |
US20140101608A1 (en) | 2012-10-05 | 2014-04-10 | Google Inc. | User Interfaces for Head-Mountable Devices |
US20140104692A1 (en) | 2012-10-11 | 2014-04-17 | Sony Computer Entertainment Europe Limited | Head mountable display |
US8750541B1 (en) | 2012-10-31 | 2014-06-10 | Google Inc. | Parametric array for a head-mountable device |
US20140129328A1 (en) | 2012-11-07 | 2014-05-08 | Microsoft Corporation | Providing augmented purchase schemes |
KR101385681B1 (en) | 2012-11-08 | 2014-04-15 | 삼성전자 주식회사 | Head-mount type display apparatus and control method thereof |
US8743052B1 (en) | 2012-11-24 | 2014-06-03 | Eric Jeffrey Keller | Computing interface system |
US20140146394A1 (en) | 2012-11-28 | 2014-05-29 | Nigel David Tout | Peripheral display for a near-eye display device |
US9189021B2 (en) | 2012-11-29 | 2015-11-17 | Microsoft Technology Licensing, Llc | Wearable food nutrition feedback system |
US20140152558A1 (en) | 2012-11-30 | 2014-06-05 | Tom Salter | Direct hologram manipulation using imu |
US20140152676A1 (en) | 2012-11-30 | 2014-06-05 | Dave Rohn | Low latency image display on multi-display device |
US8867139B2 (en) | 2012-11-30 | 2014-10-21 | Google Inc. | Dual axis internal optical beam tilt for eyepiece of an HMD |
US20140152530A1 (en) | 2012-12-03 | 2014-06-05 | Honeywell International Inc. | Multimedia near to eye display system |
US20140160170A1 (en) | 2012-12-06 | 2014-06-12 | Nokia Corporation | Provision of an Image Element on a Display Worn by a User |
US20140160157A1 (en) | 2012-12-11 | 2014-06-12 | Adam G. Poulos | People-triggered holographic reminders |
US20140160055A1 (en) | 2012-12-12 | 2014-06-12 | Jeffrey Margolis | Wearable multi-modal input device for augmented reality |
US9081210B2 (en) | 2012-12-12 | 2015-07-14 | Microsoft Technology Licensing, Llc | Head worn device having temple arms to provide long axis compression |
US9448407B2 (en) | 2012-12-13 | 2016-09-20 | Seiko Epson Corporation | Head-mounted display device, control method for head-mounted display device, and work supporting system |
US10146053B2 (en) | 2012-12-19 | 2018-12-04 | Microsoft Technology Licensing, Llc | Multiplexed hologram tiling in a waveguide display |
US20140176591A1 (en) | 2012-12-26 | 2014-06-26 | Georg Klein | Low-latency fusing of color image data |
US20140204759A1 (en) | 2013-01-21 | 2014-07-24 | Mitsubishi Electric Research Laboratories, Inc. | Load Balanced Routing for Low Power and Lossy Networks |
US20140225814A1 (en) | 2013-02-14 | 2014-08-14 | Apx Labs, Llc | Method and system for representing and interacting with geo-located markers |
US9223139B2 (en) | 2013-02-15 | 2015-12-29 | Google Inc. | Cascading optics in optical combiners of head mounted displays |
US10685487B2 (en) * | 2013-03-06 | 2020-06-16 | Qualcomm Incorporated | Disabling augmented reality (AR) devices at speed |
AU2014248874B2 (en) | 2013-03-11 | 2019-07-11 | Magic Leap, Inc. | System and method for augmented and virtual reality |
US9779517B2 (en) * | 2013-03-15 | 2017-10-03 | Upskill, Inc. | Method and system for representing and interacting with augmented reality content |
US9176582B1 (en) | 2013-04-10 | 2015-11-03 | Google Inc. | Input system |
US9443354B2 (en) | 2013-04-29 | 2016-09-13 | Microsoft Technology Licensing, Llc | Mixed reality interactions |
US9129157B2 (en) | 2013-04-30 | 2015-09-08 | Qualcomm Incorporated | Method for image-based status determination |
US20140363797A1 (en) | 2013-05-28 | 2014-12-11 | Lark Technologies, Inc. | Method for providing wellness-related directives to a user |
US9329682B2 (en) | 2013-06-18 | 2016-05-03 | Microsoft Technology Licensing, Llc | Multi-step virtual object selection |
US20140375542A1 (en) | 2013-06-25 | 2014-12-25 | Steve Robbins | Adjusting a near-eye display device |
US9129430B2 (en) | 2013-06-25 | 2015-09-08 | Microsoft Technology Licensing, Llc | Indicating out-of-view augmented reality images |
USD738373S1 (en) | 2013-08-09 | 2015-09-08 | Kopin Corporation | Eyewear viewing device |
US9158115B1 (en) | 2013-09-16 | 2015-10-13 | Amazon Technologies, Inc. | Touch control for immersion in a tablet goggles accessory |
USD728573S1 (en) | 2013-09-20 | 2015-05-05 | Jianbo Deng | Computer input device |
US20150097719A1 (en) | 2013-10-03 | 2015-04-09 | Sulon Technologies Inc. | System and method for active reference positioning in an augmented reality environment |
US9569669B2 (en) | 2013-11-27 | 2017-02-14 | International Business Machines Corporation | Centralized video surveillance data in head mounted device |
CN107300769B (en) | 2013-11-27 | 2019-12-13 | 奇跃公司 | Virtual and augmented reality systems and methods |
US20150161913A1 (en) | 2013-12-10 | 2015-06-11 | At&T Mobility Ii Llc | Method, computer-readable storage device and apparatus for providing a recommendation in a vehicle |
US20160085071A1 (en) | 2014-09-18 | 2016-03-24 | Osterhout Group, Inc. | See-through computer display systems |
US20160048019A1 (en) | 2014-08-12 | 2016-02-18 | Osterhout Group, Inc. | Content presentation in head worn computing |
US9594246B2 (en) | 2014-01-21 | 2017-03-14 | Osterhout Group, Inc. | See-through computer display systems |
US9939934B2 (en) | 2014-01-17 | 2018-04-10 | Osterhout Group, Inc. | External user interface for head worn computing |
US9529195B2 (en) | 2014-01-21 | 2016-12-27 | Osterhout Group, Inc. | See-through computer display systems |
US20150228119A1 (en) | 2014-02-11 | 2015-08-13 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US20150205351A1 (en) | 2014-01-17 | 2015-07-23 | Osterhout Group, Inc. | External user interface for head worn computing |
US10649220B2 (en) | 2014-06-09 | 2020-05-12 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US9299194B2 (en) | 2014-02-14 | 2016-03-29 | Osterhout Group, Inc. | Secure sharing in head worn computing |
US9366868B2 (en) | 2014-09-26 | 2016-06-14 | Osterhout Group, Inc. | See-through computer display systems |
US11103122B2 (en) | 2014-07-15 | 2021-08-31 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US10254856B2 (en) | 2014-01-17 | 2019-04-09 | Osterhout Group, Inc. | External user interface for head worn computing |
US11227294B2 (en) | 2014-04-03 | 2022-01-18 | Mentor Acquisition One, Llc | Sight information collection in head worn computing |
US20150294156A1 (en) | 2014-04-14 | 2015-10-15 | Osterhout Group, Inc. | Sight information collection in head worn computing |
US20150309562A1 (en) | 2014-04-25 | 2015-10-29 | Osterhout Group, Inc. | In-vehicle use in head worn computing |
US9671613B2 (en) | 2014-09-26 | 2017-06-06 | Osterhout Group, Inc. | See-through computer display systems |
WO2015109145A1 (en) | 2014-01-17 | 2015-07-23 | Osterhout Group, Inc. | See-through computer display systems |
US20160019715A1 (en) | 2014-07-15 | 2016-01-21 | Osterhout Group, Inc. | Content presentation in head worn computing |
US9575321B2 (en) | 2014-06-09 | 2017-02-21 | Osterhout Group, Inc. | Content presentation in head worn computing |
US9746686B2 (en) | 2014-05-19 | 2017-08-29 | Osterhout Group, Inc. | Content position calibration in head worn computing |
US10191279B2 (en) * | 2014-03-17 | 2019-01-29 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9841599B2 (en) | 2014-06-05 | 2017-12-12 | Osterhout Group, Inc. | Optical configurations for head-worn see-through displays |
US9810906B2 (en) | 2014-06-17 | 2017-11-07 | Osterhout Group, Inc. | External user interface for head worn computing |
US9829707B2 (en) | 2014-08-12 | 2017-11-28 | Osterhout Group, Inc. | Measuring content brightness in head worn computing |
US20150277118A1 (en) | 2014-03-28 | 2015-10-01 | Osterhout Group, Inc. | Sensor dependent content position in head worn computing |
US9523856B2 (en) | 2014-01-21 | 2016-12-20 | Osterhout Group, Inc. | See-through computer display systems |
US20150205135A1 (en) | 2014-01-21 | 2015-07-23 | Osterhout Group, Inc. | See-through computer display systems |
US9532714B2 (en) | 2014-01-21 | 2017-01-03 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9550419B2 (en) | 2014-01-21 | 2017-01-24 | Honda Motor Co., Ltd. | System and method for providing an augmented reality vehicle interface |
US9740280B2 (en) | 2014-01-21 | 2017-08-22 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9310610B2 (en) | 2014-01-21 | 2016-04-12 | Osterhout Group, Inc. | See-through computer display systems |
US9494800B2 (en) | 2014-01-21 | 2016-11-15 | Osterhout Group, Inc. | See-through computer display systems |
US20160018653A1 (en) | 2014-01-24 | 2016-01-21 | Osterhout Group, Inc. | See-through computer display systems |
US9524588B2 (en) | 2014-01-24 | 2016-12-20 | Avaya Inc. | Enhanced communication between remote participants using augmented and virtual reality |
US9852545B2 (en) | 2014-02-11 | 2017-12-26 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US9401540B2 (en) | 2014-02-11 | 2016-07-26 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US20150228120A1 (en) | 2014-02-11 | 2015-08-13 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US20150241963A1 (en) | 2014-02-11 | 2015-08-27 | Osterhout Group, Inc. | Eye imaging in head worn computing |
KR102292192B1 (en) | 2014-02-17 | 2021-08-23 | 엘지전자 주식회사 | The Apparatus and Method for Display System displaying Augmented Reality image |
US20150309534A1 (en) | 2014-04-25 | 2015-10-29 | Osterhout Group, Inc. | Ear horn assembly for headworn computer |
US9423842B2 (en) | 2014-09-18 | 2016-08-23 | Osterhout Group, Inc. | Thermal management for head-worn computer |
US9651787B2 (en) | 2014-04-25 | 2017-05-16 | Osterhout Group, Inc. | Speaker assembly for headworn computer |
US9672210B2 (en) | 2014-04-25 | 2017-06-06 | Osterhout Group, Inc. | Language translation with head-worn computing |
US20160025977A1 (en) | 2014-07-22 | 2016-01-28 | Osterhout Group, Inc. | External user interface for head worn computing |
WO2015179877A2 (en) | 2014-05-19 | 2015-11-26 | Osterhout Group, Inc. | External user interface for head worn computing |
US20160025979A1 (en) | 2014-08-28 | 2016-01-28 | Osterhout Group, Inc. | External user interface for head worn computing |
US20160062118A1 (en) | 2014-07-22 | 2016-03-03 | Osterhout Group, Inc. | External user interface for head worn computing |
US20160027414A1 (en) | 2014-07-22 | 2016-01-28 | Osterhout Group, Inc. | External user interface for head worn computing |
JP1511166S (en) | 2014-05-21 | 2014-11-10 | ||
US9143693B1 (en) | 2014-06-10 | 2015-09-22 | Google Inc. | Systems and methods for push-button slow motion |
CN106575151A (en) | 2014-06-17 | 2017-04-19 | 奥斯特豪特集团有限公司 | External user interface for head worn computing |
USD716813S1 (en) | 2014-07-28 | 2014-11-04 | Jianbo Deng | Computer mouse |
USD743963S1 (en) | 2014-12-22 | 2015-11-24 | Osterhout Group, Inc. | Air mouse |
USD751552S1 (en) | 2014-12-31 | 2016-03-15 | Osterhout Group, Inc. | Computer glasses |
-
2014
- 2014-03-28 US US14/228,526 patent/US20150277118A1/en not_active Abandoned
- 2014-05-29 US US14/290,023 patent/US20150279104A1/en not_active Abandoned
- 2014-10-15 US US14/515,315 patent/US20150277122A1/en not_active Abandoned
- 2014-10-16 US US14/516,009 patent/US20150277549A1/en not_active Abandoned
- 2014-10-16 US US14/516,013 patent/US20150277113A1/en not_active Abandoned
- 2014-10-16 US US14/516,017 patent/US20150279107A1/en not_active Abandoned
- 2014-11-19 US US14/547,134 patent/US9423612B2/en active Active
Patent Citations (7)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20110170061A1 (en) * | 2010-01-08 | 2011-07-14 | Gordon Gary B | Gaze Point Tracking Using Polarized Light |
US20120212398A1 (en) * | 2010-02-28 | 2012-08-23 | Osterhout Group, Inc. | See-through near-eye display glasses including a partially reflective, partially transmitting optical element |
US20130083009A1 (en) * | 2011-09-30 | 2013-04-04 | Kevin A. Geisner | Exercising applications for personal audio/visual system |
US20130154918A1 (en) * | 2011-12-20 | 2013-06-20 | Benjamin Isaac Vaught | Enhanced user eye gaze estimation |
US20140247286A1 (en) * | 2012-02-20 | 2014-09-04 | Google Inc. | Active Stabilization for Heads-Up Displays |
US8947323B1 (en) * | 2012-03-20 | 2015-02-03 | Hayes Solos Raffle | Content display methods |
US20150205494A1 (en) * | 2014-01-23 | 2015-07-23 | Jason Scott | Gaze swipe selection |
Cited By (141)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9965681B2 (en) | 2008-12-16 | 2018-05-08 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9939934B2 (en) | 2014-01-17 | 2018-04-10 | Osterhout Group, Inc. | External user interface for head worn computing |
US11782529B2 (en) | 2014-01-17 | 2023-10-10 | Mentor Acquisition One, Llc | External user interface for head worn computing |
US11507208B2 (en) | 2014-01-17 | 2022-11-22 | Mentor Acquisition One, Llc | External user interface for head worn computing |
US11231817B2 (en) | 2014-01-17 | 2022-01-25 | Mentor Acquisition One, Llc | External user interface for head worn computing |
US11169623B2 (en) | 2014-01-17 | 2021-11-09 | Mentor Acquisition One, Llc | External user interface for head worn computing |
US10254856B2 (en) | 2014-01-17 | 2019-04-09 | Osterhout Group, Inc. | External user interface for head worn computing |
US10139632B2 (en) | 2014-01-21 | 2018-11-27 | Osterhout Group, Inc. | See-through computer display systems |
US9811152B2 (en) | 2014-01-21 | 2017-11-07 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9529199B2 (en) | 2014-01-21 | 2016-12-27 | Osterhout Group, Inc. | See-through computer display systems |
US9529192B2 (en) | 2014-01-21 | 2016-12-27 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9529195B2 (en) | 2014-01-21 | 2016-12-27 | Osterhout Group, Inc. | See-through computer display systems |
US9532715B2 (en) | 2014-01-21 | 2017-01-03 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9532714B2 (en) | 2014-01-21 | 2017-01-03 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9538915B2 (en) | 2014-01-21 | 2017-01-10 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US11099380B2 (en) | 2014-01-21 | 2021-08-24 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US11947126B2 (en) | 2014-01-21 | 2024-04-02 | Mentor Acquisition One, Llc | See-through computer display systems |
US9594246B2 (en) | 2014-01-21 | 2017-03-14 | Osterhout Group, Inc. | See-through computer display systems |
US9615742B2 (en) | 2014-01-21 | 2017-04-11 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9651784B2 (en) | 2014-01-21 | 2017-05-16 | Osterhout Group, Inc. | See-through computer display systems |
US10866420B2 (en) | 2014-01-21 | 2020-12-15 | Mentor Acquisition One, Llc | See-through computer display systems |
US9651783B2 (en) | 2014-01-21 | 2017-05-16 | Osterhout Group, Inc. | See-through computer display systems |
US9651788B2 (en) | 2014-01-21 | 2017-05-16 | Osterhout Group, Inc. | See-through computer display systems |
US9651789B2 (en) | 2014-01-21 | 2017-05-16 | Osterhout Group, Inc. | See-Through computer display systems |
US9658457B2 (en) | 2014-01-21 | 2017-05-23 | Osterhout Group, Inc. | See-through computer display systems |
US9658458B2 (en) | 2014-01-21 | 2017-05-23 | Osterhout Group, Inc. | See-through computer display systems |
US11892644B2 (en) | 2014-01-21 | 2024-02-06 | Mentor Acquisition One, Llc | See-through computer display systems |
US11103132B2 (en) | 2014-01-21 | 2021-08-31 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US9684165B2 (en) | 2014-01-21 | 2017-06-20 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9684171B2 (en) | 2014-01-21 | 2017-06-20 | Osterhout Group, Inc. | See-through computer display systems |
US10698223B2 (en) | 2014-01-21 | 2020-06-30 | Mentor Acquisition One, Llc | See-through computer display systems |
US11126003B2 (en) | 2014-01-21 | 2021-09-21 | Mentor Acquisition One, Llc | See-through computer display systems |
US10579140B2 (en) | 2014-01-21 | 2020-03-03 | Mentor Acquisition One, Llc | Eye glint imaging in see-through computer display systems |
US9720235B2 (en) | 2014-01-21 | 2017-08-01 | Osterhout Group, Inc. | See-through computer display systems |
US9720227B2 (en) | 2014-01-21 | 2017-08-01 | Osterhout Group, Inc. | See-through computer display systems |
US11796805B2 (en) | 2014-01-21 | 2023-10-24 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US9720234B2 (en) | 2014-01-21 | 2017-08-01 | Osterhout Group, Inc. | See-through computer display systems |
US9377625B2 (en) | 2014-01-21 | 2016-06-28 | Osterhout Group, Inc. | Optical configurations for head worn computing |
US9740280B2 (en) | 2014-01-21 | 2017-08-22 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9740012B2 (en) | 2014-01-21 | 2017-08-22 | Osterhout Group, Inc. | See-through computer display systems |
US11737666B2 (en) | 2014-01-21 | 2023-08-29 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US9746676B2 (en) | 2014-01-21 | 2017-08-29 | Osterhout Group, Inc. | See-through computer display systems |
US9523856B2 (en) | 2014-01-21 | 2016-12-20 | Osterhout Group, Inc. | See-through computer display systems |
US9753288B2 (en) | 2014-01-21 | 2017-09-05 | Osterhout Group, Inc. | See-through computer display systems |
US9766463B2 (en) | 2014-01-21 | 2017-09-19 | Osterhout Group, Inc. | See-through computer display systems |
US9772492B2 (en) | 2014-01-21 | 2017-09-26 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US11353957B2 (en) | 2014-01-21 | 2022-06-07 | Mentor Acquisition One, Llc | Eye glint imaging in see-through computer display systems |
US9811159B2 (en) | 2014-01-21 | 2017-11-07 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US11487110B2 (en) | 2014-01-21 | 2022-11-01 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US11054902B2 (en) | 2014-01-21 | 2021-07-06 | Mentor Acquisition One, Llc | Eye glint imaging in see-through computer display systems |
US9829703B2 (en) | 2014-01-21 | 2017-11-28 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US11669163B2 (en) | 2014-01-21 | 2023-06-06 | Mentor Acquisition One, Llc | Eye glint imaging in see-through computer display systems |
US9836122B2 (en) | 2014-01-21 | 2017-12-05 | Osterhout Group, Inc. | Eye glint imaging in see-through computer display systems |
US9715112B2 (en) | 2014-01-21 | 2017-07-25 | Osterhout Group, Inc. | Suppression of stray light in head worn computing |
US9436006B2 (en) | 2014-01-21 | 2016-09-06 | Osterhout Group, Inc. | See-through computer display systems |
US11619820B2 (en) | 2014-01-21 | 2023-04-04 | Mentor Acquisition One, Llc | See-through computer display systems |
US10001644B2 (en) | 2014-01-21 | 2018-06-19 | Osterhout Group, Inc. | See-through computer display systems |
US9885868B2 (en) | 2014-01-21 | 2018-02-06 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9927612B2 (en) | 2014-01-21 | 2018-03-27 | Osterhout Group, Inc. | See-through computer display systems |
US11622426B2 (en) | 2014-01-21 | 2023-04-04 | Mentor Acquisition One, Llc | See-through computer display systems |
US9933622B2 (en) | 2014-01-21 | 2018-04-03 | Osterhout Group, Inc. | See-through computer display systems |
US9494800B2 (en) | 2014-01-21 | 2016-11-15 | Osterhout Group, Inc. | See-through computer display systems |
US9958674B2 (en) | 2014-01-21 | 2018-05-01 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9952664B2 (en) | 2014-01-21 | 2018-04-24 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US11822090B2 (en) | 2014-01-24 | 2023-11-21 | Mentor Acquisition One, Llc | Haptic systems for head-worn computers |
US9939646B2 (en) | 2014-01-24 | 2018-04-10 | Osterhout Group, Inc. | Stray light suppression for head worn computing |
US10558050B2 (en) | 2014-01-24 | 2020-02-11 | Mentor Acquisition One, Llc | Haptic systems for head-worn computers |
US9401540B2 (en) | 2014-02-11 | 2016-07-26 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US9852545B2 (en) | 2014-02-11 | 2017-12-26 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US9286728B2 (en) | 2014-02-11 | 2016-03-15 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US9784973B2 (en) | 2014-02-11 | 2017-10-10 | Osterhout Group, Inc. | Micro doppler presentations in head worn computing |
US9843093B2 (en) | 2014-02-11 | 2017-12-12 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US10558420B2 (en) | 2014-02-11 | 2020-02-11 | Mentor Acquisition One, Llc | Spatial location presentation in head worn computing |
US9841602B2 (en) | 2014-02-11 | 2017-12-12 | Osterhout Group, Inc. | Location indicating avatar in head worn computing |
US11599326B2 (en) | 2014-02-11 | 2023-03-07 | Mentor Acquisition One, Llc | Spatial location presentation in head worn computing |
US9547465B2 (en) | 2014-02-14 | 2017-01-17 | Osterhout Group, Inc. | Object shadowing in head worn computing |
US9928019B2 (en) | 2014-02-14 | 2018-03-27 | Osterhout Group, Inc. | Object shadowing in head worn computing |
US10191279B2 (en) | 2014-03-17 | 2019-01-29 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US11104272B2 (en) | 2014-03-28 | 2021-08-31 | Mentor Acquisition One, Llc | System for assisted operator safety using an HMD |
US9423612B2 (en) | 2014-03-28 | 2016-08-23 | Osterhout Group, Inc. | Sensor dependent content position in head worn computing |
US11227294B2 (en) | 2014-04-03 | 2022-01-18 | Mentor Acquisition One, Llc | Sight information collection in head worn computing |
US10634922B2 (en) | 2014-04-25 | 2020-04-28 | Mentor Acquisition One, Llc | Speaker assembly for headworn computer |
US11880041B2 (en) | 2014-04-25 | 2024-01-23 | Mentor Acquisition One, Llc | Speaker assembly for headworn computer |
US9672210B2 (en) | 2014-04-25 | 2017-06-06 | Osterhout Group, Inc. | Language translation with head-worn computing |
US10853589B2 (en) | 2014-04-25 | 2020-12-01 | Mentor Acquisition One, Llc | Language translation with head-worn computing |
US9651787B2 (en) | 2014-04-25 | 2017-05-16 | Osterhout Group, Inc. | Speaker assembly for headworn computer |
US11474360B2 (en) | 2014-04-25 | 2022-10-18 | Mentor Acquisition One, Llc | Speaker assembly for headworn computer |
US11727223B2 (en) | 2014-04-25 | 2023-08-15 | Mentor Acquisition One, Llc | Language translation with head-worn computing |
US9746686B2 (en) | 2014-05-19 | 2017-08-29 | Osterhout Group, Inc. | Content position calibration in head worn computing |
US11402639B2 (en) | 2014-06-05 | 2022-08-02 | Mentor Acquisition One, Llc | Optical configurations for head-worn see-through displays |
US10877270B2 (en) | 2014-06-05 | 2020-12-29 | Mentor Acquisition One, Llc | Optical configurations for head-worn see-through displays |
US11960089B2 (en) | 2014-06-05 | 2024-04-16 | Mentor Acquisition One, Llc | Optical configurations for head-worn see-through displays |
US9841599B2 (en) | 2014-06-05 | 2017-12-12 | Osterhout Group, Inc. | Optical configurations for head-worn see-through displays |
US9575321B2 (en) | 2014-06-09 | 2017-02-21 | Osterhout Group, Inc. | Content presentation in head worn computing |
US11790617B2 (en) | 2014-06-09 | 2023-10-17 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US11022810B2 (en) | 2014-06-09 | 2021-06-01 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US10976559B2 (en) | 2014-06-09 | 2021-04-13 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US11663794B2 (en) | 2014-06-09 | 2023-05-30 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US10139635B2 (en) | 2014-06-09 | 2018-11-27 | Osterhout Group, Inc. | Content presentation in head worn computing |
US11887265B2 (en) | 2014-06-09 | 2024-01-30 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US10649220B2 (en) | 2014-06-09 | 2020-05-12 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US9720241B2 (en) | 2014-06-09 | 2017-08-01 | Osterhout Group, Inc. | Content presentation in head worn computing |
US11360318B2 (en) | 2014-06-09 | 2022-06-14 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US10663740B2 (en) | 2014-06-09 | 2020-05-26 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US11327323B2 (en) | 2014-06-09 | 2022-05-10 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US10698212B2 (en) | 2014-06-17 | 2020-06-30 | Mentor Acquisition One, Llc | External user interface for head worn computing |
US11294180B2 (en) | 2014-06-17 | 2022-04-05 | Mentor Acquisition One, Llc | External user interface for head worn computing |
US11054645B2 (en) | 2014-06-17 | 2021-07-06 | Mentor Acquisition One, Llc | External user interface for head worn computing |
US9810906B2 (en) | 2014-06-17 | 2017-11-07 | Osterhout Group, Inc. | External user interface for head worn computing |
US11789267B2 (en) | 2014-06-17 | 2023-10-17 | Mentor Acquisition One, Llc | External user interface for head worn computing |
US11103122B2 (en) | 2014-07-15 | 2021-08-31 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US11786105B2 (en) | 2014-07-15 | 2023-10-17 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US11269182B2 (en) | 2014-07-15 | 2022-03-08 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US11630315B2 (en) | 2014-08-12 | 2023-04-18 | Mentor Acquisition One, Llc | Measuring content brightness in head worn computing |
US11360314B2 (en) | 2014-08-12 | 2022-06-14 | Mentor Acquisition One, Llc | Measuring content brightness in head worn computing |
US10908422B2 (en) | 2014-08-12 | 2021-02-02 | Mentor Acquisition One, Llc | Measuring content brightness in head worn computing |
US9829707B2 (en) | 2014-08-12 | 2017-11-28 | Osterhout Group, Inc. | Measuring content brightness in head worn computing |
US9423842B2 (en) | 2014-09-18 | 2016-08-23 | Osterhout Group, Inc. | Thermal management for head-worn computer |
US9671613B2 (en) | 2014-09-26 | 2017-06-06 | Osterhout Group, Inc. | See-through computer display systems |
US9448409B2 (en) | 2014-11-26 | 2016-09-20 | Osterhout Group, Inc. | See-through computer display systems |
US10684687B2 (en) | 2014-12-03 | 2020-06-16 | Mentor Acquisition One, Llc | See-through computer display systems |
US9684172B2 (en) | 2014-12-03 | 2017-06-20 | Osterhout Group, Inc. | Head worn computer display systems |
US11809628B2 (en) | 2014-12-03 | 2023-11-07 | Mentor Acquisition One, Llc | See-through computer display systems |
US11262846B2 (en) | 2014-12-03 | 2022-03-01 | Mentor Acquisition One, Llc | See-through computer display systems |
USD792400S1 (en) | 2014-12-31 | 2017-07-18 | Osterhout Group, Inc. | Computer glasses |
USD794637S1 (en) | 2015-01-05 | 2017-08-15 | Osterhout Group, Inc. | Air mouse |
US10062182B2 (en) | 2015-02-17 | 2018-08-28 | Osterhout Group, Inc. | See-through computer display systems |
US11721303B2 (en) | 2015-02-17 | 2023-08-08 | Mentor Acquisition One, Llc | See-through computer display systems |
US10878775B2 (en) | 2015-02-17 | 2020-12-29 | Mentor Acquisition One, Llc | See-through computer display systems |
WO2017143128A1 (en) * | 2016-02-18 | 2017-08-24 | Osterhout Group, Inc. | Haptic systems for head-worn computers |
US11298288B2 (en) | 2016-02-29 | 2022-04-12 | Mentor Acquisition One, Llc | Providing enhanced images for navigation |
US11654074B2 (en) | 2016-02-29 | 2023-05-23 | Mentor Acquisition One, Llc | Providing enhanced images for navigation |
US10667981B2 (en) | 2016-02-29 | 2020-06-02 | Mentor Acquisition One, Llc | Reading assistance system for visually impaired |
US10849817B2 (en) | 2016-02-29 | 2020-12-01 | Mentor Acquisition One, Llc | Providing enhanced images for navigation |
US11592669B2 (en) | 2016-03-02 | 2023-02-28 | Mentor Acquisition One, Llc | Optical systems for head-worn computers |
US10591728B2 (en) | 2016-03-02 | 2020-03-17 | Mentor Acquisition One, Llc | Optical systems for head-worn computers |
US11156834B2 (en) | 2016-03-02 | 2021-10-26 | Mentor Acquisition One, Llc | Optical systems for head-worn computers |
US11024091B2 (en) | 2016-12-21 | 2021-06-01 | Pcms Holdings, Inc. | System and method for placement of augmented reality information for users based on their activity |
US11288878B2 (en) | 2016-12-21 | 2022-03-29 | Pcms Holdings, Inc. | System and method for placement of augmented reality information for users based on their activity |
US11232530B2 (en) * | 2017-02-28 | 2022-01-25 | Nec Corporation | Inspection assistance device, inspection assistance method, and recording medium |
US11615724B2 (en) | 2020-10-05 | 2023-03-28 | Volkswagen Aktiengesellschaft | Horizontal position determination for displaying virtual objects in a head-up display |
Also Published As
Publication number | Publication date |
---|---|
US20150279104A1 (en) | 2015-10-01 |
US9423612B2 (en) | 2016-08-23 |
US20150277122A1 (en) | 2015-10-01 |
US20150277118A1 (en) | 2015-10-01 |
US20150279108A1 (en) | 2015-10-01 |
US20150277549A1 (en) | 2015-10-01 |
US20150279107A1 (en) | 2015-10-01 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US20220164809A1 (en) | Sight information collection in head worn computing | |
US9423612B2 (en) | Sensor dependent content position in head worn computing | |
US10877270B2 (en) | Optical configurations for head-worn see-through displays | |
US9811159B2 (en) | Eye imaging in head worn computing | |
US9811152B2 (en) | Eye imaging in head worn computing | |
US9958674B2 (en) | Eye imaging in head worn computing | |
US10191279B2 (en) | Eye imaging in head worn computing | |
US20230400684A1 (en) | Eye imaging in head worn computing |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: OSTERHOUT GROUP, INC., CALIFORNIA Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:BORDER, JOHN N.;LOHSE, ROBERT MICHAEL;SIGNING DATES FROM 20140814 TO 20140818;REEL/FRAME:033966/0624 |
|
AS | Assignment |
Owner name: 21ST CENTURY FOX AMERICA, INC., NEW YORK Free format text: SECURITY INTEREST;ASSIGNOR:OSTERHOUT GROUP, INC.;REEL/FRAME:044052/0367 Effective date: 20170928 |
|
AS | Assignment |
Owner name: O-FILM GLOBAL (HK) TRADING LIMITED, CHINA Free format text: SECURITY INTEREST;ASSIGNOR:OSTERHOUT GROUP, INC.;REEL/FRAME:044127/0501 Effective date: 20170929 |
|
AS | Assignment |
Owner name: JGB COLLATERAL, LLC, CONNECTICUT Free format text: SECURITY INTEREST;ASSIGNOR:OSTERHOUT GROUP, INC.;REEL/FRAME:045606/0295 Effective date: 20180313 |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |
|
AS | Assignment |
Owner name: JP MORGAN CHASE BANK, N.A., NEW YORK Free format text: PATENT SECURITY AGREEMENT;ASSIGNORS:MAGIC LEAP, INC.;MOLECULAR IMPRINTS, INC.;MENTOR ACQUISITION ONE, LLC;REEL/FRAME:050138/0287 Effective date: 20190820 |
|
AS | Assignment |
Owner name: CITIBANK, N.A., NEW YORK Free format text: ASSIGNMENT OF SECURITY INTEREST IN PATENTS;ASSIGNOR:JPMORGAN CHASE BANK, N.A.;REEL/FRAME:050967/0138 Effective date: 20191106 |