EP2617202A2 - Bluetooth or other wireless interface with power management for head mounted display - Google Patents
Bluetooth or other wireless interface with power management for head mounted displayInfo
- Publication number
- EP2617202A2 EP2617202A2 EP11827249.1A EP11827249A EP2617202A2 EP 2617202 A2 EP2617202 A2 EP 2617202A2 EP 11827249 A EP11827249 A EP 11827249A EP 2617202 A2 EP2617202 A2 EP 2617202A2
- Authority
- EP
- European Patent Office
- Prior art keywords
- user
- processor
- state
- command
- commands
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Withdrawn
Links
Classifications
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/42203—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] sound input device, e.g. microphone
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/017—Head mounted
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/26—Power supply means, e.g. regulation thereof
- G06F1/32—Means for saving power
- G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
- G06F1/3206—Monitoring of events, devices or parameters that trigger a change in power modality
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/26—Power supply means, e.g. regulation thereof
- G06F1/32—Means for saving power
- G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
- G06F1/3234—Power saving characterised by the action undertaken
- G06F1/325—Power saving in peripheral device
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/26—Power supply means, e.g. regulation thereof
- G06F1/32—Means for saving power
- G06F1/3203—Power management, i.e. event-based initiation of a power-saving mode
- G06F1/3234—Power saving characterised by the action undertaken
- G06F1/3287—Power saving characterised by the action undertaken by switching off individual functional units in the computer system
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/012—Head tracking input arrangements
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
- G06F3/0304—Detection arrangements using opto-electronic means
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/4104—Peripherals receiving signals from specially adapted client devices
- H04N21/4122—Peripherals receiving signals from specially adapted client devices additional display device, e.g. video projector
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/41—Structure of client; Structure of client peripherals
- H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
- H04N21/4223—Cameras
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/439—Processing of audio elementary streams
- H04N21/4394—Processing of audio elementary streams involving operations for analysing the audio stream, e.g. detecting features or characteristics in audio streams
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/44—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs
- H04N21/44008—Processing of video elementary streams, e.g. splicing a video clip retrieved from local storage with an incoming video stream, rendering scenes according to MPEG-4 scene graphs involving operations for analysing video streams, e.g. detecting features or characteristics in the video stream
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
- H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
- H04N21/43—Processing of content or additional data, e.g. demultiplexing additional data from a digital video stream; Elementary client operations, e.g. monitoring of home network or synchronising decoder's clock; Client middleware
- H04N21/443—OS processes, e.g. booting an STB, implementing a Java virtual machine in an STB or power management in an STB
- H04N21/4436—Power management, e.g. shutting down unused components of the receiver
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0101—Head-up displays characterised by optical features
- G02B2027/014—Head-up displays characterised by optical features comprising information/image processing systems
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/01—Head-up displays
- G02B27/0179—Display position adjusting means not related to the information to be displayed
- G02B2027/0187—Display position adjusting means not related to the information to be displayed slaved to motion of at least a part of the body of the user, e.g. head, eye
-
- Y—GENERAL TAGGING OF NEW TECHNOLOGICAL DEVELOPMENTS; GENERAL TAGGING OF CROSS-SECTIONAL TECHNOLOGIES SPANNING OVER SEVERAL SECTIONS OF THE IPC; TECHNICAL SUBJECTS COVERED BY FORMER USPC CROSS-REFERENCE ART COLLECTIONS [XRACs] AND DIGESTS
- Y02—TECHNOLOGIES OR APPLICATIONS FOR MITIGATION OR ADAPTATION AGAINST CLIMATE CHANGE
- Y02D—CLIMATE CHANGE MITIGATION TECHNOLOGIES IN INFORMATION AND COMMUNICATION TECHNOLOGIES [ICT], I.E. INFORMATION AND COMMUNICATION TECHNOLOGIES AIMING AT THE REDUCTION OF THEIR OWN ENERGY USE
- Y02D10/00—Energy efficient computing, e.g. low power processors, power management or thermal management
Definitions
- the present disclosure relates to a head mounted display (HMD) system or other video eyewear apparatus having a wireless front end that controls the activation of other system components.
- HMD head mounted display
- HMD head mounted display
- the electronics enclosed in such devices have become extremely sophisticated and can now include integrated data processors, wireless interfaces, and other input devices such as head tracking accelerometers, cameras, voice recognition circuits and software and other components.
- a head mounted display (HMD) system includes a high resolution microdisplay, microphone(s) and speaker(s), auxiliary interfaces such as wireless interfaces, wireless network interfaces, cameras (visible, infrared, etc.), audio devices, keypads, game controller devices and other peripherals.
- HMD head mounted display
- a wireless interface such as a Bluetooth, WiFi, cellular or other wireless interface
- a wireless interface becomes a front end for the system and controls when the other system components are activated.
- the HMD system When the HMD system is first switched on it can act with minimal functionality such as similar to a Bluetooth headset to provide only speaker and microphone functions.
- a paired device such as a Bluetooth compatible mobile phone (i.e., an Apple iPhone or Google Android Smartphone) can be used to feed and be fed audio conversations in a hands free mode. In this mode, other functions of the HMD system remain in hibernation
- the Bluetooth front end detects an incoming data stream with a certain audio component
- the audio component is passed to a voice recognition function embodied as a circuit or software. If the voice recognition function determines the audio component is a verbal command to the system that requires video output, the front end will cause the embedded data processor to turn on any other peripheral devices such as the video display. When the video display is no longer needed, the processor and video driver circuitry are placed back in sleep mode.
- the user may speak a command that requires more sophisticated functionality of the onboard processor.
- the user may request the reading a list of today's "to do" tasks.
- the list may be generated by an application program such as Microsoft Outlook running on a host computer.
- Carrying out this command requires not only waking up the embedded processor to access a local mass storage device to fetch the to do list information but also text to speech circuitry that generates an audio signal that corresponds to the spoken words.
- the user may request viewing an agenda document that was attached as a Microsoft Word document to a webmail message. This may require the embedded processor to wake up, activate the local microdisplay, and make a connection to an external network, such as through a Bluetooth, Wireless Local Area Network (WLAN) circuit and/or cellular modem circuit. Those circuits will then only be energized at that point.
- the embedded processor then contacts a host machine, such as a Personal Computer, to retrieve and display the web information.
- a host machine such as a Personal Computer
- the wireless front end has contained therein a table or other logic that enables it to not only perform speech to text functions, or convert gestures to text, but then to interpret these voice or gesture commands and consult a list of components that require energizing to carry out the spoken or gesture commands.
- Fig. 1 is a head mounted display (HMD) system that includes a microdisplay as well as other peripheral interfaces.
- HMD head mounted display
- Fig. 2 is another perspective view showing the HMD system as worn on the user's head with a camera peripheral installed.
- Fig. 3 is a high level functional diagram showing the HMD and its use as a controller for a host processor such as a personal computer.
- Fig. 4 is a high level block diagram of the components internal to the HMD assembly.
- Fig. 5 is a more detailed diagram showing the electronic components used in one particular embodiment.
- Fig. 6 is a high level flow diagram showing one example of how the
- Bluetooth front end acts as a system controller to control when various other HMD system components become active.
- Fig. 7 is a list of spoken commands, gesture commands or head motion commands and those portions of the subsystem that are activated.
- Fig. 1 shows a head mounted display (HMD) system 100 that generally includes a frame 1000, a strap 1002, a back section 1004, a speaker 1006, cantilever or arm 1008 and microdisplay subassembly 1010.
- HMD head mounted display
- the HMD 100 may be based on Golden-iTM hands free communication and control device available from Kopin Corporation of Taunton, Massachusetts.
- One side of the device, particularly the side opposite the microdisplay subassembly 1010, may include a "hot shoe" peripheral port 1020 that can provide corresponding connections to accessories, as explained further below, so that a user can remove the attached various components to the HMD 100.
- Fig. 2 is a view of the HMD 100 worn on the head of a user where a camera accessory 1060 has been placed in the port 1020.
- the camera 1060 can include both audio and video sensing and recording capability and may for example, be in a small form factor similar to a "bullet cam".
- user commands which may be a head motion or voice commands are provided by the user to control the settings of the display 1010, the camera 1020, and to perform other functions.
- the HMD system may principally function as either a stand alone personal computer or "smart phone” device and/or as remote control for a host personal computer, or other device.
- various electronic circuits including as will be understood shortly, a microcomputer, one or more wireless interfaces, associated memory or other storage devices, position, motion and/or gesture sensors.
- the camera position, motion and/or gesture sensors are used to track the position, motion and/or gesture of the users head or hands and the microphone receives audio input commands.
- the wireless interface enables HMD system 100 to be used as a smart phone itself or as a remote control for a host computing device 200 (see Fig. 3).
- the host 200 may for example, be a laptop, cell phone, RIM Blackberry, Apple iPhone, Google Android or other smart phone device having similar or greater computational complexity and similar or greater network connectivity than the remote control HMD system 100 itself.
- the host 200 may be further connected to other networks, such as through a Wide Area Network (WAN) connection 210 to the Internet.
- WAN Wide Area Network
- the HMD system 100 and host 200 are connected to one another through suitable wireless connections, such as are provided by a Bluetooth link or Wireless Local Area Network (WLAN) 150.
- WLAN Wireless Local Area Network
- the HMD system 100 receives inputs from the user such as voice input via the microphone, hand movements via the camera or other optical sensors, and/or head movement inputs via the accelerometers or other head motion tracking circuitry. These are translated by software in system 100 (and/or cooperating with software in host 200) into device 100 and/or host 200 system recognizable commands. If these are commands for HMD system 100, it carries them and if they are command for host 200, they are sent over the Bluetooth and/or WiFi interface 150 to the host 200. The host 200 then interprets these translated commands in accordance with its own operating system/application software to perform various functions.
- commands may be a command to fetch and display a document.
- Among these commands may be to answer a telephone call.
- commands may be to retrieve information from the Internet and display it, such as an email or a web page.
- Among other commands may be to select a field of view within a virtual display and then return that selected screen to the remote device. It should also be understood that a very large format to virtual display area may be associated with application software running on or in an operating system running on the host 200 for only a portion of that large virtual display area within the field of view is selected by various voice commands and/or head movements supplied to the display 1010 on the HMD 100.
- the HMD system 100 may provide other functions to other accessories beyond the camera such as additional speakers, additional cameras, light sources and any other electronic components that may be put into the hot shoe 1020 or battery self powered.
- Fig. 4 is a simplified high level block diagram of a non-limiting example embodiment of the HMD system 100. These components are integrated into the HMD housing in various places as shown in Fig. 1.
- the HMD system 100 may include an eye pod assembly 4000 that includes the aforementioned microdisplay 4010, and one or more microphones 4020.
- One or more speakers 4030 are positioned in the housing earpiece near the user's ear (see item 1006 in Fig. 1).
- the system electronics can be placed on or in the frame in an appropriate location (such as back section 1004) and include an Open Media Application Platform (OMAP) processor 41 10, a power/audio companion chip 4102, a display driver 4104, a head tracker 4105, a circuit board 4106 and wireless LAN/Bluetooth interface 4108.
- OMAP Open Media Application Platform
- a power/audio companion chip 4102 a display driver 4104
- head tracker 4105 a circuit board 4106
- wireless LAN/Bluetooth interface 4108 wireless LAN/Bluet
- the head tracker circuitry 4106 may include circuits to detect head movements and gestures such as lateral movements along and rotation gestures around the X, Y and Z axes using Hall effect sensors, MIM diodes, accelerometers, gyros and/or transducers. HMD system 100 may also receive inputs from external input devices such as a wireless mouse, track ball, or keyboard that may be wirelessly connected through the Bluetooth interface 4108. Software in the WLAN/BT front end 4108, the OMAP 4100 and/or host may be used to interpret hand gestures detected by the camera.
- a camera board 4060 may optionally provide video input.
- the OMAP processor may include a central processing unit, and on-chip memory such as Random Access Memory (RAM) that may include non volatile memory and/or Read Only Memory (ROM).
- RAM Random Access Memory
- ROM Read Only Memory
- the OMAP may be a Texas Instruments Inc.
- the OMAP is generally a more powerful, and more power consuming processor then the WLAN/BT interface 4108.
- a TPS 65950 power/audio companion chip also available from Texas Instruments, provides audio, USB, keypad control and battery charging functions to the system.
- the WLAN/BT interface 4108 may be a model LBEE 1 W8 NEC-interface circuit or newer radio module with similar or greater capabilities.
- the display driver may be a model KCD-A 910 display driver available from Kopin Corporation of Westborough, Massachusetts.
- the microdisplay 4010 can include models 113LV, 152LV, 230LV, WQVGA, or other manufactures accesptable micro- displays.
- An NCS module 4400 processes the received microphone signals to provide voice recognition functions and produce an audio signal to the audio companion chip 4102.
- Fig. 5 is a more detailed block diagram showing the connectivity between the various components of Fig. 4.
- the WLAN/BT front end 4108 can be use a UART interface. It is programmed to control the power on or power off state of the OMAP processor 4100and accordingly the audio and power companion chip 4102 and the power applied to other peripheral functions such as the cameram, display, WLAN modem, etc.
- Fig. 6 is a flow chart illustrating a few example functions where the
- WLAN/Bluetooth front end 4108 may control application of power to other system components.
- Fig. 7 is an example table that may be kept by front end 4108 to determine when to power on/power off various devices, and should be referred to in the discussion of Fig. 6 that follows.
- the system components may initially be in a hibernate mode with only the Bluetooth headset functions (e.g. microphone 4020 and speaker 4030) enabled.
- This state 6000 may be entered when the HMD system 100 is first switched on.
- state 6100 if an incoming telephone call occurs, the audio input and output functions remain active in state 6110 (e.g. leaving microphones 4020 on and enabling speaker 4030). But in state 6120 the OMAP 4100, display 4010, camera 4060 and other component remain in a hibernate mode.
- processing returns to state 6000.
- the audio circuitry may detect a specific user voice command, such as "read my to-do list”. From this state
- the OMAP is enabled in state 6210.
- the OMAP in state 6220, may then access a local storage area such as an SDRAM memory to access a file which is associated with the user's local task list to obtain an itemized list of tasks to do for the present day.
- the to-do list data is returned from the OMAP 4100 to the audio interface in state 6240 where the to-do list text is converted to speech and then heard on the speakers of the headset 100.
- a state 6500 is entered where the system then returns to the hibernate state 6000, where the system again functions only as a Bluetooth headset, awaiting another user command.
- a user speaks a command indicating that she wishes to view the view the New York Times homepage at www.NYTimes.com.
- state 6310 On receipt of this command, in state 6310 the OMAP is enabled.
- a WLAN interface is activated enabling the OMAP to issue commands to the host processor, such as was described in connection with Fig. 3, to obtain the HTML content for the New York Times homepage.
- state 6330 is entered where the microdisplay 1010 is also enabled, anticipating that the user wishes to display a webpage.
- the HTML webpage for the New York Times, com can be fetched in other ways, such as for example, if the headset itself had the ability to access the Internet through an onboard Bluetooth, WLAN circuit (Wi Fi) or through auxiliary cellular modems.
- Wi Fi WLAN circuit
- auxiliary cellular modems auxiliary cellular modems.
- the user may issue another command to disable the display (or the command may time out), at which point the hibernate function 6000 is again entered.
- the Bluetooth and/or WiFi circuit automatically wakes up the OMAP and other peripheral devices but only those many as are actually necessary to carry out the command.
- the Bluetooth and/or WiFi front end thus retains information, such as in the form of a table or database entries, indicating which peripheral devices and/or OMAP need to be energized, if any, to carry out each of many different user commands.
- user hand gestures may be detected by the camera and provide input to control the application of power to specific components of the system 100.
- the audio components are initially enabled, but detect a spoken command to read and interpret hand gestures by activating at least camera 4060 and OMAP 4100.
- the user may use head movements as an input.
- head motion tracking 4106 and display are related components enabled, such as when the user is using head movements to pan, zoom, or otherwise navigate a large format virtual display area.
- HMD system functions of the HMD system are controlled via appropriate spoken commands, gesture commands or commands from a host computer, and interpreted by the Bluetooth and/or WiFi interface, waking up only that portion of the subsystem electronics as required.
- another spoken command or time out can return the HMD system back to the sleep mode.
- the Bluetooth and/or WiFi circuit goes back into a hibernation state waiting for an external contact signal, such as a phone call, another voice command, or computer interface command via the WLAN.
Abstract
Description
Claims
Applications Claiming Priority (3)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US38441910P | 2010-09-20 | 2010-09-20 | |
US38454310P | 2010-09-20 | 2010-09-20 | |
PCT/US2011/051760 WO2012040030A2 (en) | 2010-09-20 | 2011-09-15 | Bluetooth or other wireless interface with power management for head mounted display |
Publications (2)
Publication Number | Publication Date |
---|---|
EP2617202A2 true EP2617202A2 (en) | 2013-07-24 |
EP2617202A4 EP2617202A4 (en) | 2015-01-21 |
Family
ID=45874290
Family Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
EP11827249.1A Withdrawn EP2617202A4 (en) | 2010-09-20 | 2011-09-15 | Bluetooth or other wireless interface with power management for head mounted display |
Country Status (4)
Country | Link |
---|---|
EP (1) | EP2617202A4 (en) |
JP (1) | JP5974008B2 (en) |
CN (1) | CN103890836B (en) |
WO (1) | WO2012040030A2 (en) |
Families Citing this family (64)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9715112B2 (en) | 2014-01-21 | 2017-07-25 | Osterhout Group, Inc. | Suppression of stray light in head worn computing |
US20150205111A1 (en) | 2014-01-21 | 2015-07-23 | Osterhout Group, Inc. | Optical configurations for head worn computing |
US9952664B2 (en) | 2014-01-21 | 2018-04-24 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9965681B2 (en) | 2008-12-16 | 2018-05-08 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9298007B2 (en) | 2014-01-21 | 2016-03-29 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9400390B2 (en) | 2014-01-24 | 2016-07-26 | Osterhout Group, Inc. | Peripheral lighting for head worn computing |
US9229233B2 (en) | 2014-02-11 | 2016-01-05 | Osterhout Group, Inc. | Micro Doppler presentations in head worn computing |
US9851787B2 (en) | 2012-11-29 | 2017-12-26 | Microsoft Technology Licensing, Llc | Display resource management |
US20140282273A1 (en) * | 2013-03-15 | 2014-09-18 | Glen J. Anderson | System and method for assigning voice and gesture command areas |
US10254856B2 (en) | 2014-01-17 | 2019-04-09 | Osterhout Group, Inc. | External user interface for head worn computing |
US20160019715A1 (en) | 2014-07-15 | 2016-01-21 | Osterhout Group, Inc. | Content presentation in head worn computing |
US20150277118A1 (en) | 2014-03-28 | 2015-10-01 | Osterhout Group, Inc. | Sensor dependent content position in head worn computing |
US9575321B2 (en) | 2014-06-09 | 2017-02-21 | Osterhout Group, Inc. | Content presentation in head worn computing |
US10684687B2 (en) | 2014-12-03 | 2020-06-16 | Mentor Acquisition One, Llc | See-through computer display systems |
US9841599B2 (en) | 2014-06-05 | 2017-12-12 | Osterhout Group, Inc. | Optical configurations for head-worn see-through displays |
US9810906B2 (en) | 2014-06-17 | 2017-11-07 | Osterhout Group, Inc. | External user interface for head worn computing |
US9299194B2 (en) | 2014-02-14 | 2016-03-29 | Osterhout Group, Inc. | Secure sharing in head worn computing |
US11103122B2 (en) | 2014-07-15 | 2021-08-31 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US9671613B2 (en) | 2014-09-26 | 2017-06-06 | Osterhout Group, Inc. | See-through computer display systems |
US9939934B2 (en) | 2014-01-17 | 2018-04-10 | Osterhout Group, Inc. | External user interface for head worn computing |
US9448409B2 (en) | 2014-11-26 | 2016-09-20 | Osterhout Group, Inc. | See-through computer display systems |
US9829707B2 (en) | 2014-08-12 | 2017-11-28 | Osterhout Group, Inc. | Measuring content brightness in head worn computing |
US11227294B2 (en) | 2014-04-03 | 2022-01-18 | Mentor Acquisition One, Llc | Sight information collection in head worn computing |
US9594246B2 (en) | 2014-01-21 | 2017-03-14 | Osterhout Group, Inc. | See-through computer display systems |
US9529195B2 (en) | 2014-01-21 | 2016-12-27 | Osterhout Group, Inc. | See-through computer display systems |
US10649220B2 (en) | 2014-06-09 | 2020-05-12 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US10191279B2 (en) | 2014-03-17 | 2019-01-29 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9746686B2 (en) | 2014-05-19 | 2017-08-29 | Osterhout Group, Inc. | Content position calibration in head worn computing |
US9836122B2 (en) | 2014-01-21 | 2017-12-05 | Osterhout Group, Inc. | Eye glint imaging in see-through computer display systems |
US9766463B2 (en) | 2014-01-21 | 2017-09-19 | Osterhout Group, Inc. | See-through computer display systems |
US11487110B2 (en) | 2014-01-21 | 2022-11-01 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US9811159B2 (en) | 2014-01-21 | 2017-11-07 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US9651784B2 (en) | 2014-01-21 | 2017-05-16 | Osterhout Group, Inc. | See-through computer display systems |
US20150205135A1 (en) | 2014-01-21 | 2015-07-23 | Osterhout Group, Inc. | See-through computer display systems |
US9651788B2 (en) | 2014-01-21 | 2017-05-16 | Osterhout Group, Inc. | See-through computer display systems |
US9753288B2 (en) | 2014-01-21 | 2017-09-05 | Osterhout Group, Inc. | See-through computer display systems |
US9494800B2 (en) | 2014-01-21 | 2016-11-15 | Osterhout Group, Inc. | See-through computer display systems |
US11737666B2 (en) | 2014-01-21 | 2023-08-29 | Mentor Acquisition One, Llc | Eye imaging in head worn computing |
US11892644B2 (en) | 2014-01-21 | 2024-02-06 | Mentor Acquisition One, Llc | See-through computer display systems |
US9532714B2 (en) | 2014-01-21 | 2017-01-03 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US11669163B2 (en) | 2014-01-21 | 2023-06-06 | Mentor Acquisition One, Llc | Eye glint imaging in see-through computer display systems |
US9846308B2 (en) | 2014-01-24 | 2017-12-19 | Osterhout Group, Inc. | Haptic systems for head-worn computers |
US9401540B2 (en) | 2014-02-11 | 2016-07-26 | Osterhout Group, Inc. | Spatial location presentation in head worn computing |
US20150241963A1 (en) | 2014-02-11 | 2015-08-27 | Osterhout Group, Inc. | Eye imaging in head worn computing |
US20160187651A1 (en) | 2014-03-28 | 2016-06-30 | Osterhout Group, Inc. | Safety for a vehicle operator with an hmd |
US10853589B2 (en) | 2014-04-25 | 2020-12-01 | Mentor Acquisition One, Llc | Language translation with head-worn computing |
US9651787B2 (en) | 2014-04-25 | 2017-05-16 | Osterhout Group, Inc. | Speaker assembly for headworn computer |
US9672210B2 (en) | 2014-04-25 | 2017-06-06 | Osterhout Group, Inc. | Language translation with head-worn computing |
US9423842B2 (en) | 2014-09-18 | 2016-08-23 | Osterhout Group, Inc. | Thermal management for head-worn computer |
WO2015179877A2 (en) * | 2014-05-19 | 2015-11-26 | Osterhout Group, Inc. | External user interface for head worn computing |
US10663740B2 (en) | 2014-06-09 | 2020-05-26 | Mentor Acquisition One, Llc | Content presentation in head worn computing |
US9684172B2 (en) | 2014-12-03 | 2017-06-20 | Osterhout Group, Inc. | Head worn computer display systems |
JP6516464B2 (en) * | 2014-12-15 | 2019-05-22 | Kddi株式会社 | Wearable search system |
USD751552S1 (en) | 2014-12-31 | 2016-03-15 | Osterhout Group, Inc. | Computer glasses |
USD753114S1 (en) | 2015-01-05 | 2016-04-05 | Osterhout Group, Inc. | Air mouse |
CN104615243A (en) * | 2015-01-15 | 2015-05-13 | 深圳市掌网立体时代视讯技术有限公司 | Head-wearable type multi-channel interaction system and multi-channel interaction method |
US9933995B2 (en) * | 2015-01-25 | 2018-04-03 | Harman International Industries, Incorporated | Headphones with integral image display |
US20160239985A1 (en) | 2015-02-17 | 2016-08-18 | Osterhout Group, Inc. | See-through computer display systems |
ES2835877T3 (en) * | 2015-06-09 | 2021-06-23 | Nokia Technologies Oy | Transition of a virtual reality headset from operation in a first power mode to operation in a second power mode |
US9298283B1 (en) | 2015-09-10 | 2016-03-29 | Connectivity Labs Inc. | Sedentary virtual reality method and systems |
CN105797366A (en) * | 2016-03-25 | 2016-07-27 | 中国传媒大学 | Head-wearing type interactive audio game terminal based on sound source location |
DE102016207009A1 (en) * | 2016-04-26 | 2017-10-26 | Krones Aktiengesellschaft | Operating system for a machine of the food industry, in particular the beverage industry |
TWI599966B (en) * | 2016-05-10 | 2017-09-21 | H P B Optoelectronic Co Ltd | Gesture control modular system |
USD864959S1 (en) | 2017-01-04 | 2019-10-29 | Mentor Acquisition One, Llc | Computer glasses |
Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040104864A1 (en) * | 2002-11-28 | 2004-06-03 | Nec Corporation | Glasses type display and controlling method thereof |
US20080300886A1 (en) * | 2007-05-17 | 2008-12-04 | Kimberly Patch | Systems and methods of a structured grammar for a speech recognition command system |
WO2009054619A2 (en) * | 2007-10-22 | 2009-04-30 | Moon Key Lee | Augmented reality computer device |
EP2133729A1 (en) * | 2008-06-11 | 2009-12-16 | Honeywell International Inc. | Method and system for operating a near-to-eye display |
Family Cites Families (15)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
JPH10312631A (en) * | 1997-05-14 | 1998-11-24 | Nec Niigata Ltd | Sound reproducing device and information processing device |
US6408257B1 (en) * | 1999-08-31 | 2002-06-18 | Xerox Corporation | Augmented-reality display method and system |
DE60045473D1 (en) * | 1999-11-09 | 2011-02-17 | Nuance Comm Austria Gmbh | LANGUAGE RECOGNITION METHOD FOR ACTIVATING INTERNET HYPERLINKS |
AU2003239385A1 (en) * | 2002-05-10 | 2003-11-11 | Richard R. Reisman | Method and apparatus for browsing using multiple coordinated device |
JP2004349915A (en) * | 2003-05-21 | 2004-12-09 | Matsushita Electric Ind Co Ltd | Remote control unit |
KR100594117B1 (en) * | 2004-09-20 | 2006-06-28 | 삼성전자주식회사 | Apparatus and method for inputting key using biosignal in HMD information terminal |
JP4364861B2 (en) * | 2005-11-01 | 2009-11-18 | 株式会社ソニー・コンピュータエンタテインメント | Information display device |
JP2008054085A (en) * | 2006-08-25 | 2008-03-06 | Hitachi Ltd | Broadcast receiving apparatus and starting method thereof |
JP5228305B2 (en) * | 2006-09-08 | 2013-07-03 | ソニー株式会社 | Display device and display method |
US8386259B2 (en) * | 2006-12-28 | 2013-02-26 | Intel Corporation | Voice interface to NFC applications |
KR101441873B1 (en) * | 2007-01-12 | 2014-11-04 | 코핀 코포레이션 | Head mounted monocular display device |
US9116340B2 (en) * | 2007-05-14 | 2015-08-25 | Kopin Corporation | Mobile wireless display for accessing data from a host and method for controlling |
IL183637A (en) * | 2007-06-04 | 2013-06-27 | Zvi Lapidot | Distributed head-mounted display |
JP5217386B2 (en) * | 2007-11-21 | 2013-06-19 | 株式会社ニコン | Control device, head mounted display device, program, and control method |
US8957835B2 (en) * | 2008-09-30 | 2015-02-17 | Apple Inc. | Head-mounted display apparatus for retaining a portable electronic device with display |
-
2011
- 2011-09-15 EP EP11827249.1A patent/EP2617202A4/en not_active Withdrawn
- 2011-09-15 JP JP2013529323A patent/JP5974008B2/en active Active
- 2011-09-15 WO PCT/US2011/051760 patent/WO2012040030A2/en active Application Filing
- 2011-09-15 CN CN201180050791.3A patent/CN103890836B/en active Active
Patent Citations (4)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20040104864A1 (en) * | 2002-11-28 | 2004-06-03 | Nec Corporation | Glasses type display and controlling method thereof |
US20080300886A1 (en) * | 2007-05-17 | 2008-12-04 | Kimberly Patch | Systems and methods of a structured grammar for a speech recognition command system |
WO2009054619A2 (en) * | 2007-10-22 | 2009-04-30 | Moon Key Lee | Augmented reality computer device |
EP2133729A1 (en) * | 2008-06-11 | 2009-12-16 | Honeywell International Inc. | Method and system for operating a near-to-eye display |
Non-Patent Citations (1)
Title |
---|
See also references of WO2012040030A2 * |
Also Published As
Publication number | Publication date |
---|---|
WO2012040030A2 (en) | 2012-03-29 |
WO2012040030A3 (en) | 2014-03-27 |
CN103890836B (en) | 2017-03-15 |
JP5974008B2 (en) | 2016-08-23 |
EP2617202A4 (en) | 2015-01-21 |
CN103890836A (en) | 2014-06-25 |
JP2014503085A (en) | 2014-02-06 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US9152378B2 (en) | Bluetooth or other wireless interface with power management for head mounted display | |
EP2617202A2 (en) | Bluetooth or other wireless interface with power management for head mounted display | |
US10535320B2 (en) | Head-mounted display apparatus | |
EP2845075B1 (en) | Headset computer (hsc) as auxiliary display with asr and ht input | |
US10013976B2 (en) | Context sensitive overlays in voice controlled headset computer displays | |
EP2616907B1 (en) | Control of applications on a head-mounted display using gesture and motion commands | |
EP3411780B1 (en) | Intelligent electronic device and method of operating the same | |
US11258893B2 (en) | Method for prompting notification message and mobile terminal | |
US9378028B2 (en) | Headset computer (HSC) with docking station and dual personality | |
US20120075177A1 (en) | Lapel microphone micro-display system incorporating mobile information access | |
KR102492560B1 (en) | Electronic device and method for controlling input thereof | |
GB2534274A (en) | Gaze triggered voice recognition | |
CN107767839B (en) | Brightness adjusting method and related product | |
KR20190032026A (en) | Method for providing natural language expression and electronic device supporting the same | |
JP2018032440A (en) | Controllable headset computer displays | |
KR102345883B1 (en) | Electronic device for ouputting graphical indication | |
US11144091B2 (en) | Power save mode for wearable device | |
CN113782025B (en) | Speech recognition method, device, terminal and storage medium | |
US20190373072A1 (en) | Event notification | |
CN110989963B (en) | Wake-up word recommendation method and device and storage medium | |
KR20150014254A (en) | Mobile terminal and operation method thereof | |
KR101716153B1 (en) | Mobile terminal and operation method thereof | |
CN117332061A (en) | Interaction method and device for dialogue interface, electronic equipment and storage medium | |
KR20110139595A (en) | Mobile terminal and operation method thereof | |
KR20150024145A (en) | Method of operating a Mobile Terminal |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
PUAI | Public reference made under article 153(3) epc to a published international application that has entered the european phase |
Free format text: ORIGINAL CODE: 0009012 |
|
17P | Request for examination filed |
Effective date: 20130418 |
|
AK | Designated contracting states |
Kind code of ref document: A2 Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR |
|
DAX | Request for extension of the european patent (deleted) | ||
R17D | Deferred search report published (corrected) |
Effective date: 20140327 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G09F 5/00 20060101AFI20140522BHEP |
|
A4 | Supplementary search report drawn up and despatched |
Effective date: 20150105 |
|
RIC1 | Information provided on ipc code assigned before grant |
Ipc: G06F 3/01 20060101ALI20141218BHEP Ipc: G09F 5/00 20060101AFI20141218BHEP Ipc: G06F 3/03 20060101ALI20141218BHEP Ipc: G02B 27/01 20060101ALI20141218BHEP Ipc: H04N 21/422 20110101ALI20141218BHEP Ipc: G06F 1/32 20060101ALI20141218BHEP Ipc: H04N 21/443 20110101ALI20141218BHEP Ipc: H04N 21/439 20110101ALI20141218BHEP Ipc: H04N 21/44 20110101ALI20141218BHEP Ipc: H04N 21/41 20110101ALI20141218BHEP Ipc: H04N 21/4223 20110101ALI20141218BHEP |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE |
|
STAA | Information on the status of an ep patent application or granted ep patent |
Free format text: STATUS: THE APPLICATION IS DEEMED TO BE WITHDRAWN |
|
18D | Application deemed to be withdrawn |
Effective date: 20170401 |