WO2008109619A3 - Interface to convert mental states and facial expressions to application input - Google Patents

Interface to convert mental states and facial expressions to application input Download PDF

Info

Publication number
WO2008109619A3
WO2008109619A3 PCT/US2008/055827 US2008055827W WO2008109619A3 WO 2008109619 A3 WO2008109619 A3 WO 2008109619A3 US 2008055827 W US2008055827 W US 2008055827W WO 2008109619 A3 WO2008109619 A3 WO 2008109619A3
Authority
WO
WIPO (PCT)
Prior art keywords
convert
interface
facial expressions
user
application input
Prior art date
Application number
PCT/US2008/055827
Other languages
French (fr)
Other versions
WO2008109619A2 (en
Inventor
Randall P Breen
Tan Thi Thai Le
Original Assignee
Emotiv Systems Pty Ltd
Randall P Breen
Tan Thi Thai Le
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Emotiv Systems Pty Ltd, Randall P Breen, Tan Thi Thai Le filed Critical Emotiv Systems Pty Ltd
Publication of WO2008109619A2 publication Critical patent/WO2008109619A2/en
Publication of WO2008109619A3 publication Critical patent/WO2008109619A3/en

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection

Abstract

A method of interacting with an application includes receiving, in a processor, data generated based on signals from one or more bio-signal detectors on a user, the data representing a mental state or facial expression of the user, generating an input event based on the data representing the mental state or facial expression of the user of the user, and passing the input event to an application.
PCT/US2008/055827 2007-03-05 2008-03-04 Interface to convert mental states and facial expressions to application input WO2008109619A2 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US11/682,300 US20080218472A1 (en) 2007-03-05 2007-03-05 Interface to convert mental states and facial expressions to application input
US11/682,300 2007-03-05

Publications (2)

Publication Number Publication Date
WO2008109619A2 WO2008109619A2 (en) 2008-09-12
WO2008109619A3 true WO2008109619A3 (en) 2008-10-30

Family

ID=39739071

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2008/055827 WO2008109619A2 (en) 2007-03-05 2008-03-04 Interface to convert mental states and facial expressions to application input

Country Status (3)

Country Link
US (1) US20080218472A1 (en)
TW (1) TW200844797A (en)
WO (1) WO2008109619A2 (en)

Families Citing this family (110)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20060257834A1 (en) * 2005-05-10 2006-11-16 Lee Linda M Quantitative EEG as an identifier of learning modality
WO2006122398A1 (en) * 2005-05-16 2006-11-23 Cerebral Diagnostics Canada Incorporated Near-real time three-dimensional localization, display , recording , and analysis of electrical activity in the cerebral cortex
US9351658B2 (en) 2005-09-02 2016-05-31 The Nielsen Company (Us), Llc Device and method for sensing electrical activity in tissue
US20090070798A1 (en) * 2007-03-02 2009-03-12 Lee Hans C System and Method for Detecting Viewer Attention to Media Delivery Devices
US9215996B2 (en) 2007-03-02 2015-12-22 The Nielsen Company (Us), Llc Apparatus and method for objectively determining human response to media
US8230457B2 (en) * 2007-03-07 2012-07-24 The Nielsen Company (Us), Llc. Method and system for using coherence of biological responses as a measure of performance of a media
US8473044B2 (en) 2007-03-07 2013-06-25 The Nielsen Company (Us), Llc Method and system for measuring and ranking a positive or negative response to audiovisual or interactive media, products or activities using physiological signals
US8764652B2 (en) * 2007-03-08 2014-07-01 The Nielson Company (US), LLC. Method and system for measuring and ranking an “engagement” response to audiovisual or interactive media, products, or activities using physiological signals
US8782681B2 (en) 2007-03-08 2014-07-15 The Nielsen Company (Us), Llc Method and system for rating media and events in media based on physiological data
US20090030717A1 (en) 2007-03-29 2009-01-29 Neurofocus, Inc. Intra-modality synthesis of central nervous system, autonomic nervous system, and effector data
WO2008137581A1 (en) 2007-05-01 2008-11-13 Neurofocus, Inc. Neuro-feedback based stimulus compression device
US8392253B2 (en) 2007-05-16 2013-03-05 The Nielsen Company (Us), Llc Neuro-physiology and neuro-behavioral based stimulus targeting system
JP5542051B2 (en) 2007-07-30 2014-07-09 ニューロフォーカス・インコーポレーテッド System, method, and apparatus for performing neural response stimulation and stimulation attribute resonance estimation
US8386313B2 (en) 2007-08-28 2013-02-26 The Nielsen Company (Us), Llc Stimulus placement system using subject neuro-response measurements
US8392255B2 (en) 2007-08-29 2013-03-05 The Nielsen Company (Us), Llc Content based selection and meta tagging of advertisement breaks
US8376952B2 (en) * 2007-09-07 2013-02-19 The Nielsen Company (Us), Llc. Method and apparatus for sensing blood oxygen
US20090083129A1 (en) 2007-09-20 2009-03-26 Neurofocus, Inc. Personalized content delivery using neuro-response priming data
US8151292B2 (en) * 2007-10-02 2012-04-03 Emsense Corporation System for remote access to media, and reaction and survey data from viewers of the media
WO2009059246A1 (en) 2007-10-31 2009-05-07 Emsense Corporation Systems and methods providing en mass collection and centralized processing of physiological responses from viewers
US20090150919A1 (en) * 2007-11-30 2009-06-11 Lee Michael J Correlating Media Instance Information With Physiological Responses From Participating Subjects
US8347326B2 (en) 2007-12-18 2013-01-01 The Nielsen Company (US) Identifying key media events and modeling causal relationships between key events and reported feelings
US20090299960A1 (en) * 2007-12-21 2009-12-03 Lineberger William B Methods, systems, and computer program products for automatically modifying a virtual environment based on user profile information
US20090310290A1 (en) * 2008-06-11 2009-12-17 Tennent James Wearable display media
US20090310187A1 (en) * 2008-06-12 2009-12-17 Harris Scott C Face Simulation in Networking
US8326408B2 (en) * 2008-06-18 2012-12-04 Green George H Method and apparatus of neurological feedback systems to control physical objects for therapeutic and other reasons
BRPI0920821B8 (en) 2008-10-01 2021-06-22 Hua Sherwin bone stabilization system and screw for use in bone stabilization
US8032628B2 (en) 2008-11-21 2011-10-04 The Invention Science Fund I, Llc Soliciting data indicating at least one objective occurrence in response to acquisition of data indicating at least one subjective user state
US8028063B2 (en) * 2008-11-21 2011-09-27 The Invention Science Fund I, Llc Soliciting data indicating at least one objective occurrence in response to acquisition of data indicating at least one subjective user state
US8260729B2 (en) * 2008-11-21 2012-09-04 The Invention Science Fund I, Llc Soliciting data indicating at least one subjective user state in response to acquisition of data indicating at least one objective occurrence
US8260912B2 (en) * 2008-11-21 2012-09-04 The Invention Science Fund I, Llc Hypothesis based solicitation of data indicating at least one subjective user state
US8224842B2 (en) * 2008-11-21 2012-07-17 The Invention Science Fund I, Llc Hypothesis selection and presentation of one or more advisories
US8180890B2 (en) * 2008-11-21 2012-05-15 The Invention Science Fund I, Llc Hypothesis based solicitation of data indicating at least one subjective user state
US20100131334A1 (en) * 2008-11-21 2010-05-27 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Hypothesis development based on selective reported events
US8180830B2 (en) * 2008-11-21 2012-05-15 The Invention Science Fund I, Llc Action execution based on user modified hypothesis
US8046455B2 (en) * 2008-11-21 2011-10-25 The Invention Science Fund I, Llc Correlating subjective user states with objective occurrences associated with a user
US8103613B2 (en) * 2008-11-21 2012-01-24 The Invention Science Fund I, Llc Hypothesis based solicitation of data indicating at least one objective occurrence
US8010663B2 (en) * 2008-11-21 2011-08-30 The Invention Science Fund I, Llc Correlating data indicating subjective user states associated with multiple users with data indicating objective occurrences
US8005948B2 (en) * 2008-11-21 2011-08-23 The Invention Science Fund I, Llc Correlating subjective user states with objective occurrences associated with a user
US8224956B2 (en) 2008-11-21 2012-07-17 The Invention Science Fund I, Llc Hypothesis selection and presentation of one or more advisories
US7937465B2 (en) * 2008-11-21 2011-05-03 The Invention Science Fund I, Llc Correlating data indicating at least one subjective user state with data indicating at least one objective occurrence associated with a user
US8127002B2 (en) 2008-11-21 2012-02-28 The Invention Science Fund I, Llc Hypothesis development based on user and sensing device data
US8086668B2 (en) 2008-11-21 2011-12-27 The Invention Science Fund I, Llc Hypothesis based solicitation of data indicating at least one objective occurrence
US8239488B2 (en) 2008-11-21 2012-08-07 The Invention Science Fund I, Llc Hypothesis development based on user and sensing device data
US7945632B2 (en) 2008-11-21 2011-05-17 The Invention Science Fund I, Llc Correlating data indicating at least one subjective user state with data indicating at least one objective occurrence associated with a user
US8244858B2 (en) * 2008-11-21 2012-08-14 The Invention Science Fund I, Llc Action execution based on user modified hypothesis
US8010662B2 (en) * 2008-11-21 2011-08-30 The Invention Science Fund I, Llc Soliciting data indicating at least one subjective user state in response to acquisition of data indicating at least one objective occurrence
US20100177116A1 (en) * 2009-01-09 2010-07-15 Sony Ericsson Mobile Communications Ab Method and arrangement for handling non-textual information
US20100250325A1 (en) 2009-03-24 2010-09-30 Neurofocus, Inc. Neurological profiles for market matching and stimulus presentation
US8493286B1 (en) 2009-04-21 2013-07-23 Mark T. Agrama Facial movement measurement and stimulation apparatus and method
US10987015B2 (en) 2009-08-24 2021-04-27 Nielsen Consumer Llc Dry electrodes for electroencephalography
WO2011028307A1 (en) 2009-09-01 2011-03-10 Exxonmobil Upstream Research Company Method of using human physiological responses as inputs to hydrocarbon management decisions
US8121618B2 (en) 2009-10-28 2012-02-21 Digimarc Corporation Intuitive computing methods and systems
US8175617B2 (en) 2009-10-28 2012-05-08 Digimarc Corporation Sensor-based mobile search, related methods and systems
US20110106750A1 (en) 2009-10-29 2011-05-05 Neurofocus, Inc. Generating ratings predictions using neuro-response data
US9560984B2 (en) 2009-10-29 2017-02-07 The Nielsen Company (Us), Llc Analysis of controlled and automatic attention for introduction of stimulus material
KR20130004253A (en) 2009-12-21 2013-01-09 셔윈 화 Insertion of medical devices through non-orthogonal and orthogonal trajectories within the cranium and methods of using
WO2011133548A2 (en) 2010-04-19 2011-10-27 Innerscope Research, Inc. Short imagery task (sit) research method
US8655428B2 (en) 2010-05-12 2014-02-18 The Nielsen Company (Us), Llc Neuro-response data synchronization
US9723992B2 (en) * 2010-06-07 2017-08-08 Affectiva, Inc. Mental state analysis using blink rate
US10074024B2 (en) 2010-06-07 2018-09-11 Affectiva, Inc. Mental state analysis using blink rate for vehicles
WO2012100081A2 (en) * 2011-01-19 2012-07-26 California Institute Of Technology Aggregation of bio-signals from multiple individuals to achieve a collective outcome
US8760551B2 (en) 2011-03-02 2014-06-24 Canon Kabushiki Kaisha Systems and methods for image capturing based on user interest
US9013264B2 (en) 2011-03-12 2015-04-21 Perceptive Devices, Llc Multipurpose controller for electronic devices, facial expressions management and drowsiness detection
WO2013064914A1 (en) * 2011-10-31 2013-05-10 Sony Ericsson Mobile Communications Ab Amplifying audio-visual data based on user's head orientation
US9292858B2 (en) 2012-02-27 2016-03-22 The Nielsen Company (Us), Llc Data collection system for aggregating biologically based measures in asynchronous geographically distributed public environments
US9569986B2 (en) 2012-02-27 2017-02-14 The Nielsen Company (Us), Llc System and method for gathering and analyzing biometric user feedback for use in social media and advertising applications
US9451303B2 (en) 2012-02-27 2016-09-20 The Nielsen Company (Us), Llc Method and system for gathering and computing an audience's neurologically-based reactions in a distributed framework involving remote storage and computing
US20130243270A1 (en) * 2012-03-16 2013-09-19 Gila Kamhi System and method for dynamic adaption of media based on implicit user input and behavior
US9814426B2 (en) 2012-06-14 2017-11-14 Medibotics Llc Mobile wearable electromagnetic brain activity monitor
US10130277B2 (en) 2014-01-28 2018-11-20 Medibotics Llc Willpower glasses (TM)—a wearable food consumption monitor
GB201211703D0 (en) 2012-07-02 2012-08-15 Charles Nduka Plastic Surgery Ltd Biofeedback system
US9060671B2 (en) 2012-08-17 2015-06-23 The Nielsen Company (Us), Llc Systems and methods to gather and analyze electroencephalographic data
US9015087B2 (en) * 2012-10-09 2015-04-21 At&T Intellectual Property I, L.P. Methods, systems, and products for interfacing with neurological and biological networks
TWI582708B (en) * 2012-11-22 2017-05-11 緯創資通股份有限公司 Facial expression control system, facial expression control method, and computer system thereof
US9320450B2 (en) 2013-03-14 2016-04-26 The Nielsen Company (Us), Llc Methods and apparatus to gather and analyze electroencephalographic data
US8918339B2 (en) * 2013-03-15 2014-12-23 Facebook, Inc. Associating an indication of user emotional reaction with content items presented by a social networking system
US9383819B2 (en) * 2013-06-03 2016-07-05 Daqri, Llc Manipulation of virtual object in augmented reality via intent
US9354702B2 (en) 2013-06-03 2016-05-31 Daqri, Llc Manipulation of virtual object in augmented reality via thought
WO2015023952A1 (en) * 2013-08-16 2015-02-19 Affectiva, Inc. Mental state analysis using an application programming interface
CN104750241B (en) * 2013-12-26 2018-10-02 财团法人工业技术研究院 Head-mounted device and related simulation system and simulation method thereof
US9311639B2 (en) 2014-02-11 2016-04-12 Digimarc Corporation Methods, apparatus and arrangements for device to device communication
US10275583B2 (en) * 2014-03-10 2019-04-30 FaceToFace Biometrics, Inc. Expression recognition in messaging systems
US9817960B2 (en) 2014-03-10 2017-11-14 FaceToFace Biometrics, Inc. Message sender security in messaging system
US9622702B2 (en) 2014-04-03 2017-04-18 The Nielsen Company (Us), Llc Methods and apparatus to gather and analyze electroencephalographic data
JP6641298B2 (en) 2014-05-30 2020-02-05 ザ リージェンツ オブ ザ ユニバーシティー オブ ミシガン Brain computer interface to facilitate direct selection of multiple choice answers and identification of state changes
US9778736B2 (en) * 2014-09-22 2017-10-03 Rovi Guides, Inc. Methods and systems for calibrating user devices
US9936250B2 (en) 2015-05-19 2018-04-03 The Nielsen Company (Us), Llc Methods and apparatus to adjust content presented to an individual
JP6742628B2 (en) * 2016-08-10 2020-08-19 国立大学法人広島大学 Brain islet cortex activity extraction method
KR20180044685A (en) * 2016-10-24 2018-05-03 엘지전자 주식회사 Head mounted display device
US10943100B2 (en) 2017-01-19 2021-03-09 Mindmaze Holding Sa Systems, methods, devices and apparatuses for detecting facial expression
US20190025919A1 (en) * 2017-01-19 2019-01-24 Mindmaze Holding Sa System, method and apparatus for detecting facial expression in an augmented reality system
US10515474B2 (en) 2017-01-19 2019-12-24 Mindmaze Holding Sa System, method and apparatus for detecting facial expression in a virtual reality system
CN110300945A (en) * 2017-02-08 2019-10-01 赛伯有限公司 In order to which the motion transform of the personnel of use device to be detected to the device of movement into Virtual Space
US10862927B2 (en) 2017-05-15 2020-12-08 Forcepoint, LLC Dividing events into sessions during adaptive trust profile operations
US10447718B2 (en) * 2017-05-15 2019-10-15 Forcepoint Llc User profile definition and management
US10999296B2 (en) 2017-05-15 2021-05-04 Forcepoint, LLC Generating adaptive trust profiles using information derived from similarly situated organizations
US10917423B2 (en) 2017-05-15 2021-02-09 Forcepoint, LLC Intelligently differentiating between different types of states and attributes when using an adaptive trust profile
US10129269B1 (en) 2017-05-15 2018-11-13 Forcepoint, LLC Managing blockchain access to user profile information
US9882918B1 (en) 2017-05-15 2018-01-30 Forcepoint, LLC User behavior profile in a blockchain
US10915643B2 (en) 2017-05-15 2021-02-09 Forcepoint, LLC Adaptive trust profile endpoint architecture
US10999297B2 (en) 2017-05-15 2021-05-04 Forcepoint, LLC Using expected behavior of an entity when prepopulating an adaptive trust profile
US10623431B2 (en) 2017-05-15 2020-04-14 Forcepoint Llc Discerning psychological state from correlated user behavior and contextual information
CN111629653A (en) 2017-08-23 2020-09-04 神经股份有限公司 Brain-computer interface with high speed eye tracking features
US11328533B1 (en) 2018-01-09 2022-05-10 Mindmaze Holding Sa System, method and apparatus for detecting facial expression for motion capture
WO2019157360A1 (en) 2018-02-09 2019-08-15 Starkey Laboratories, Inc. Use of periauricular muscle signals to estimate a direction of a user's auditory attention locus
US10664050B2 (en) 2018-09-21 2020-05-26 Neurable Inc. Human-computer interface using high-speed and accurate tracking of user interactions
US11160580B2 (en) 2019-04-24 2021-11-02 Spine23 Inc. Systems and methods for pedicle screw stabilization of spinal vertebrae
US10997295B2 (en) 2019-04-26 2021-05-04 Forcepoint, LLC Adaptive trust profile reference architecture
US11553871B2 (en) 2019-06-04 2023-01-17 Lab NINE, Inc. System and apparatus for non-invasive measurement of transcranial electrical signals, and method of calibrating and/or using same for various applications
US11960784B2 (en) * 2021-12-07 2024-04-16 Snap Inc. Shared augmented reality unboxing experience

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010056225A1 (en) * 1995-08-02 2001-12-27 Devito Drew Method and apparatus for measuring and analyzing physiological signals for active or passive control of physical and virtual spaces and the contents therein
US20040138578A1 (en) * 2002-07-25 2004-07-15 Pineda Jaime A. Method and system for a real time adaptive system for effecting changes in cognitive-emotive profiles

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070060830A1 (en) * 2005-09-12 2007-03-15 Le Tan Thi T Method and system for detecting and classifying facial muscle movements
CN101331490A (en) * 2005-09-12 2008-12-24 埃默迪弗系统股份有限公司 Detection of and interaction using mental states
US7865235B2 (en) * 2005-09-12 2011-01-04 Tan Thi Thai Le Method and system for detecting and classifying the mental state of a subject

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20010056225A1 (en) * 1995-08-02 2001-12-27 Devito Drew Method and apparatus for measuring and analyzing physiological signals for active or passive control of physical and virtual spaces and the contents therein
US20040138578A1 (en) * 2002-07-25 2004-07-15 Pineda Jaime A. Method and system for a real time adaptive system for effecting changes in cognitive-emotive profiles

Also Published As

Publication number Publication date
US20080218472A1 (en) 2008-09-11
TW200844797A (en) 2008-11-16
WO2008109619A2 (en) 2008-09-12

Similar Documents

Publication Publication Date Title
WO2008109619A3 (en) Interface to convert mental states and facial expressions to application input
WO2008001202A3 (en) Touchless gesture based input
WO2010068447A3 (en) Financial gadgets
WO2013126905A3 (en) Gesture recognition devices and methods
WO2009032856A3 (en) Customized today module
WO2010033461A3 (en) Alzheimer's cognitive enabler
WO2010018577A3 (en) Data entry system
WO2008070054A3 (en) Medical device including a slider assembly
WO2005121938A3 (en) Input system
WO2007030503A3 (en) Graphical user interfaces
WO2010126841A3 (en) Altering a view perspective within a display environment
WO2009134927A3 (en) Business software application system and method
WO2014093525A8 (en) Wearable multi-modal input device for augmented reality
WO2010093780A3 (en) Communication and skills training using interactive virtual humans
WO2014028529A3 (en) Maintaining a discrete data representation that corresponds to information contained in free-form text
WO2010137859A3 (en) Apparatus and method for unlocking a locking mode of portable terminal
WO2010014845A3 (en) Multi-purpose detector-based input feature for a computing device
WO2010065752A3 (en) Providing selected data through a locked display
WO2010018579A3 (en) Improved data entry system
WO2008123247A1 (en) Information processing device and its control method
WO2011017658A3 (en) Document layout system
EP2472341A3 (en) Electronic device and method providing improved indication that an alarm clock is in an on condition
WO2007127166A3 (en) Systems and methods for generating media content using microtrends
WO2012045017A3 (en) Choosing recognized text from a background environment
WO2007076509A3 (en) Generation of names related to organization actions

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 08731376

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 08731376

Country of ref document: EP

Kind code of ref document: A2