US20130293580A1 - System and method for selecting targets in an augmented reality environment - Google Patents

System and method for selecting targets in an augmented reality environment Download PDF

Info

Publication number
US20130293580A1
US20130293580A1 US13/874,421 US201313874421A US2013293580A1 US 20130293580 A1 US20130293580 A1 US 20130293580A1 US 201313874421 A US201313874421 A US 201313874421A US 2013293580 A1 US2013293580 A1 US 2013293580A1
Authority
US
United States
Prior art keywords
user
mobile device
augmented reality
movements
processor
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US13/874,421
Inventor
Nova T. Spivack
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Augmented Reality Holdings 2 LLC
Original Assignee
Zambala LLLP
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zambala LLLP filed Critical Zambala LLLP
Priority to US13/874,421 priority Critical patent/US20130293580A1/en
Priority to PCT/US2013/039123 priority patent/WO2013166191A2/en
Assigned to ZAMBALA LLLP reassignment ZAMBALA LLLP ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: SPIVACK, NOVA T.
Publication of US20130293580A1 publication Critical patent/US20130293580A1/en
Priority to US14/939,889 priority patent/US10388070B2/en
Assigned to AUGMENTED REALITY HOLDINGS, LLC reassignment AUGMENTED REALITY HOLDINGS, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: ZAMBALA, LLLP
Assigned to AUGMENTED REALITY HOLDINGS 2, LLC reassignment AUGMENTED REALITY HOLDINGS 2, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: AUGMENTED REALITY HOLDINGS, LLC
Priority to US16/543,011 priority patent/US10878636B2/en
Priority to US17/101,429 priority patent/US11417066B2/en
Priority to US17/887,075 priority patent/US20220392178A1/en
Abandoned legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06TIMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00Manipulating 3D models or images for computer graphics
    • G06T19/006Mixed reality
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/013Eye tracking input arrangements
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017Gesture based interaction, e.g. based on a set of recognized hand gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/048Interaction techniques based on graphical user interfaces [GUI]
    • G06F3/0487Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
    • G06F3/0488Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0623Item investigation
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0639Item locations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0641Shopping interfaces
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06QINFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES; SYSTEMS OR METHODS SPECIALLY ADAPTED FOR ADMINISTRATIVE, COMMERCIAL, FINANCIAL, MANAGERIAL OR SUPERVISORY PURPOSES, NOT OTHERWISE PROVIDED FOR
    • G06Q30/00Commerce
    • G06Q30/06Buying, selling or leasing transactions
    • G06Q30/0601Electronic shopping [e-shopping]
    • G06Q30/0641Shopping interfaces
    • G06Q30/0643Graphical representation of items or shoppers
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/20Scenes; Scene-specific elements in augmented reality scenes
    • GPHYSICS
    • G09EDUCATION; CRYPTOGRAPHY; DISPLAY; ADVERTISING; SEALS
    • G09GARRANGEMENTS OR CIRCUITS FOR CONTROL OF INDICATING DEVICES USING STATIC MEANS TO PRESENT VARIABLE INFORMATION
    • G09G5/00Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators
    • G09G5/36Control arrangements or circuits for visual indicators common to cathode-ray tube indicators and other visual indicators characterised by the display of a graphic pattern, e.g. using an all-points-addressable [APA] memory
    • G09G5/37Details of the operation on graphic patterns
    • G09G5/377Details of the operation on graphic patterns for mixing or overlaying two or more graphic patterns
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01Head-up displays
    • G02B27/017Head mounted
    • G02B2027/0178Eyeglass type

Definitions

  • portable electronics or other electronics devices now generally include a large variety of sensing capabilities. These capabilities can be utilized to further enhance the users' augmented reality experiences.
  • FIG. 1 illustrates an example block diagram of mobile devices able to communicate with a host server that generates and controls access to simulated objects and/or facilitates transactions of a physical product or real life service via an augmented reality environment (e.g., an augmented reality marketplace) through a network;
  • a host server that generates and controls access to simulated objects and/or facilitates transactions of a physical product or real life service via an augmented reality environment (e.g., an augmented reality marketplace) through a network;
  • an augmented reality environment e.g., an augmented reality marketplace
  • FIG. 2 depicts an example block diagram of the components of a host server that generates information that facilitates transactions of a physical product or real life service via an augmented reality environment;
  • FIG. 3A depicts an example functional block diagram of the host server that facilitates transactions of a physical product or real life service via an augmented reality environment
  • FIG. 3B depicts an example block diagram illustrating the components of the host server that facilitates transactions of a physical product or real life service via an augmented reality environment;
  • FIG. 4A depicts an example functional block diagram of a mobile device that presents augmented information with merchandise objects to a user and processes interactions with the objects in the augmented reality marketplace based on the information;
  • FIG. 4B depicts an example block diagram of the mobile device that presents augmented information with merchandise objects in the augmented reality marketplace to a user and facilitates user interactions with the objects based on the information;
  • FIG. 5A illustrates an example application where mobile devices such as a smart phone or a smart goggle having perspective interfaces which are used to facilitate electronic commerce in an augmented reality marketplace;
  • FIG. 5B illustrates an example screenshot on a mobile device displaying an augmented reality environment with objects associated with physical objects and/or real life services in a real world environment;
  • FIG. 6 depicts a flow chart illustrating an example process for facilitating user interaction with a merchandise object that is identified in an augmented reality environment
  • FIG. 7 depicts a flow chart illustrating an example process for selecting/identifying a merchandise object in an augmented reality environment.
  • FIG. 8 depicts a diagrammatic representation of a machine in the example form of a computer system within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed, according to some embodiments.
  • a method comprises detecting, by a mobile device, presence of the physical product or the real life service; and presenting, on the mobile device, information to conduct the transaction of a physical product or a real life service via an augmented reality environment.
  • the mobile device facilitates the augmented reality environment.
  • a method comprises detecting one or more targets in the augmented reality platform using a select area in a perspective of a user, the perspective being captured by a mobile device; and prompting the user to choose an object of interest from the one or more detected targets.
  • Embodiments of the present disclosure include systems and methods for facilitating transactions of a physical product or real life service via an augmented reality environment. Additionally, embodiments of the present disclosure include systems and methods for enabling selection of a physical product or a real life service in an augmented reality platform via a mobile device.
  • FIG. 1 illustrates an example block diagram of mobile devices 102 A-N able to communicate with a host server 124 that generates and controls access to simulated objects and/or facilitates transactions of a physical product or real life service via an augmented reality environment through a network 106 .
  • the mobile devices 102 A-N can be any system and/or device, and/or any combination of devices/systems that is able to establish a connection with another device, a server and/or other systems.
  • the mobile devices 102 A-N typically include a display and/or other output functionalities to present information and data exchanged between/among the devices 102 A-N and the host server 124 .
  • the mobile devices 102 A-N can be any of, but are not limited to, a server desktop, a desktop computer, a computer cluster, or portable devices including, a notebook, a laptop computer, a handheld computer, a palmtop computer, a mobile phone, a cell phone, a smart phone, a PDA, a Blackberry device, a Treo, an iPhone, cover headsets, heads-up displays, helmet mounted display, head-mounted display, scanned-beam display, wearable computer such as mobile enabled watches, and/or any other mobile interfaces and viewing devices, etc.
  • a server desktop a desktop computer, a computer cluster, or portable devices including, a notebook, a laptop computer, a handheld computer, a palmtop computer, a mobile phone, a cell phone, a smart phone, a PDA, a Blackberry device, a Treo, an iPhone, cover headsets, heads-up displays, helmet mounted display, head-mounted display, scanned-beam display, wearable computer such as mobile enabled watches, and/or any
  • mobile devices 102 A-N can include a Google Glass, smart glasses/goggles, a smart phone, or other suitable mobile computing devices that are configured to facilitate or enable an augmented reality environment or platform.
  • the mobile devices 102 A-N may be location-aware devices that are able to determine their own location or identify location information from an external source.
  • the mobile devices 102 A-N are coupled to a network 106 .
  • the devices 102 A-N and host server 124 may be directly connected to one another.
  • the host server 124 is operable to provide simulated objects (e.g., objects, computer-controlled objects, or simulated objects) that correspond to real world physical locations to be presented to users on mobile devices 102 A-N.
  • the simulated objects are typically software entities or occurrences that are controlled by computer programs and can be generated upon request when certain criteria are met.
  • the host server 124 also processes interactions of simulated object with one another and actions on simulated objects caused by stimulus from a real user and/or the real world environment.
  • the host server 124 is operable, in conjunction with the mobile devices 102 A-N, to facilitate electronic commercial transactions in an augmented reality environment or platform, such as an augmented reality marketplace. Services and functions provided by the host server 124 and the components therein are described in detail with further references to the examples of FIG. 3A-3B .
  • the mobile devices 102 A-N are generally operable to provide access (e.g., visible access, audible access) to the simulated objects to users, for example via user interface 104 A-N displayed on the display units.
  • the devices 102 A-N may be able to detect simulated objects based on location and/or timing data and provide those objects authorized by the user for access via the devices.
  • the mobile devices 102 A-N can facilitate electronic commercial transactions in an augmented reality environment.
  • the devices 102 A-N communicate with the host server 124 in facilitating the transactions. Services and functions provided by the mobile devices 102 A-N and the components therein are described in detail with further references to the examples of FIG. 4A-4B .
  • the network 106 over which the mobile devices 102 A-N and the host server 124 communicate, may be a telephonic network, an open network, such as the Internet, or a private network, such as an intranet and/or the extranet.
  • the Internet can provide file transfer, remote log in, email, news, RSS, and other services through any known or convenient protocol, such as, but is not limited to the TCP/IP protocol, Open System Interconnections (OSI), FTP, UPnP, iSCSI, NSF, ISDN, PDH, RS-232, SDH, SONET, etc.
  • OSI Open System Interconnections
  • the network 106 can be any collection of distinct networks operating wholly or partially in conjunction to provide connectivity to the mobile devices 102 A-N and the host server 124 and may appear as one or more networks to the serviced systems and devices.
  • communications to and from the mobile devices 102 A-N can be achieved by, an open network, such as the Internet, or a private network, such as an intranet and/or the extranet.
  • communications can be achieved by a secure communications protocol, such as secure sockets layer (SSL), or transport layer security (TLS).
  • SSL secure sockets layer
  • TLS transport layer security
  • communications can be achieved via one or more wireless networks, such as, but is not limited to, one or more of a Local Area Network (LAN), Wireless Local Area Network (WLAN), a Personal area network (PAN), a Campus area network (CAN), a Metropolitan area network (MAN), a Wide area network (WAN), a Wireless wide area network (WWAN), Global System for Mobile Communications (GSM), Personal Communications Service (PCS), Digital Advanced Mobile Phone Service (D-Amps), Bluetooth, Wi-Fi, Fixed Wireless Data, 2G, 2.5G, 3G (e.g., WCDMA/UTMS based 3G networks), 3.5G, 4G, IMT-Advanced, pre-4G, LTE Advanced, mobile WiMax, WiMax 2, WirelessMAN-Advanced networks, enhanced data rates for GSM evolution (EDGE), General packet radio service (GPRS), enhanced GPRS, iBurst, UMTS, HSPDA, HSPA, HSPA+, UMTS-TDD, 1xRTT, EV-DO, messaging protocols such as, T
  • the host server 124 may include or be coupled to a user repository 128 and/or a simulated object repository 130 and or a merchandise object repository 132 .
  • the data repositories 128 , 130 , 132 can store software, descriptive data, images, system information, drivers, and/or any other data item utilized by other components of the host server 124 and/or any other servers for operation.
  • the data repositories 128 , 130 , 132 may be managed by a database management system (DBMS), for example but not limited to, Oracle, DB2, Microsoft Access, Microsoft SQL Server, PostgreSQL, MySQL, FileMaker, etc.
  • DBMS database management system
  • the repositories 128 , 130 , 132 can be implemented via object-oriented technology and/or via text files, and can be managed by a distributed database management system, an object-oriented database management system (OODBMS) (e.g., ConceptBase, FastDB Main Memory Database Management System, JDOInstruments, ObjectDB, etc.), an object-relational database management system (ORDBMS) (e.g., Informix, OpenLink Virtuoso, VMDS, etc.), a file system, and/or any other convenient or known database management package.
  • OODBMS object-oriented database management system
  • ORDBMS object-relational database management system
  • the host server 124 is able to provide data to be stored in the user data repository 128 , the simulated object repository 130 , the merchandise object repository 132 , and/or the can retrieve data stored in the user data repository 128 and/or the simulated object repository 130 and/or the merchandise object repository 132 .
  • the user data repository 128 can store user information, user preferences, user profile, user list of interest, user favorite, user bookmark, user annotation, user tags, access permissions associated with the users, device information, hardware information, etc.
  • the simulated object repository 130 can store software entities (e.g., computer programs) that control simulated objects and the simulated environments in which they are presented for visual/audible access or control/manipulation.
  • the simulated object repository 130 may further include simulated objects and their associated data structures with metadata defining the simulated object including its associated access permission.
  • the merchandise object repository 132 can store merchandise information including, for example, merchandise name, alias, attributes, shape, size, dimension, or other physical characteristics or recognition patterns or augmented reality markers, location (if suitable), vendor(s), price, SKU code, QR code, 1-dimensional or 2-dimensional or multi-dimensional barcode, RFID code, metadata, tags, annotations, sales contact information.
  • merchandise information including, for example, merchandise name, alias, attributes, shape, size, dimension, or other physical characteristics or recognition patterns or augmented reality markers, location (if suitable), vendor(s), price, SKU code, QR code, 1-dimensional or 2-dimensional or multi-dimensional barcode, RFID code, metadata, tags, annotations, sales contact information.
  • FIG. 2 depicts an example block diagram of the components of a host server 224 that generates information that facilitates transactions of a physical product or real life service via an augmented reality environment.
  • the host server 224 includes a network controller 202 , a firewall 204 , a multimedia server 206 , a web application server 212 , an electronic commerce (e-commerce) server 213 , and a database including a database storage 216 and database software 218 .
  • the network controller 202 can be a networking device that enables the host server 224 to mediate data in a network with an entity that is external to the host server 224 , through any known and/or convenient communications protocol supported by the host and the external entity.
  • the network controller 202 can include one or more of a network adaptor card, a wireless network interface card, a router, an access point, a wireless router, a switch, a multilayer switch, a protocol converter, a gateway, a bridge, bridge router, a hub, a digital media receiver, and/or a repeater.
  • the firewall 204 can, in some embodiments, govern and/or manage permission to access/proxy data in a computer network, and track varying levels of trust between different machines and/or applications.
  • the firewall 204 can be any number of modules having any combination of hardware and/or software components able to enforce a predetermined set of access rights between a particular set of machines and applications, machines and machines, and/or applications and applications, for example, to regulate the flow of traffic and resource sharing between these varying entities.
  • the firewall 204 may additionally manage and/or have access to an access control list which details permissions including for example, the access and operation rights of an object by an individual, a machine, and/or an application, and the circumstances under which the permission rights stand.
  • firewall 204 can be, for example, but are not limited to, intrusion-prevention, intrusion detection, next-generation firewall, personal firewall, etc. without deviating from the novel art of this disclosure.
  • the functionalities of the network controller 202 and the firewall 204 are partially or wholly combined and the functions of which can be implemented in any combination of software and/or hardware, in part or in whole.
  • the host server 224 includes the multimedia server 206 or a combination of multimedia servers to manage images, photographs, animation, video, audio content, graphical content, documents, and/or other types of multimedia data for use in or to supplement simulated content such as simulated objects and their associated deployment environment (e.g., a simulated environment).
  • the multimedia server 206 is any software suitable for delivering messages to facilitate retrieval/transmission of multimedia data among servers to be provided to other components and/or systems of the host server 224 , for example when rendering a web page, an augmented reality environment, and/or augmenting physical objects using multimedia content (e.g., the augmented information includes a short video clip introducing physical products or real life services).
  • the multimedia server 206 can facilitate transmission/receipt of streaming data such as streaming images, audio, and/or video.
  • the multimedia server 206 can be configured separately or together with the web application server 212 , depending on a desired scalability of the host server 224 .
  • Examples of graphics file formats that can be managed by the multimedia server 206 include but are not limited to, ADRG, ADRI, AI, GIF, IMA, GS, JPG, JP2, PNG, PSD, PSP, TIFF, and/or BMP, etc.
  • the web application server 212 can include any combination of software agents and/or hardware modules for accepting Hypertext Transfer Protocol (HTTP) requests from end users, external systems, and/or external mobile devices and responding to the request by providing the requestors with web pages, such as HTML documents and objects that can include static and/or dynamic content (e.g., via one or more supported interfaces, such as the Common Gateway Interface (CGI), Simple CGI (SCGI), PHP, JavaServer Pages (JSP), Active Server Pages (ASP), ASP.NET, etc.).
  • CGI Common Gateway Interface
  • SCGI Simple CGI
  • JSP JavaServer Pages
  • ASP Active Server Pages
  • ASP ASP.NET
  • SSL and/or TLS can be established by the web application server 212 .
  • the e-commerce server 213 can include any combination of software agents and/or hardware modules for facilitating electronic commercial transactions to be initiated, processed, and completed with mobile devices (e.g., devices 102 A-N, FIG. 1 ).
  • the e-commerce server 213 typically includes at least one processor and a memory, and may be further connected to one or more computers (not shown in FIG. 2 for simplicity) that manage inventory, logistics and/or other commercial functions.
  • the e-commerce server 213 can establish secured connections using one or more well-known security protocols.
  • the host server 224 assists, by interactively or proactively providing relevant information, mobile devices such as device 102 A-N to generate the augmented reality user interfaces, which facilitate the augmented reality marketplace disclosed herein, as shown in the example screenshots of FIG. 5B-FIG . 5 C.
  • the augmented information which is provided by the host server 224 to client users/end devices (e.g., onto the user interface screens 104 A- 104 N on mobile devices 102 A- 102 N for display).
  • the web application server 212 and/or the e-commerce server 213 also perform authentication processes before responding to requests for conducting transactions in the augmented reality marketplace.
  • the databases 216 , 218 can store software, descriptive data, images, system information, drivers, and/or any other data item utilized by other components of the host server for operation.
  • the databases 216 , 218 may be managed by a database management system (DBMS), for example but not limited to, Oracle, DB2, Microsoft Access, Microsoft SQL Server, PostgreSQL, MySQL, FileMaker, etc.
  • DBMS database management system
  • the databases 216 , 218 can be implemented via object-oriented technology and/or via text files, and can be managed by a distributed database management system, an object-oriented database management system (OODBMS) (e.g., ConceptBase, FastDB Main Memory Database Management System, JDOInstruments, ObjectDB, etc.), an object-relational database management system (ORDBMS) (e.g., Informix, OpenLink Virtuoso, VMDS, etc.), a file system, and/or any other convenient or known database management package.
  • OODBMS object-oriented database management system
  • ORDBMS object-relational database management system
  • the host server 224 includes components (e.g., a network controller, a firewall, a storage server, a multimedia server, an e-commerce server, and/or a database including a database storage and database software, etc.) coupled to one another and each component is illustrated as being individual and distinct. However, in some embodiments, some or all of the components, and/or the functions represented by each of the components can be combined in any convenient or known manner. Furthermore, the functions represented by the devices can be implemented individually or in any combination thereof, in hardware, software, or a combination of hardware and software.
  • components e.g., a network controller, a firewall, a storage server, a multimedia server, an e-commerce server, and/or a database including a database storage and database software, etc.
  • FIG. 3A depicts an example functional block diagram of the host server 324 that facilitates transactions of a physical product or real life service via an augmented reality environment.
  • the host server 324 includes a network interface 302 , a repository manager 304 , and/or a search engine 322 .
  • the host server 324 is also coupled to a user data repository 328 and/or a merchandise object repository 332 .
  • the user data repository 328 and merchandise object repository 332 are described with further reference to the example of FIG. 1 .
  • the host server 324 can also include a variety of modules including, for example, a pricing management module 312 , a physical characteristics management module 314 , an annotation management module 316 , a metadata management module 318 , an advertisement/coupon management module 320 , and/or a recognition/disambiguation process module 323 .
  • the management modules 312 - 320 can be coupled to the repository manager 304 .
  • the recognition/disambiguation process module 323 can be coupled to the search engine 322 .
  • each module in the example of FIG. 3A can include any number and combination of sub-modules, and systems, implemented with any combination of hardware and/or software modules.
  • the host server 324 although illustrated as comprised of distributed components (physically distributed and/or functionally distributed), could be implemented as a collective element.
  • some or all of the modules, and/or the functions represented by each of the modules can be combined in any convenient or known manner.
  • the functions represented by the modules can be implemented individually or in any combination thereof, partially or wholly, in hardware, software, or a combination of hardware and software.
  • the network interface 302 can be a networking device that enables the host server 324 to mediate data in a network with an entity that is external to the host server, through any known and/or convenient communications protocol supported by the host and the external entity.
  • the network interface 302 can include one or more of a network adaptor card, a wireless network interface card, a router, an access point, a wireless router, a switch, a multilayer switch, a protocol converter, a gateway, a bridge, bridge router, a hub, a digital media receiver, and/or a repeater.
  • the merchandise object repository 332 can be a database that comprises any combination of software agents and/or hardware modules able to store, modify, update, adjust, edit, and/or delete various physical as well as other attributes of a merchandise object (e.g., a physical product, or a real life service).
  • the management functions of these physical and other attributes of merchandise objects can be performed by the repository manager 304 .
  • a merchandise object can include a physical entity or goods (such as a store, a merchant, commercial product, a living object, an inanimate object, etc.) or a real life service (such as a restaurant, a spa, a beauty salon, a medical doctor, an art performance, etc.).
  • merchandise objects can be identified/recognized/detected using the various physical and/or other characteristics of the objects, and information relevant to commercial transactions of the physical product or real life service can be presented/displayed/projected in the augmented reality environment so as to enable the user to conduct such transactions.
  • Characteristics and attributes of merchandise objects that help the identification/recognition/detection can include what can be perceived by users in the augmented reality via a physical device (e.g., a mobile device or device 102 in the example of FIG. 1 ).
  • a physical device e.g., a mobile device or device 102 in the example of FIG. 1
  • they can include merchandise name, alias, attributes, shape, size, dimension, or other physical characteristics or recognition patterns or augmented reality markers, or location (if suitable).
  • They can also include other identification information that can be detected by the device 102 , such as vendor(s) name, SKU code, QR code, 1-dimensional or 2-dimensional or multi-dimensional barcode, RFID code.
  • the host server 324 includes a search engine 322 .
  • the search engine 322 can be any combination of software agents and/or hardware modules able to search, detect, and/or identify merchandise objects.
  • attributes/clues/characteristics observed by the mobile device can be sent to the search engine 322 .
  • the search engine 322 can perform textual or imagery or other suitable types of search to identify the merchandise object.
  • the search engine 322 can search or detect objects either automatically or in response to user request.
  • the user can request access to merchandise objects' information by perform a search request.
  • the search request parameters can include, one or more of, at least a partial name input of the merchandise object from the user, the user's location, the current time or a time period.
  • the search that is performed can automatically detect the merchandise object(s) being viewing by or otherwise available for access to the user in the augmented reality.
  • the merchandise objects are further filtered based on preferences or profiles or interest of the user.
  • the search engine 322 can further utilize the recognition/disambiguation process module 323 to prompt the user to assist in disambiguation of the merchandise object.
  • the module 323 can generate a plurality of choices in forms of name, picture, locational information, and/or description that can help the recognition.
  • the module 323 can prompt the user to enter a partial or a full name.
  • the module 323 can ask the user to take a plurality of pictures of the merchandise object.
  • the host server 324 can access relevant information including, for example, purchase information of the product, price from the vendor of the exact product that the user is viewing in the augmented reality, price from another vendor of the same or similar product, availability of the product, any metadata or tags of the product, annotations or reviews of the product added by another user of the augmented reality environment, images or video clips that are shared by other users or the merchant, and/or sales contact information.
  • relevant information including, for example, purchase information of the product, price from the vendor of the exact product that the user is viewing in the augmented reality, price from another vendor of the same or similar product, availability of the product, any metadata or tags of the product, annotations or reviews of the product added by another user of the augmented reality environment, images or video clips that are shared by other users or the merchant, and/or sales contact information.
  • These information can be generated, maintained and updated using the modules 312 - 320 coupled to the repository manager 304 (e.g., by the merchant, by the user, by another user, or by an administrator of the host server 324
  • the merchandise object is stored in the merchandise object repository 332 using a data structure having metadata.
  • the metadata can properties of the simulated object, location and/or timing parameters (e.g., sale event, availability, store hours, etc.)
  • the location data can be specified with longitude and latitude coordinates, GPS coordinates, and/or relative position.
  • the object is associated with a unique identifier or augmented reality (AR) markers.
  • AR augmented reality
  • the unique identifier or markers may be further associated with a location data structure having a set of location data that includes the qualifying location data for the merchandise object.
  • the metadata includes a link to data from an external source (e.g., the store's or merchant's website on the Internet).
  • this information is automatically provided to the user when he or she arrives at or near the merchandise object's location (e.g., implicit request) via the device.
  • the information is provided upon request by the user (e.g., explicit request).
  • merchandise objects associated with various attractions for example, in a shopping mall or a tourist location in the real world, can be selected by the user (e.g., via input to the device).
  • an advertisement (e.g., an electronic advertisement) that is relevant to the merchandise object is delivered by the advertisement/coupon management module 320 to the user that qualifies one or more criteria.
  • the user may qualify on a basis of a location, identity, and/or a timing parameter.
  • the user may be provided with advertisements of local pizza shops or other late night dining options when the user is driving around town during late night hours when other dining options may not be available.
  • an electronic coupon can be presented to a user using the device based on the merchandise object identification result, for example, when the device is located at the location during a certain period of time that satisfies the criteria. The electronic coupon may be redeemed by the user for conducting transactions of the physical product or the real life service.
  • FIG. 3B depicts an example block diagram illustrating the components of the host server 324 that facilitates transactions of a physical product or real life service via an augmented reality environment.
  • host server 334 includes a network interface 302 , a processing unit 334 , a memory unit 336 , a storage unit 338 , a location sensor 340 , and/or a timing module 342 . Additional or less units or modules may be included.
  • the host server 334 can be any combination of hardware components and/or software agents for generating information for merchandise objects in facilitating transactions in an augmented reality marketplace environment.
  • the network interface 302 has been described in the example of FIG. 3A .
  • One embodiment of the host server 324 further includes a processing unit 334 .
  • the data received from the network interface 302 , location sensor 340 , and/or the timing module 342 can be input to a processing unit 334 .
  • the location sensor 340 can include GPS receivers, RF transceiver, an optical rangefinder, etc.
  • the timing module 342 can include an internal clock, a connection to a time server (via NTP), an atomic clock, a GPS master clock, etc.
  • the processing unit 334 can include one or more processors, CPUs, microcontrollers, FPGAs, ASICs, DSPs, or any combination of the above. Data that is input to the host server 334 can be processed by the processing unit 334 and output to a display and/or output via a wired or wireless connection to an external device, such as a mobile phone, a portable device, a host or server computer by way of a communications component.
  • an external device such as a mobile phone, a portable device, a host or server computer by way of a communications component.
  • One embodiment of the host server 324 further includes a memory unit 336 and a storage unit 338 .
  • the memory unit 335 and a storage unit 338 are, in some embodiments, coupled to the processing unit 334 .
  • the memory unit can include volatile and/or non-volatile memory.
  • the processing unit 334 may perform one or more processes related to identifying/recognizing/detecting the merchandise objects and/or generating information for conducting transactions of such objects.
  • any portion of or all of the functions described of the various example modules in the host server 324 of the example of FIG. 3A can be performed by the processing unit 334 .
  • FIG. 4A depicts an example functional block diagram of a mobile device 402 that presents augmented information with merchandise objects to a user and processes interactions with the objects in the augmented reality marketplace.
  • the mobile device 402 includes a network interface 404 , a timing module 406 , a location sensor 408 , an identification verifier module 410 , an object identifier module 412 , a user selection module 414 , a user stimulus sensor 416 , a motion/gesture sensor 418 , an environmental stimulus sensor 420 , and/or an audio/video output module 422 .
  • the mobile device 402 may be any electronic device such as the devices described in conjunction with the mobile devices 102 A-N in the example of FIG.
  • a computer including but not limited to portable devices, a computer, a server, location-aware devices, mobile phones, PDAs, laptops, palmtops, iPhones, cover headsets, heads-up displays, helmet mounted display, head-mounted display, scanned-beam display, Google Glass, smart glasses/goggles, wearable computer such as mobile enabled watches or eyewear, and/or any other mobile interfaces and viewing devices, etc.
  • the mobile device 402 is coupled to a merchandise object repository 431 for identifying/detecting/recognizing merchandise objects (e.g., physical product or real life service) in the augmented reality environment.
  • the merchandise object repository 431 may be internal to or coupled to the mobile device 402 but the contents stored therein can be illustrated with reference to the example of a merchandise object repository 132 described in the example of FIG. 1 .
  • each module in the example of FIG. 4A can include any number and combination of sub-modules, and systems, implemented with any combination of hardware and/or software modules.
  • the mobile device 402 although illustrated as comprised of distributed components (physically distributed and/or functionally distributed), could be implemented as a collective element.
  • some or all of the modules, and/or the functions represented by each of the modules can be combined in any convenient or known manner.
  • the functions represented by the modules can be implemented individually or in any combination thereof, partially or wholly, in hardware, software, or a combination of hardware and software.
  • the network interface 404 can be a networking device that enables the mobile device 402 to mediate data in a network with an entity that is external to the host server, through any known and/or convenient communications protocol supported by the host and the external entity.
  • the network interface 404 can include one or more of a network adaptor card, a wireless network interface card, a router, an access point, a wireless router, a switch, a multilayer switch, a protocol converter, a gateway, a bridge, bridge router, a hub, a digital media receiver, and/or a repeater.
  • the mobile device 402 can facilitate transactions (e.g., e-commerce transactions) of a merchandise object (e.g., a physical product or a real life service) via an augmented reality environment provided by the mobile device 402 .
  • the mobile device 402 can detect presence of the physical product or the real life service when the product or service is seen/viewed/looked at by the user or in vicinity of the user.
  • the mobile device 402 automatically identifies the product or service from a perspective of a user. The perspective can be captured or sensed by the mobile device 402 via, for example, a camera or an image sensor.
  • the mobile device 402 can access information relevant to the transaction of the merchandise object(s) from a merchandise object repository (e.g., repository 431 in mobile device 402 and/or repository 332 in host server 324 ), and can present the relevant information to the user to conduct the transaction of the product or service via the augmented reality environment.
  • a merchandise object repository e.g., repository 431 in mobile device 402 and/or repository 332 in host server 324
  • the mobile device 402 can render on a display of the device 402
  • the mobile device 402 includes a timing module 406 .
  • the timing module 406 can be any combination of software agents and/or hardware modules able to identify, detect, transmit, compute, a current time, a time range, and/or a relative time of a request related to simulated objects/environments.
  • the timing module 406 can include a local clock, timer, or a connection to a remote time server to determine time.
  • the timing module 406 can be implemented via any known and/or convenient manner including but not limited to, electronic oscillator, clock oscillator, or various types of crystal oscillators.
  • the timing module 406 can provide time data which may be used to assist identification, recognition or detection by the mobile device 402 of the merchandise object. Examples that the mobile device 402 can consider include the time of day of a request, the date of the request, a relative time to another event, the time of year of the request, and/or the time span of a request or activity pertaining to merchandise objects.
  • the mobile device 402 includes a radio frequency (RF) or other types of wireless sensor 407 .
  • the RF sensor 407 can be any combination of software agents and/or hardware modules able to detect or identify an RF identification chip or other similar wireless identification tags.
  • the mobile device 402 includes a location sensor 408 .
  • the location sensor 408 can be any combination of software agents and/or hardware modules able to identify, detect, transmit, compute, a current location, a previous location, a range of locations, a location at or in a certain time period, and/or a relative location of the mobile device 402 .
  • the location sensor 408 can include a local sensor (e.g., GPS sensor) or a connection to an external entity (e.g., a base station) to determine the location information.
  • the location sensor 408 can determine location or relative location of the mobile device 402 via any known or convenient manner including but not limited to, GPS, cell phone tower triangulation, mesh network triangulation, relative distance from another location or device, RF signals, RF fields, optical range finders or grids, etc.
  • the location sensor 408 can identify location data and determine whether the location data satisfies the location parameter of a certain criteria.
  • the location sensor 408 provides location data to the host server (e.g., host server 324 of FIG. 3A ) for the host server to determine whether the criteria is satisfied.
  • the mobile device 402 includes a camera or an image sensor 409 .
  • the image sensor 409 can be any combination of software agents and/or hardware modules able to record or capture still images such as photographs or moving images such as video clips or movies.
  • sensor modules 407 - 409 are exemplary, as the mobile device 402 can include other sensors 410 including, for example, accelerometers, gyroscopic gauges, proximity sensors, light sensors, microphones, temperature sensors, heart beat sensors, touch sensitive sensors including touch-screens, fingerprint sensors, and so forth.
  • the mobile device 402 utilizes suitable sensors to detect presence of the physical product or the real life service in the augmented reality environment or platform. In some embodiments, the mobile device automatically identifies or detects the product(s) or service(s) that falls within a perspective of the user, that is, being viewed/looked at/seen by the user. The product is recognized/identified/detected by using one of several mechanisms as employed by the object identifier module 412 , and by querying the merchandise object repository 431 .
  • mobile device 402 includes the object identifier module 412 .
  • the object identifier module 412 can be any combination of software agents and/or hardware modules able to identify, detect, and/or recognize, merchandise objects for presentation to a user.
  • the object identifier module 412 in one embodiment, is coupled to the timing module 406 , the RF sensor 407 , the location sensor 408 , the image sensor 409 , and/or other sensors 410 .
  • the identifier module 412 can also receive data signals from the motion/gesture sensor 418 , the user stimulus sensor 416 , the environmental stimulus sensor 420 , and is coupled to the merchandise object repository 431 via the network interface 404 .
  • the object identifier module 412 is operable to identify the merchandise objects available for access using the device 402 .
  • the presence of the product is detected or identified in the augmented reality environment by a shape or other physical characteristics for presentation via the mobile device to facilitate transactions.
  • the product or service can have an unique combination of shape, size, color, or other physical dimension or characteristics (as registered in the merchandise object repository 431 ) so that the object identifier module 412 can recognize it.
  • shape or physical characteristics can be detected from, for example, the image sensor 409 .
  • the entries in the repository 431 can also include recognition patterns or photos or videos or metadata that may help identifying the product.
  • the presence of the product or service is detected or identified in the augmented reality environment by a geographic location of the product or service.
  • the location of the mobile device 402 can assist in identifying (e.g., by pinpointing or ruling out) certain product or service that is only available in that location.
  • location data can be detected from, for example, the location sensor 408 .
  • the presence of the product is detected or identified in the augmented reality environment by one or more of: (i) a visual marker; (ii) a marker or tag; (iii) a one-dimensional barcode; or (iv) a multi-dimensional barcode, on the product.
  • a marker on the object such as a QR code or other augmented reality marker can be presented for identification or detection (e.g., via the image sensor 409 ).
  • a barcode representing a stock-keeping unit number (SKU) may be present.
  • the barcode can be one-dimensional or multi-dimensional.
  • the presence of the product is detected or identified in the augmented reality environment by a radio frequency identification (RFID) chip coupled to the product.
  • RFID radio frequency identification
  • the RFID chip can be embedded in the product, or on a label attached to the product, or attached on a surface or a nearby location of the product.
  • the RFID chip can be detected and read, for example, via the RF sensor/RFID reader 407 .
  • the presence of the product is detected or identified in the augmented reality environment by optical character recognition (OCR) of written materials on or nearby the product.
  • OCR optical character recognition
  • the image sensor 409 can capture text on labels or other marketing materials on or near the product
  • the object identifier module 412 can use well-known OCR techniques to convert the written materials into text for identification (e.g., via a search using the search engine 322 to search merchandise repositories 332 , 431 ).
  • the object identifier module 412 can receive input from the user to disambiguate an identity of the product or service.
  • the user's input can include at least partially a name of the product or service.
  • the user can be prompted to name the product in assisting the disambiguation process proactively or interactively.
  • Such user input can be received from, for example, the motion/gesture sensor 418 or the user stimulus sensor 416 .
  • Some embodiments of the object identifier module 412 can also receive advisory data associated with the product or service to assist the identifying/recognizing/detecting the product or service.
  • the advisory data being added by another user of the augmented reality environment, or by an administrator or merchant of the augmented reality marketplace.
  • the advisory data can include one or more of: (i) metadata or tags relevant to the product or service, or (ii) an annotation that is descriptive of the product or service.
  • the metadata about the product can be added by another previous user who already identified the product, and/or tagged the product at the location that the mobile device 402 is currently situated.
  • the motion/gesture sensor 418 is operable to detect motion of the device 402 .
  • the detected motion is used by the user selection module 414 to perform actions responsive to the presented information according to the detected motion of the device.
  • Motion detecting can include detecting velocity and/or acceleration of the device 402 or a gesture of the user handling the device 402 .
  • the motion/gesture sensor 418 can include for example, an accelerometer.
  • the mobile device 402 can present (e.g., using the user selection module 414 and/or audio/video output module 422 ) information relevant to the merchandise product or service to the user so that the user can initiate various actions responsive to the presented information, for example, initiating the transaction of the product or service via the mobile device 402 .
  • Other example actions can include, for example, get price, compare price, read reviews, get availability, add to shopping cart, call a sales representative (e.g., a local sales person or a virtual one) for assistance, buy the product, tag the product or add metadata for the product, share the product with another user, or bookmark the product for later consideration.
  • the relevant information can be transmitted from the repository 431 , which may be local or may be hosted by an online server (e.g., host server 324 , 224 , which may also include or be coupled to e-commerce server 213 ) that enables electronic commerce.
  • an online server e.g., host server 324 , 224 , which may also include or be coupled to e-commerce server 213 .
  • a user interface generator in the user selection module 414 can be used to create buttons/links/etc. that are selectable by the user.
  • the interactions or actions from the user can be detected or determined using the user selection module 414 (e.g., via interpreting signals generated from the motion/gesture sensor 418 , user stimulus sensor 416 , and/or the environmental stimulus sensor 420 ). Any response to the user's selection or action can be reflected (e.g., highlighting a selected merchandise, blinking of a selected button, displaying an annotation or review by another user, or displaying a short video clip of the selected merchandise) in the augmented reality environment via the audio/video output module 422 .
  • the information is rendered translucently and disposed adjacently or partially overlaid with the product or service depicted in the augmented reality environment on the mobile device 402 .
  • the information includes advisory data associated with the product or service. Examples of advisory data can include metadata or tags that are deemed (either automatically by the object identifier module 412 , or by another user, or by a merchant or an administrator) relevant to the product or service, or (ii) an annotation that is descriptive of the product or service.
  • the mobile device 402 can also prompt the user to enter an advisory data for the product or service by, for example, operating a virtual keyboard using the user stimulus sensor 416 and/or motion/gesture sensor 418 . Then, the user can select to share (e.g., via network interface 404 ) the entry to another user of the augmented reality environment.
  • the content of the interactions can also be decided based on environmental factors including, for example, physical location of the user and/or the merchandise, user action, user feedback/annotation/review/tags/etc. as provided by the user of the device 402 or another user of the augmented reality platform using another device and/or at another physical location, motion/movement of the device 402 , gesture of the user using the device 402 .
  • an advertisement relevant to the product or service can be identified and displayed in the augmented reality environment via the mobile device 402 .
  • the advertisement is identified or detected based on one or more of: (i) location, (ii) time, (iii) interest or profile of a user of the mobile device 402 .
  • the environmental stimulus sensor 420 can detect environmental factors or changes in environmental factors surrounding the real environment in which the device 402 is located, and the advertisement can be provided based on these factors or changes.
  • Environmental factors can include, weather, temperature, topographical characters, density, surrounding businesses, buildings, living objects, etc. These factors or changes in them can also affect the positioning and/or characters of the presented information for the merchandise objects in the augmented reality in which they are presented to a user via the device 402 .
  • the advertisement is identified or detected based on a pointing direction of the mobile device 402 . In some embodiments, the advertisement is identified or detected based on velocity or speed of movement of the mobile device, or based on an ambient temperature in which the mobile device physically resides, or based on a plurality of sensors (e.g., sensors 406 - 410 ) on the mobile device 402 . Further, the advertisement is identified or detected (e.g., by a combined reading and interpretation of data signals from the location sensor 408 , the image sensor 409 , and a directional sensor which may be included among other sensors 410 ) based on a landmark which the mobile device is pointed at or in a vicinity of.
  • sensors e.g., sensors 406 - 410
  • the information includes a contact information of a sales representative of the product or service
  • the mobile device 402 prompts the user to contact the sale representative based on the contact information.
  • the mobile device 402 allows the user to bookmark the product or service, and in some embodiments, the mobile device 402 allows the user to share the bookmark with another user of the augmented reality environment.
  • the mobile device 402 in providing the augmented reality environment, can enable the user to select a physical product or service that is in the augmented reality.
  • the mobile device 402 can detect one or more targets in the augmented reality platform using a select area on a perspective of a user.
  • the perspective can be captured, for example, by a camera of the mobile device 402 .
  • the mobile device 402 prompts the user to choose an object of interest from the one or more detected targets.
  • the mobile device 402 can detect one or more targets in the augmented reality platform using a select area in a perspective of a user, and prompt the user to choose an object of interest from the one or more detected targets.
  • the perspective can be captured by the mobile device 402 (e.g., via the image sensor 409 .
  • some embodiments disclosed herein enable selection of real and/or virtual items in an augmented reality environment/platform/marketplace, which in turn enables actions to be taken in relation to the selected items (e.g., the aforementioned transactions of or interactions with the merchandise objects).
  • the detection of targets can be performed, for example, by object identifier module 412 of the mobile device 402 .
  • the detection of targets in the augmented reality environment can be performed by pointing the device 402 at the target(s).
  • the detection of targets can be performed by moving a pointer or a select area on the display of the device 402 to point at or frame the object in a reticle or a circular or rectangular frame (e.g., select area 520 , 525 , described below with respect to FIG. 5A ). It is noted that the select area can be any suitable shape and sizes.
  • the mobile device 402 can provide (e.g., via audio/video output module 422 to a display of the mobile device 402 ) the user with a lasso or a selection tool in the perspective to surround a respective target so as to form the select area. Additionally, the mobile device 402 can prompt the user to choose the object of interest from a set of choices such as a number of targets that are recognized in the perspective.
  • the mobile device 402 can prompt the user, or the user can select, to enter a learning mode in which the mobile device 402 can receive assistance from the user to perform the detection or recognition of the target.
  • the assistance can include one or more of (i) entering at least partially a name, (ii) entering metadata, (iii) defining a boundary via a pointer, or (iv) taking snapshots from a plurality of views to develop a recognition pattern, for a respective target.
  • the teaching can be done by aiming at the target or circling around the target with the interface (e.g., using the select area or the lasso tool, provided by the interface generator module in the user selection module 414 ) on the device 402 , and then by selecting a “learn” function or button on the interface in the augmented reality environment. Then, the mobile device 402 can ask the user for a name or other metadata for the target, and may ask for assistance in defining the boundaries of the object (e.g., via the lasso tool). The device 402 can also ask the user to take a series of snapshot views (e.g., via camera or image sensor 409 ) of the target to develop a pattern recognition database for it, as well as potentially a series of tests to improve recognition.
  • a series of snapshot views e.g., via camera or image sensor 409
  • targets After targets are recognized, they can then be selected by the user for actions or interactions.
  • the mobile device 402 can receive the user's choice of the object of interest by detecting movement and selection by a selection tool or a pointer, or movement and selection represented by a gesture (e.g., via the motion/gesture sensor 418 , user stimulus sensor 416 , and/or hand/finger/other gestures captured by the image sensor 409 ).
  • the mobile device 402 can perform capturing, via its various sensors, the gesture from one or more of: (i) movements or non-movements of an eye of the user, (ii) locations of a focal point of an eye of the user, or (iii) movements of an eye lid of the user.
  • the mobile device 402 can capture the gesture from one or more of: (i) movements of hand or finger as recognized by a camera of the mobile device, (ii) movements of a virtual pointer controlled by a wireless peripheral device, (iv) movements of a virtual pointer controlled by a touch-sensitive display of the mobile device, (v) movements of the mobile device itself, or (vi) movements of the user's head, limbs, or torso.
  • the capturing can be further based on a speed or velocity of the movements.
  • the present embodiments can capture or identify gestures from, for example, winking of the user and/or an eye focus or eye foci of the user.
  • gesture controlling can include finger or arm gesturing as captured by camera and/or distance detector/proximity detectors, so that the user can perform “spatial” or “virtual” gesturing in the air or other detectable spaces with similar gestures as those well-known gestures applicable to a mobile phone's touch screen.
  • gesture controlling can include eye ball motion tracking and/or eye focal point tracking
  • the user of mobile device 402 may operate various selection mechanisms, for example, using his or her eyes (e.g., via eye movement tracking) or by moving his or her hands/arms/fingers in the perspective to make specific gestures such as pointing or tracing the outline of some object, or by operating a virtual pointer in the scene using a handheld peripheral such as a wireless pointing device or a mouse equivalent, or by touching a touch-sensitive display on a mobile device and gesturing on it to indicate actions and selections, or by moving the device itself with specific gestures and velocities to use the device as a pointer or selection tool.
  • Additional gestures may include eye tracking and determining a focus of the eye for targeting things, and/or blinking to select a target that is in the focal point, to take a photo, or to select a button, etc.
  • the mobile device 402 can optionally confirm with the user of the choice of the object of interest.
  • the confirmation can include highlighting or outlining the target in the augmented reality platform by employing, for example, the audio/video output module 422 .
  • FIG. 4B depicts an example block diagram of the mobile device 402 that presents augmented information with merchandise objects in the augmented reality marketplace to a user and facilitates user interactions with the objects.
  • mobile device 402 includes a network interface 432 , a processing unit 434 , a memory unit 436 , a storage unit 438 , a location sensor 440 , an accelerometer/motion sensor 442 , an audio output unit/speakers 446 , a display unit 450 , an image capture unit 452 , a pointing device/sensor 454 , a input device 456 , and/or a touch screen sensor 458 . Additional or less units or modules may be included.
  • the mobile device 402 can be any combination of hardware components and/or software agents for that presenting information with respect to merchandise objects to a user and facilitating user interactions with the merchandise objects.
  • the network interface 432 has been described in the example of FIG. 4A .
  • One embodiment of the mobile device 402 further includes a processing unit 434 .
  • the location sensor 440 , accelerometer/motion sensor 442 , and timer 444 have been described with reference to the example of FIG. 4A .
  • the processing unit 434 can include one or more processors, CPUs, microcontrollers, FPGAs, ASICs, DSPs, or any combination of the above.
  • Data that is input to the mobile device 402 for example, via the image capture unit 452 , pointing device/sensor 554 , input device 456 (e.g., keyboard), and/or the touch screen sensor 458 can be processed by the processing unit 434 and output to the display unit 450 , audio output unit/speakers 446 and/or output via a wired or wireless connection to an external device, such as a host or server computer that generates and controls access to simulated objects by way of a communications component.
  • an external device such as a host or server computer that generates and controls access to simulated objects by way of a communications component.
  • One embodiment of the mobile device 402 further includes a memory unit 436 and a storage unit 438 .
  • the memory unit 436 and a storage unit 438 are, in some embodiments, coupled to the processing unit 434 .
  • the memory unit can include volatile and/or non-volatile memory.
  • the processing unit 434 may perform one or more processes related to presenting information with respect to merchandise objects to a user and/or facilitating user interactions with the merchandise objects based on the presented information.
  • any portion of or all of the functions described of the various example modules in the mobile device 402 of the example of FIG. 4A can be performed by the processing unit 434 .
  • various sensors and/or modules can be performed via any of the combinations of modules in the control subsystem that are not illustrated, including, but not limited to, the processing unit 434 and/or the memory unit 436 .
  • FIG. 5A illustrates an example application where mobile devices 502 such as a smart phone 502 A or a smart goggle 502 B having perspective interfaces 510 , 515 which can be used to facilitate electronic commerce in an augmented reality marketplace.
  • mobile devices 502 such as a smart phone 502 A or a smart goggle 502 B having perspective interfaces 510 , 515 which can be used to facilitate electronic commerce in an augmented reality marketplace.
  • the user when the user is doing “window shopping,” that is, when the user is looking at a merchandise product in a store window, or on a shelf, or when the user is near a physical store, the user can utilize the mobile devices 502 A, 502 B to access information 504 A, 504 B regarding the identified/detected/recognized merchandise objects.
  • the merchandise objects can be selected using select areas 520 , 525 in ways that are discussed above. Then, the user can select to perform actions 506 (including transactions) to the merchandise objects in ways that are discussed above.
  • the user can get price of, add comments/annotations/tags for, read reviews of, get availability of, or select to purchase the selected merchandise object.
  • the objects can be identified by ways discussed above including, for example, a barcode 508 attached to the merchandise object.
  • Written materials for identifying objects and/or other relevant information can also be recognized/detected by the mobile devices 502 A, 502 B.
  • augmented information can be provided to the user to inform the user if the store is now open, what is on sale now, etc.
  • other sales information including advertisement, video clips, images, or sales representative's contact information can be presented or otherwise made accessible to the user.
  • the information can also include entries from other users of the augmented reality, including but not limited to, their names and profiles, recent blog or microblog posts, recent photos or videos or links or documents added or annotated by them, their recent locations, their professions, etc.
  • FIG. 5B illustrates an example screenshot on a device 502 displaying an augmented reality environment 520 with objects 522 (including merchandise objects) associated with physical objects and/or real life services 526 in a real world environment 524 .
  • augmented reality environment 520 reflects a perspective of the user as captured by the mobile device 502 (e.g., via a camera of the device 502 ).
  • the display and movement of the object 522 in the augmented reality environment 520 corresponds to the display and movement of the physical object 526 (e.g., car) in the real world 524 .
  • relevant information regarding the physical object 526 can be rendered or presented with the object 522 so as to “augment” the object 522 .
  • Such information or data can be about the people or entities that are in or near a particular geographical location.
  • the mobile device 502 can facilitate transactions of the real physical product or service 526 based on the information, so that transactions can be conducted by users via devices when the user is physically near or at the physical product or the real life service.
  • FIG. 6 depicts a flow chart illustrating an example process 600 for facilitating user interaction with a merchandise object that is identified in an augmented reality environment.
  • process 600 is now described.
  • a mobile device detects ( 610 ) presence of a physical product or a real life service.
  • the mobile device 102 , 402 , 502 provides the augmented reality environment (e.g., through perspective interfaces 510 , 515 , FIG. 5A ) to the user.
  • the device 102 , 402 , 502 automatically identifies ( 615 ) the product or service from the perspective 510 , 515 of the user captured by the mobile device (e.g., via image sensor 409 , FIG. 4A ).
  • the mobile device 102 , 402 , 502 can query a merchandise object repository (e.g., repository 132 , FIG. 1 ; repository 332 , FIG. 3A ; repository 431 , FIG. 4A ) in identifying the merchandise product or service and obtaining relevant information. Thereafter, the mobile device 102 , 402 , 502 presents ( 620 ), on the mobile device (e.g., on user interface 104 A- 104 N, FIG. 1 , or on perspective 510 , 515 ), information (e.g., information 504 A, 504 B, FIG. 5A ) to conduct the transaction of the product or service via the augmented reality environment.
  • a merchandise object repository e.g., repository 132 , FIG. 1 ; repository 332 , FIG. 3A ; repository 431 , FIG. 4A
  • the mobile device 102 , 402 , 502 presents ( 620 ), on the mobile device (e.g., on user interface 104 A- 104 N,
  • the device 102 , 402 , 502 renders ( 625 ), on a display of the mobile device (e.g., via audio/video output module 422 , FIG. 4A ), a transaction information associated with the product or service.
  • the mobile device 102 , 402 , 502 can receive ( 630 ) input from the user to disambiguate an identity of the product or service. Further, in some embodiments, the mobile device 102 , 402 , 502 can receive ( 640 ) advisory data associated with the product or service. The advisory data can be added, for example, by another user of the augmented reality environment.
  • the mobile device 102 , 402 , 502 can initiate ( 650 ) the transaction of the product or service via the mobile device based on responses to the presented information (e.g., as detected via various sensors 406 - 410 , 416 , 418 , 420 of device 402 and determined by the user selection module 414 , FIG. 4A ) from the user.
  • responses to the presented information e.g., as detected via various sensors 406 - 410 , 416 , 418 , 420 of device 402 and determined by the user selection module 414 , FIG. 4A .
  • FIG. 7 depicts a flow chart illustrating an example process 700 for selecting/identifying a merchandise object in an augmented reality environment.
  • process 700 is now described.
  • a mobile device detects ( 710 ) one or more targets in an augmented reality platform using a select area (e.g., select areas 520 , 525 , FIG. 5A ) in a perspective (e.g., perspective 510 , 515 , FIG. 5A ) of a user.
  • the perspective 510 , 515 is captured by the mobile device 102 , 402 , 502 (e.g., via image sensor 409 , FIG. 4A ).
  • the mobile device 102 , 402 , 502 provides ( 712 ), via a display of the mobile device (e.g., via audio/video output module 422 , FIG. 4A ), the user with a lasso tool in the perspective 510 , 515 to surround a respective target so as to form the select area 520 , 525 .
  • the mobile device 102 , 402 , 502 prompts ( 714 ), via audio/video output module 422 , the user to enter a learning mode in which the mobile device 102 , 402 , 502 receives ( 714 ) an assistance from the user to perform the detection.
  • the mobile device 102 , 402 , 502 After detecting the targets, the mobile device 102 , 402 , 502 prompts ( 720 ) the user to choose an object of interest from the one or more detected targets. Then, the mobile device 102 , 402 , 502 receives ( 730 ) the user's choice of the object of interest (e.g., via motion/gesture sensor 418 , user stimulus sensor 416 , and/or other gestures captured by image sensor 409 , as determined by the user selection module 414 , FIG. 4A ). The mobile device 102 , 402 , 502 can detect ( 732 ) movement and selection by a selection tool or a pointer, and/or movement and selection represented by a gesture. Additionally, the mobile device 102 , 402 , 502 can confirm ( 740 ) with the user of the choice of the object of interest.
  • the mobile device 102 , 402 , 502 can confirm ( 740 ) with the user of the choice of the object of
  • FIG. 8 shows a diagrammatic representation of a machine in the example form of a computer system 800 within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed.
  • the machine operates as a standalone device or may be connected (e.g., networked) to other machines.
  • the machine may operate in the capacity of a server or a client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.
  • the machine may be a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.
  • PC personal computer
  • PDA personal digital assistant
  • machine-readable medium is shown in an exemplary embodiment to be a single medium, the term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions.
  • the term “machine-readable medium” shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure.
  • routines executed to implement the embodiments of the disclosure may be implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions referred to as “computer programs.”
  • the computer programs typically comprise one or more instructions set at various times in various memory and storage devices in a computer, and that, when read and executed by one or more processors in a computer, cause the computer to perform operations to execute elements involving the various aspects of the disclosure.
  • machine or computer-readable media include but are not limited to recordable type media such as volatile and non-volatile memory devices, floppy and other removable disks, hard disk drives, optical disks (e.g., Compact Disk Read-Only Memory (CD ROMS), Digital Versatile Disks, (DVDs), etc.), among others, and transmission type media such as digital and analog communication links.
  • recordable type media such as volatile and non-volatile memory devices, floppy and other removable disks, hard disk drives, optical disks (e.g., Compact Disk Read-Only Memory (CD ROMS), Digital Versatile Disks, (DVDs), etc.
  • CD ROMS Compact Disk Read-Only Memory
  • DVDs Digital Versatile Disks
  • transmission type media such as digital and analog communication links.
  • the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense; that is to say, in the sense of “including, but not limited to.”
  • the terms “connected,” “coupled,” or any variant thereof means any connection or coupling, either direct or indirect, between two or more elements; the coupling of connection between the elements can be physical, logical, or a combination thereof.
  • the words “herein,” “above,” “below,” and words of similar import when used in this application, shall refer to this application as a whole and not to any particular portions of this application.
  • words in the above Detailed Description using the singular or plural number may also include the plural or singular number respectively.
  • the word “or,” in reference to a list of two or more items, covers all of the following interpretations of the word: any of the items in the list, all of the items in the list, and any combination of the items in the list.

Abstract

Techniques are disclosed for facilitating electronic commerce in an augmented reality environment. In some embodiments, a method comprises detecting, by a mobile device, presence of the physical product or the real life service; and presenting, on the mobile device, information to conduct the transaction of a physical product or a real life service via the augmented reality environment. In some embodiments, a method comprises detecting one or more targets in the augmented reality platform using a select area in a perspective of a user, the perspective being captured by a mobile device; and prompting the user to choose an object of interest from the one or more detected targets. Among other advantages, embodiments disclosed herein provide an intuitive and integrated user experience in shopping using augmented reality devices, thereby reducing the consumer user's effort in conducting such activities, reducing necessary sales personnel and their working hours, and increasing sales.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS AND EFFECTIVE FILING DATE ENTITLEMENT
  • This application is entitled to the benefit of and/or the right of priority to U.S. Provisional Application No. 61/640,856, entitled “AUGMENTED REALITY MARKETPLACE”(Attorney Docket No. 86220-8004.US00), filed May 1, 2012, which is hereby incorporated by reference in its entirety. This application is therefore entitled to an effective filing date of May 1, 2012.
  • This application is related to U.S. patent application Ser. No. 12/473,143, entitled “SYSTEM AND METHOD FOR CONTROL OF A SIMULATED OBJECT THAT IS ASSOCIATED WITH A PHYSICAL LOCATION IN THE REAL WORLD ENVIRONMENT”(Attorney Docket No. 86220-8001.US01), filed May 27, 2009; to U.S. patent application Ser. No. 12/473,171, entitled “SYSTEM AND METHOD FOR FACILITATING USER INTERACTION WITH A SIMULATED OBJECT ASSOCIATED WITH A PHYSICAL LOCATION”(Attorney Docket No. 86220-8001.U502), filed May 27, 2009; to U.S. patent application Ser. No. 12/473,182, entitled “SYSTEM AND METHOD OF SIMULATED OBJECTS AND APPLICATIONS THEREOF”(Attorney Docket No. 86220-8001.U503), filed May 27, 2009, now issued as U.S. Pat. No. 8,303,387; to U.S. patent application Ser. No. 13/631,778, entitled “SIMULATED ENVIRONMENTS FOR MARKETPLACES, GAMING, SPORTING EVENTS, AND PERFORMANCE EVENTS”(Attorney Docket No. 86220-8001.U504), filed Sep. 28, 2012; all of which are hereby incorporated by reference in their entirety.
  • This application is also related to U.S. patent application Ser. No.______, entitled “SYSTEM AND METHOD FOR FACILITATING TRANSACTIONS OF A PHYSICAL PRODUCT OR REAL LIFE SERVICE VIA AN AUGMENTED REALITY ENVIRONMENT”(Attorney Docket No. 86220-8004.US01), also claiming priority to U.S. Provisional Application No. 61/640,856, filed May 1, 2012, concurrently filed herewith and is also hereby incorporated by reference.
  • BACKGROUND
  • Miniaturization of consumer electronics with sophisticated graphics capabilities and expansive computing power has augmented the activities one can engage in via consumer electronics and in particular, portable electronics such as smart phones, tablets, and the like. The advancements in technologies also have brought science fiction into reality.
  • Further, portable electronics or other electronics devices now generally include a large variety of sensing capabilities. These capabilities can be utilized to further enhance the users' augmented reality experiences.
  • BRIEF DESCRIPTION OF THE DRAWINGS
  • The present embodiments are illustrated by way of example and are not intended to be limited by the figures of the accompanying drawings. In the drawings:
  • FIG. 1 illustrates an example block diagram of mobile devices able to communicate with a host server that generates and controls access to simulated objects and/or facilitates transactions of a physical product or real life service via an augmented reality environment (e.g., an augmented reality marketplace) through a network;
  • FIG. 2 depicts an example block diagram of the components of a host server that generates information that facilitates transactions of a physical product or real life service via an augmented reality environment;
  • FIG. 3A depicts an example functional block diagram of the host server that facilitates transactions of a physical product or real life service via an augmented reality environment;
  • FIG. 3B depicts an example block diagram illustrating the components of the host server that facilitates transactions of a physical product or real life service via an augmented reality environment;
  • FIG. 4A depicts an example functional block diagram of a mobile device that presents augmented information with merchandise objects to a user and processes interactions with the objects in the augmented reality marketplace based on the information;
  • FIG. 4B depicts an example block diagram of the mobile device that presents augmented information with merchandise objects in the augmented reality marketplace to a user and facilitates user interactions with the objects based on the information;
  • FIG. 5A illustrates an example application where mobile devices such as a smart phone or a smart goggle having perspective interfaces which are used to facilitate electronic commerce in an augmented reality marketplace;
  • FIG. 5B illustrates an example screenshot on a mobile device displaying an augmented reality environment with objects associated with physical objects and/or real life services in a real world environment;
  • FIG. 6 depicts a flow chart illustrating an example process for facilitating user interaction with a merchandise object that is identified in an augmented reality environment;
  • FIG. 7 depicts a flow chart illustrating an example process for selecting/identifying a merchandise object in an augmented reality environment; and
  • FIG. 8 depicts a diagrammatic representation of a machine in the example form of a computer system within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed, according to some embodiments.
  • The same reference numbers and any acronyms identify elements or acts with the same or similar structure or functionality throughout the drawings and specification for ease of understanding and convenience.
  • DETAILED DESCRIPTION
  • Techniques are disclosed for facilitating electronic commerce in an augmented reality environment. In some embodiments, a method comprises detecting, by a mobile device, presence of the physical product or the real life service; and presenting, on the mobile device, information to conduct the transaction of a physical product or a real life service via an augmented reality environment. The mobile device facilitates the augmented reality environment. In some embodiments, a method comprises detecting one or more targets in the augmented reality platform using a select area in a perspective of a user, the perspective being captured by a mobile device; and prompting the user to choose an object of interest from the one or more detected targets. Among other advantages, embodiments disclosed herein provide an intuitive and integrated user experience in shopping using augmented reality devices, thereby reducing the consumer user's effort in conducting such activities, reducing necessary sales personnel and their working hours, and increasing sales.
  • Reference in this specification to “one embodiment” or “an embodiment” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, nor are separate or alternative embodiments mutually exclusive of other embodiments. Moreover, various features are described which may be exhibited by some embodiments and not by others. Similarly, various requirements are described which may be requirements for some embodiments but not other embodiments.
  • The terms used in this specification generally have their ordinary meanings in the art, within the context of the disclosure, and in the specific context where each term is used. Certain terms that are used to describe the disclosure are discussed below, or elsewhere in the specification, to provide additional guidance to the practitioner regarding the description of the disclosure. For convenience, certain terms may be highlighted, for example using italics and/or quotation marks. The use of highlighting has no influence on the scope and meaning of a term; the scope and meaning of a term is the same, in the same context, whether or not it is highlighted. It will be appreciated that same thing can be said in more than one way.
  • Consequently, alternative language and synonyms may be used for any one or more of the terms discussed herein, nor is any special significance to be placed upon whether or not a term is elaborated or discussed herein. Synonyms for certain terms are provided. A recital of one or more synonyms does not exclude the use of other synonyms. The use of examples anywhere in this specification including examples of any terms discussed herein is illustrative only, and is not intended to further limit the scope and meaning of the disclosure or of any exemplified term. Likewise, the disclosure is not limited to various embodiments given in this specification.
  • Without intent to further limit the scope of the disclosure, examples of instruments, apparatus, methods and their related results according to the embodiments of the present disclosure are given below. Note that titles or subtitles may be used in the examples for convenience of a reader, which in no way should limit the scope of the disclosure. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. In the case of conflict, the present document, including definitions will control.
  • Mobile devices, especially the wearable ones which may be in forms of eyewear, mobile-enabled wrist watches, head-mounted displays, are now available to provide augmented reality experiences to users. Embodiments of the present disclosure include systems and methods for facilitating transactions of a physical product or real life service via an augmented reality environment. Additionally, embodiments of the present disclosure include systems and methods for enabling selection of a physical product or a real life service in an augmented reality platform via a mobile device.
  • FIG. 1 illustrates an example block diagram of mobile devices 102A-N able to communicate with a host server 124 that generates and controls access to simulated objects and/or facilitates transactions of a physical product or real life service via an augmented reality environment through a network 106.
  • The mobile devices 102A-N can be any system and/or device, and/or any combination of devices/systems that is able to establish a connection with another device, a server and/or other systems. The mobile devices 102A-N typically include a display and/or other output functionalities to present information and data exchanged between/among the devices 102A-N and the host server 124. For example, the mobile devices 102A-N can be any of, but are not limited to, a server desktop, a desktop computer, a computer cluster, or portable devices including, a notebook, a laptop computer, a handheld computer, a palmtop computer, a mobile phone, a cell phone, a smart phone, a PDA, a Blackberry device, a Treo, an iPhone, cover headsets, heads-up displays, helmet mounted display, head-mounted display, scanned-beam display, wearable computer such as mobile enabled watches, and/or any other mobile interfaces and viewing devices, etc. Additional examples of mobile devices 102A-N can include a Google Glass, smart glasses/goggles, a smart phone, or other suitable mobile computing devices that are configured to facilitate or enable an augmented reality environment or platform. The mobile devices 102A-N may be location-aware devices that are able to determine their own location or identify location information from an external source. In one embodiment, the mobile devices 102A-N are coupled to a network 106. In some embodiments, the devices 102A-N and host server 124 may be directly connected to one another.
  • In one embodiment, the host server 124 is operable to provide simulated objects (e.g., objects, computer-controlled objects, or simulated objects) that correspond to real world physical locations to be presented to users on mobile devices 102A-N. The simulated objects are typically software entities or occurrences that are controlled by computer programs and can be generated upon request when certain criteria are met. The host server 124 also processes interactions of simulated object with one another and actions on simulated objects caused by stimulus from a real user and/or the real world environment.
  • In some embodiments, the host server 124 is operable, in conjunction with the mobile devices 102A-N, to facilitate electronic commercial transactions in an augmented reality environment or platform, such as an augmented reality marketplace. Services and functions provided by the host server 124 and the components therein are described in detail with further references to the examples of FIG. 3A-3B.
  • The mobile devices 102A-N are generally operable to provide access (e.g., visible access, audible access) to the simulated objects to users, for example via user interface 104A-N displayed on the display units. The devices 102A-N may be able to detect simulated objects based on location and/or timing data and provide those objects authorized by the user for access via the devices. In some embodiments, the mobile devices 102A-N can facilitate electronic commercial transactions in an augmented reality environment. In some embodiments, the devices 102A-N communicate with the host server 124 in facilitating the transactions. Services and functions provided by the mobile devices 102A-N and the components therein are described in detail with further references to the examples of FIG. 4A-4B.
  • The network 106, over which the mobile devices 102A-N and the host server 124 communicate, may be a telephonic network, an open network, such as the Internet, or a private network, such as an intranet and/or the extranet. For example, the Internet can provide file transfer, remote log in, email, news, RSS, and other services through any known or convenient protocol, such as, but is not limited to the TCP/IP protocol, Open System Interconnections (OSI), FTP, UPnP, iSCSI, NSF, ISDN, PDH, RS-232, SDH, SONET, etc.
  • The network 106 can be any collection of distinct networks operating wholly or partially in conjunction to provide connectivity to the mobile devices 102A-N and the host server 124 and may appear as one or more networks to the serviced systems and devices. In one embodiment, communications to and from the mobile devices 102A-N can be achieved by, an open network, such as the Internet, or a private network, such as an intranet and/or the extranet. In one embodiment, communications can be achieved by a secure communications protocol, such as secure sockets layer (SSL), or transport layer security (TLS).
  • In addition, communications can be achieved via one or more wireless networks, such as, but is not limited to, one or more of a Local Area Network (LAN), Wireless Local Area Network (WLAN), a Personal area network (PAN), a Campus area network (CAN), a Metropolitan area network (MAN), a Wide area network (WAN), a Wireless wide area network (WWAN), Global System for Mobile Communications (GSM), Personal Communications Service (PCS), Digital Advanced Mobile Phone Service (D-Amps), Bluetooth, Wi-Fi, Fixed Wireless Data, 2G, 2.5G, 3G (e.g., WCDMA/UTMS based 3G networks), 3.5G, 4G, IMT-Advanced, pre-4G, LTE Advanced, mobile WiMax, WiMax 2, WirelessMAN-Advanced networks, enhanced data rates for GSM evolution (EDGE), General packet radio service (GPRS), enhanced GPRS, iBurst, UMTS, HSPDA, HSPA, HSPA+, UMTS-TDD, 1xRTT, EV-DO, messaging protocols such as, TCP/IP, SMS, MMS, extensible messaging and presence protocol (XMPP), real time messaging protocol (RTMP), instant messaging and presence protocol (IMPP), instant messaging, USSD, IRC, or any other wireless data networks or messaging protocols.
  • The host server 124 may include or be coupled to a user repository 128 and/or a simulated object repository 130 and or a merchandise object repository 132. The data repositories 128, 130, 132 can store software, descriptive data, images, system information, drivers, and/or any other data item utilized by other components of the host server 124 and/or any other servers for operation. The data repositories 128, 130, 132 may be managed by a database management system (DBMS), for example but not limited to, Oracle, DB2, Microsoft Access, Microsoft SQL Server, PostgreSQL, MySQL, FileMaker, etc.
  • The repositories 128, 130, 132 can be implemented via object-oriented technology and/or via text files, and can be managed by a distributed database management system, an object-oriented database management system (OODBMS) (e.g., ConceptBase, FastDB Main Memory Database Management System, JDOInstruments, ObjectDB, etc.), an object-relational database management system (ORDBMS) (e.g., Informix, OpenLink Virtuoso, VMDS, etc.), a file system, and/or any other convenient or known database management package.
  • In some embodiments, the host server 124 is able to provide data to be stored in the user data repository 128 , the simulated object repository 130, the merchandise object repository 132, and/or the can retrieve data stored in the user data repository 128 and/or the simulated object repository 130 and/or the merchandise object repository 132.
  • The user data repository 128 can store user information, user preferences, user profile, user list of interest, user favorite, user bookmark, user annotation, user tags, access permissions associated with the users, device information, hardware information, etc. The simulated object repository 130 can store software entities (e.g., computer programs) that control simulated objects and the simulated environments in which they are presented for visual/audible access or control/manipulation.
  • The simulated object repository 130 may further include simulated objects and their associated data structures with metadata defining the simulated object including its associated access permission.
  • The merchandise object repository 132 can store merchandise information including, for example, merchandise name, alias, attributes, shape, size, dimension, or other physical characteristics or recognition patterns or augmented reality markers, location (if suitable), vendor(s), price, SKU code, QR code, 1-dimensional or 2-dimensional or multi-dimensional barcode, RFID code, metadata, tags, annotations, sales contact information.
  • FIG. 2 depicts an example block diagram of the components of a host server 224 that generates information that facilitates transactions of a physical product or real life service via an augmented reality environment.
  • In the example of FIG. 2, the host server 224 includes a network controller 202, a firewall 204, a multimedia server 206, a web application server 212, an electronic commerce (e-commerce) server 213, and a database including a database storage 216 and database software 218.
  • In the example of FIG. 2, the network controller 202 can be a networking device that enables the host server 224 to mediate data in a network with an entity that is external to the host server 224, through any known and/or convenient communications protocol supported by the host and the external entity. The network controller 202 can include one or more of a network adaptor card, a wireless network interface card, a router, an access point, a wireless router, a switch, a multilayer switch, a protocol converter, a gateway, a bridge, bridge router, a hub, a digital media receiver, and/or a repeater.
  • The firewall 204, can, in some embodiments, govern and/or manage permission to access/proxy data in a computer network, and track varying levels of trust between different machines and/or applications. The firewall 204 can be any number of modules having any combination of hardware and/or software components able to enforce a predetermined set of access rights between a particular set of machines and applications, machines and machines, and/or applications and applications, for example, to regulate the flow of traffic and resource sharing between these varying entities. The firewall 204 may additionally manage and/or have access to an access control list which details permissions including for example, the access and operation rights of an object by an individual, a machine, and/or an application, and the circumstances under which the permission rights stand.
  • Other network security functions can be performed or included in the functions of the firewall 204, can be, for example, but are not limited to, intrusion-prevention, intrusion detection, next-generation firewall, personal firewall, etc. without deviating from the novel art of this disclosure. In some embodiments, the functionalities of the network controller 202 and the firewall 204 are partially or wholly combined and the functions of which can be implemented in any combination of software and/or hardware, in part or in whole.
  • In the example of FIG. 2, the host server 224 includes the multimedia server 206 or a combination of multimedia servers to manage images, photographs, animation, video, audio content, graphical content, documents, and/or other types of multimedia data for use in or to supplement simulated content such as simulated objects and their associated deployment environment (e.g., a simulated environment). The multimedia server 206 is any software suitable for delivering messages to facilitate retrieval/transmission of multimedia data among servers to be provided to other components and/or systems of the host server 224, for example when rendering a web page, an augmented reality environment, and/or augmenting physical objects using multimedia content (e.g., the augmented information includes a short video clip introducing physical products or real life services).
  • In addition, the multimedia server 206 can facilitate transmission/receipt of streaming data such as streaming images, audio, and/or video. The multimedia server 206 can be configured separately or together with the web application server 212, depending on a desired scalability of the host server 224. Examples of graphics file formats that can be managed by the multimedia server 206 include but are not limited to, ADRG, ADRI, AI, GIF, IMA, GS, JPG, JP2, PNG, PSD, PSP, TIFF, and/or BMP, etc.
  • The web application server 212 can include any combination of software agents and/or hardware modules for accepting Hypertext Transfer Protocol (HTTP) requests from end users, external systems, and/or external mobile devices and responding to the request by providing the requestors with web pages, such as HTML documents and objects that can include static and/or dynamic content (e.g., via one or more supported interfaces, such as the Common Gateway Interface (CGI), Simple CGI (SCGI), PHP, JavaServer Pages (JSP), Active Server Pages (ASP), ASP.NET, etc.). In addition, a secure connection, SSL and/or TLS can be established by the web application server 212.
  • The e-commerce server 213 can include any combination of software agents and/or hardware modules for facilitating electronic commercial transactions to be initiated, processed, and completed with mobile devices (e.g., devices 102A-N, FIG. 1). The e-commerce server 213 typically includes at least one processor and a memory, and may be further connected to one or more computers (not shown in FIG. 2 for simplicity) that manage inventory, logistics and/or other commercial functions. In some embodiments, the e-commerce server 213 can establish secured connections using one or more well-known security protocols.
  • In general, the host server 224 assists, by interactively or proactively providing relevant information, mobile devices such as device 102A-N to generate the augmented reality user interfaces, which facilitate the augmented reality marketplace disclosed herein, as shown in the example screenshots of FIG. 5B-FIG. 5C. The augmented information, which is provided by the host server 224 to client users/end devices (e.g., onto the user interface screens 104A-104N on mobile devices 102A-102N for display). In some embodiments, the web application server 212 and/or the e-commerce server 213 also perform authentication processes before responding to requests for conducting transactions in the augmented reality marketplace.
  • The databases 216, 218 can store software, descriptive data, images, system information, drivers, and/or any other data item utilized by other components of the host server for operation. The databases 216, 218 may be managed by a database management system (DBMS), for example but not limited to, Oracle, DB2, Microsoft Access, Microsoft SQL Server, PostgreSQL, MySQL, FileMaker, etc. The databases 216, 218 can be implemented via object-oriented technology and/or via text files, and can be managed by a distributed database management system, an object-oriented database management system (OODBMS) (e.g., ConceptBase, FastDB Main Memory Database Management System, JDOInstruments, ObjectDB, etc.), an object-relational database management system (ORDBMS) (e.g., Informix, OpenLink Virtuoso, VMDS, etc.), a file system, and/or any other convenient or known database management package.
  • In the example of FIG. 2, the host server 224 includes components (e.g., a network controller, a firewall, a storage server, a multimedia server, an e-commerce server, and/or a database including a database storage and database software, etc.) coupled to one another and each component is illustrated as being individual and distinct. However, in some embodiments, some or all of the components, and/or the functions represented by each of the components can be combined in any convenient or known manner. Furthermore, the functions represented by the devices can be implemented individually or in any combination thereof, in hardware, software, or a combination of hardware and software.
  • FIG. 3A depicts an example functional block diagram of the host server 324 that facilitates transactions of a physical product or real life service via an augmented reality environment.
  • The host server 324 includes a network interface 302, a repository manager 304, and/or a search engine 322. The host server 324 is also coupled to a user data repository 328 and/or a merchandise object repository 332. The user data repository 328 and merchandise object repository 332 are described with further reference to the example of FIG. 1. The host server 324 can also include a variety of modules including, for example, a pricing management module 312, a physical characteristics management module 314, an annotation management module 316, a metadata management module 318, an advertisement/coupon management module 320, and/or a recognition/disambiguation process module 323. The management modules 312-320 can be coupled to the repository manager 304. The recognition/disambiguation process module 323 can be coupled to the search engine 322.
  • Additional or less modules can be included without deviating from the techniques discussed in this disclosure. In addition, each module in the example of FIG. 3A can include any number and combination of sub-modules, and systems, implemented with any combination of hardware and/or software modules.
  • The host server 324, although illustrated as comprised of distributed components (physically distributed and/or functionally distributed), could be implemented as a collective element. In some embodiments, some or all of the modules, and/or the functions represented by each of the modules can be combined in any convenient or known manner. Furthermore, the functions represented by the modules can be implemented individually or in any combination thereof, partially or wholly, in hardware, software, or a combination of hardware and software.
  • In the example of FIG. 3A, the network interface 302 can be a networking device that enables the host server 324 to mediate data in a network with an entity that is external to the host server, through any known and/or convenient communications protocol supported by the host and the external entity. The network interface 302 can include one or more of a network adaptor card, a wireless network interface card, a router, an access point, a wireless router, a switch, a multilayer switch, a protocol converter, a gateway, a bridge, bridge router, a hub, a digital media receiver, and/or a repeater.
  • The merchandise object repository 332 can be a database that comprises any combination of software agents and/or hardware modules able to store, modify, update, adjust, edit, and/or delete various physical as well as other attributes of a merchandise object (e.g., a physical product, or a real life service). The management functions of these physical and other attributes of merchandise objects can be performed by the repository manager 304.
  • A merchandise object can include a physical entity or goods (such as a store, a merchant, commercial product, a living object, an inanimate object, etc.) or a real life service (such as a restaurant, a spa, a beauty salon, a medical doctor, an art performance, etc.). In some embodiments, merchandise objects can be identified/recognized/detected using the various physical and/or other characteristics of the objects, and information relevant to commercial transactions of the physical product or real life service can be presented/displayed/projected in the augmented reality environment so as to enable the user to conduct such transactions.
  • Characteristics and attributes of merchandise objects that help the identification/recognition/detection can include what can be perceived by users in the augmented reality via a physical device (e.g., a mobile device or device 102 in the example of FIG. 1). For example, they can include merchandise name, alias, attributes, shape, size, dimension, or other physical characteristics or recognition patterns or augmented reality markers, or location (if suitable). They can also include other identification information that can be detected by the device 102, such as vendor(s) name, SKU code, QR code, 1-dimensional or 2-dimensional or multi-dimensional barcode, RFID code.
  • One embodiment of the host server 324 includes a search engine 322. The search engine 322 can be any combination of software agents and/or hardware modules able to search, detect, and/or identify merchandise objects.
  • More specifically, attributes/clues/characteristics observed by the mobile device can be sent to the search engine 322. The search engine 322 can perform textual or imagery or other suitable types of search to identify the merchandise object. The search engine 322 can search or detect objects either automatically or in response to user request. For example, the user can request access to merchandise objects' information by perform a search request. The search request parameters can include, one or more of, at least a partial name input of the merchandise object from the user, the user's location, the current time or a time period. The search that is performed can automatically detect the merchandise object(s) being viewing by or otherwise available for access to the user in the augmented reality. In one embodiment, the merchandise objects are further filtered based on preferences or profiles or interest of the user.
  • The search engine 322 can further utilize the recognition/disambiguation process module 323 to prompt the user to assist in disambiguation of the merchandise object. For example, the module 323 can generate a plurality of choices in forms of name, picture, locational information, and/or description that can help the recognition. For another example, the module 323 can prompt the user to enter a partial or a full name. For yet another example, the module 323 can ask the user to take a plurality of pictures of the merchandise object.
  • Once identified or detected the presence of the merchandise object or product, the host server 324 can access relevant information including, for example, purchase information of the product, price from the vendor of the exact product that the user is viewing in the augmented reality, price from another vendor of the same or similar product, availability of the product, any metadata or tags of the product, annotations or reviews of the product added by another user of the augmented reality environment, images or video clips that are shared by other users or the merchant, and/or sales contact information. These information can be generated, maintained and updated using the modules 312-320 coupled to the repository manager 304 (e.g., by the merchant, by the user, by another user, or by an administrator of the host server 324).
  • In some embodiments, the merchandise object is stored in the merchandise object repository 332 using a data structure having metadata. The metadata can properties of the simulated object, location and/or timing parameters (e.g., sale event, availability, store hours, etc.) The location data can be specified with longitude and latitude coordinates, GPS coordinates, and/or relative position. In one embodiment, the object is associated with a unique identifier or augmented reality (AR) markers. The unique identifier or markers may be further associated with a location data structure having a set of location data that includes the qualifying location data for the merchandise object. In one embodiment, the metadata includes a link to data from an external source (e.g., the store's or merchant's website on the Internet).
  • In one embodiment, this information is automatically provided to the user when he or she arrives at or near the merchandise object's location (e.g., implicit request) via the device. Alternatively, the information is provided upon request by the user (e.g., explicit request). For example, merchandise objects associated with various attractions, for example, in a shopping mall or a tourist location in the real world, can be selected by the user (e.g., via input to the device).
  • Additionally, an advertisement (e.g., an electronic advertisement) that is relevant to the merchandise object is delivered by the advertisement/coupon management module 320 to the user that qualifies one or more criteria. The user may qualify on a basis of a location, identity, and/or a timing parameter. For example, the user may be provided with advertisements of local pizza shops or other late night dining options when the user is driving around town during late night hours when other dining options may not be available. In another example, an electronic coupon can be presented to a user using the device based on the merchandise object identification result, for example, when the device is located at the location during a certain period of time that satisfies the criteria. The electronic coupon may be redeemed by the user for conducting transactions of the physical product or the real life service.
  • FIG. 3B depicts an example block diagram illustrating the components of the host server 324 that facilitates transactions of a physical product or real life service via an augmented reality environment.
  • In one embodiment, host server 334 includes a network interface 302, a processing unit 334, a memory unit 336, a storage unit 338, a location sensor 340, and/or a timing module 342. Additional or less units or modules may be included. The host server 334 can be any combination of hardware components and/or software agents for generating information for merchandise objects in facilitating transactions in an augmented reality marketplace environment. The network interface 302 has been described in the example of FIG. 3A.
  • One embodiment of the host server 324 further includes a processing unit 334. The data received from the network interface 302, location sensor 340, and/or the timing module 342 can be input to a processing unit 334. The location sensor 340 can include GPS receivers, RF transceiver, an optical rangefinder, etc. The timing module 342 can include an internal clock, a connection to a time server (via NTP), an atomic clock, a GPS master clock, etc.
  • The processing unit 334 can include one or more processors, CPUs, microcontrollers, FPGAs, ASICs, DSPs, or any combination of the above. Data that is input to the host server 334 can be processed by the processing unit 334 and output to a display and/or output via a wired or wireless connection to an external device, such as a mobile phone, a portable device, a host or server computer by way of a communications component.
  • One embodiment of the host server 324 further includes a memory unit 336 and a storage unit 338. The memory unit 335 and a storage unit 338 are, in some embodiments, coupled to the processing unit 334. The memory unit can include volatile and/or non-volatile memory. In receiving characteristics of and generating relevant information for the merchandise objects, the processing unit 334 may perform one or more processes related to identifying/recognizing/detecting the merchandise objects and/or generating information for conducting transactions of such objects.
  • In some embodiments, any portion of or all of the functions described of the various example modules in the host server 324 of the example of FIG. 3A can be performed by the processing unit 334.
  • FIG. 4A depicts an example functional block diagram of a mobile device 402 that presents augmented information with merchandise objects to a user and processes interactions with the objects in the augmented reality marketplace.
  • The mobile device 402 includes a network interface 404, a timing module 406, a location sensor 408, an identification verifier module 410, an object identifier module 412, a user selection module 414, a user stimulus sensor 416, a motion/gesture sensor 418, an environmental stimulus sensor 420, and/or an audio/video output module 422. The mobile device 402 may be any electronic device such as the devices described in conjunction with the mobile devices 102A-N in the example of FIG. 1 including but not limited to portable devices, a computer, a server, location-aware devices, mobile phones, PDAs, laptops, palmtops, iPhones, cover headsets, heads-up displays, helmet mounted display, head-mounted display, scanned-beam display, Google Glass, smart glasses/goggles, wearable computer such as mobile enabled watches or eyewear, and/or any other mobile interfaces and viewing devices, etc.
  • In one embodiment, the mobile device 402 is coupled to a merchandise object repository 431 for identifying/detecting/recognizing merchandise objects (e.g., physical product or real life service) in the augmented reality environment. The merchandise object repository 431 may be internal to or coupled to the mobile device 402 but the contents stored therein can be illustrated with reference to the example of a merchandise object repository 132 described in the example of FIG. 1.
  • Additional or less modules can be included without deviating from the novel art of this disclosure. In addition, each module in the example of FIG. 4A can include any number and combination of sub-modules, and systems, implemented with any combination of hardware and/or software modules.
  • The mobile device 402, although illustrated as comprised of distributed components (physically distributed and/or functionally distributed), could be implemented as a collective element. In some embodiments, some or all of the modules, and/or the functions represented by each of the modules can be combined in any convenient or known manner. Furthermore, the functions represented by the modules can be implemented individually or in any combination thereof, partially or wholly, in hardware, software, or a combination of hardware and software.
  • In the example of FIG. 4A, the network interface 404 can be a networking device that enables the mobile device 402 to mediate data in a network with an entity that is external to the host server, through any known and/or convenient communications protocol supported by the host and the external entity. The network interface 404 can include one or more of a network adaptor card, a wireless network interface card, a router, an access point, a wireless router, a switch, a multilayer switch, a protocol converter, a gateway, a bridge, bridge router, a hub, a digital media receiver, and/or a repeater.
  • According to the embodiments disclosed herein, the mobile device 402 can facilitate transactions (e.g., e-commerce transactions) of a merchandise object (e.g., a physical product or a real life service) via an augmented reality environment provided by the mobile device 402. The mobile device 402 can detect presence of the physical product or the real life service when the product or service is seen/viewed/looked at by the user or in vicinity of the user. In some embodiments, the mobile device 402 automatically identifies the product or service from a perspective of a user. The perspective can be captured or sensed by the mobile device 402 via, for example, a camera or an image sensor.
  • After detecting/identifying/recognizing the merchandise object, the mobile device 402 can access information relevant to the transaction of the merchandise object(s) from a merchandise object repository (e.g., repository 431 in mobile device 402 and/or repository 332 in host server 324), and can present the relevant information to the user to conduct the transaction of the product or service via the augmented reality environment. For example, the mobile device 402 can render on a display of the device 402
  • More specifically, one embodiment of the mobile device 402 includes a timing module 406. The timing module 406 can be any combination of software agents and/or hardware modules able to identify, detect, transmit, compute, a current time, a time range, and/or a relative time of a request related to simulated objects/environments.
  • The timing module 406 can include a local clock, timer, or a connection to a remote time server to determine time. The timing module 406 can be implemented via any known and/or convenient manner including but not limited to, electronic oscillator, clock oscillator, or various types of crystal oscillators. According to some embodiments, the timing module 406 can provide time data which may be used to assist identification, recognition or detection by the mobile device 402 of the merchandise object. Examples that the mobile device 402 can consider include the time of day of a request, the date of the request, a relative time to another event, the time of year of the request, and/or the time span of a request or activity pertaining to merchandise objects.
  • One embodiment of the mobile device 402 includes a radio frequency (RF) or other types of wireless sensor 407. The RF sensor 407 can be any combination of software agents and/or hardware modules able to detect or identify an RF identification chip or other similar wireless identification tags.
  • One embodiment of the mobile device 402 includes a location sensor 408. The location sensor 408 can be any combination of software agents and/or hardware modules able to identify, detect, transmit, compute, a current location, a previous location, a range of locations, a location at or in a certain time period, and/or a relative location of the mobile device 402. The location sensor 408 can include a local sensor (e.g., GPS sensor) or a connection to an external entity (e.g., a base station) to determine the location information. The location sensor 408 can determine location or relative location of the mobile device 402 via any known or convenient manner including but not limited to, GPS, cell phone tower triangulation, mesh network triangulation, relative distance from another location or device, RF signals, RF fields, optical range finders or grids, etc. The location sensor 408 can identify location data and determine whether the location data satisfies the location parameter of a certain criteria. In some embodiments, the location sensor 408 provides location data to the host server (e.g., host server 324 of FIG. 3A) for the host server to determine whether the criteria is satisfied.
  • One embodiment of the mobile device 402 includes a camera or an image sensor 409. The image sensor 409 can be any combination of software agents and/or hardware modules able to record or capture still images such as photographs or moving images such as video clips or movies.
  • It is noted that a person having ordinary skills in the art will understand that sensor modules 407-409 are exemplary, as the mobile device 402 can include other sensors 410 including, for example, accelerometers, gyroscopic gauges, proximity sensors, light sensors, microphones, temperature sensors, heart beat sensors, touch sensitive sensors including touch-screens, fingerprint sensors, and so forth.
  • The mobile device 402 utilizes suitable sensors to detect presence of the physical product or the real life service in the augmented reality environment or platform. In some embodiments, the mobile device automatically identifies or detects the product(s) or service(s) that falls within a perspective of the user, that is, being viewed/looked at/seen by the user. The product is recognized/identified/detected by using one of several mechanisms as employed by the object identifier module 412, and by querying the merchandise object repository 431.
  • More specifically, mobile device 402 includes the object identifier module 412. The object identifier module 412 can be any combination of software agents and/or hardware modules able to identify, detect, and/or recognize, merchandise objects for presentation to a user. The object identifier module 412, in one embodiment, is coupled to the timing module 406, the RF sensor 407, the location sensor 408, the image sensor 409, and/or other sensors 410. The identifier module 412 can also receive data signals from the motion/gesture sensor 418, the user stimulus sensor 416, the environmental stimulus sensor 420, and is coupled to the merchandise object repository 431 via the network interface 404. The object identifier module 412 is operable to identify the merchandise objects available for access using the device 402.
  • In some embodiments, the presence of the product is detected or identified in the augmented reality environment by a shape or other physical characteristics for presentation via the mobile device to facilitate transactions. For example, the product or service can have an unique combination of shape, size, color, or other physical dimension or characteristics (as registered in the merchandise object repository 431) so that the object identifier module 412 can recognize it. Such shape or physical characteristics can be detected from, for example, the image sensor 409. As previously mentioned, the entries in the repository 431 can also include recognition patterns or photos or videos or metadata that may help identifying the product.
  • In some embodiments, the presence of the product or service is detected or identified in the augmented reality environment by a geographic location of the product or service. For example, the location of the mobile device 402 can assist in identifying (e.g., by pinpointing or ruling out) certain product or service that is only available in that location. Such location data can be detected from, for example, the location sensor 408.
  • In some embodiments, the presence of the product is detected or identified in the augmented reality environment by one or more of: (i) a visual marker; (ii) a marker or tag; (iii) a one-dimensional barcode; or (iv) a multi-dimensional barcode, on the product. For example, a marker on the object such as a QR code or other augmented reality marker can be presented for identification or detection (e.g., via the image sensor 409). In another example, a barcode representing a stock-keeping unit number (SKU) may be present. The barcode can be one-dimensional or multi-dimensional.
  • In some embodiments, the presence of the product is detected or identified in the augmented reality environment by a radio frequency identification (RFID) chip coupled to the product. For example, the RFID chip can be embedded in the product, or on a label attached to the product, or attached on a surface or a nearby location of the product. The RFID chip can be detected and read, for example, via the RF sensor/RFID reader 407.
  • In some embodiments, the presence of the product is detected or identified in the augmented reality environment by optical character recognition (OCR) of written materials on or nearby the product. For example, the image sensor 409 can capture text on labels or other marketing materials on or near the product, and the object identifier module 412 can use well-known OCR techniques to convert the written materials into text for identification (e.g., via a search using the search engine 322 to search merchandise repositories 332, 431).
  • Further, as an additional or an alternative embodiment, the object identifier module 412 can receive input from the user to disambiguate an identity of the product or service. For example, the user's input can include at least partially a name of the product or service. The user can be prompted to name the product in assisting the disambiguation process proactively or interactively. Such user input can be received from, for example, the motion/gesture sensor 418 or the user stimulus sensor 416.
  • Some embodiments of the object identifier module 412 can also receive advisory data associated with the product or service to assist the identifying/recognizing/detecting the product or service. The advisory data being added by another user of the augmented reality environment, or by an administrator or merchant of the augmented reality marketplace. The advisory data can include one or more of: (i) metadata or tags relevant to the product or service, or (ii) an annotation that is descriptive of the product or service. The metadata about the product can be added by another previous user who already identified the product, and/or tagged the product at the location that the mobile device 402 is currently situated.
  • After identifying/detecting/recognizing merchandise objects, external actions can be performed by the user in the real world to interact with the detected merchandise objects. In one embodiment, such actions or requests from the user to perform a requested action on a merchandise object can be received by the user stimulus sensor 416. In one embodiment, the motion/gesture sensor 418 is operable to detect motion of the device 402. The detected motion is used by the user selection module 414 to perform actions responsive to the presented information according to the detected motion of the device. Motion detecting can include detecting velocity and/or acceleration of the device 402 or a gesture of the user handling the device 402. The motion/gesture sensor 418 can include for example, an accelerometer.
  • More specifically, with the merchandise object(s) identified, the mobile device 402 can present (e.g., using the user selection module 414 and/or audio/video output module 422) information relevant to the merchandise product or service to the user so that the user can initiate various actions responsive to the presented information, for example, initiating the transaction of the product or service via the mobile device 402. Other example actions can include, for example, get price, compare price, read reviews, get availability, add to shopping cart, call a sales representative (e.g., a local sales person or a virtual one) for assistance, buy the product, tag the product or add metadata for the product, share the product with another user, or bookmark the product for later consideration. The relevant information, including transaction information, can be transmitted from the repository 431, which may be local or may be hosted by an online server (e.g., host server 324, 224, which may also include or be coupled to e-commerce server 213) that enables electronic commerce.
  • In some embodiments, a user interface generator in the user selection module 414 can be used to create buttons/links/etc. that are selectable by the user. The interactions or actions from the user can be detected or determined using the user selection module 414 (e.g., via interpreting signals generated from the motion/gesture sensor 418, user stimulus sensor 416, and/or the environmental stimulus sensor 420). Any response to the user's selection or action can be reflected (e.g., highlighting a selected merchandise, blinking of a selected button, displaying an annotation or review by another user, or displaying a short video clip of the selected merchandise) in the augmented reality environment via the audio/video output module 422.
  • In some embodiments, the information is rendered translucently and disposed adjacently or partially overlaid with the product or service depicted in the augmented reality environment on the mobile device 402. In some embodiments, the information includes advisory data associated with the product or service. Examples of advisory data can include metadata or tags that are deemed (either automatically by the object identifier module 412, or by another user, or by a merchant or an administrator) relevant to the product or service, or (ii) an annotation that is descriptive of the product or service.
  • The mobile device 402 can also prompt the user to enter an advisory data for the product or service by, for example, operating a virtual keyboard using the user stimulus sensor 416 and/or motion/gesture sensor 418. Then, the user can select to share (e.g., via network interface 404) the entry to another user of the augmented reality environment.
  • The content of the interactions can also be decided based on environmental factors including, for example, physical location of the user and/or the merchandise, user action, user feedback/annotation/review/tags/etc. as provided by the user of the device 402 or another user of the augmented reality platform using another device and/or at another physical location, motion/movement of the device 402, gesture of the user using the device 402.
  • In some embodiments, an advertisement relevant to the product or service can be identified and displayed in the augmented reality environment via the mobile device 402. The advertisement is identified or detected based on one or more of: (i) location, (ii) time, (iii) interest or profile of a user of the mobile device 402. For example, the environmental stimulus sensor 420 can detect environmental factors or changes in environmental factors surrounding the real environment in which the device 402 is located, and the advertisement can be provided based on these factors or changes. Environmental factors can include, weather, temperature, topographical characters, density, surrounding businesses, buildings, living objects, etc. These factors or changes in them can also affect the positioning and/or characters of the presented information for the merchandise objects in the augmented reality in which they are presented to a user via the device 402.
  • In some embodiments, the advertisement is identified or detected based on a pointing direction of the mobile device 402. In some embodiments, the advertisement is identified or detected based on velocity or speed of movement of the mobile device, or based on an ambient temperature in which the mobile device physically resides, or based on a plurality of sensors (e.g., sensors 406-410) on the mobile device 402. Further, the advertisement is identified or detected (e.g., by a combined reading and interpretation of data signals from the location sensor 408, the image sensor 409, and a directional sensor which may be included among other sensors 410) based on a landmark which the mobile device is pointed at or in a vicinity of.
  • In some embodiments, the information includes a contact information of a sales representative of the product or service, and the mobile device 402 prompts the user to contact the sale representative based on the contact information.
  • Additionally, in some embodiments, the mobile device 402 allows the user to bookmark the product or service, and in some embodiments, the mobile device 402 allows the user to share the bookmark with another user of the augmented reality environment.
  • According to some additional or alternative embodiments, the mobile device 402, in providing the augmented reality environment, can enable the user to select a physical product or service that is in the augmented reality. The mobile device 402 can detect one or more targets in the augmented reality platform using a select area on a perspective of a user. The perspective can be captured, for example, by a camera of the mobile device 402. After the targets are detected, the mobile device 402 prompts the user to choose an object of interest from the one or more detected targets.
  • More specifically, the mobile device 402 can detect one or more targets in the augmented reality platform using a select area in a perspective of a user, and prompt the user to choose an object of interest from the one or more detected targets. As previously mentioned, the perspective can be captured by the mobile device 402 (e.g., via the image sensor 409. For example, just as a user using a mouse to select items on a personal computer's screen, some embodiments disclosed herein enable selection of real and/or virtual items in an augmented reality environment/platform/marketplace, which in turn enables actions to be taken in relation to the selected items (e.g., the aforementioned transactions of or interactions with the merchandise objects).
  • The detection of targets can be performed, for example, by object identifier module 412 of the mobile device 402. According to some embodiments, the detection of targets in the augmented reality environment can be performed by pointing the device 402 at the target(s). In some embodiments, the detection of targets can be performed by moving a pointer or a select area on the display of the device 402 to point at or frame the object in a reticle or a circular or rectangular frame (e.g., select area 520, 525, described below with respect to FIG. 5A). It is noted that the select area can be any suitable shape and sizes. In some embodiments, the mobile device 402 can provide (e.g., via audio/video output module 422 to a display of the mobile device 402) the user with a lasso or a selection tool in the perspective to surround a respective target so as to form the select area. Additionally, the mobile device 402 can prompt the user to choose the object of interest from a set of choices such as a number of targets that are recognized in the perspective.
  • In some instances, for example, when target is not recognized automatically or when the user so desires, the mobile device 402 can prompt the user, or the user can select, to enter a learning mode in which the mobile device 402 can receive assistance from the user to perform the detection or recognition of the target. The assistance can include one or more of (i) entering at least partially a name, (ii) entering metadata, (iii) defining a boundary via a pointer, or (iv) taking snapshots from a plurality of views to develop a recognition pattern, for a respective target. For example, the teaching can be done by aiming at the target or circling around the target with the interface (e.g., using the select area or the lasso tool, provided by the interface generator module in the user selection module 414) on the device 402, and then by selecting a “learn” function or button on the interface in the augmented reality environment. Then, the mobile device 402 can ask the user for a name or other metadata for the target, and may ask for assistance in defining the boundaries of the object (e.g., via the lasso tool). The device 402 can also ask the user to take a series of snapshot views (e.g., via camera or image sensor 409) of the target to develop a pattern recognition database for it, as well as potentially a series of tests to improve recognition.
  • After targets are recognized, they can then be selected by the user for actions or interactions. The mobile device 402 can receive the user's choice of the object of interest by detecting movement and selection by a selection tool or a pointer, or movement and selection represented by a gesture (e.g., via the motion/gesture sensor 418, user stimulus sensor 416, and/or hand/finger/other gestures captured by the image sensor 409).
  • More specifically, in detecting the user's gesture to move a pointer or targeting or selection tool, and/or to select an object, the mobile device 402 can perform capturing, via its various sensors, the gesture from one or more of: (i) movements or non-movements of an eye of the user, (ii) locations of a focal point of an eye of the user, or (iii) movements of an eye lid of the user. Additionally, the mobile device 402 can capture the gesture from one or more of: (i) movements of hand or finger as recognized by a camera of the mobile device, (ii) movements of a virtual pointer controlled by a wireless peripheral device, (iv) movements of a virtual pointer controlled by a touch-sensitive display of the mobile device, (v) movements of the mobile device itself, or (vi) movements of the user's head, limbs, or torso. The capturing can be further based on a speed or velocity of the movements. As such, in light of the above, the present embodiments can capture or identify gestures from, for example, winking of the user and/or an eye focus or eye foci of the user. Another example of gesture controlling can include finger or arm gesturing as captured by camera and/or distance detector/proximity detectors, so that the user can perform “spatial” or “virtual” gesturing in the air or other detectable spaces with similar gestures as those well-known gestures applicable to a mobile phone's touch screen. Yet another example of gesture controlling can include eye ball motion tracking and/or eye focal point tracking In this way, the user of mobile device 402 may operate various selection mechanisms, for example, using his or her eyes (e.g., via eye movement tracking) or by moving his or her hands/arms/fingers in the perspective to make specific gestures such as pointing or tracing the outline of some object, or by operating a virtual pointer in the scene using a handheld peripheral such as a wireless pointing device or a mouse equivalent, or by touching a touch-sensitive display on a mobile device and gesturing on it to indicate actions and selections, or by moving the device itself with specific gestures and velocities to use the device as a pointer or selection tool. Additional gestures may include eye tracking and determining a focus of the eye for targeting things, and/or blinking to select a target that is in the focal point, to take a photo, or to select a button, etc.
  • After receiving the user's selection or choice of the object of interest, the mobile device 402 can optionally confirm with the user of the choice of the object of interest. The confirmation can include highlighting or outlining the target in the augmented reality platform by employing, for example, the audio/video output module 422.
  • FIG. 4B depicts an example block diagram of the mobile device 402 that presents augmented information with merchandise objects in the augmented reality marketplace to a user and facilitates user interactions with the objects.
  • In one embodiment, mobile device 402 includes a network interface 432, a processing unit 434, a memory unit 436, a storage unit 438, a location sensor 440, an accelerometer/motion sensor 442, an audio output unit/speakers 446, a display unit 450, an image capture unit 452, a pointing device/sensor 454, a input device 456, and/or a touch screen sensor 458. Additional or less units or modules may be included. The mobile device 402 can be any combination of hardware components and/or software agents for that presenting information with respect to merchandise objects to a user and facilitating user interactions with the merchandise objects. The network interface 432 has been described in the example of FIG. 4A.
  • One embodiment of the mobile device 402 further includes a processing unit 434. The location sensor 440, accelerometer/motion sensor 442, and timer 444 have been described with reference to the example of FIG. 4A.
  • The processing unit 434 can include one or more processors, CPUs, microcontrollers, FPGAs, ASICs, DSPs, or any combination of the above. Data that is input to the mobile device 402 for example, via the image capture unit 452, pointing device/sensor 554, input device 456 (e.g., keyboard), and/or the touch screen sensor 458 can be processed by the processing unit 434 and output to the display unit 450, audio output unit/speakers 446 and/or output via a wired or wireless connection to an external device, such as a host or server computer that generates and controls access to simulated objects by way of a communications component.
  • One embodiment of the mobile device 402 further includes a memory unit 436 and a storage unit 438. The memory unit 436 and a storage unit 438 are, in some embodiments, coupled to the processing unit 434. The memory unit can include volatile and/or non-volatile memory. In generating and controlling access to the simulated objects, the processing unit 434 may perform one or more processes related to presenting information with respect to merchandise objects to a user and/or facilitating user interactions with the merchandise objects based on the presented information.
  • In some embodiments, any portion of or all of the functions described of the various example modules in the mobile device 402 of the example of FIG. 4A can be performed by the processing unit 434. In particular, with reference to the mobile device illustrated in FIG. 4A, various sensors and/or modules can be performed via any of the combinations of modules in the control subsystem that are not illustrated, including, but not limited to, the processing unit 434 and/or the memory unit 436.
  • FIG. 5A illustrates an example application where mobile devices 502 such as a smart phone 502A or a smart goggle 502B having perspective interfaces 510, 515 which can be used to facilitate electronic commerce in an augmented reality marketplace.
  • With the embodiments disclosed herein, for example, when the user is doing “window shopping,” that is, when the user is looking at a merchandise product in a store window, or on a shelf, or when the user is near a physical store, the user can utilize the mobile devices 502A, 502B to access information 504A, 504B regarding the identified/detected/recognized merchandise objects. The merchandise objects can be selected using select areas 520, 525 in ways that are discussed above. Then, the user can select to perform actions 506 (including transactions) to the merchandise objects in ways that are discussed above. Thus, for example, through the perspective interface 510, 515 in the augmented reality environment, the user can get price of, add comments/annotations/tags for, read reviews of, get availability of, or select to purchase the selected merchandise object. The objects can be identified by ways discussed above including, for example, a barcode 508 attached to the merchandise object. Written materials for identifying objects and/or other relevant information (such as sales event information 530 as illustrated in FIG. 5A) can also be recognized/detected by the mobile devices 502A, 502B.
  • Other example augmented information can be provided to the user to inform the user if the store is now open, what is on sale now, etc. Similarly, other sales information including advertisement, video clips, images, or sales representative's contact information can be presented or otherwise made accessible to the user. The information can also include entries from other users of the augmented reality, including but not limited to, their names and profiles, recent blog or microblog posts, recent photos or videos or links or documents added or annotated by them, their recent locations, their professions, etc.
  • FIG. 5B illustrates an example screenshot on a device 502 displaying an augmented reality environment 520 with objects 522 (including merchandise objects) associated with physical objects and/or real life services 526 in a real world environment 524.
  • As aforementioned, augmented reality environment 520 reflects a perspective of the user as captured by the mobile device 502 (e.g., via a camera of the device 502). As such, the display and movement of the object 522 in the augmented reality environment 520 corresponds to the display and movement of the physical object 526 (e.g., car) in the real world 524.
  • In augmented reality environment 502, relevant information regarding the physical object 526 can be rendered or presented with the object 522 so as to “augment” the object 522. Such information or data can be about the people or entities that are in or near a particular geographical location. Further, the mobile device 502 can facilitate transactions of the real physical product or service 526 based on the information, so that transactions can be conducted by users via devices when the user is physically near or at the physical product or the real life service.
  • FIG. 6 depicts a flow chart illustrating an example process 600 for facilitating user interaction with a merchandise object that is identified in an augmented reality environment. With reference to FIGS. 1, 3A, 4A, and 5A, process 600 is now described.
  • First, a mobile device (e.g., devices 102A-102N, FIG. 1; device 402, FIG. 4; devices 502A, 502B, FIG. 5A) detects (610) presence of a physical product or a real life service. The mobile device 102, 402, 502 provides the augmented reality environment (e.g., through perspective interfaces 510, 515, FIG. 5A) to the user. In some embodiments, the device 102, 402, 502 automatically identifies (615) the product or service from the perspective 510, 515 of the user captured by the mobile device (e.g., via image sensor 409, FIG. 4A).
  • Then, the mobile device 102, 402, 502 can query a merchandise object repository (e.g., repository 132, FIG. 1; repository 332, FIG. 3A; repository 431, FIG. 4A) in identifying the merchandise product or service and obtaining relevant information. Thereafter, the mobile device 102, 402, 502 presents (620), on the mobile device (e.g., on user interface 104A-104N, FIG. 1, or on perspective 510, 515), information (e.g., information 504A, 504B, FIG. 5A) to conduct the transaction of the product or service via the augmented reality environment. In some embodiments, the device 102, 402, 502 renders (625), on a display of the mobile device (e.g., via audio/video output module 422, FIG. 4A), a transaction information associated with the product or service.
  • Next, in some embodiments, the mobile device 102, 402, 502 can receive (630) input from the user to disambiguate an identity of the product or service. Further, in some embodiments, the mobile device 102, 402, 502 can receive (640) advisory data associated with the product or service. The advisory data can be added, for example, by another user of the augmented reality environment.
  • After the information is presented, the mobile device 102, 402, 502 can initiate (650) the transaction of the product or service via the mobile device based on responses to the presented information (e.g., as detected via various sensors 406-410, 416, 418, 420 of device 402 and determined by the user selection module 414, FIG. 4A) from the user.
  • FIG. 7 depicts a flow chart illustrating an example process 700 for selecting/identifying a merchandise object in an augmented reality environment. With reference to FIGS. 1, 3A, 4A, and 5A, process 700 is now described.
  • First, a mobile device (e.g., devices 102A-102N, FIG. 1; device 402, FIG. 4; devices 502A, 502B, FIG. 5A) detects (710) one or more targets in an augmented reality platform using a select area (e.g., select areas 520, 525, FIG. 5A) in a perspective (e.g., perspective 510, 515, FIG. 5A) of a user. The perspective 510, 515 is captured by the mobile device 102, 402, 502 (e.g., via image sensor 409, FIG. 4A). In some embodiments, the mobile device 102, 402, 502 provides (712), via a display of the mobile device (e.g., via audio/video output module 422, FIG. 4A), the user with a lasso tool in the perspective 510, 515 to surround a respective target so as to form the select area 520, 525. In accordance with some embodiments, the mobile device 102, 402, 502 prompts (714), via audio/video output module 422, the user to enter a learning mode in which the mobile device 102, 402, 502 receives (714) an assistance from the user to perform the detection.
  • After detecting the targets, the mobile device 102, 402, 502 prompts (720) the user to choose an object of interest from the one or more detected targets. Then, the mobile device 102, 402, 502 receives (730) the user's choice of the object of interest (e.g., via motion/gesture sensor 418, user stimulus sensor 416, and/or other gestures captured by image sensor 409, as determined by the user selection module 414, FIG. 4A). The mobile device 102, 402, 502 can detect (732) movement and selection by a selection tool or a pointer, and/or movement and selection represented by a gesture. Additionally, the mobile device 102, 402, 502 can confirm (740) with the user of the choice of the object of interest.
  • FIG. 8 shows a diagrammatic representation of a machine in the example form of a computer system 800 within which a set of instructions, for causing the machine to perform any one or more of the methodologies discussed herein, may be executed.
  • In alternative embodiments, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server or a client machine in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment.
  • The machine may be a server computer, a client computer, a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a cellular telephone, a web appliance, a network router, switch or bridge, or any machine capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by that machine.
  • While the machine-readable medium is shown in an exemplary embodiment to be a single medium, the term “machine-readable medium” should be taken to include a single medium or multiple media (e.g., a centralized or distributed database, and/or associated caches and servers) that store the one or more sets of instructions. The term “machine-readable medium” shall also be taken to include any medium that is capable of storing, encoding or carrying a set of instructions for execution by the machine and that cause the machine to perform any one or more of the methodologies of the present disclosure.
  • In general, the routines executed to implement the embodiments of the disclosure, may be implemented as part of an operating system or a specific application, component, program, object, module or sequence of instructions referred to as “computer programs.” The computer programs typically comprise one or more instructions set at various times in various memory and storage devices in a computer, and that, when read and executed by one or more processors in a computer, cause the computer to perform operations to execute elements involving the various aspects of the disclosure.
  • Moreover, while embodiments have been described in the context of fully functioning computers and computer systems, those skilled in the art will appreciate that the various embodiments are capable of being distributed as a program product in a variety of forms, and that the disclosure applies equally regardless of the particular type of machine or computer-readable media used to actually effect the distribution.
  • Further examples of machine or computer-readable media include but are not limited to recordable type media such as volatile and non-volatile memory devices, floppy and other removable disks, hard disk drives, optical disks (e.g., Compact Disk Read-Only Memory (CD ROMS), Digital Versatile Disks, (DVDs), etc.), among others, and transmission type media such as digital and analog communication links.
  • Unless the context clearly requires otherwise, throughout the description and the claims, the words “comprise,” “comprising,” and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense; that is to say, in the sense of “including, but not limited to.” As used herein, the terms “connected,” “coupled,” or any variant thereof, means any connection or coupling, either direct or indirect, between two or more elements; the coupling of connection between the elements can be physical, logical, or a combination thereof. Additionally, the words “herein,” “above,” “below,” and words of similar import, when used in this application, shall refer to this application as a whole and not to any particular portions of this application. Where the context permits, words in the above Detailed Description using the singular or plural number may also include the plural or singular number respectively. The word “or,” in reference to a list of two or more items, covers all of the following interpretations of the word: any of the items in the list, all of the items in the list, and any combination of the items in the list.
  • The above detailed description of embodiments of the disclosure is not intended to be exhaustive or to limit the teachings to the precise form disclosed above. While specific embodiments of, and examples for, the disclosure are described above for illustrative purposes, various equivalent modifications are possible within the scope of the disclosure, as those skilled in the relevant art will recognize. For example, while processes or blocks are presented in a given order, alternative embodiments may perform routines having steps, or employ systems having blocks, in a different order, and some processes or blocks may be deleted, moved, added, subdivided, combined, and/or modified to provide alternative or subcombinations. Each of these processes or blocks may be implemented in a variety of different ways. Also, while processes or blocks are at times shown as being performed in series, these processes or blocks may instead be performed in parallel, or may be performed at different times. Further any specific numbers noted herein are only examples: alternative implementations may employ differing values or ranges.
  • The teachings of the disclosure provided herein can be applied to other systems, not necessarily the system described above. The elements and acts of the various embodiments described above can be combined to provide further embodiments.
  • Any patents and applications and other references noted above, including any that may be listed in accompanying filing papers, are incorporated herein by reference. Aspects of the disclosure can be modified, if necessary, to employ the systems, functions, and concepts of the various references described above to provide yet further embodiments of the disclosure.
  • These and other changes can be made to the disclosure in light of the above Detailed Description. While the above description describes certain embodiments of the disclosure, and describes the best mode contemplated, no matter how detailed the above appears in text, the teachings can be practiced in many ways. Details of the system may vary considerably in its implementation details, while still being encompassed by the subject matter disclosed herein. As noted above, particular terminology used when describing certain features or aspects of the disclosure should not be taken to imply that the terminology is being redefined herein to be restricted to any specific characteristics, features, or aspects of the disclosure with which that terminology is associated. In general, the terms used in the following claims should not be construed to limit the disclosure to the specific embodiments disclosed in the specification, unless the above Detailed Description section explicitly defines such terms. Accordingly, the actual scope of the disclosure encompasses not only the disclosed embodiments, but also all equivalent ways of practicing or implementing the disclosure under the claims.
  • While certain aspects of the disclosure are presented below in certain claim forms, the inventors contemplate the various aspects of the disclosure in any number of claim forms. For example, while only one aspect of the disclosure is recited as a means-plus-function claim under 35 U.S.C. § 112, ¶ 6, other aspects may likewise be embodied as a means-plus-function claim, or in other forms, such as being embodied in a computer-readable medium. (Any claims intended to be treated under 35 U.S.C. §112, ¶6 begins with the words “means for”.) Accordingly, the applicant reserves the right to add additional claims after filing the application to pursue such additional claim forms for other aspects of the disclosure.

Claims (26)

What is claimed is:
1. A method for selecting a physical product or a real life service in an augmented reality platform via a mobile device, the method comprising:
detecting one or more targets in the augmented reality platform using a select area in a perspective of a user, the perspective being captured by a mobile device; and
prompting the user to choose an object of interest from the one or more detected targets.
2. The method of claim 1, further comprising:
providing, via a display of the mobile device, the user with a lasso tool in the perspective to surround a respective target so as to form the select area.
3. The method of claim 1, further comprising:
prompting, via a display of the mobile device, the user to enter a learning mode in which the mobile device receives an assistance from the user to perform the detection.
4. The method of claim 3, wherein the assistance includes one or more of:
(i) entering at least partially a name, (ii) entering metadata, (iii) defining a boundary via a pointer, or (iv) taking snapshots from a plurality of views to develop a recognition pattern, for a respective target.
5. The method of claim 1, further comprising:
receiving, via the mobile device, the user's choice of the object of interest.
6. The method of claim 5, further comprising:
detecting, via the mobile device, one or more of: (i) movement and selection by a selection tool or a pointer, or (ii) movement and selection represented by a gesture.
7. The method of claim 6, further comprising:
capturing, via the mobile device, the gesture from one or more of: (i) movements or non-movements of an eye of the user, (ii) locations of a focal point of an eye of the user, or (iii) movements of an eye lid of the user.
8. The method of claim 6, further comprising:
capturing, via the mobile device, the gesture from one or more of: (i) movements of hand or finger as recognized by a camera of the mobile device, (ii) movements of a virtual pointer controlled by a wireless peripheral device, (iv) movements of a virtual pointer controlled by a touch-sensitive display of the mobile device, or (v) movements of the mobile device itself.
9. The method of claim 8, wherein the capturing is further based on a speed or velocity of the movements.
10. The method of claim 5, further comprising:
confirming, via the mobile device, with the user of the choice of the object of interest.
11. The method of claim 5, wherein the confirmation comprises highlighting or outlining the target in the augmented reality platform.
12. The method of claim 1, wherein the device is in form of an electronic eyewear or a head-mounted device.
13. The method of claim 1, wherein the target is a physical or a virtual goods.
14. A mobile device for an augmented reality platform, the device comprising:
a processor; and
a memory coupled to the processor and storing a plurality of instructions which, when executed by the processor, cause the processor to:
detect one or more targets in the augmented reality platform using a select area in a perspective of a user, the perspective being captured by the mobile device; and
prompt the user to choose an object of interest from the one or more detected targets.
15. The device of claim 14, wherein the processor is further instructed to:
provide, via a display of the mobile device, the user with a lasso tool in the perspective to surround a respective target so as to form the select area.
16. The device of claim 14, wherein the processor is further instructed to:
prompt, via a display of the mobile device, the user to enter a learning mode in which the mobile device receives an assistance from the user to perform the detection.
17. The device of claim 16, wherein the assistance includes one or more of:
(i) entering at least partially a name, (ii) entering metadata, (iii) defining a boundary via a pointer, or (iv) taking snapshots from a plurality of views to develop a recognition pattern, for a respective target.
18. The device of claim 14, wherein the processor is further instructed to:
receive, via the mobile device, the user's choice of the object of interest.
19. The device of claim 18, wherein the processor is further instructed to:
detect, via the mobile device, one or more of: (i) movement and selection by a selection tool or a pointer, or (ii) movement and selection represented by a gesture.
20. The device of claim 19, wherein the processor is further instructed to:
capture, via the mobile device, the gesture from one or more of: (i) movements or non-movements of an eye of the user, (ii) locations of a focal point of an eye of the user, or (iii) movements of an eye lid of the user.
21. The device of claim 19, wherein the processor is further instructed to:
capture, via the mobile device, the gesture from one or more of: (i) movements of hand or finger as recognized by a camera of the mobile device, (ii) movements of a virtual pointer controlled by a wireless peripheral device, (iv) movements of a virtual pointer controlled by a touch-sensitive display of the mobile device, or (v) movements of the mobile device itself.
22. The device of claim 21, wherein the processor is further instructed to:
capture based on a speed or velocity of the movements.
23. The device of claim 18, wherein the processor is further instructed to:
confirm, via the mobile device, with the user of the choice of the object of interest.
24. The device of claim 23, wherein the confirmation comprises highlighting or outlining the target in the augmented reality platform.
25. The device of claim 14, wherein the device is in form of an electronic eyewear or a head-mounted device.
26. The device of claim 14, wherein the target is a physical or a virtual goods.
US13/874,421 2012-05-01 2013-04-30 System and method for selecting targets in an augmented reality environment Abandoned US20130293580A1 (en)

Priority Applications (6)

Application Number Priority Date Filing Date Title
US13/874,421 US20130293580A1 (en) 2012-05-01 2013-04-30 System and method for selecting targets in an augmented reality environment
PCT/US2013/039123 WO2013166191A2 (en) 2012-05-01 2013-05-01 System and method for facilitating transactions of a physical product or real life service via an augmented reality environment
US14/939,889 US10388070B2 (en) 2012-05-01 2015-11-12 System and method for selecting targets in an augmented reality environment
US16/543,011 US10878636B2 (en) 2012-05-01 2019-08-16 System and method for selecting targets in an augmented reality environment
US17/101,429 US11417066B2 (en) 2012-05-01 2020-11-23 System and method for selecting targets in an augmented reality environment
US17/887,075 US20220392178A1 (en) 2012-05-01 2022-08-12 System and method for selecting targets in an augmented reality environment

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201261640856P 2012-05-01 2012-05-01
US13/874,421 US20130293580A1 (en) 2012-05-01 2013-04-30 System and method for selecting targets in an augmented reality environment

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US14/939,889 Continuation US10388070B2 (en) 2012-05-01 2015-11-12 System and method for selecting targets in an augmented reality environment

Publications (1)

Publication Number Publication Date
US20130293580A1 true US20130293580A1 (en) 2013-11-07

Family

ID=49512197

Family Applications (9)

Application Number Title Priority Date Filing Date
US13/874,421 Abandoned US20130293580A1 (en) 2012-05-01 2013-04-30 System and method for selecting targets in an augmented reality environment
US13/874,404 Abandoned US20130297460A1 (en) 2012-05-01 2013-04-30 System and method for facilitating transactions of a physical product or real life service via an augmented reality environment
US14/249,541 Active 2034-06-10 US9665983B2 (en) 2012-05-01 2014-04-10 Method, medium, and system for facilitating electronic commercial transactions in an augmented reality environment
US14/755,553 Abandoned US20150302517A1 (en) 2012-05-01 2015-06-30 System and method for facilitating transactions of a physical product or real life service via an augmented reality environment
US14/939,889 Active US10388070B2 (en) 2012-05-01 2015-11-12 System and method for selecting targets in an augmented reality environment
US15/863,698 Active US10127735B2 (en) 2012-05-01 2018-01-05 System, method and apparatus of eye tracking or gaze detection applications including facilitating action on or interaction with a simulated object
US16/543,011 Active US10878636B2 (en) 2012-05-01 2019-08-16 System and method for selecting targets in an augmented reality environment
US17/101,429 Active US11417066B2 (en) 2012-05-01 2020-11-23 System and method for selecting targets in an augmented reality environment
US17/887,075 Pending US20220392178A1 (en) 2012-05-01 2022-08-12 System and method for selecting targets in an augmented reality environment

Family Applications After (8)

Application Number Title Priority Date Filing Date
US13/874,404 Abandoned US20130297460A1 (en) 2012-05-01 2013-04-30 System and method for facilitating transactions of a physical product or real life service via an augmented reality environment
US14/249,541 Active 2034-06-10 US9665983B2 (en) 2012-05-01 2014-04-10 Method, medium, and system for facilitating electronic commercial transactions in an augmented reality environment
US14/755,553 Abandoned US20150302517A1 (en) 2012-05-01 2015-06-30 System and method for facilitating transactions of a physical product or real life service via an augmented reality environment
US14/939,889 Active US10388070B2 (en) 2012-05-01 2015-11-12 System and method for selecting targets in an augmented reality environment
US15/863,698 Active US10127735B2 (en) 2012-05-01 2018-01-05 System, method and apparatus of eye tracking or gaze detection applications including facilitating action on or interaction with a simulated object
US16/543,011 Active US10878636B2 (en) 2012-05-01 2019-08-16 System and method for selecting targets in an augmented reality environment
US17/101,429 Active US11417066B2 (en) 2012-05-01 2020-11-23 System and method for selecting targets in an augmented reality environment
US17/887,075 Pending US20220392178A1 (en) 2012-05-01 2022-08-12 System and method for selecting targets in an augmented reality environment

Country Status (2)

Country Link
US (9) US20130293580A1 (en)
WO (1) WO2013166191A2 (en)

Cited By (185)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130307872A1 (en) * 2012-05-17 2013-11-21 International Business Machines Corporation Integrating Remote Content with Local Content
US20130328926A1 (en) * 2012-06-08 2013-12-12 Samsung Electronics Co., Ltd Augmented reality arrangement of nearby location information
US20130328925A1 (en) * 2012-06-12 2013-12-12 Stephen G. Latta Object focus in a mixed reality environment
US20140002643A1 (en) * 2012-06-27 2014-01-02 International Business Machines Corporation Presentation of augmented reality images on mobile computing devices
US20140198035A1 (en) * 2013-01-14 2014-07-17 Thalmic Labs Inc. Wearable muscle interface systems, devices and methods that interact with content displayed on an electronic display
US20140214597A1 (en) * 2013-01-30 2014-07-31 Wal-Mart Stores, Inc. Method And System For Managing An Electronic Shopping List With Gestures
US20140236652A1 (en) * 2013-02-19 2014-08-21 Wal-Mart Stores, Inc. Remote sales assistance system
US20140292636A1 (en) * 2013-03-29 2014-10-02 Plantronics, Inc. Head-Worn Infrared-Based Mobile User-Interface
CN104102412A (en) * 2014-07-24 2014-10-15 央数文化(上海)股份有限公司 Augmented reality technology-based handheld reading equipment and reading method thereof
US20150049113A1 (en) * 2013-08-19 2015-02-19 Qualcomm Incorporated Visual search in real world using optical see-through head mounted display with augmented reality and user interaction tracking
US9007401B1 (en) 2014-02-20 2015-04-14 Lg Electronics Inc. Head mounted display and method for controlling the same
US20150124106A1 (en) * 2013-11-05 2015-05-07 Sony Computer Entertainment Inc. Terminal apparatus, additional information managing apparatus, additional information managing method, and program
WO2015130773A1 (en) * 2014-02-28 2015-09-03 Infomotion Sports Technologies, Inc. Sporting device and wearable computer interaction
US20150294389A1 (en) * 2014-04-10 2015-10-15 Bank Of America Corporation Providing comparison shopping experiences through an optical head-mounted displays in a wearable computer
US20150294284A1 (en) * 2011-11-21 2015-10-15 Nant Holdings Ip, Llc Subscription Bill Service, Systems and Methods
US20150317835A1 (en) * 2014-05-02 2015-11-05 Cisco Technology, Inc. Automated patron guidance
US20150317836A1 (en) * 2014-05-05 2015-11-05 Here Global B.V. Method and apparatus for contextual query based on visual elements and user input in augmented reality at a device
US20150329122A1 (en) * 2012-08-08 2015-11-19 C/O Sony Corporation Mobile object, system, and storage medium
US20150348427A1 (en) * 2014-06-03 2015-12-03 David Peregrim Exploiting Smart Vision Devices to Train Sports Team Tactics
US20160018648A1 (en) * 2014-01-21 2016-01-21 Osterhout Group, Inc. See-through computer display systems
USD748123S1 (en) * 2012-02-03 2016-01-26 Symantec Corporation Display screen with graphical user interface
US20160048024A1 (en) * 2014-08-13 2016-02-18 Beijing Lenovo Software Ltd. Information processing method and electronic device
US20160066147A1 (en) * 2014-09-02 2016-03-03 Qualcomm Incorporated Techniques for allocating positioning signal acquisition tasks among a plurality of co-located mobile devices
US20160078512A1 (en) * 2014-09-11 2016-03-17 Oracle International Corporation Character personal shopper system
US9298283B1 (en) * 2015-09-10 2016-03-29 Connectivity Labs Inc. Sedentary virtual reality method and systems
US20160140759A1 (en) * 2014-11-13 2016-05-19 Mastercard International Incorporated Augmented reality security feeds system, method and apparatus
US9354711B2 (en) * 2014-09-30 2016-05-31 Xerox Corporation Dynamic hand-gesture-based region of interest localization
JP2016110565A (en) * 2014-12-10 2016-06-20 富士通株式会社 Display control method, information processing program and information processing device
US20160224110A1 (en) * 2013-10-14 2016-08-04 Suricog Method of interaction by gaze and associated device
CN106062783A (en) * 2013-12-17 2016-10-26 亚马逊科技公司 Pointer tracking for eye-level scanners and displays
US9494800B2 (en) 2014-01-21 2016-11-15 Osterhout Group, Inc. See-through computer display systems
US9529195B2 (en) 2014-01-21 2016-12-27 Osterhout Group, Inc. See-through computer display systems
US9529192B2 (en) 2014-01-21 2016-12-27 Osterhout Group, Inc. Eye imaging in head worn computing
US9547465B2 (en) 2014-02-14 2017-01-17 Osterhout Group, Inc. Object shadowing in head worn computing
US9575321B2 (en) 2014-06-09 2017-02-21 Osterhout Group, Inc. Content presentation in head worn computing
JP2017049833A (en) * 2015-09-02 2017-03-09 株式会社東芝 Electronic apparatus and support method
US9594246B2 (en) 2014-01-21 2017-03-14 Osterhout Group, Inc. See-through computer display systems
US9606701B1 (en) 2013-10-14 2017-03-28 Benko, LLC Automated recommended joining data with presented methods for joining in computer-modeled structures
US20170090850A1 (en) * 2015-09-29 2017-03-30 International Business Machines Corporation Enabling proximity-aware visual identification
US9613020B1 (en) 2014-09-15 2017-04-04 Benko, LLC Natural language user interface for computer-aided design systems
US9615742B2 (en) 2014-01-21 2017-04-11 Osterhout Group, Inc. Eye imaging in head worn computing
CN106601043A (en) * 2016-11-07 2017-04-26 爱可目(北京)科技股份有限公司 Multimedia interaction education device and multimedia interaction education method based on augmented reality
US9651787B2 (en) 2014-04-25 2017-05-16 Osterhout Group, Inc. Speaker assembly for headworn computer
US9651784B2 (en) 2014-01-21 2017-05-16 Osterhout Group, Inc. See-through computer display systems
US9665983B2 (en) 2012-05-01 2017-05-30 Zambala, Lllp Method, medium, and system for facilitating electronic commercial transactions in an augmented reality environment
US9672210B2 (en) 2014-04-25 2017-06-06 Osterhout Group, Inc. Language translation with head-worn computing
US9671613B2 (en) 2014-09-26 2017-06-06 Osterhout Group, Inc. See-through computer display systems
US9684172B2 (en) 2014-12-03 2017-06-20 Osterhout Group, Inc. Head worn computer display systems
USD792400S1 (en) 2014-12-31 2017-07-18 Osterhout Group, Inc. Computer glasses
US9715112B2 (en) 2014-01-21 2017-07-25 Osterhout Group, Inc. Suppression of stray light in head worn computing
US9720234B2 (en) 2014-01-21 2017-08-01 Osterhout Group, Inc. See-through computer display systems
USD794637S1 (en) 2015-01-05 2017-08-15 Osterhout Group, Inc. Air mouse
US9740280B2 (en) 2014-01-21 2017-08-22 Osterhout Group, Inc. Eye imaging in head worn computing
US9746686B2 (en) 2014-05-19 2017-08-29 Osterhout Group, Inc. Content position calibration in head worn computing
US9753288B2 (en) 2014-01-21 2017-09-05 Osterhout Group, Inc. See-through computer display systems
US9766463B2 (en) 2014-01-21 2017-09-19 Osterhout Group, Inc. See-through computer display systems
US9778750B2 (en) 2014-09-30 2017-10-03 Xerox Corporation Hand-gesture-based region of interest localization
US9784973B2 (en) 2014-02-11 2017-10-10 Osterhout Group, Inc. Micro doppler presentations in head worn computing
US9810906B2 (en) 2014-06-17 2017-11-07 Osterhout Group, Inc. External user interface for head worn computing
US9811152B2 (en) 2014-01-21 2017-11-07 Osterhout Group, Inc. Eye imaging in head worn computing
US9829707B2 (en) 2014-08-12 2017-11-28 Osterhout Group, Inc. Measuring content brightness in head worn computing
US9836122B2 (en) 2014-01-21 2017-12-05 Osterhout Group, Inc. Eye glint imaging in see-through computer display systems
KR101804884B1 (en) * 2016-08-08 2017-12-06 한국정보통신주식회사 A method for providing virtual reality contents, and an appratus using the method
US9841599B2 (en) 2014-06-05 2017-12-12 Osterhout Group, Inc. Optical configurations for head-worn see-through displays
US9843093B2 (en) 2014-02-11 2017-12-12 Osterhout Group, Inc. Spatial location presentation in head worn computing
US20180040161A1 (en) * 2016-08-02 2018-02-08 Qualtrics, Llc Conducting digital surveys utilizing virtual reality and augmented reality devices
US9897822B2 (en) 2014-04-25 2018-02-20 Osterhout Group, Inc. Temple and ear horn assembly for headworn computer
US9939646B2 (en) 2014-01-24 2018-04-10 Osterhout Group, Inc. Stray light suppression for head worn computing
US9939934B2 (en) 2014-01-17 2018-04-10 Osterhout Group, Inc. External user interface for head worn computing
US20180101550A1 (en) * 2016-10-11 2018-04-12 International Business Machines Corporation Real time object description service integrated with knowledge center on augmented reality (ar) and virtual reality (vr) devices
US9952664B2 (en) 2014-01-21 2018-04-24 Osterhout Group, Inc. Eye imaging in head worn computing
US9965681B2 (en) 2008-12-16 2018-05-08 Osterhout Group, Inc. Eye imaging in head worn computing
US20180143436A1 (en) * 2015-07-20 2018-05-24 Peishu XIE Head-operated digital eyeglasses
US10008042B2 (en) * 2016-09-30 2018-06-26 Intel Corporation Augmented reality rendered structured content
US20180190033A1 (en) * 2016-12-30 2018-07-05 Facebook, Inc. Systems and methods for providing augmented reality effects and three-dimensional mapping associated with interior spaces
US10025805B1 (en) 2014-06-24 2018-07-17 Benko, LLC Systems and methods for automated help
US20180204266A1 (en) * 2017-01-19 2018-07-19 Samsung Electronics Co., Ltd System and method for virtual reality content rating using biometric data
US20180240259A1 (en) * 2013-03-14 2018-08-23 Paypal, Inc. Using augmented reality for electronic commerce transactions
US10062182B2 (en) 2015-02-17 2018-08-28 Osterhout Group, Inc. See-through computer display systems
US10073439B1 (en) 2014-10-31 2018-09-11 Desprez, Llc Methods, systems, and software for processing expedited production or supply of designed products
US10095217B2 (en) 2014-09-15 2018-10-09 Desprez, Llc Natural language user interface for computer-aided design systems
US10108846B2 (en) * 2016-09-30 2018-10-23 Autodesk, Inc. Scanning for materials and scale information
US10115122B2 (en) * 2011-11-21 2018-10-30 Nant Holdings Ip, Llc Subscription bill service, systems and methods
EP3410261A1 (en) * 2017-06-01 2018-12-05 VDEh-Betriebsforschungsinstitut GmbH Visualisation of quality information
US20180345129A1 (en) * 2018-07-27 2018-12-06 Yogesh Rathod Display virtual objects within predefined geofence or receiving of unique code from closest beacon
US20180349837A1 (en) * 2017-05-19 2018-12-06 Hcl Technologies Limited System and method for inventory management within a warehouse
US10162337B2 (en) 2014-09-15 2018-12-25 Desprez, Llc Natural language user interface for computer-aided design systems
US10191279B2 (en) 2014-03-17 2019-01-29 Osterhout Group, Inc. Eye imaging in head worn computing
USD840395S1 (en) 2016-10-17 2019-02-12 Osterhout Group, Inc. Head-worn computer
US20190080171A1 (en) * 2017-09-14 2019-03-14 Ebay Inc. Camera Platform and Object Inventory Control
US10235009B1 (en) 2014-10-31 2019-03-19 Desprez, Llc Product variable optimization for manufacture or supply of designed products
WO2019067035A1 (en) * 2017-09-29 2019-04-04 Microsoft Technology Licensing, Llc Entity attribute identification
US10254856B2 (en) 2014-01-17 2019-04-09 Osterhout Group, Inc. External user interface for head worn computing
WO2019099585A1 (en) * 2017-11-17 2019-05-23 Ebay Inc. Rendering virtual content based on items recognized in a real-world environment
US10318011B2 (en) * 2017-01-06 2019-06-11 Lumini Corporation Gesture-controlled augmented reality experience using a mobile communications device
US10325568B2 (en) 2015-08-03 2019-06-18 Qualtrics, Llc Providing a display based electronic survey
US20190205646A1 (en) * 2017-12-29 2019-07-04 Ebay Inc. Computer Vision, User Segment, and Missing Item Determination
US10373183B1 (en) 2013-10-16 2019-08-06 Alekhine, Llc Automatic firm fabrication price quoting and fabrication ordering for computer-modeled joining features and related structures
US10401953B2 (en) * 2015-10-26 2019-09-03 Pillantas Inc. Systems and methods for eye vergence control in real and augmented reality environments
US10401824B2 (en) 2016-04-14 2019-09-03 The Rapid Manufacturing Group LLC Methods and software for reducing machining equipment usage when machining multiple objects from a single workpiece
US10430778B2 (en) * 2017-09-29 2019-10-01 Paypal, Inc. Using augmented reality for secure transactions
US10430018B2 (en) * 2013-06-07 2019-10-01 Sony Interactive Entertainment Inc. Systems and methods for providing user tagging of content within a virtual scene
US10460342B1 (en) 2014-08-12 2019-10-29 Benko, LLC Methods and software for providing targeted advertising to a product program
USD864959S1 (en) 2017-01-04 2019-10-29 Mentor Acquisition One, Llc Computer glasses
US10466492B2 (en) 2014-04-25 2019-11-05 Mentor Acquisition One, Llc Ear horn assembly for headworn computer
US10466491B2 (en) 2016-06-01 2019-11-05 Mentor Acquisition One, Llc Modular systems for head-worn computers
US20190378334A1 (en) * 2018-06-08 2019-12-12 Vulcan Inc. Augmented reality portal-based applications
US10520996B2 (en) 2014-09-18 2019-12-31 Mentor Acquisition One, Llc Thermal management for head-worn computer
CN110646938A (en) * 2018-06-27 2020-01-03 脸谱科技有限责任公司 Near-eye display system
US10528998B2 (en) * 2018-04-11 2020-01-07 Trivver, Inc. Systems and methods for presenting information related to products or services being shown on a second display device on a first display device using augmented reality technology
US10532248B2 (en) 2009-03-27 2020-01-14 Russell Brands, Llc Monitoring of physical training events
US20200026413A1 (en) * 2018-06-29 2020-01-23 Vulcan Inc. Augmented reality cursors
US10547709B2 (en) 2015-06-18 2020-01-28 Qualtrics, Llc Recomposing survey questions for distribution via multiple distribution channels
US10545481B2 (en) 2016-12-28 2020-01-28 Proto Labs Inc Methods and software for providing graphical representations of a plurality of objects in a central through opening
US10552882B1 (en) 2014-05-20 2020-02-04 Desprez, Llc Methods and software for enabling custom pricing in an electronic commerce system
US10558050B2 (en) 2014-01-24 2020-02-11 Mentor Acquisition One, Llc Haptic systems for head-worn computers
US10556309B1 (en) 2016-03-24 2020-02-11 Proto Labs Inc. Methods of subtractively manufacturing a plurality of discrete objects from a single workpiece using a removable fixating material
US10579230B2 (en) * 2018-06-21 2020-03-03 Google Llc Digital supplement association and retrieval for visual search
US10592929B2 (en) * 2014-02-19 2020-03-17 VP Holdings, Inc. Systems and methods for delivering content
US10591988B2 (en) * 2016-06-28 2020-03-17 Hiscene Information Technology Co., Ltd Method for displaying user interface of head-mounted display device
US10628814B2 (en) 2014-07-31 2020-04-21 Walmart Apollo, Llc Systems and methods for managing self check out services
US10636063B1 (en) 2016-11-08 2020-04-28 Wells Fargo Bank, N.A. Method for an augmented reality value advisor
US10649220B2 (en) 2014-06-09 2020-05-12 Mentor Acquisition One, Llc Content presentation in head worn computing
US10663740B2 (en) 2014-06-09 2020-05-26 Mentor Acquisition One, Llc Content presentation in head worn computing
US10684478B2 (en) 2016-05-09 2020-06-16 Mentor Acquisition One, Llc User interface systems for head-worn computers
US10684692B2 (en) 2014-06-19 2020-06-16 Facebook Technologies, Llc Systems, devices, and methods for gesture identification
US10684687B2 (en) 2014-12-03 2020-06-16 Mentor Acquisition One, Llc See-through computer display systems
US10690936B2 (en) 2016-08-29 2020-06-23 Mentor Acquisition One, Llc Adjustable nose bridge assembly for headworn computer
US10702743B2 (en) 2014-02-28 2020-07-07 Russell Brands, Llc Data processing inside gaming device
US10713394B1 (en) 2014-06-12 2020-07-14 Benko, LLC Filtering components compatible with a computer-modeled structure
US10768500B2 (en) 2016-09-08 2020-09-08 Mentor Acquisition One, Llc Electrochromic systems for head-worn computer systems
US10768951B2 (en) * 2018-08-29 2020-09-08 Bank Of America Corporation Providing augmented reality user interfaces and controlling automated systems based on user activity information and pre-staging information
US10803501B1 (en) 2015-03-17 2020-10-13 Desprez, Llc Systems, methods, and software for generating, customizing, and automatedly e-mailing a request for quotation for fabricating a computer-modeled structure from within a CAD program
US10824253B2 (en) 2016-05-09 2020-11-03 Mentor Acquisition One, Llc User interface systems for head-worn computers
US10836110B2 (en) 2014-10-31 2020-11-17 Desprez, Llc Method and system for ordering expedited production or supply of designed products
US10853589B2 (en) 2014-04-25 2020-12-01 Mentor Acquisition One, Llc Language translation with head-worn computing
US10855683B2 (en) 2009-05-27 2020-12-01 Samsung Electronics Co., Ltd. System and method for facilitating user interaction with a simulated object associated with a physical location
US10878037B2 (en) 2018-06-21 2020-12-29 Google Llc Digital supplement association and retrieval for visual search
US10929904B1 (en) 2012-10-23 2021-02-23 Protolabs, Inc. Automated fabrication price quoting and fabrication ordering for computer-modeled structures
US10970936B2 (en) * 2018-10-05 2021-04-06 Facebook Technologies, Llc Use of neuromuscular signals to provide enhanced interactions with physical objects in an augmented reality environment
US10976804B1 (en) * 2019-07-09 2021-04-13 Facebook Technologies, Llc Pointer-based interaction with a virtual surface using a peripheral device in artificial reality environments
US11003868B2 (en) * 2016-11-07 2021-05-11 ' Rockwell Automation Technologies, Inc. Filtering display data
US11004126B1 (en) 2016-03-17 2021-05-11 Desprez, Llc Systems, methods, and software for generating, customizing, and automatedly e-mailing a request for quotation for fabricating a computer-modeled structure from within a CAD program
US11023036B1 (en) 2019-07-09 2021-06-01 Facebook Technologies, Llc Virtual drawing surface interaction using a peripheral device in artificial reality environments
US11023934B1 (en) 2014-10-30 2021-06-01 Desprez, Llc Business variable optimization for manufacture or supply of designed products
US11023035B1 (en) 2019-07-09 2021-06-01 Facebook Technologies, Llc Virtual pinboard interaction using a peripheral device in artificial reality environments
US11073901B2 (en) * 2015-07-07 2021-07-27 Seiko Epson Corporation Display device, control method for display device, and computer program
US11079846B2 (en) 2013-11-12 2021-08-03 Facebook Technologies, Llc Systems, articles, and methods for capacitive electromyography sensors
US11087539B2 (en) 2018-08-21 2021-08-10 Mastercard International Incorporated Systems and methods for generating augmented reality-based profiles
US11094017B1 (en) 2016-12-01 2021-08-17 Wells Fargo Bank, N.A. Visualizing investment changes using augmented reality
US11090561B2 (en) 2019-02-15 2021-08-17 Microsoft Technology Licensing, Llc Aligning location for a shared augmented reality experience
US11097194B2 (en) * 2019-05-16 2021-08-24 Microsoft Technology Licensing, Llc Shared augmented reality game within a shared coordinate space
US11104272B2 (en) 2014-03-28 2021-08-31 Mentor Acquisition One, Llc System for assisted operator safety using an HMD
US11103122B2 (en) 2014-07-15 2021-08-31 Mentor Acquisition One, Llc Content presentation in head worn computing
US11145397B1 (en) 2020-01-31 2021-10-12 Express Scripts Strategie Development, Inc. System and method for augmented reality detection of loose pharmacy items
US11221819B2 (en) * 2014-03-31 2022-01-11 Amazon Technologies, Inc. Extendable architecture for augmented reality system
US20220012789A1 (en) * 2020-07-07 2022-01-13 W.W. Grainger, Inc. System and method for providing real-time visual search
US11227294B2 (en) 2014-04-03 2022-01-18 Mentor Acquisition One, Llc Sight information collection in head worn computing
US11269411B1 (en) * 2020-12-10 2022-03-08 A9.Com, Inc. Gaze dependent ocular mode controller for mixed reality
US11269182B2 (en) 2014-07-15 2022-03-08 Mentor Acquisition One, Llc Content presentation in head worn computing
US11276095B1 (en) 2014-10-30 2022-03-15 Desprez, Llc Methods and software for a pricing-method-agnostic ecommerce marketplace for manufacturing services
US11301877B2 (en) 2016-09-01 2022-04-12 Qualtrics, Llc Providing analysis of perception data over time for events
US11373342B2 (en) * 2020-01-14 2022-06-28 Sony Interactive Entertainment Inc. Social and scene target awareness and adaptation of an occlusion system for increased social and scene interaction in an optical see-through augmented reality head mounted display
US11392396B1 (en) 2014-06-24 2022-07-19 Desprez, Llc Systems and methods for automated help
US11410224B1 (en) 2014-03-28 2022-08-09 Desprez, Llc Methods and software for requesting a pricing in an electronic marketplace using a user-modifiable spectrum interface
US11415961B1 (en) 2014-10-31 2022-08-16 Desprez, Llc Automated correlation of modeled product and preferred manufacturers
US11423449B1 (en) 2016-03-23 2022-08-23 Desprez, Llc Electronic pricing machine configured to generate prices based on supplier willingness and a user interface therefor
US11481031B1 (en) 2019-04-30 2022-10-25 Meta Platforms Technologies, Llc Devices, systems, and methods for controlling computing devices via neuromuscular signals of users
US11481030B2 (en) 2019-03-29 2022-10-25 Meta Platforms Technologies, Llc Methods and apparatus for gesture detection and classification
US11487110B2 (en) 2014-01-21 2022-11-01 Mentor Acquisition One, Llc Eye imaging in head worn computing
US11493993B2 (en) 2019-09-04 2022-11-08 Meta Platforms Technologies, Llc Systems, methods, and interfaces for performing inputs based on neuromuscular control
US11537765B1 (en) 2014-02-20 2022-12-27 Benko, LLC Placement and pricing of part marks in computer-modeled structures
US11567573B2 (en) 2018-09-20 2023-01-31 Meta Platforms Technologies, Llc Neuromuscular text entry, writing and drawing in augmented reality systems
WO2023009432A1 (en) * 2021-07-29 2023-02-02 La Barrie Simeon System and method of data routing for videotelephonic shopping
US11599086B2 (en) 2014-09-15 2023-03-07 Desprez, Llc Natural language user interface for computer-aided design systems
US11635736B2 (en) 2017-10-19 2023-04-25 Meta Platforms Technologies, Llc Systems and methods for identifying biological structures associated with neuromuscular source signals
US11644799B2 (en) 2013-10-04 2023-05-09 Meta Platforms Technologies, Llc Systems, articles and methods for wearable electronic devices employing contact sensors
US11669163B2 (en) 2014-01-21 2023-06-06 Mentor Acquisition One, Llc Eye glint imaging in see-through computer display systems
US11666264B1 (en) 2013-11-27 2023-06-06 Meta Platforms Technologies, Llc Systems, articles, and methods for electromyography sensors
US11737666B2 (en) 2014-01-21 2023-08-29 Mentor Acquisition One, Llc Eye imaging in head worn computing
US11797087B2 (en) 2018-11-27 2023-10-24 Meta Platforms Technologies, Llc Methods and apparatus for autocalibration of a wearable electrode sensor system
US11868531B1 (en) 2021-04-08 2024-01-09 Meta Platforms Technologies, Llc Wearable device providing for thumb-to-finger-based input gestures detected based on neuromuscular signals, and systems and methods of use thereof
US11892644B2 (en) 2014-01-21 2024-02-06 Mentor Acquisition One, Llc See-through computer display systems
US11907423B2 (en) 2019-11-25 2024-02-20 Meta Platforms Technologies, Llc Systems and methods for contextualized interactions with an environment
US11921471B2 (en) 2013-08-16 2024-03-05 Meta Platforms Technologies, Llc Systems, articles, and methods for wearable devices having secondary power sources in links of a band for providing secondary power in addition to a primary power source

Families Citing this family (80)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9955286B2 (en) * 2013-05-08 2018-04-24 Natalya Segal Smart wearable devices and system therefor
US9086286B2 (en) * 2013-07-21 2015-07-21 Rodney Wayne Long Goggles for use in water
US10290031B2 (en) * 2013-07-24 2019-05-14 Gregorio Reid Method and system for automated retail checkout using context recognition
KR102138511B1 (en) * 2013-08-28 2020-07-28 엘지전자 주식회사 Apparatus and Method for Portable Device transmitting marker information for videotelephony of Head Mounted Display
US9240077B1 (en) * 2014-03-19 2016-01-19 A9.Com, Inc. Real-time visual effects for a live camera view
US9804753B2 (en) * 2014-03-20 2017-10-31 Microsoft Technology Licensing, Llc Selection using eye gaze evaluation over time
US10012506B1 (en) 2014-06-01 2018-07-03 DNP Technology Navigation guidance system and method of use
WO2016017945A1 (en) * 2014-07-29 2016-02-04 Samsung Electronics Co., Ltd. Mobile device and method of pairing the same with electronic device
US20160034543A1 (en) * 2014-08-04 2016-02-04 Vinay Rajashekar Nagar Generating item listings according to mapped sensor data
CN105487649B (en) * 2014-09-19 2019-01-08 华为技术有限公司 A kind of reminding method and mobile terminal
US10497053B2 (en) 2015-02-27 2019-12-03 3D Product Imaging Inc. Augmented reality E-commerce
US10113877B1 (en) * 2015-09-11 2018-10-30 Philip Raymond Schaefer System and method for providing directional information
US10922733B2 (en) * 2015-10-26 2021-02-16 Sk Planet Co., Ltd. Payment information providing system using wearable device and payment information providing method using the same
US20170309070A1 (en) * 2016-04-20 2017-10-26 Sangiovanni John System and method for very large-scale communication and asynchronous documentation in virtual reality and augmented reality environments
WO2017206061A1 (en) * 2016-05-31 2017-12-07 深圳市赛亿科技开发有限公司 System for virtual tourist scenery
WO2017206059A1 (en) * 2016-05-31 2017-12-07 深圳市赛亿科技开发有限公司 Helmet for virtual tourist scenery
US11108708B2 (en) 2016-06-06 2021-08-31 Global Tel*Link Corporation Personalized chatbots for inmates
CN106200918B (en) * 2016-06-28 2019-10-01 Oppo广东移动通信有限公司 A kind of information display method based on AR, device and mobile terminal
GB2552371B (en) * 2016-07-22 2022-04-20 Advanced Risc Mach Ltd Data processing
CN109478339A (en) * 2016-07-29 2019-03-15 三菱电机株式会社 Display device, display control unit and display control method
US11790382B2 (en) 2016-09-15 2023-10-17 Circlesx Llc Method to transmit geolocation exchange based markets
US20190272589A1 (en) 2016-09-15 2019-09-05 Erik M. Simpson Securitization of transportation units
US11810023B2 (en) 2018-10-22 2023-11-07 Circlesx Llc System and method for a transportation or freight capacity exchange for one or more transportation or freight capacity units
US11823090B2 (en) 2016-09-15 2023-11-21 Circlesx Llc Transportation and freight and parking and tolling and curb capacity unit IPO method and system
US11740777B2 (en) * 2016-09-15 2023-08-29 Circlesx Llc Multi-dimension information service helmet method and system
US10460520B2 (en) 2017-01-13 2019-10-29 Simpsx Technologies Llc Computer ball device for mixed reality, virtual reality, or augmented reality
US11861527B2 (en) 2018-11-07 2024-01-02 Circlesx Llc Financial swap payment structure method and system on transportation capacity unit assets
US11215466B2 (en) 2016-09-15 2022-01-04 Circlesx Llc Route community objects with price-time priority queues for transformed transportation units
US11880883B2 (en) 2016-09-15 2024-01-23 Circlesx Llc Systems and methods for geolocation portfolio exchanges
US20190228351A1 (en) 2018-01-23 2019-07-25 Erik M. Simpson Electronic forward market exchange for transportation seats and capacity in transportation spaces and vehicles
US10540491B1 (en) * 2016-10-25 2020-01-21 Wells Fargo Bank, N.A. Virtual and augmented reality signatures
CN106888203B (en) 2016-12-13 2020-03-24 阿里巴巴集团控股有限公司 Virtual object distribution method and device based on augmented reality
US10152738B2 (en) * 2016-12-22 2018-12-11 Capital One Services, Llc Systems and methods for providing an interactive virtual environment
US10404804B2 (en) 2017-01-30 2019-09-03 Global Tel*Link Corporation System and method for personalized virtual reality experience in a controlled environment
US20180276895A1 (en) * 2017-03-27 2018-09-27 Global Tel*Link Corporation Personalized augmented reality in a controlled environment
US10408624B2 (en) 2017-04-18 2019-09-10 Microsoft Technology Licensing, Llc Providing familiarizing directional information
KR102358543B1 (en) * 2017-04-27 2022-02-03 지멘스 악티엔게젤샤프트 Authoring of augmented reality experiences using augmented reality and virtual reality
CN107135387A (en) * 2017-05-05 2017-09-05 厦门汇利伟业科技有限公司 Online Customer Reception method and its system based on VR technologies
CN107330239A (en) * 2017-05-19 2017-11-07 深圳市海睿广告有限公司 A kind of method, mobile terminal and the cloud server of shared home medical apparatus
US10448684B2 (en) * 2017-06-21 2019-10-22 Loubert S. Suddaby Protective head support assembly
US10719870B2 (en) * 2017-06-27 2020-07-21 Microsoft Technology Licensing, Llc Mixed reality world integration of holographic buttons in a mixed reality device
US10803832B2 (en) 2017-08-01 2020-10-13 Samsung Electronics Co., Ltd. Synchronizing holographic displays and 3D objects with physical video panels
JP6795471B2 (en) * 2017-08-25 2020-12-02 ファナック株式会社 Robot system
CN107493515B (en) * 2017-08-30 2021-01-01 香港乐蜜有限公司 Event reminding method and device based on live broadcast
JP6953247B2 (en) * 2017-09-08 2021-10-27 ラピスセミコンダクタ株式会社 Goggles type display device, line-of-sight detection method and line-of-sight detection system
US11249714B2 (en) 2017-09-13 2022-02-15 Magical Technologies, Llc Systems and methods of shareable virtual objects and virtual objects as message objects to facilitate communications sessions in an augmented reality environment
US10878632B2 (en) 2017-09-29 2020-12-29 Youar Inc. Planet-scale positioning of augmented reality content
US10255728B1 (en) * 2017-09-29 2019-04-09 Youar Inc. Planet-scale positioning of augmented reality content
WO2019079826A1 (en) 2017-10-22 2019-04-25 Magical Technologies, Llc Systems, methods and apparatuses of digital assistants in an augmented reality environment and local determination of virtual object placement and apparatuses of single or multi-directional lens as portals between a physical world and a digital world component of the augmented reality environment
US10134084B1 (en) 2017-11-17 2018-11-20 Capital One Services, Llc Augmented reality systems for facilitating a purchasing process at a merchant location
US10904374B2 (en) 2018-01-24 2021-01-26 Magical Technologies, Llc Systems, methods and apparatuses to facilitate gradual or instantaneous adjustment in levels of perceptibility of virtual objects or reality object in a digital scene
US11398088B2 (en) * 2018-01-30 2022-07-26 Magical Technologies, Llc Systems, methods and apparatuses to generate a fingerprint of a physical location for placement of virtual objects
US10838600B2 (en) 2018-02-12 2020-11-17 Wayfair Llc Systems and methods for providing an extended reality interface
WO2019168780A1 (en) * 2018-02-27 2019-09-06 Thin Film Electronics Asa System and method for providing augmented reality experience to objects using wireless tags
US11086474B2 (en) * 2018-04-09 2021-08-10 Spatial Systems Inc. Augmented reality computing environments—mobile device join and load
TWI656447B (en) * 2018-04-13 2019-04-11 中華電信股份有限公司 Method and system for augmenting reality
US20190347635A1 (en) * 2018-05-10 2019-11-14 Adobe Inc. Configuring a physical environment based on electronically detected interactions
US11557060B2 (en) 2018-11-05 2023-01-17 Wayfair Llc Systems and methods for scanning three-dimensional objects
CN109508090B (en) * 2018-11-06 2021-01-08 燕山大学 Augmented reality panel system with interchangeability
JP7232030B2 (en) * 2018-12-07 2023-03-02 株式会社メルカリ Information processing program, information processing method, and information processing apparatus
JP2020091764A (en) * 2018-12-07 2020-06-11 株式会社メルカリ Information processing program, information processing method, and information processing device
US11544760B1 (en) * 2018-12-13 2023-01-03 Sunrise R&D Holdings, Llc Systems and methods for providing personalized information
US11631119B2 (en) 2019-01-11 2023-04-18 Target Brands, Inc. Electronic product recognition
US11436826B2 (en) 2019-01-11 2022-09-06 Target Brands, Inc. Augmented reality experience for shopping
CN110020909A (en) * 2019-01-14 2019-07-16 启云科技股份有限公司 Using the purchase system of virtual reality technology
US11467656B2 (en) 2019-03-04 2022-10-11 Magical Technologies, Llc Virtual object control of a physical device and/or physical device control of a virtual object
US11521262B2 (en) * 2019-05-28 2022-12-06 Capital One Services, Llc NFC enhanced augmented reality information overlays
CN110264320B (en) * 2019-06-27 2021-06-08 北京小米移动软件有限公司 Information display method and device based on reality augmentation equipment and storage medium
US10990251B1 (en) * 2019-11-08 2021-04-27 Sap Se Smart augmented reality selector
US11301681B2 (en) * 2019-12-26 2022-04-12 Paypal, Inc. Securing virtual objects tracked in an augmented reality experience between multiple devices
US11386652B2 (en) * 2019-12-26 2022-07-12 Paypal, Inc. Tagging objects in augmented reality to track object data
CN111915737B (en) * 2020-08-11 2024-03-01 厦门长辉实业有限公司 Human-object interaction system based on augmented reality
WO2022046754A1 (en) 2020-08-24 2022-03-03 Fd Ip & Licensing Llc Previsualization devices and systems for the film industry
JP7405735B2 (en) * 2020-12-21 2023-12-26 トヨタ自動車株式会社 Display system, display device, and program
US11468638B2 (en) 2020-12-22 2022-10-11 Kyndryl, Inc. Augmented reality object movement assistant
US11900392B2 (en) * 2021-05-17 2024-02-13 Capital One Services, Llc Mobile device with in-person assistance
KR102594976B1 (en) * 2021-08-13 2023-10-26 백진욱 Apparatus of selecting video content, user terminal and method of providing video content for augmented reality
US11868583B2 (en) * 2022-03-28 2024-01-09 Google Llc Tangible six-degree-of-freedom interfaces for augmented reality
US20230394561A1 (en) * 2022-06-02 2023-12-07 Snap Inc. Augmented reality self-scanning and self-checkout
US11875492B1 (en) 2023-05-01 2024-01-16 Fd Ip & Licensing Llc Systems and methods for digital compositing

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090237546A1 (en) * 2008-03-24 2009-09-24 Sony Ericsson Mobile Communications Ab Mobile Device with Image Recognition Processing Capability
US20090304283A1 (en) * 2008-06-06 2009-12-10 Microsoft Corporation Corrections for recognizers
US20100331041A1 (en) * 2009-06-26 2010-12-30 Fuji Xerox Co., Ltd. System and method for language-independent manipulations of digital copies of documents through a camera phone
US20110138317A1 (en) * 2009-12-04 2011-06-09 Lg Electronics Inc. Augmented remote controller, method for operating the augmented remote controller, and system for the same
US20110213664A1 (en) * 2010-02-28 2011-09-01 Osterhout Group, Inc. Local advertising content on an interactive head-mounted eyepiece
US20120038670A1 (en) * 2010-08-13 2012-02-16 Pantech Co., Ltd. Apparatus and method for providing augmented reality information
US20120050324A1 (en) * 2010-08-24 2012-03-01 Lg Electronics Inc. Mobile terminal and controlling method thereof
US20120143361A1 (en) * 2010-12-02 2012-06-07 Empire Technology Development Llc Augmented reality system
US20120230538A1 (en) * 2011-03-08 2012-09-13 Bank Of America Corporation Providing information associated with an identified representation of an object
US20120249586A1 (en) * 2011-03-31 2012-10-04 Nokia Corporation Method and apparatus for providing collaboration between remote and on-site users of indirect augmented reality
US20130050258A1 (en) * 2011-08-25 2013-02-28 James Chia-Ming Liu Portals: Registered Objects As Virtualized, Personalized Displays
US20130147837A1 (en) * 2011-12-13 2013-06-13 Matei Stroila Augmented reality personalization
US20130155105A1 (en) * 2011-12-19 2013-06-20 Nokia Corporation Method and apparatus for providing seamless interaction in mixed reality

Family Cites Families (230)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4413719A (en) 1981-05-28 1983-11-08 White Carl J Method and apparatus for entrapment prevention and lateral guidance in passenger conveyor systems
US4829899A (en) 1988-02-11 1989-05-16 The United States Of America As Represented By The Adminstrator National Aeronautics And Space Administration Timing control system
US5009598A (en) 1988-11-23 1991-04-23 Bennington Thomas E Flight simulator apparatus using an inoperative aircraft
US5415549A (en) 1991-03-21 1995-05-16 Atari Games Corporation Method for coloring a polygon on a video display
JPH06301657A (en) 1993-02-22 1994-10-28 Internatl Business Mach Corp <Ibm> Method for parallel management
US7859551B2 (en) 1993-10-15 2010-12-28 Bulman Richard L Object customization and presentation system
US5461711A (en) 1993-12-22 1995-10-24 Interval Research Corporation Method and system for spatial accessing of time-based information
US5604907A (en) 1993-12-30 1997-02-18 International Business Machines Corporation Computer system for executing action slots including multiple action object classes
US5623657A (en) 1993-12-30 1997-04-22 International Business Machines Corporation System for processing application programs including a language independent context management technique
US5616030A (en) 1994-06-01 1997-04-01 Watson; Bruce L. Flight simulator employing an actual aircraft
US5715468A (en) 1994-09-30 1998-02-03 Budzinski; Robert Lucius Memory system for storing and retrieving experience and knowledge with natural language
US20100131081A1 (en) 1995-05-30 2010-05-27 Brown David W Systems and methods for motion control
US6430997B1 (en) 1995-11-06 2002-08-13 Trazer Technologies, Inc. System and method for tracking and assessing movement skills in multidimensional space
US6028593A (en) 1995-12-01 2000-02-22 Immersion Corporation Method and apparatus for providing simulated physical interactions within computer generated environments
US6219032B1 (en) 1995-12-01 2001-04-17 Immersion Corporation Method for providing force feedback to a user of an interface device based on interactions of a controlled cursor with graphical elements in a graphical user interface
US5862259A (en) * 1996-03-27 1999-01-19 Caere Corporation Pattern recognition employing arbitrary segmentation and compound probabilistic evaluation
US6050898A (en) 1996-05-15 2000-04-18 Vr-1, Inc. Initiating and scaling massive concurrent data transaction
US6122345A (en) 1996-06-25 2000-09-19 Mci Worldcom, Inc. System and method for developing and processing automatic response unit (ARU) services
US6421047B1 (en) 1996-09-09 2002-07-16 De Groot Marc Multi-user virtual reality system for simulating a three-dimensional environment
US6080063A (en) 1997-01-06 2000-06-27 Khosla; Vinod Simulated real time game play with live event
SE510485C2 (en) 1997-02-24 1999-05-25 Ericsson Telefon Ab L M An apparatus, system and method for handling communication
US6763376B1 (en) 1997-09-26 2004-07-13 Mci Communications Corporation Integrated customer interface system for communications network management
WO1999023262A1 (en) 1997-11-04 1999-05-14 Nkk Corporation Method of operating blast furnace
US6023270A (en) 1997-11-17 2000-02-08 International Business Machines Corporation Delivery of objects in a virtual world using a descriptive container
US6965870B1 (en) * 1997-12-24 2005-11-15 Nortel Networks Limited Method and system for activity responsive telemarketing
GB9800397D0 (en) 1998-01-09 1998-03-04 Philips Electronics Nv Virtual environment viewpoint control
US8046338B2 (en) 1998-01-26 2011-10-25 At&T Intellectual Property Ii, L.P. System and method of organizing data to facilitate access and streaming
US6529210B1 (en) 1998-04-08 2003-03-04 Altor Systems, Inc. Indirect object manipulation in a simulation
US7065553B1 (en) 1998-06-01 2006-06-20 Microsoft Corporation Presentation system with distributed object oriented multi-user domain and separate view and model objects
US6292798B1 (en) 1998-09-09 2001-09-18 International Business Machines Corporation Method and system for controlling access to data resources and protecting computing system resources from unauthorized access
US6101489A (en) 1998-12-22 2000-08-08 Ac Properties, B.V. System, method and article of manufacture for a goal based system utilizing a time based model
US7054848B1 (en) 1999-02-08 2006-05-30 Accenture, Llp Goal based system utilizing a time based model
US6677858B1 (en) 1999-02-26 2004-01-13 Reveo, Inc. Internet-based method of and system for monitoring space-time coordinate information and biophysiological state information collected from an animate object along a course through the space-time continuum
US6317718B1 (en) * 1999-02-26 2001-11-13 Accenture Properties (2) B.V. System, method and article of manufacture for location-based filtering for shopping agent in the physical world
US6842175B1 (en) * 1999-04-22 2005-01-11 Fraunhofer Usa, Inc. Tools for interacting with virtual environments
US6424410B1 (en) 1999-08-27 2002-07-23 Maui Innovative Peripherals, Inc. 3D navigation system using complementary head-mounted and stationary infrared beam detection units
US6680909B1 (en) 1999-11-04 2004-01-20 International Business Machines Corporation Media access control scheduling methodology in master driven time division duplex wireless Pico-cellular systems
US20010053968A1 (en) 2000-01-10 2001-12-20 Iaskweb, Inc. System, method, and computer program product for responding to natural language queries
JP3363861B2 (en) 2000-01-13 2003-01-08 キヤノン株式会社 Mixed reality presentation device, mixed reality presentation method, and storage medium
US20020010734A1 (en) 2000-02-03 2002-01-24 Ebersole John Franklin Internetworked augmented reality system and method
JP2001246165A (en) 2000-03-07 2001-09-11 Konami Co Ltd Rotational operation device for game machine
US7545225B2 (en) 2000-05-11 2009-06-09 Multigig Inc. Regeneration device for rotary traveling wave oscillator
US6954728B1 (en) 2000-05-15 2005-10-11 Avatizing, Llc System and method for consumer-selected advertising and branding in interactive media
US7996264B2 (en) 2000-05-15 2011-08-09 Avatizing, Llc System and method for consumer-selected advertising and branding in interactive media
WO2001093117A2 (en) 2000-06-01 2001-12-06 Siemens Dematic Electronics Assembly Systems, Inc. Electronics assembly systems customer benefit modeling tools and methods
US6572380B1 (en) 2000-07-12 2003-06-03 Kathryn Sue Buckley Game apparatus and method for teaching favorable behavior patterns
US8064887B2 (en) 2000-08-29 2011-11-22 Logan James D Communication and control system using location aware devices for audio message storage and transmission operating under rule-based control
US6721706B1 (en) 2000-10-30 2004-04-13 Koninklijke Philips Electronics N.V. Environment-responsive user interface/entertainment device that simulates personal interaction
SE521874C2 (en) 2001-01-10 2003-12-16 Saab Ab battle Simulation
US7904194B2 (en) 2001-02-09 2011-03-08 Roy-G-Biv Corporation Event management systems and methods for motion control systems
US20020133325A1 (en) 2001-02-09 2002-09-19 Hoare Raymond R. Discrete event simulator
US7181017B1 (en) 2001-03-23 2007-02-20 David Felsher System and method for secure three-party communications
US7155496B2 (en) 2001-05-15 2006-12-26 Occam Networks Configuration management utilizing generalized markup language
US7685508B2 (en) 2001-05-15 2010-03-23 Occam Networks Device monitoring via generalized markup language
US20020184516A1 (en) 2001-05-29 2002-12-05 Hale Douglas Lavell Virtual object access control mediator
MXPA03011976A (en) 2001-06-22 2005-07-01 Nervana Inc System and method for knowledge retrieval, management, delivery and presentation.
US20030064712A1 (en) 2001-09-28 2003-04-03 Jason Gaston Interactive real world event system via computer networks
EP2375689A3 (en) 2002-03-01 2012-01-11 Enterasys Networks, Inc. Location discovery in a data network
KR100974200B1 (en) 2002-03-08 2010-08-06 레베래이션즈 인 디자인, 엘피 Electric device control apparatus
EP1502080B1 (en) 2002-04-30 2013-05-22 Telmap Ltd. Navigation system using corridor maps
US7158995B2 (en) 2002-05-08 2007-01-02 Oracle International Corporation Method for managing pointers to external objects in a run-time environment
US20050009608A1 (en) 2002-05-13 2005-01-13 Consolidated Global Fun Unlimited Commerce-enabled environment for interacting with simulated phenomena
US20070265089A1 (en) 2002-05-13 2007-11-15 Consolidated Global Fun Unlimited Simulated phenomena interaction game
US20040002843A1 (en) 2002-05-13 2004-01-01 Consolidated Global Fun Unlimited, Llc Method and system for interacting with simulated phenomena
US20030224855A1 (en) 2002-05-31 2003-12-04 Robert Cunningham Optimizing location-based mobile gaming applications
US7298740B2 (en) 2002-07-11 2007-11-20 Sprint Communications Company L.P. Centralized service control for a telecommunication system
US8454428B2 (en) 2002-09-12 2013-06-04 Wms Gaming Inc. Gaming machine performing real-time 3D rendering of gaming events
US20040095311A1 (en) 2002-11-19 2004-05-20 Motorola, Inc. Body-centric virtual interactive apparatus and method
US7640267B2 (en) 2002-11-20 2009-12-29 Radar Networks, Inc. Methods and systems for managing entities in a computing device using semantic objects
US8307273B2 (en) 2002-12-30 2012-11-06 The Board Of Trustees Of The Leland Stanford Junior University Methods and apparatus for interactive network sharing of digital video content
US7543238B2 (en) 2003-01-21 2009-06-02 Microsoft Corporation System and method for directly accessing functionality provided by an application
DE10307408B3 (en) 2003-02-20 2004-09-02 Radioplan Gmbh Process for the sequential control of sequential object-oriented system simulations of communication in mobile radio networks
US7570261B1 (en) 2003-03-06 2009-08-04 Xdyne, Inc. Apparatus and method for creating a virtual three-dimensional environment, and method of generating revenue therefrom
US7313402B1 (en) 2003-06-24 2007-12-25 Verizon Corporate Services Group Inc. System and method for evaluating accuracy of an automatic location identification system
US11033821B2 (en) 2003-09-02 2021-06-15 Jeffrey D. Mullen Systems and methods for location based games and employment of the same on location enabled devices
US7739479B2 (en) 2003-10-02 2010-06-15 Nvidia Corporation Method for providing physics simulation data
US7702693B1 (en) 2003-10-30 2010-04-20 Cisco Technology, Inc. Role-based access control enforced by filesystem of an operating system
US20050208457A1 (en) * 2004-01-05 2005-09-22 Wolfgang Fink Digital object recognition audio-assistant for the visually impaired
US7266806B2 (en) 2004-03-02 2007-09-04 International Business Machines Corporation Portlet template based on a state design pattern
US7173604B2 (en) 2004-03-23 2007-02-06 Fujitsu Limited Gesture identification of controlled devices
US20050219223A1 (en) 2004-03-31 2005-10-06 Kotzin Michael D Method and apparatus for determining the context of a device
US20050246275A1 (en) 2004-04-30 2005-11-03 Nelson John R Real-time FBO management method & system
US7516052B2 (en) 2004-05-27 2009-04-07 Robert Allen Hatcherson Container-based architecture for simulation of entities in a time domain
US20050286421A1 (en) 2004-06-24 2005-12-29 Thomas Janacek Location determination for mobile devices for location-based services
US8113517B2 (en) 2004-07-30 2012-02-14 Wms Gaming Inc. Gaming machine chair
US20060230073A1 (en) 2004-08-31 2006-10-12 Gopalakrishnan Kumar C Information Services for Real World Augmentation
GB2417694A (en) 2004-09-02 2006-03-08 Sec Dep Acting Through Ordnanc Real-world interactive game
US7720570B2 (en) * 2004-10-01 2010-05-18 Redzone Robotics, Inc. Network architecture for remote robot with interchangeable tools
US7606375B2 (en) 2004-10-12 2009-10-20 Microsoft Corporation Method and system for automatically generating world environmental reverberation from game geometry
US7576725B2 (en) 2004-10-19 2009-08-18 Microsoft Corporation Using clear-coded, see-through objects to manipulate virtual objects
US7487177B2 (en) 2004-11-08 2009-02-03 Sap Aktiengesellschaft Set identifiers for objects
US20090005167A1 (en) 2004-11-29 2009-01-01 Juha Arrasvuori Mobile Gaming with External Devices in Single and Multiplayer Games
US20060189386A1 (en) 2005-01-28 2006-08-24 Outland Research, L.L.C. Device, system and method for outdoor computer gaming
US8639629B1 (en) 2005-02-02 2014-01-28 Nexus Payments, LLC System and method for accessing an online user account registry via a thin-client unique user code
US8768838B1 (en) 2005-02-02 2014-07-01 Nexus Payments, LLC Financial transactions using a rule-module nexus and a user account registry
US20090070180A1 (en) 2005-02-04 2009-03-12 Searete Llc A Limited Liability Corporation Of The State Of Delaware Variant rating plans for virtual world environment
US20150120533A1 (en) 2005-02-04 2015-04-30 Searete Llc Real-world profile data for making virtual world contacts
US20090043682A1 (en) 2005-02-04 2009-02-12 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Real-world profile data for making virtual world contacts
US20090138355A1 (en) 2005-02-04 2009-05-28 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Real-world profile data for making virtual world contacts
US20090144148A1 (en) 2005-02-04 2009-06-04 Searete Llc, A Limited Liability Corporation Of The State Of Delaware Attribute enhancement in virtual world environments
US8271365B2 (en) 2005-02-04 2012-09-18 The Invention Science Fund I, Llc Real-world profile data for making virtual world contacts
US7890419B2 (en) 2005-02-04 2011-02-15 The Invention Science Fund I, Llc Virtual credit in simulated environments
US20060178968A1 (en) 2005-02-04 2006-08-10 Jung Edward K Virtual world interconnection technique
US20060192852A1 (en) 2005-02-09 2006-08-31 Sally Rosenthal System, method, software arrangement and computer-accessible medium for providing audio and/or visual information
JP5160407B2 (en) 2005-03-10 2013-03-13 コーニンクレッカ フィリップス エレクトロニクス エヌ ヴィ Method and apparatus for displaying virtual objects
US20060223635A1 (en) 2005-04-04 2006-10-05 Outland Research method and apparatus for an on-screen/off-screen first person gaming experience
EP1712981A1 (en) 2005-04-15 2006-10-18 Herman Bailey Interactive augmented reality system
US8732025B2 (en) 2005-05-09 2014-05-20 Google Inc. System and method for enabling image recognition and searching of remote content on display
US20060277466A1 (en) 2005-05-13 2006-12-07 Anderson Thomas G Bimodal user interaction with a simulated object
US7711478B2 (en) 2005-06-21 2010-05-04 Mappick Technologies, Llc Navigation system and method
US7955168B2 (en) 2005-06-24 2011-06-07 Disney Enterprises, Inc. Amusement ride and video game
US10510214B2 (en) 2005-07-08 2019-12-17 Cfph, Llc System and method for peer-to-peer wireless gaming
US8275397B2 (en) 2005-07-14 2012-09-25 Huston Charles D GPS based friend location and identification system and method
US8249626B2 (en) 2005-07-14 2012-08-21 Huston Charles D GPS based friend location and identification system and method
US20080026838A1 (en) 2005-08-22 2008-01-31 Dunstan James E Multi-player non-role-playing virtual world games: method for two-way interaction between participants and multi-player virtual world games
US20070097832A1 (en) 2005-10-19 2007-05-03 Nokia Corporation Interoperation between virtual gaming environment and real-world environments
CN100547590C (en) 2005-12-05 2009-10-07 北京书生国际信息技术有限公司 DPS (Document Processing System)
US20070196809A1 (en) 2006-02-21 2007-08-23 Mr. Prabir Sen Digital Reality Sports, Games Events and Activities in three dimensional and interactive space display environment and information processing medium
US8204751B1 (en) 2006-03-03 2012-06-19 At&T Intellectual Property Ii, L.P. Relevance recognition for a human machine dialog system contextual question answering based on a normalization of the length of the user input
US8477154B2 (en) 2006-03-20 2013-07-02 Siemens Energy, Inc. Method and system for interactive virtual inspection of modeled objects
US7593354B2 (en) 2006-03-22 2009-09-22 Musigy Usa, Inc. Method and system for low latency high quality music conferencing
EP1870210A1 (en) 2006-06-22 2007-12-26 Honda Research Institute Europe GmbH Evaluating visual proto-objects for robot interaction
KR100883907B1 (en) 2006-09-15 2009-02-17 엔에이치엔(주) Method and system for controlling game using in multi-player online game
US8191121B2 (en) 2006-11-10 2012-05-29 Bally Gaming, Inc. Methods and systems for controlling access to resources in a gaming network
US9665966B2 (en) 2006-12-01 2017-05-30 Lucasfilm Entertainment Company Ltd. Simulation object connections
US20080220397A1 (en) 2006-12-07 2008-09-11 Livesight Target Systems Inc. Method of Firearms and/or Use of Force Training, Target, and Training Simulator
US7824268B2 (en) 2006-12-19 2010-11-02 Electronic Arts, Inc. Live hosted online multiplayer game
US20080162707A1 (en) 2006-12-28 2008-07-03 Microsoft Corporation Time Based Permissioning
US8468244B2 (en) 2007-01-05 2013-06-18 Digital Doors, Inc. Digital information infrastructure and method for security designated data and with granular data stores
US8655939B2 (en) 2007-01-05 2014-02-18 Digital Doors, Inc. Electromagnetic pulse (EMP) hardened information infrastructure with extractor, cloud dispersal, secure storage, content analysis and classification and method therefor
US20090262074A1 (en) 2007-01-05 2009-10-22 Invensense Inc. Controlling and accessing content using motion processing on mobile devices
US8595635B2 (en) * 2007-01-25 2013-11-26 Salesforce.Com, Inc. System, method and apparatus for selecting content from web sources and posting content to web logs
RU2428808C2 (en) 2007-01-26 2011-09-10 Интердиджитал Текнолоджи Корпорейшн Method and device to arrange protection of location information and to control access with application of location information
US7873710B2 (en) 2007-02-06 2011-01-18 5O9, Inc. Contextual data communication platform
US7958104B2 (en) 2007-03-08 2011-06-07 O'donnell Shawn C Context based data searching
US7991706B2 (en) 2007-06-12 2011-08-02 Neopost Technologies Virtual mailing system
US8302033B2 (en) 2007-06-22 2012-10-30 Apple Inc. Touch screen device, method, and graphical user interface for providing maps, directions, and location-based information
US8675017B2 (en) 2007-06-26 2014-03-18 Qualcomm Incorporated Real world gaming framework
US8205092B2 (en) 2007-06-26 2012-06-19 Novell, Inc. Time-based method for authorizing access to resources
US20090005018A1 (en) 2007-06-28 2009-01-01 Apple Inc. Route Sharing and Location
JP2009025871A (en) 2007-07-17 2009-02-05 Hewlett-Packard Development Co Lp Access restriction device and its method
US20090036186A1 (en) 2007-08-03 2009-02-05 Lucent Technologies Inc. Interactive real world gaming supported over internet protocol multimedia subsystem
US8500533B2 (en) 2007-08-29 2013-08-06 Cfph, Llc Game with chance element and strategy component that can be copied
US8644842B2 (en) * 2007-09-04 2014-02-04 Nokia Corporation Personal augmented reality advertising
US9277351B2 (en) 2007-09-07 2016-03-01 International Business Machines Corporation Wireless transmission duration and location-based services
US8898691B2 (en) 2007-09-27 2014-11-25 At&T Intellectual Property I, L.P. Control of access to multimedia content
US20090102616A1 (en) 2007-10-22 2009-04-23 Microsoft Corporation Time-based access control for an entertainment console
US8201229B2 (en) 2007-11-12 2012-06-12 Bally Gaming, Inc. User authorization system and methods
US8244593B2 (en) 2007-11-20 2012-08-14 Pure Verticals, Inc. Method and system for monetizing content
US8386918B2 (en) 2007-12-06 2013-02-26 International Business Machines Corporation Rendering of real world objects and interactions into a virtual universe
US8615383B2 (en) 2008-01-18 2013-12-24 Lockheed Martin Corporation Immersive collaborative environment using motion capture, head mounted display, and cave
US8138930B1 (en) * 2008-01-22 2012-03-20 Google Inc. Advertising based on environmental conditions
US8719077B2 (en) 2008-01-29 2014-05-06 Microsoft Corporation Real world and virtual world cross-promotion
US9147080B2 (en) 2008-02-06 2015-09-29 International Business Machines Corporation System and methods for granular access control
WO2009119762A1 (en) * 2008-03-27 2009-10-01 三菱電機株式会社 Connector port
US8480499B2 (en) 2008-04-30 2013-07-09 Scientific Games Holdings Limited System and method for game brokering
US8406531B2 (en) * 2008-05-15 2013-03-26 Yahoo! Inc. Data access based on content of image recorded by a mobile device
US8249263B2 (en) 2008-05-15 2012-08-21 International Business Machines Corporation Method and apparatus for providing audio motion feedback in a simulated three-dimensional environment
US8839140B2 (en) 2008-05-23 2014-09-16 Microsoft Corporation Pivot search results by time and location
US9403087B2 (en) 2008-06-09 2016-08-02 Disney Enterprises, Inc. System and method of providing access to virtual spaces that are associated with physical analogues in the real world
US20100017820A1 (en) 2008-07-18 2010-01-21 Telephoto Technologies Inc. Realtime insertion of video content in live broadcasting
US8805110B2 (en) 2008-08-19 2014-08-12 Digimarc Corporation Methods and systems for content processing
US20100050100A1 (en) 2008-08-21 2010-02-25 Dettinger Richard D Virtual World Object Presentation, Recommendations and Navigation
JP5372157B2 (en) * 2008-09-17 2013-12-18 ノキア コーポレイション User interface for augmented reality
US8957835B2 (en) 2008-09-30 2015-02-17 Apple Inc. Head-mounted display apparatus for retaining a portable electronic device with display
US9100249B2 (en) 2008-10-10 2015-08-04 Metaplace, Inc. System and method for providing virtual spaces for access by users via the web
US8094834B1 (en) 2008-11-14 2012-01-10 The United States Of America As Represented By The Secretary Of The Air Force Remote auditory spatial communication aid
US8972048B2 (en) 2008-11-20 2015-03-03 Disney Enterprises, Inc. Self-service beverage and snack dispensing using identity-based access control
US8229718B2 (en) 2008-12-23 2012-07-24 Microsoft Corporation Use of scientific models in environmental simulation
US9600306B2 (en) 2009-01-31 2017-03-21 International Business Machines Corporation Client-side simulated virtual universe environment
US8192283B2 (en) 2009-03-10 2012-06-05 Bally Gaming, Inc. Networked gaming system including a live floor view module
US8364984B2 (en) 2009-03-13 2013-01-29 Microsoft Corporation Portable secure data files
US20100251185A1 (en) 2009-03-31 2010-09-30 Codemasters Software Company Ltd. Virtual object appearance control
US9195898B2 (en) * 2009-04-14 2015-11-24 Qualcomm Incorporated Systems and methods for image recognition using mobile devices
US8246467B2 (en) 2009-04-29 2012-08-21 Apple Inc. Interactive gaming with co-located, networked direction and location aware devices
US8303387B2 (en) 2009-05-27 2012-11-06 Zambala Lllp System and method of simulated objects and applications thereof
US20100306825A1 (en) 2009-05-27 2010-12-02 Lucid Ventures, Inc. System and method for facilitating user interaction with a simulated object associated with a physical location
US8745494B2 (en) * 2009-05-27 2014-06-03 Zambala Lllp System and method for control of a simulated object that is associated with a physical location in the real world environment
US8645220B2 (en) * 2009-08-28 2014-02-04 Homer Tlc, Inc. Method and system for creating an augmented reality experience in connection with a stored value token
US9001252B2 (en) 2009-11-02 2015-04-07 Empire Technology Development Llc Image matching to augment reality
US20110184972A1 (en) * 2009-12-23 2011-07-28 Cbs Interactive Inc. System and method for navigating a product catalog
US8947455B2 (en) * 2010-02-22 2015-02-03 Nike, Inc. Augmented reality design system
US20110214082A1 (en) * 2010-02-28 2011-09-01 Osterhout Group, Inc. Projection triggering through an external marker in an augmented reality eyepiece
US8610771B2 (en) 2010-03-08 2013-12-17 Empire Technology Development Llc Broadband passive tracking for augmented reality
US20110225069A1 (en) * 2010-03-12 2011-09-15 Cramer Donald M Purchase and Delivery of Goods and Services, and Payment Gateway in An Augmented Reality-Enabled Distribution Network
US8682879B2 (en) 2010-04-16 2014-03-25 Bizmodeline Co., Ltd. Marker search system for augmented reality service
KR101682218B1 (en) 2010-06-07 2016-12-02 엘지전자 주식회사 Mobile terminal and control method thereof
US20110310227A1 (en) * 2010-06-17 2011-12-22 Qualcomm Incorporated Mobile device based content mapping for augmented reality environment
US20120011142A1 (en) * 2010-07-08 2012-01-12 Qualcomm Incorporated Feedback to improve object recognition
US9031809B1 (en) 2010-07-14 2015-05-12 Sri International Method and apparatus for generating three-dimensional pose using multi-modal sensor fusion
US8593375B2 (en) * 2010-07-23 2013-11-26 Gregory A Maltz Eye gaze user interface and method
US8405550B2 (en) * 2010-07-30 2013-03-26 Raytheon Applied Signal Technology, Inc. Near-vertical direction finding and geolocation system
US20120038668A1 (en) * 2010-08-16 2012-02-16 Lg Electronics Inc. Method for display information and mobile terminal using the same
US8866847B2 (en) * 2010-09-14 2014-10-21 International Business Machines Corporation Providing augmented reality information
JP5592014B2 (en) 2010-09-30 2014-09-17 エンパイア テクノロジー ディベロップメント エルエルシー Projecting patterns for high resolution texture extraction
US10121133B2 (en) * 2010-10-13 2018-11-06 Walmart Apollo, Llc Method for self-checkout with a mobile device
US9304319B2 (en) * 2010-11-18 2016-04-05 Microsoft Technology Licensing, Llc Automatic focus improvement for augmented reality displays
CN103210360B (en) 2011-02-25 2016-05-04 英派尔科技开发有限公司 For the method, apparatus and system of augmented reality performance
US9317835B2 (en) * 2011-03-08 2016-04-19 Bank Of America Corporation Populating budgets and/or wish lists using real-time video image analysis
US20120229624A1 (en) * 2011-03-08 2012-09-13 Bank Of America Corporation Real-time image analysis for providing health related information
US20120239469A1 (en) * 2011-03-15 2012-09-20 Videodeals.com S.A. System and method for marketing
JP5765019B2 (en) * 2011-03-31 2015-08-19 ソニー株式会社 Display control apparatus, display control method, and program
US8732193B2 (en) * 2011-06-13 2014-05-20 Opus Deli, Inc. Multi-media management and streaming techniques implemented over a computer network
US20120326966A1 (en) * 2011-06-21 2012-12-27 Qualcomm Incorporated Gesture-controlled technique to expand interaction radius in computer vision applications
US8292743B1 (en) 2011-06-30 2012-10-23 Zynga Inc. Changing virtual items based on location-based actions
AU2011205223C1 (en) * 2011-08-09 2013-03-28 Microsoft Technology Licensing, Llc Physical interaction with virtual objects for DRM
US8959440B2 (en) * 2011-09-08 2015-02-17 Citizen, Inc. Inner-circle social sourcing
US9285871B2 (en) * 2011-09-30 2016-03-15 Microsoft Technology Licensing, Llc Personal audio/visual system for providing an adaptable augmented reality environment
US9255813B2 (en) * 2011-10-14 2016-02-09 Microsoft Technology Licensing, Llc User controlled real object disappearance in a mixed reality display
US8970452B2 (en) * 2011-11-02 2015-03-03 Google Inc. Imaging method
JP5948429B2 (en) 2011-11-09 2016-07-06 エンパイア テクノロジー ディベロップメント エルエルシー Virtual and augmented reality
US9182815B2 (en) * 2011-12-07 2015-11-10 Microsoft Technology Licensing, Llc Making static printed content dynamic with virtual data
US9135508B2 (en) * 2011-12-20 2015-09-15 Microsoft Technology Licensing, Llc. Enhanced user eye gaze estimation
US20150097772A1 (en) 2012-01-06 2015-04-09 Thad Eugene Starner Gaze Signal Based on Physical Characteristics of the Eye
US9262780B2 (en) * 2012-01-09 2016-02-16 Google Inc. Method and apparatus for enabling real-time product and vendor identification
US8947456B2 (en) 2012-03-22 2015-02-03 Empire Technology Development Llc Augmented reality process for sorting materials
WO2013147815A1 (en) 2012-03-29 2013-10-03 Empire Technology Development, Llc Enabling location-based applications to work with imaginary locations
US8620021B2 (en) * 2012-03-29 2013-12-31 Digimarc Corporation Image-related methods and arrangements
US20130293580A1 (en) 2012-05-01 2013-11-07 Zambala Lllp System and method for selecting targets in an augmented reality environment
KR20140136517A (en) 2012-05-02 2014-11-28 엠파이어 테크놀로지 디벨롭먼트 엘엘씨 Four dimensional image registration using dynamical model for augmented reality in medical applications
US9196094B2 (en) 2012-11-30 2015-11-24 Empire Technology Develoment Llc Method and apparatus for augmented reality
CN105188516B (en) 2013-03-11 2017-12-22 奇跃公司 For strengthening the System and method for virtual reality
US10268276B2 (en) 2013-03-15 2019-04-23 Eyecam, LLC Autonomous computing and telecommunications head-up displays glasses
KR102063076B1 (en) 2013-07-10 2020-01-07 엘지전자 주식회사 The mobile device and controlling method thereof, the head mounted display and controlling method thereof
US10296663B2 (en) * 2014-05-13 2019-05-21 Atheer, Inc. Method for moving and aligning 3D objects in a plane within the 2D environment
US9579577B2 (en) 2014-06-20 2017-02-28 Samsung Electronics Co., Ltd. Electronic system with challenge mechanism and method of operation thereof
US9609383B1 (en) 2015-03-23 2017-03-28 Amazon Technologies, Inc. Directional audio for virtual environments
WO2019028479A1 (en) * 2017-08-04 2019-02-07 Magical Technologies, Llc Systems, methods and apparatuses for deployment and targeting of context-aware virtual objects and behavior modeling of virtual objects based on physical principles
US10783428B2 (en) * 2018-07-05 2020-09-22 Accenture Global Solutions Limited Holographic virtual assistant
US11682178B2 (en) * 2021-11-16 2023-06-20 Meta Platforms Technologies, Llc Alternating perceived realities in a virtual world based on first person preferences and a relative coordinate system

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090237546A1 (en) * 2008-03-24 2009-09-24 Sony Ericsson Mobile Communications Ab Mobile Device with Image Recognition Processing Capability
US20090304283A1 (en) * 2008-06-06 2009-12-10 Microsoft Corporation Corrections for recognizers
US20100331041A1 (en) * 2009-06-26 2010-12-30 Fuji Xerox Co., Ltd. System and method for language-independent manipulations of digital copies of documents through a camera phone
US20110138317A1 (en) * 2009-12-04 2011-06-09 Lg Electronics Inc. Augmented remote controller, method for operating the augmented remote controller, and system for the same
US20110213664A1 (en) * 2010-02-28 2011-09-01 Osterhout Group, Inc. Local advertising content on an interactive head-mounted eyepiece
US20120038670A1 (en) * 2010-08-13 2012-02-16 Pantech Co., Ltd. Apparatus and method for providing augmented reality information
US20120050324A1 (en) * 2010-08-24 2012-03-01 Lg Electronics Inc. Mobile terminal and controlling method thereof
US20120143361A1 (en) * 2010-12-02 2012-06-07 Empire Technology Development Llc Augmented reality system
US20120230538A1 (en) * 2011-03-08 2012-09-13 Bank Of America Corporation Providing information associated with an identified representation of an object
US20120249586A1 (en) * 2011-03-31 2012-10-04 Nokia Corporation Method and apparatus for providing collaboration between remote and on-site users of indirect augmented reality
US20130050258A1 (en) * 2011-08-25 2013-02-28 James Chia-Ming Liu Portals: Registered Objects As Virtualized, Personalized Displays
US20130147837A1 (en) * 2011-12-13 2013-06-13 Matei Stroila Augmented reality personalization
US20130155105A1 (en) * 2011-12-19 2013-06-20 Nokia Corporation Method and apparatus for providing seamless interaction in mixed reality

Cited By (363)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US11506912B2 (en) 2008-01-02 2022-11-22 Mentor Acquisition One, Llc Temple and ear horn assembly for headworn computer
US9965681B2 (en) 2008-12-16 2018-05-08 Osterhout Group, Inc. Eye imaging in head worn computing
US10532248B2 (en) 2009-03-27 2020-01-14 Russell Brands, Llc Monitoring of physical training events
US10855683B2 (en) 2009-05-27 2020-12-01 Samsung Electronics Co., Ltd. System and method for facilitating user interaction with a simulated object associated with a physical location
US11765175B2 (en) 2009-05-27 2023-09-19 Samsung Electronics Co., Ltd. System and method for facilitating user interaction with a simulated object associated with a physical location
US10147113B2 (en) * 2011-11-21 2018-12-04 Nant Holdings Ip, Llc Subscription bill service, systems and methods
US9805385B2 (en) * 2011-11-21 2017-10-31 Nant Holdings Ip, Llc Subscription bill service, systems and methods
US20150294284A1 (en) * 2011-11-21 2015-10-15 Nant Holdings Ip, Llc Subscription Bill Service, Systems and Methods
US10115122B2 (en) * 2011-11-21 2018-10-30 Nant Holdings Ip, Llc Subscription bill service, systems and methods
US11004102B2 (en) * 2011-11-21 2021-05-11 Nant Holdings Ip, Llc Methods and systems for reconciling a transaction within a computer-based game
US10304073B2 (en) * 2011-11-21 2019-05-28 Nant Holdings Ip, Llc Subscription bill service, systems and methods
USD748123S1 (en) * 2012-02-03 2016-01-26 Symantec Corporation Display screen with graphical user interface
US10878636B2 (en) 2012-05-01 2020-12-29 Samsung Electronics Co., Ltd. System and method for selecting targets in an augmented reality environment
US10127735B2 (en) 2012-05-01 2018-11-13 Augmented Reality Holdings 2, Llc System, method and apparatus of eye tracking or gaze detection applications including facilitating action on or interaction with a simulated object
US11417066B2 (en) 2012-05-01 2022-08-16 Samsung Electronics Co., Ltd. System and method for selecting targets in an augmented reality environment
US10388070B2 (en) 2012-05-01 2019-08-20 Samsung Electronics Co., Ltd. System and method for selecting targets in an augmented reality environment
US9665983B2 (en) 2012-05-01 2017-05-30 Zambala, Lllp Method, medium, and system for facilitating electronic commercial transactions in an augmented reality environment
US20130307872A1 (en) * 2012-05-17 2013-11-21 International Business Machines Corporation Integrating Remote Content with Local Content
US20130328926A1 (en) * 2012-06-08 2013-12-12 Samsung Electronics Co., Ltd Augmented reality arrangement of nearby location information
US20130328925A1 (en) * 2012-06-12 2013-12-12 Stephen G. Latta Object focus in a mixed reality environment
US20140002643A1 (en) * 2012-06-27 2014-01-02 International Business Machines Corporation Presentation of augmented reality images on mobile computing devices
US20150329122A1 (en) * 2012-08-08 2015-11-19 C/O Sony Corporation Mobile object, system, and storage medium
US9616901B2 (en) * 2012-08-08 2017-04-11 Sony Corporation Mobile object, system, and storage medium
US10929904B1 (en) 2012-10-23 2021-02-23 Protolabs, Inc. Automated fabrication price quoting and fabrication ordering for computer-modeled structures
US20140198035A1 (en) * 2013-01-14 2014-07-17 Thalmic Labs Inc. Wearable muscle interface systems, devices and methods that interact with content displayed on an electronic display
US10528135B2 (en) * 2013-01-14 2020-01-07 Ctrl-Labs Corporation Wearable muscle interface systems, devices and methods that interact with content displayed on an electronic display
US11009951B2 (en) 2013-01-14 2021-05-18 Facebook Technologies, Llc Wearable muscle interface systems, devices and methods that interact with content displayed on an electronic display
US9449340B2 (en) * 2013-01-30 2016-09-20 Wal-Mart Stores, Inc. Method and system for managing an electronic shopping list with gestures
US20140214597A1 (en) * 2013-01-30 2014-07-31 Wal-Mart Stores, Inc. Method And System For Managing An Electronic Shopping List With Gestures
US20140236652A1 (en) * 2013-02-19 2014-08-21 Wal-Mart Stores, Inc. Remote sales assistance system
US9082149B2 (en) * 2013-02-19 2015-07-14 Wal-Mart Stores, Inc. System and method for providing sales assistance to a consumer wearing an augmented reality device in a physical store
US10930043B2 (en) 2013-03-14 2021-02-23 Paypal, Inc. Using augmented reality for electronic commerce transactions
US20180240259A1 (en) * 2013-03-14 2018-08-23 Paypal, Inc. Using augmented reality for electronic commerce transactions
US10529105B2 (en) * 2013-03-14 2020-01-07 Paypal, Inc. Using augmented reality for electronic commerce transactions
US11748735B2 (en) 2013-03-14 2023-09-05 Paypal, Inc. Using augmented reality for electronic commerce transactions
US20140292636A1 (en) * 2013-03-29 2014-10-02 Plantronics, Inc. Head-Worn Infrared-Based Mobile User-Interface
US10430018B2 (en) * 2013-06-07 2019-10-01 Sony Interactive Entertainment Inc. Systems and methods for providing user tagging of content within a virtual scene
US11921471B2 (en) 2013-08-16 2024-03-05 Meta Platforms Technologies, Llc Systems, articles, and methods for wearable devices having secondary power sources in links of a band for providing secondary power in addition to a primary power source
US20150049113A1 (en) * 2013-08-19 2015-02-19 Qualcomm Incorporated Visual search in real world using optical see-through head mounted display with augmented reality and user interaction tracking
US10152495B2 (en) * 2013-08-19 2018-12-11 Qualcomm Incorporated Visual search in real world using optical see-through head mounted display with augmented reality and user interaction tracking
US11068531B2 (en) 2013-08-19 2021-07-20 Qualcomm Incorporated Visual search in real world using optical see-through head mounted display with augmented reality and user interaction tracking
US10372751B2 (en) 2013-08-19 2019-08-06 Qualcomm Incorporated Visual search in real world using optical see-through head mounted display with augmented reality and user interaction tracking
US11734336B2 (en) 2013-08-19 2023-08-22 Qualcomm Incorporated Method and apparatus for image processing and associated user interaction
US11644799B2 (en) 2013-10-04 2023-05-09 Meta Platforms Technologies, Llc Systems, articles and methods for wearable electronic devices employing contact sensors
US10007338B2 (en) * 2013-10-14 2018-06-26 Suricog Method of interaction by gaze and associated device
US9606701B1 (en) 2013-10-14 2017-03-28 Benko, LLC Automated recommended joining data with presented methods for joining in computer-modeled structures
US20180275755A1 (en) * 2013-10-14 2018-09-27 Suricog Method of interaction by gaze and associated device
US20160224110A1 (en) * 2013-10-14 2016-08-04 Suricog Method of interaction by gaze and associated device
US10373183B1 (en) 2013-10-16 2019-08-06 Alekhine, Llc Automatic firm fabrication price quoting and fabrication ordering for computer-modeled joining features and related structures
US9558593B2 (en) * 2013-11-05 2017-01-31 Sony Corporation Terminal apparatus, additional information managing apparatus, additional information managing method, and program
US20150124106A1 (en) * 2013-11-05 2015-05-07 Sony Computer Entertainment Inc. Terminal apparatus, additional information managing apparatus, additional information managing method, and program
US11079846B2 (en) 2013-11-12 2021-08-03 Facebook Technologies, Llc Systems, articles, and methods for capacitive electromyography sensors
US11666264B1 (en) 2013-11-27 2023-06-06 Meta Platforms Technologies, Llc Systems, articles, and methods for electromyography sensors
EP3084684A4 (en) * 2013-12-17 2017-06-14 Amazon Technologies Inc. Pointer tracking for eye-level scanners and displays
CN106062783A (en) * 2013-12-17 2016-10-26 亚马逊科技公司 Pointer tracking for eye-level scanners and displays
US9971154B1 (en) 2013-12-17 2018-05-15 Amazon Technologies, Inc. Pointer tracking for eye-level scanners and displays
US10254856B2 (en) 2014-01-17 2019-04-09 Osterhout Group, Inc. External user interface for head worn computing
US11782529B2 (en) 2014-01-17 2023-10-10 Mentor Acquisition One, Llc External user interface for head worn computing
US11231817B2 (en) 2014-01-17 2022-01-25 Mentor Acquisition One, Llc External user interface for head worn computing
US11169623B2 (en) 2014-01-17 2021-11-09 Mentor Acquisition One, Llc External user interface for head worn computing
US9939934B2 (en) 2014-01-17 2018-04-10 Osterhout Group, Inc. External user interface for head worn computing
US11507208B2 (en) 2014-01-17 2022-11-22 Mentor Acquisition One, Llc External user interface for head worn computing
US9658457B2 (en) 2014-01-21 2017-05-23 Osterhout Group, Inc. See-through computer display systems
US9958674B2 (en) 2014-01-21 2018-05-01 Osterhout Group, Inc. Eye imaging in head worn computing
US9684171B2 (en) 2014-01-21 2017-06-20 Osterhout Group, Inc. See-through computer display systems
US11054902B2 (en) 2014-01-21 2021-07-06 Mentor Acquisition One, Llc Eye glint imaging in see-through computer display systems
US11619820B2 (en) 2014-01-21 2023-04-04 Mentor Acquisition One, Llc See-through computer display systems
US9715112B2 (en) 2014-01-21 2017-07-25 Osterhout Group, Inc. Suppression of stray light in head worn computing
US9720235B2 (en) 2014-01-21 2017-08-01 Osterhout Group, Inc. See-through computer display systems
US9720227B2 (en) 2014-01-21 2017-08-01 Osterhout Group, Inc. See-through computer display systems
US9529199B2 (en) 2014-01-21 2016-12-27 Osterhout Group, Inc. See-through computer display systems
US9720234B2 (en) 2014-01-21 2017-08-01 Osterhout Group, Inc. See-through computer display systems
US20170220865A1 (en) * 2014-01-21 2017-08-03 Osterhout Group, Inc. See-through computer display systems
US9529192B2 (en) 2014-01-21 2016-12-27 Osterhout Group, Inc. Eye imaging in head worn computing
US9740012B2 (en) 2014-01-21 2017-08-22 Osterhout Group, Inc. See-through computer display systems
US9740280B2 (en) 2014-01-21 2017-08-22 Osterhout Group, Inc. Eye imaging in head worn computing
US9746676B2 (en) * 2014-01-21 2017-08-29 Osterhout Group, Inc. See-through computer display systems
US11099380B2 (en) 2014-01-21 2021-08-24 Mentor Acquisition One, Llc Eye imaging in head worn computing
US9753288B2 (en) 2014-01-21 2017-09-05 Osterhout Group, Inc. See-through computer display systems
US9766463B2 (en) 2014-01-21 2017-09-19 Osterhout Group, Inc. See-through computer display systems
US9772492B2 (en) 2014-01-21 2017-09-26 Osterhout Group, Inc. Eye imaging in head worn computing
US10379365B2 (en) * 2014-01-21 2019-08-13 Mentor Acquisition One, Llc See-through computer display systems
US9529195B2 (en) 2014-01-21 2016-12-27 Osterhout Group, Inc. See-through computer display systems
US11669163B2 (en) 2014-01-21 2023-06-06 Mentor Acquisition One, Llc Eye glint imaging in see-through computer display systems
US9684165B2 (en) 2014-01-21 2017-06-20 Osterhout Group, Inc. Eye imaging in head worn computing
US9523856B2 (en) 2014-01-21 2016-12-20 Osterhout Group, Inc. See-through computer display systems
US11103132B2 (en) 2014-01-21 2021-08-31 Mentor Acquisition One, Llc Eye imaging in head worn computing
US9811159B2 (en) 2014-01-21 2017-11-07 Osterhout Group, Inc. Eye imaging in head worn computing
US9811152B2 (en) 2014-01-21 2017-11-07 Osterhout Group, Inc. Eye imaging in head worn computing
US11892644B2 (en) 2014-01-21 2024-02-06 Mentor Acquisition One, Llc See-through computer display systems
US9829703B2 (en) 2014-01-21 2017-11-28 Osterhout Group, Inc. Eye imaging in head worn computing
US9836122B2 (en) 2014-01-21 2017-12-05 Osterhout Group, Inc. Eye glint imaging in see-through computer display systems
US11719934B2 (en) 2014-01-21 2023-08-08 Mentor Acquisition One, Llc Suppression of stray light in head worn computing
US11487110B2 (en) 2014-01-21 2022-11-01 Mentor Acquisition One, Llc Eye imaging in head worn computing
US9494800B2 (en) 2014-01-21 2016-11-15 Osterhout Group, Inc. See-through computer display systems
US9594246B2 (en) 2014-01-21 2017-03-14 Osterhout Group, Inc. See-through computer display systems
US9885868B2 (en) 2014-01-21 2018-02-06 Osterhout Group, Inc. Eye imaging in head worn computing
US11126003B2 (en) 2014-01-21 2021-09-21 Mentor Acquisition One, Llc See-through computer display systems
US11796805B2 (en) 2014-01-21 2023-10-24 Mentor Acquisition One, Llc Eye imaging in head worn computing
US11737666B2 (en) 2014-01-21 2023-08-29 Mentor Acquisition One, Llc Eye imaging in head worn computing
US9927612B2 (en) 2014-01-21 2018-03-27 Osterhout Group, Inc. See-through computer display systems
US9615742B2 (en) 2014-01-21 2017-04-11 Osterhout Group, Inc. Eye imaging in head worn computing
US9933622B2 (en) 2014-01-21 2018-04-03 Osterhout Group, Inc. See-through computer display systems
US9651783B2 (en) 2014-01-21 2017-05-16 Osterhout Group, Inc. See-through computer display systems
US10866420B2 (en) 2014-01-21 2020-12-15 Mentor Acquisition One, Llc See-through computer display systems
US20160018648A1 (en) * 2014-01-21 2016-01-21 Osterhout Group, Inc. See-through computer display systems
US9952664B2 (en) 2014-01-21 2018-04-24 Osterhout Group, Inc. Eye imaging in head worn computing
US11622426B2 (en) 2014-01-21 2023-04-04 Mentor Acquisition One, Llc See-through computer display systems
US9651789B2 (en) 2014-01-21 2017-05-16 Osterhout Group, Inc. See-Through computer display systems
US10579140B2 (en) 2014-01-21 2020-03-03 Mentor Acquisition One, Llc Eye glint imaging in see-through computer display systems
US11353957B2 (en) 2014-01-21 2022-06-07 Mentor Acquisition One, Llc Eye glint imaging in see-through computer display systems
US9651784B2 (en) 2014-01-21 2017-05-16 Osterhout Group, Inc. See-through computer display systems
US10001644B2 (en) 2014-01-21 2018-06-19 Osterhout Group, Inc. See-through computer display systems
US9658458B2 (en) 2014-01-21 2017-05-23 Osterhout Group, Inc. See-through computer display systems
US9651788B2 (en) 2014-01-21 2017-05-16 Osterhout Group, Inc. See-through computer display systems
US10705339B2 (en) 2014-01-21 2020-07-07 Mentor Acquisition One, Llc Suppression of stray light in head worn computing
US10698223B2 (en) 2014-01-21 2020-06-30 Mentor Acquisition One, Llc See-through computer display systems
US10558050B2 (en) 2014-01-24 2020-02-11 Mentor Acquisition One, Llc Haptic systems for head-worn computers
US11822090B2 (en) 2014-01-24 2023-11-21 Mentor Acquisition One, Llc Haptic systems for head-worn computers
US9939646B2 (en) 2014-01-24 2018-04-10 Osterhout Group, Inc. Stray light suppression for head worn computing
US9784973B2 (en) 2014-02-11 2017-10-10 Osterhout Group, Inc. Micro doppler presentations in head worn computing
US9841602B2 (en) 2014-02-11 2017-12-12 Osterhout Group, Inc. Location indicating avatar in head worn computing
US9843093B2 (en) 2014-02-11 2017-12-12 Osterhout Group, Inc. Spatial location presentation in head worn computing
US9547465B2 (en) 2014-02-14 2017-01-17 Osterhout Group, Inc. Object shadowing in head worn computing
US9928019B2 (en) 2014-02-14 2018-03-27 Osterhout Group, Inc. Object shadowing in head worn computing
US10592929B2 (en) * 2014-02-19 2020-03-17 VP Holdings, Inc. Systems and methods for delivering content
US9007401B1 (en) 2014-02-20 2015-04-14 Lg Electronics Inc. Head mounted display and method for controlling the same
US11537765B1 (en) 2014-02-20 2022-12-27 Benko, LLC Placement and pricing of part marks in computer-modeled structures
WO2015126006A1 (en) * 2014-02-20 2015-08-27 Lg Electronics Inc. Head mounted display and method for controlling the same
WO2015130773A1 (en) * 2014-02-28 2015-09-03 Infomotion Sports Technologies, Inc. Sporting device and wearable computer interaction
US10702743B2 (en) 2014-02-28 2020-07-07 Russell Brands, Llc Data processing inside gaming device
US10191279B2 (en) 2014-03-17 2019-01-29 Osterhout Group, Inc. Eye imaging in head worn computing
US11410224B1 (en) 2014-03-28 2022-08-09 Desprez, Llc Methods and software for requesting a pricing in an electronic marketplace using a user-modifiable spectrum interface
US11104272B2 (en) 2014-03-28 2021-08-31 Mentor Acquisition One, Llc System for assisted operator safety using an HMD
US11221819B2 (en) * 2014-03-31 2022-01-11 Amazon Technologies, Inc. Extendable architecture for augmented reality system
US11227294B2 (en) 2014-04-03 2022-01-18 Mentor Acquisition One, Llc Sight information collection in head worn computing
US20150294389A1 (en) * 2014-04-10 2015-10-15 Bank Of America Corporation Providing comparison shopping experiences through an optical head-mounted displays in a wearable computer
US9785994B2 (en) * 2014-04-10 2017-10-10 Bank Of America Corporation Providing comparison shopping experiences through an optical head-mounted displays in a wearable computer
US10101588B2 (en) 2014-04-25 2018-10-16 Osterhout Group, Inc. Speaker assembly for headworn computer
US9651787B2 (en) 2014-04-25 2017-05-16 Osterhout Group, Inc. Speaker assembly for headworn computer
US10732434B2 (en) 2014-04-25 2020-08-04 Mentor Acquisition One, Llc Temple and ear horn assembly for headworn computer
US11809022B2 (en) 2014-04-25 2023-11-07 Mentor Acquisition One, Llc Temple and ear horn assembly for headworn computer
US10634922B2 (en) 2014-04-25 2020-04-28 Mentor Acquisition One, Llc Speaker assembly for headworn computer
US11474360B2 (en) 2014-04-25 2022-10-18 Mentor Acquisition One, Llc Speaker assembly for headworn computer
US9897822B2 (en) 2014-04-25 2018-02-20 Osterhout Group, Inc. Temple and ear horn assembly for headworn computer
US10466492B2 (en) 2014-04-25 2019-11-05 Mentor Acquisition One, Llc Ear horn assembly for headworn computer
US10146772B2 (en) 2014-04-25 2018-12-04 Osterhout Group, Inc. Language translation with head-worn computing
US11880041B2 (en) 2014-04-25 2024-01-23 Mentor Acquisition One, Llc Speaker assembly for headworn computer
US11727223B2 (en) 2014-04-25 2023-08-15 Mentor Acquisition One, Llc Language translation with head-worn computing
US9672210B2 (en) 2014-04-25 2017-06-06 Osterhout Group, Inc. Language translation with head-worn computing
US10853589B2 (en) 2014-04-25 2020-12-01 Mentor Acquisition One, Llc Language translation with head-worn computing
US9508137B2 (en) * 2014-05-02 2016-11-29 Cisco Technology, Inc. Automated patron guidance
US20150317835A1 (en) * 2014-05-02 2015-11-05 Cisco Technology, Inc. Automated patron guidance
US9558716B2 (en) * 2014-05-05 2017-01-31 Here Global B.V. Method and apparatus for contextual query based on visual elements and user input in augmented reality at a device
US20150317836A1 (en) * 2014-05-05 2015-11-05 Here Global B.V. Method and apparatus for contextual query based on visual elements and user input in augmented reality at a device
US9746686B2 (en) 2014-05-19 2017-08-29 Osterhout Group, Inc. Content position calibration in head worn computing
US10552882B1 (en) 2014-05-20 2020-02-04 Desprez, Llc Methods and software for enabling custom pricing in an electronic commerce system
US20150348427A1 (en) * 2014-06-03 2015-12-03 David Peregrim Exploiting Smart Vision Devices to Train Sports Team Tactics
US11402639B2 (en) 2014-06-05 2022-08-02 Mentor Acquisition One, Llc Optical configurations for head-worn see-through displays
US10877270B2 (en) 2014-06-05 2020-12-29 Mentor Acquisition One, Llc Optical configurations for head-worn see-through displays
US9841599B2 (en) 2014-06-05 2017-12-12 Osterhout Group, Inc. Optical configurations for head-worn see-through displays
US11022810B2 (en) 2014-06-09 2021-06-01 Mentor Acquisition One, Llc Content presentation in head worn computing
US10139635B2 (en) 2014-06-09 2018-11-27 Osterhout Group, Inc. Content presentation in head worn computing
US11360318B2 (en) 2014-06-09 2022-06-14 Mentor Acquisition One, Llc Content presentation in head worn computing
US10976559B2 (en) 2014-06-09 2021-04-13 Mentor Acquisition One, Llc Content presentation in head worn computing
US11887265B2 (en) 2014-06-09 2024-01-30 Mentor Acquisition One, Llc Content presentation in head worn computing
US9720241B2 (en) 2014-06-09 2017-08-01 Osterhout Group, Inc. Content presentation in head worn computing
US11663794B2 (en) 2014-06-09 2023-05-30 Mentor Acquisition One, Llc Content presentation in head worn computing
US9575321B2 (en) 2014-06-09 2017-02-21 Osterhout Group, Inc. Content presentation in head worn computing
US10649220B2 (en) 2014-06-09 2020-05-12 Mentor Acquisition One, Llc Content presentation in head worn computing
US11327323B2 (en) 2014-06-09 2022-05-10 Mentor Acquisition One, Llc Content presentation in head worn computing
US10663740B2 (en) 2014-06-09 2020-05-26 Mentor Acquisition One, Llc Content presentation in head worn computing
US11790617B2 (en) 2014-06-09 2023-10-17 Mentor Acquisition One, Llc Content presentation in head worn computing
US10713394B1 (en) 2014-06-12 2020-07-14 Benko, LLC Filtering components compatible with a computer-modeled structure
US10698212B2 (en) 2014-06-17 2020-06-30 Mentor Acquisition One, Llc External user interface for head worn computing
US11054645B2 (en) 2014-06-17 2021-07-06 Mentor Acquisition One, Llc External user interface for head worn computing
US11789267B2 (en) 2014-06-17 2023-10-17 Mentor Acquisition One, Llc External user interface for head worn computing
US11294180B2 (en) 2014-06-17 2022-04-05 Mentor Acquisition One, Llc External user interface for head worn computing
US9810906B2 (en) 2014-06-17 2017-11-07 Osterhout Group, Inc. External user interface for head worn computing
US10684692B2 (en) 2014-06-19 2020-06-16 Facebook Technologies, Llc Systems, devices, and methods for gesture identification
US10025805B1 (en) 2014-06-24 2018-07-17 Benko, LLC Systems and methods for automated help
US11392396B1 (en) 2014-06-24 2022-07-19 Desprez, Llc Systems and methods for automated help
US11103122B2 (en) 2014-07-15 2021-08-31 Mentor Acquisition One, Llc Content presentation in head worn computing
US11269182B2 (en) 2014-07-15 2022-03-08 Mentor Acquisition One, Llc Content presentation in head worn computing
US11786105B2 (en) 2014-07-15 2023-10-17 Mentor Acquisition One, Llc Content presentation in head worn computing
CN104102412A (en) * 2014-07-24 2014-10-15 央数文化(上海)股份有限公司 Augmented reality technology-based handheld reading equipment and reading method thereof
US20220084003A1 (en) * 2014-07-31 2022-03-17 Walmart Apollo, Llc Systems and methods for managing self check out services
US11790341B2 (en) * 2014-07-31 2023-10-17 Walmart Apollo, Llc Systems and methods for managing self check out services
US11182760B2 (en) * 2014-07-31 2021-11-23 Walmart Apollo, Llc Systems and methods for managing self check out services
US10628814B2 (en) 2014-07-31 2020-04-21 Walmart Apollo, Llc Systems and methods for managing self check out services
US10908422B2 (en) 2014-08-12 2021-02-02 Mentor Acquisition One, Llc Measuring content brightness in head worn computing
US11360314B2 (en) 2014-08-12 2022-06-14 Mentor Acquisition One, Llc Measuring content brightness in head worn computing
US10460342B1 (en) 2014-08-12 2019-10-29 Benko, LLC Methods and software for providing targeted advertising to a product program
US9829707B2 (en) 2014-08-12 2017-11-28 Osterhout Group, Inc. Measuring content brightness in head worn computing
US11630315B2 (en) 2014-08-12 2023-04-18 Mentor Acquisition One, Llc Measuring content brightness in head worn computing
US9696551B2 (en) * 2014-08-13 2017-07-04 Beijing Lenovo Software Ltd. Information processing method and electronic device
US20160048024A1 (en) * 2014-08-13 2016-02-18 Beijing Lenovo Software Ltd. Information processing method and electronic device
US10477352B2 (en) 2014-09-02 2019-11-12 Qualcomm Incorporated Techniques for allocating positioning signal acquisition tasks among a plurality of co-located mobile devices
US20160066147A1 (en) * 2014-09-02 2016-03-03 Qualcomm Incorporated Techniques for allocating positioning signal acquisition tasks among a plurality of co-located mobile devices
US10536807B1 (en) 2014-09-02 2020-01-14 Qualcomm Incorporated Techniques for allocating positioning tasks among a plurality of mobile devices
US10237687B2 (en) * 2014-09-02 2019-03-19 Qualcomm Incorporated Techniques for allocating positioning signal acquisition tasks among a plurality of co-located mobile devices
US20160078512A1 (en) * 2014-09-11 2016-03-17 Oracle International Corporation Character personal shopper system
US9996874B2 (en) * 2014-09-11 2018-06-12 Oracle International Corporation Character personal shopper system
US10162337B2 (en) 2014-09-15 2018-12-25 Desprez, Llc Natural language user interface for computer-aided design systems
US10095217B2 (en) 2014-09-15 2018-10-09 Desprez, Llc Natural language user interface for computer-aided design systems
US10079016B2 (en) 2014-09-15 2018-09-18 Desprez, Llc Natural language user interface for computer-aided design systems
US11599086B2 (en) 2014-09-15 2023-03-07 Desprez, Llc Natural language user interface for computer-aided design systems
US10229679B1 (en) 2014-09-15 2019-03-12 Benko, LLC Natural language user interface for computer-aided design systems
US9613020B1 (en) 2014-09-15 2017-04-04 Benko, LLC Natural language user interface for computer-aided design systems
US10520996B2 (en) 2014-09-18 2019-12-31 Mentor Acquisition One, Llc Thermal management for head-worn computer
US10963025B2 (en) 2014-09-18 2021-03-30 Mentor Acquisition One, Llc Thermal management for head-worn computer
US11474575B2 (en) 2014-09-18 2022-10-18 Mentor Acquisition One, Llc Thermal management for head-worn computer
US9671613B2 (en) 2014-09-26 2017-06-06 Osterhout Group, Inc. See-through computer display systems
US9354711B2 (en) * 2014-09-30 2016-05-31 Xerox Corporation Dynamic hand-gesture-based region of interest localization
US9778750B2 (en) 2014-09-30 2017-10-03 Xerox Corporation Hand-gesture-based region of interest localization
US11276095B1 (en) 2014-10-30 2022-03-15 Desprez, Llc Methods and software for a pricing-method-agnostic ecommerce marketplace for manufacturing services
US11023934B1 (en) 2014-10-30 2021-06-01 Desprez, Llc Business variable optimization for manufacture or supply of designed products
US10836110B2 (en) 2014-10-31 2020-11-17 Desprez, Llc Method and system for ordering expedited production or supply of designed products
US11474498B2 (en) 2014-10-31 2022-10-18 Desprez Llc Methods and systems for ordering expedited production or supply of designed products
US11415961B1 (en) 2014-10-31 2022-08-16 Desprez, Llc Automated correlation of modeled product and preferred manufacturers
US10073439B1 (en) 2014-10-31 2018-09-11 Desprez, Llc Methods, systems, and software for processing expedited production or supply of designed products
US10235009B1 (en) 2014-10-31 2019-03-19 Desprez, Llc Product variable optimization for manufacture or supply of designed products
US20160140759A1 (en) * 2014-11-13 2016-05-19 Mastercard International Incorporated Augmented reality security feeds system, method and apparatus
US10197801B2 (en) 2014-12-03 2019-02-05 Osterhout Group, Inc. Head worn computer display systems
US10684687B2 (en) 2014-12-03 2020-06-16 Mentor Acquisition One, Llc See-through computer display systems
US10018837B2 (en) 2014-12-03 2018-07-10 Osterhout Group, Inc. Head worn computer display systems
US11262846B2 (en) 2014-12-03 2022-03-01 Mentor Acquisition One, Llc See-through computer display systems
US9684172B2 (en) 2014-12-03 2017-06-20 Osterhout Group, Inc. Head worn computer display systems
US10036889B2 (en) 2014-12-03 2018-07-31 Osterhout Group, Inc. Head worn computer display systems
US11809628B2 (en) 2014-12-03 2023-11-07 Mentor Acquisition One, Llc See-through computer display systems
JP2016110565A (en) * 2014-12-10 2016-06-20 富士通株式会社 Display control method, information processing program and information processing device
USD792400S1 (en) 2014-12-31 2017-07-18 Osterhout Group, Inc. Computer glasses
USD794637S1 (en) 2015-01-05 2017-08-15 Osterhout Group, Inc. Air mouse
US10062182B2 (en) 2015-02-17 2018-08-28 Osterhout Group, Inc. See-through computer display systems
US10803501B1 (en) 2015-03-17 2020-10-13 Desprez, Llc Systems, methods, and software for generating, customizing, and automatedly e-mailing a request for quotation for fabricating a computer-modeled structure from within a CAD program
US10547709B2 (en) 2015-06-18 2020-01-28 Qualtrics, Llc Recomposing survey questions for distribution via multiple distribution channels
US11272033B2 (en) 2015-06-18 2022-03-08 Qualtrics, Llc Recomposing survey questions for distribution via multiple distribution channels
US11073901B2 (en) * 2015-07-07 2021-07-27 Seiko Epson Corporation Display device, control method for display device, and computer program
US11301034B2 (en) 2015-07-07 2022-04-12 Seiko Epson Corporation Display device, control method for display device, and computer program
US20180143436A1 (en) * 2015-07-20 2018-05-24 Peishu XIE Head-operated digital eyeglasses
US10832630B2 (en) 2015-08-03 2020-11-10 Qualtrics, Llc Providing a display based electronic survey
US10325568B2 (en) 2015-08-03 2019-06-18 Qualtrics, Llc Providing a display based electronic survey
JP2017049833A (en) * 2015-09-02 2017-03-09 株式会社東芝 Electronic apparatus and support method
US20200159017A1 (en) * 2015-09-10 2020-05-21 Connectivity Labs Inc. Sedentary virtual reality method and systems
US11803055B2 (en) 2015-09-10 2023-10-31 Connectivity Labs Inc. Sedentary virtual reality method and systems
US9298283B1 (en) * 2015-09-10 2016-03-29 Connectivity Labs Inc. Sedentary virtual reality method and systems
US11125996B2 (en) * 2015-09-10 2021-09-21 Connectivity Labs Inc. Sedentary virtual reality method and systems
US9804394B2 (en) 2015-09-10 2017-10-31 Connectivity Labs Inc. Sedentary virtual reality method and systems
US10345588B2 (en) * 2015-09-10 2019-07-09 Connectivity Labs Inc. Sedentary virtual reality method and systems
US20170090850A1 (en) * 2015-09-29 2017-03-30 International Business Machines Corporation Enabling proximity-aware visual identification
US9891879B2 (en) * 2015-09-29 2018-02-13 International Business Machines Corporation Enabling proximity-aware visual identification
US10401953B2 (en) * 2015-10-26 2019-09-03 Pillantas Inc. Systems and methods for eye vergence control in real and augmented reality environments
US11004126B1 (en) 2016-03-17 2021-05-11 Desprez, Llc Systems, methods, and software for generating, customizing, and automatedly e-mailing a request for quotation for fabricating a computer-modeled structure from within a CAD program
US11423449B1 (en) 2016-03-23 2022-08-23 Desprez, Llc Electronic pricing machine configured to generate prices based on supplier willingness and a user interface therefor
US10556309B1 (en) 2016-03-24 2020-02-11 Proto Labs Inc. Methods of subtractively manufacturing a plurality of discrete objects from a single workpiece using a removable fixating material
US10401824B2 (en) 2016-04-14 2019-09-03 The Rapid Manufacturing Group LLC Methods and software for reducing machining equipment usage when machining multiple objects from a single workpiece
US11500212B2 (en) 2016-05-09 2022-11-15 Mentor Acquisition One, Llc User interface systems for head-worn computers
US11320656B2 (en) 2016-05-09 2022-05-03 Mentor Acquisition One, Llc User interface systems for head-worn computers
US10684478B2 (en) 2016-05-09 2020-06-16 Mentor Acquisition One, Llc User interface systems for head-worn computers
US11226691B2 (en) 2016-05-09 2022-01-18 Mentor Acquisition One, Llc User interface systems for head-worn computers
US10824253B2 (en) 2016-05-09 2020-11-03 Mentor Acquisition One, Llc User interface systems for head-worn computers
US11754845B2 (en) 2016-06-01 2023-09-12 Mentor Acquisition One, Llc Modular systems for head-worn computers
US11460708B2 (en) 2016-06-01 2022-10-04 Mentor Acquisition One, Llc Modular systems for head-worn computers
US11022808B2 (en) 2016-06-01 2021-06-01 Mentor Acquisition One, Llc Modular systems for head-worn computers
US11586048B2 (en) 2016-06-01 2023-02-21 Mentor Acquisition One, Llc Modular systems for head-worn computers
US10466491B2 (en) 2016-06-01 2019-11-05 Mentor Acquisition One, Llc Modular systems for head-worn computers
US10591988B2 (en) * 2016-06-28 2020-03-17 Hiscene Information Technology Co., Ltd Method for displaying user interface of head-mounted display device
US11360551B2 (en) * 2016-06-28 2022-06-14 Hiscene Information Technology Co., Ltd Method for displaying user interface of head-mounted display device
US20180040161A1 (en) * 2016-08-02 2018-02-08 Qualtrics, Llc Conducting digital surveys utilizing virtual reality and augmented reality devices
US10176640B2 (en) * 2016-08-02 2019-01-08 Qualtrics, Llc Conducting digital surveys utilizing virtual reality and augmented reality devices
US11657576B2 (en) 2016-08-02 2023-05-23 Qualtrics, Llc Conducting digital surveys utilizing virtual reality and augmented reality devices
KR101804884B1 (en) * 2016-08-08 2017-12-06 한국정보통신주식회사 A method for providing virtual reality contents, and an appratus using the method
US11409128B2 (en) 2016-08-29 2022-08-09 Mentor Acquisition One, Llc Adjustable nose bridge assembly for headworn computer
US10690936B2 (en) 2016-08-29 2020-06-23 Mentor Acquisition One, Llc Adjustable nose bridge assembly for headworn computer
US11301877B2 (en) 2016-09-01 2022-04-12 Qualtrics, Llc Providing analysis of perception data over time for events
US11415856B2 (en) 2016-09-08 2022-08-16 Mentor Acquisition One, Llc Electrochromic systems for head-worn computer systems
US11768417B2 (en) 2016-09-08 2023-09-26 Mentor Acquisition One, Llc Electrochromic systems for head-worn computer systems
US10768500B2 (en) 2016-09-08 2020-09-08 Mentor Acquisition One, Llc Electrochromic systems for head-worn computer systems
US10008042B2 (en) * 2016-09-30 2018-06-26 Intel Corporation Augmented reality rendered structured content
US10108846B2 (en) * 2016-09-30 2018-10-23 Autodesk, Inc. Scanning for materials and scale information
US10585939B2 (en) * 2016-10-11 2020-03-10 International Business Machines Corporation Real time object description service integrated with knowledge center on augmented reality (AR) and virtual reality (VR) devices
US20180101550A1 (en) * 2016-10-11 2018-04-12 International Business Machines Corporation Real time object description service integrated with knowledge center on augmented reality (ar) and virtual reality (vr) devices
USD840395S1 (en) 2016-10-17 2019-02-12 Osterhout Group, Inc. Head-worn computer
CN106601043A (en) * 2016-11-07 2017-04-26 爱可目(北京)科技股份有限公司 Multimedia interaction education device and multimedia interaction education method based on augmented reality
US11003868B2 (en) * 2016-11-07 2021-05-11 ' Rockwell Automation Technologies, Inc. Filtering display data
US11195214B1 (en) 2016-11-08 2021-12-07 Wells Fargo Bank, N.A. Augmented reality value advisor
US10636063B1 (en) 2016-11-08 2020-04-28 Wells Fargo Bank, N.A. Method for an augmented reality value advisor
US11756124B1 (en) 2016-12-01 2023-09-12 Wells Fargo Bank, N.A. Visualizing investment changes using augmented reality
US11403709B1 (en) 2016-12-01 2022-08-02 Wells Fargo Bank, N.A. Visualizing investment changes using augmented reality
US11094017B1 (en) 2016-12-01 2021-08-17 Wells Fargo Bank, N.A. Visualizing investment changes using augmented reality
US10545481B2 (en) 2016-12-28 2020-01-28 Proto Labs Inc Methods and software for providing graphical representations of a plurality of objects in a central through opening
US11210854B2 (en) 2016-12-30 2021-12-28 Facebook, Inc. Systems and methods for providing augmented reality personalized content
US20180190033A1 (en) * 2016-12-30 2018-07-05 Facebook, Inc. Systems and methods for providing augmented reality effects and three-dimensional mapping associated with interior spaces
USD864959S1 (en) 2017-01-04 2019-10-29 Mentor Acquisition One, Llc Computer glasses
USD947186S1 (en) 2017-01-04 2022-03-29 Mentor Acquisition One, Llc Computer glasses
USD918905S1 (en) 2017-01-04 2021-05-11 Mentor Acquisition One, Llc Computer glasses
US10318011B2 (en) * 2017-01-06 2019-06-11 Lumini Corporation Gesture-controlled augmented reality experience using a mobile communications device
US10402888B2 (en) * 2017-01-19 2019-09-03 Samsung Electronics Co., Ltd. System and method for virtual reality content rating using biometric data
US20180204266A1 (en) * 2017-01-19 2018-07-19 Samsung Electronics Co., Ltd System and method for virtual reality content rating using biometric data
US20180349837A1 (en) * 2017-05-19 2018-12-06 Hcl Technologies Limited System and method for inventory management within a warehouse
EP3410261A1 (en) * 2017-06-01 2018-12-05 VDEh-Betriebsforschungsinstitut GmbH Visualisation of quality information
US10949667B2 (en) * 2017-09-14 2021-03-16 Ebay Inc. Camera platform and object inventory control
KR20220038517A (en) * 2017-09-14 2022-03-28 이베이 인크. Camera platform and object inventory control
KR102596920B1 (en) * 2017-09-14 2023-11-06 이베이 인크. Camera platform and object inventory control
US11659143B2 (en) 2017-09-14 2023-05-23 Ebay Inc. Camera platform incorporating schedule and stature
US11126849B2 (en) * 2017-09-14 2021-09-21 Ebay Inc. Camera platform incorporating schedule and stature
US11665320B2 (en) 2017-09-14 2023-05-30 Ebay Inc. Camera platform and object inventory control
CN111183449A (en) * 2017-09-14 2020-05-19 电子湾有限公司 Camera platform and object inventory control
WO2019055352A1 (en) * 2017-09-14 2019-03-21 Ebay Inc. Camera platform and object inventory control
US20190080171A1 (en) * 2017-09-14 2019-03-14 Ebay Inc. Camera Platform and Object Inventory Control
US10509962B2 (en) 2017-09-14 2019-12-17 Ebay Inc. Camera platform incorporating schedule and stature
US10430778B2 (en) * 2017-09-29 2019-10-01 Paypal, Inc. Using augmented reality for secure transactions
WO2019067035A1 (en) * 2017-09-29 2019-04-04 Microsoft Technology Licensing, Llc Entity attribute identification
US11423366B2 (en) 2017-09-29 2022-08-23 Paypal, Inc. Using augmented reality for secure transactions
US11030606B2 (en) * 2017-09-29 2021-06-08 Paypal, Inc. Using augmented reality for secure transactions
US11635736B2 (en) 2017-10-19 2023-04-25 Meta Platforms Technologies, Llc Systems and methods for identifying biological structures associated with neuromuscular source signals
US11200617B2 (en) 2017-11-17 2021-12-14 Ebay Inc. Efficient rendering of 3D models using model placement metadata
US11556980B2 (en) 2017-11-17 2023-01-17 Ebay Inc. Method, system, and computer-readable storage media for rendering of object data based on recognition and/or location matching
US10891685B2 (en) 2017-11-17 2021-01-12 Ebay Inc. Efficient rendering of 3D models using model placement metadata
US11080780B2 (en) 2017-11-17 2021-08-03 Ebay Inc. Method, system and computer-readable media for rendering of three-dimensional model data based on characteristics of objects in a real-world environment
WO2019099585A1 (en) * 2017-11-17 2019-05-23 Ebay Inc. Rendering virtual content based on items recognized in a real-world environment
US10963940B2 (en) * 2017-12-29 2021-03-30 Ebay Inc. Computer vision, user segment, and missing item determination
US20210201378A1 (en) * 2017-12-29 2021-07-01 Ebay Inc. Computer Vision, User Segment, and Missing Item Determination
US20190205646A1 (en) * 2017-12-29 2019-07-04 Ebay Inc. Computer Vision, User Segment, and Missing Item Determination
US11200611B2 (en) 2017-12-29 2021-12-14 Ebay Inc. Computer vision for unsuccessful queries and iterative search
US11636524B2 (en) * 2017-12-29 2023-04-25 Ebay Inc. Computer vision, user segment, and missing item determination
US11250487B2 (en) 2017-12-29 2022-02-15 Ebay Inc. Computer vision and image characteristic search
US20230222560A1 (en) * 2017-12-29 2023-07-13 Ebay Inc. Computer Vision, User Segment, and Missing Item Determination
US10528998B2 (en) * 2018-04-11 2020-01-07 Trivver, Inc. Systems and methods for presenting information related to products or services being shown on a second display device on a first display device using augmented reality technology
US20190378334A1 (en) * 2018-06-08 2019-12-12 Vulcan Inc. Augmented reality portal-based applications
US11195336B2 (en) 2018-06-08 2021-12-07 Vulcan Inc. Framework for augmented reality applications
US10878037B2 (en) 2018-06-21 2020-12-29 Google Llc Digital supplement association and retrieval for visual search
US11023106B2 (en) 2018-06-21 2021-06-01 Google Llc Digital supplement association and retrieval for visual search
US11640431B2 (en) 2018-06-21 2023-05-02 Google Llc Digital supplement association and retrieval for visual search
US10579230B2 (en) * 2018-06-21 2020-03-03 Google Llc Digital supplement association and retrieval for visual search
CN110646938A (en) * 2018-06-27 2020-01-03 脸谱科技有限责任公司 Near-eye display system
US10996831B2 (en) * 2018-06-29 2021-05-04 Vulcan Inc. Augmented reality cursors
US20200026413A1 (en) * 2018-06-29 2020-01-23 Vulcan Inc. Augmented reality cursors
US20180345129A1 (en) * 2018-07-27 2018-12-06 Yogesh Rathod Display virtual objects within predefined geofence or receiving of unique code from closest beacon
US11087539B2 (en) 2018-08-21 2021-08-10 Mastercard International Incorporated Systems and methods for generating augmented reality-based profiles
US10983809B2 (en) 2018-08-29 2021-04-20 Bank Of America Corporation Providing augmented reality user interfaces and controlling automated systems based on user activity information and pre-staging information
US10768951B2 (en) * 2018-08-29 2020-09-08 Bank Of America Corporation Providing augmented reality user interfaces and controlling automated systems based on user activity information and pre-staging information
US11301272B2 (en) 2018-08-29 2022-04-12 Bank Of America Corporation Providing augmented reality user interfaces and controlling automated systems based on user activity information and pre-staging information
US11567573B2 (en) 2018-09-20 2023-01-31 Meta Platforms Technologies, Llc Neuromuscular text entry, writing and drawing in augmented reality systems
US10970936B2 (en) * 2018-10-05 2021-04-06 Facebook Technologies, Llc Use of neuromuscular signals to provide enhanced interactions with physical objects in an augmented reality environment
US11797087B2 (en) 2018-11-27 2023-10-24 Meta Platforms Technologies, Llc Methods and apparatus for autocalibration of a wearable electrode sensor system
US11090561B2 (en) 2019-02-15 2021-08-17 Microsoft Technology Licensing, Llc Aligning location for a shared augmented reality experience
US11481030B2 (en) 2019-03-29 2022-10-25 Meta Platforms Technologies, Llc Methods and apparatus for gesture detection and classification
US11481031B1 (en) 2019-04-30 2022-10-25 Meta Platforms Technologies, Llc Devices, systems, and methods for controlling computing devices via neuromuscular signals of users
US11097194B2 (en) * 2019-05-16 2021-08-24 Microsoft Technology Licensing, Llc Shared augmented reality game within a shared coordinate space
US20210346810A1 (en) * 2019-05-16 2021-11-11 Microsoft Technology Licensing, Llc Shared Augmented Reality Game Within a Shared Coordinate Space
US10976804B1 (en) * 2019-07-09 2021-04-13 Facebook Technologies, Llc Pointer-based interaction with a virtual surface using a peripheral device in artificial reality environments
US11023036B1 (en) 2019-07-09 2021-06-01 Facebook Technologies, Llc Virtual drawing surface interaction using a peripheral device in artificial reality environments
US11023035B1 (en) 2019-07-09 2021-06-01 Facebook Technologies, Llc Virtual pinboard interaction using a peripheral device in artificial reality environments
US11493993B2 (en) 2019-09-04 2022-11-08 Meta Platforms Technologies, Llc Systems, methods, and interfaces for performing inputs based on neuromuscular control
US11907423B2 (en) 2019-11-25 2024-02-20 Meta Platforms Technologies, Llc Systems and methods for contextualized interactions with an environment
US11373342B2 (en) * 2020-01-14 2022-06-28 Sony Interactive Entertainment Inc. Social and scene target awareness and adaptation of an occlusion system for increased social and scene interaction in an optical see-through augmented reality head mounted display
US11145397B1 (en) 2020-01-31 2021-10-12 Express Scripts Strategie Development, Inc. System and method for augmented reality detection of loose pharmacy items
US11776673B2 (en) 2020-01-31 2023-10-03 Express Scripts Strategic Development, Inc. System and method for augmented reality detection of loose pharmacy items
US20220012789A1 (en) * 2020-07-07 2022-01-13 W.W. Grainger, Inc. System and method for providing real-time visual search
US11269411B1 (en) * 2020-12-10 2022-03-08 A9.Com, Inc. Gaze dependent ocular mode controller for mixed reality
US11868531B1 (en) 2021-04-08 2024-01-09 Meta Platforms Technologies, Llc Wearable device providing for thumb-to-finger-based input gestures detected based on neuromuscular signals, and systems and methods of use thereof
WO2023009432A1 (en) * 2021-07-29 2023-02-02 La Barrie Simeon System and method of data routing for videotelephonic shopping

Also Published As

Publication number Publication date
US11417066B2 (en) 2022-08-16
US20180158248A1 (en) 2018-06-07
US9665983B2 (en) 2017-05-30
US20150302517A1 (en) 2015-10-22
US10878636B2 (en) 2020-12-29
US20160071326A1 (en) 2016-03-10
US20140222558A1 (en) 2014-08-07
WO2013166191A3 (en) 2013-12-19
US10388070B2 (en) 2019-08-20
US20130297460A1 (en) 2013-11-07
US20210104101A1 (en) 2021-04-08
US10127735B2 (en) 2018-11-13
US20220392178A1 (en) 2022-12-08
WO2013166191A2 (en) 2013-11-07
US20190371077A1 (en) 2019-12-05

Similar Documents

Publication Publication Date Title
US11417066B2 (en) System and method for selecting targets in an augmented reality environment
US11398088B2 (en) Systems, methods and apparatuses to generate a fingerprint of a physical location for placement of virtual objects
KR102379643B1 (en) Data mesh platform
US9288079B2 (en) Virtual notes in a reality overlay
US8494215B2 (en) Augmenting a field of view in connection with vision-tracking
US20130286048A1 (en) Method and system for managing data in terminal-server environments
US20140214628A1 (en) Gesture-Based Product Wishlist And Shared Social Networking
US11782933B2 (en) Search result optimization using machine learning models
US11023106B2 (en) Digital supplement association and retrieval for visual search
CN105229566A (en) In augmented reality system, instruction is observed or visual pattern
US10878037B2 (en) Digital supplement association and retrieval for visual search
US20230353639A1 (en) Analyzing augmented reality content usage data
US11706167B2 (en) Generating and accessing video content for products
US20200068133A1 (en) Edge-Facing Camera Enabled Systems, Methods and Apparatuses
US20230111437A1 (en) System and method for content recognition and data categorization
US20200090656A1 (en) Sensor Based Semantic Object Generation
JP2022110057A (en) Digital supplement association and retrieval for visual search
KR101589957B1 (en) Method and program for selling products linked to contents
Gao Application of wireless technologies in mobile business

Legal Events

Date Code Title Description
AS Assignment

Owner name: ZAMBALA LLLP, NEVADA

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:SPIVACK, NOVA T.;REEL/FRAME:030330/0677

Effective date: 20130422

STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION

AS Assignment

Owner name: AUGMENTED REALITY HOLDINGS, LLC, DELAWARE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:ZAMBALA, LLLP;REEL/FRAME:043662/0346

Effective date: 20170918

AS Assignment

Owner name: AUGMENTED REALITY HOLDINGS 2, LLC, DELAWARE

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:AUGMENTED REALITY HOLDINGS, LLC;REEL/FRAME:045922/0001

Effective date: 20180216