US20140372856A1 - Natural Quick Functions Gestures - Google Patents
Natural Quick Functions Gestures Download PDFInfo
- Publication number
- US20140372856A1 US20140372856A1 US13/918,871 US201313918871A US2014372856A1 US 20140372856 A1 US20140372856 A1 US 20140372856A1 US 201313918871 A US201313918871 A US 201313918871A US 2014372856 A1 US2014372856 A1 US 2014372856A1
- Authority
- US
- United States
- Prior art keywords
- spreadsheet document
- gesture
- spreadsheet
- receiving
- gesture input
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Abandoned
Links
Images
Classifications
-
- G06F17/246—
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F1/00—Details not covered by groups G06F3/00 - G06F13/00 and G06F21/00
- G06F1/16—Constructional details or arrangements
- G06F1/18—Packaging or power distribution
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/048—Interaction techniques based on graphical user interfaces [GUI]
- G06F3/0487—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser
- G06F3/0488—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures
- G06F3/04883—Interaction techniques based on graphical user interfaces [GUI] using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser using a touch-screen or digitiser, e.g. input of commands through traced gestures for inputting data by handwriting, e.g. gesture or text
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F40/00—Handling natural language data
- G06F40/10—Text processing
- G06F40/166—Editing, e.g. inserting or deleting
- G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
- G06F40/18—Editing, e.g. inserting or deleting of tables; using ruled lines of spreadsheets
Definitions
- Typical spreadsheet solutions provide a grid made up of rows and columns for receiving and manipulating data.
- users interact with spreadsheets through input devices, such as mice, touchscreens, motion-detection devices, graphical user interfaces, and keyboards.
- input devices such as mice, touchscreens, motion-detection devices, graphical user interfaces, and keyboards.
- this interaction can be frustrating.
- the user interface may be designed as a mouse-focused UI. Selection of and interaction with data and functionality controls via touch may be the same as or similar to selection and interaction via a mouse, wherein differences, in some cases, may include providing larger functionality controls for touch input, or that a user's finger is used instead of a mouse cursor.
- touch or motion-sensing devices may not be suited for productivity work, and may be more suited for consumption of data.
- Current UI interactions with a spreadsheet application on touch or motion-sensing devices may not take advantage of using natural gestures to interact with the application.
- an improvement and technical advantage will be gained by providing a variety of natural gestures for interacting with an application user interface, such as a spreadsheet application user interface and associated documents.
- Embodiments of the present invention solve the above and other problems by providing natural gestures for interacting with spreadsheet documents.
- a user may interact with a spreadsheet document using a variety of touch-based gestures.
- Gestures applied to a spreadsheet document may be used for making structural and display changes to the document.
- gestures may be used on the spreadsheet document for inserting new columns and rows; for revealing hidden columns and rows; for deleting columns and rows; for extending functionalities across columns and rows; for moving objects displayed in a spreadsheet document; and for navigating various parts of a spreadsheet document.
- FIG. 1 illustrates a system architecture in which embodiments of the present invention may be practiced
- FIG. 2 illustrates a touch input system in which embodiments of the present invention may be practiced
- FIGS. 3A-3D illustrate a pulling-apart gesture
- FIGS. 6A-6B illustrate a tap-hold-pull gesture
- FIGS. 7A-7B illustrate a push-together gesture
- FIGS. 9A-9C illustrate a multi-finger swipe gesture
- FIGS. 10A-10D illustrate a touch-and-throw gesture
- FIGS. 11A-11B illustrate a touch-and-pull gesture
- FIGS. 12A-12B illustrate a pinch-and-turn gesture
- FIGS. 13A-13B illustrate a multi-finger tear-and-pull gesture
- FIG. 14 shows one embodiment of a method for providing natural quick function gestures
- FIG. 15 is a block diagram illustrating one embodiment of the physical components of a computing device with which embodiments of the invention may be practiced;
- FIGS. 16A and 16B are simplified block diagrams of a mobile computing device with which embodiments of the present invention may be practiced.
- FIG. 17 is a simplified block diagram of a distributed computing system in which embodiments of the present invention may be practiced.
- FIG. 1 illustrates a system architecture in which embodiments of the present invention may be practiced.
- the system 100 includes one or more computing devices such as a personal computer (e.g., a desktop or laptop computer) 102 A, a tablet computer 102 B, a smart phone 102 C, and a large-scale display 102 D.
- Each of the computing devices 102 may run an application 104 for displaying and editing data.
- an application 104 may include a spreadsheet application for displaying and editing a spreadsheet 106 .
- the computing devices 102 are for purposes of illustration only and are not exhaustive of the various types of computing devices that may be used in accordance with embodiments of the present invention.
- the large-scale display 102 D is illustrated as a display unit with an associated computing device 102 A
- the large-form display device 102 D may operate as an integrated display/computing device that may be used for any suitable use from large-scale display uses, for example, in a conference room, to personal computing, as desired.
- each of the computing devices 102 illustrated in FIG. 1 may receive input through a variety of suitable means including touch or gesture input, keyboard input, mouse input, voice command input, electronic inking input, and the like.
- each computing device 102 may comprise an operating system configured to include support for touch and gesture input.
- a gesture manager 108 illustrated in association with each of the computing devices and which will be described in greater detail below, may be utilized to process some/all of the touch and gesture input that is received from touch and gesture input.
- the applications 104 illustrated in association with each of the computing devices 102 are illustrative of applications 104 having sufficient computer executable instructions for enabling embodiments of the present invention as described herein.
- An example application 104 includes a spreadsheet application, such as Excel® manufactured by Microsoft Corporation of Redmond, Wash. As should be appreciated, this is but one example spreadsheet application, and any other applications 104 suitable for enabling embodiments described herein may be used.
- each application 104 may be a fully functional “thick” client application having all application functionality including the ability to send and receive data to and from other applications 102 operating on other computing devices in the collaborative work session.
- the computing devices 102 may communicate via a distributed computing network 112 , for example, the Internet.
- An Internet-based or “cloud-based” server computer 110 may be operated on one or more web services applications 114 for providing a collaborative work session.
- each of the devices 102 may operate as thin client systems, and collaborative communication between the devices may be operated at the direction of the web services application 114 operated at a remote site.
- an application 104 may operate at the remote server 110 wherein each of the devices 102 may access and receive the functionality described herein via a remote service from the server based application, for example, via the web services application 114 .
- the system 100 may comprise a gesture manager 108 configured to detect gestures that are received by a computing device 102 .
- a gesture manager 108 may be configured to sense when a gesture is received that is related to performing an operation in conjunction with an application program 104 , such as a spreadsheet application.
- a gesture may include user interaction with data displayed on or functionalities associated with a computing device 102 , wherein the user interaction may originate from a bodily movement (e.g., movement of one or more fingers, hands, arms, etc.).
- a gesture may include bodily contact with a touch input device (e.g., a touchscreen 116 ), or may include bodily movement detected by a sensor, such as a camera, an accelerometer, a motion detector, etc.
- a touch input device may be operable to recognize a presence of two or more points of contact on a sensing surface, sometimes referred to as multi-touch input.
- a gesture may include user interaction with a touchscreen 116 using bodily movements to manipulate data and/or to select functionalities, for example, data displayed in and functionalities associated with an application 104 , such as a spreadsheet application.
- the term “gesture” includes touch input.
- gesturing may enable users to communicate with a computing device 102 and interact naturally. Different types of gestures may be received. For example, a tapping gesture, a pulling gesture, a dragging gesture, a pushing gesture, a throwing gesture, panning gesture, a tearing gesture, a pinch-and-turn gesture, and the like may be received.
- the gestures may be used to change a display of information, activate/deactivate functions, and/or perform some other type of operation associated with an application 104 or some other function and/or program. Additional details regarding the gestures will be provided below.
- FIG. 2 illustrates an exemplary touch input system 200 .
- the touch input system 200 as illustrated comprises a touch panel 216 that comprises several sensors 204 integrated therein.
- the sensors 204 are Infrared (IR) sensors.
- the touch input system 200 is configured to detect objects that either in contact with a touchable surface 116 or are close to but not in actual contact with (“adjacent”) touchable surface 116 .
- the objects that are sensed may be many different types of objects such as finger, hands, or other physical objects.
- Infrared sensors 204 are distributed throughout touch panel 216 and are arranged parallel to touchable surface 116 .
- One or more of the infrared sensors 204 may detect infrared radiation reflected from objects, such as hand 208 , as indicated by the arrow.
- the touchable surface 116 is horizontal, but in a different embodiment generated by rotating system 200 clockwise by 90 degrees, the touchable surface 116 could be vertical. In that embodiment, the objects from which reflected IR radiation is detected are to the side of touchable surface 116 .
- the term “above” is intended to be applicable to all such orientations.
- the touchable surface 116 may also be changed to other orientations.
- the touch panel 216 may comprise filters 212 that absorb visible light and transmit infrared radiation and are located between touchable surface 116 and IR sensors 204 in order to shield IR sensors 204 from visible light incident on touchable surface 116 in the case where IR sensors 204 are sensitive to a broader range of wavelengths of light other than purely infrared wavelengths.
- the touch panel 216 may comprise a display that is configured to display images that are viewable via the touchable surface 116 .
- the displayed image may be images relating to an application 104 , such as a spreadsheet 106 .
- the display may be, for example, an LCD, an organic light emitting diode (OLED) display, a flexible display such as electronic paper, or any other suitable display in which an IR sensor can be integrated.
- OLED organic light emitting diode
- the system 200 may comprise a backlight 216 for the display.
- the backlight 216 may comprise at least one IR source 218 that is configured to illuminate objects in contact with or adjacent to the touchable surface 116 with infrared radiation through the touchable surface 116 , as indicated by the arrows.
- IR sensors 204 are sensitive to radiation incident from above, so IR radiation traveling directly from the backlight 216 to the IR sensors 204 is not detected.
- sensors 204 may be processed by the gesture manager 108 and/or functionality included within an operating system or some other application to detect when a physical object (e.g., a hand, a finger, a stylus, etc.) has come into physical contact with a portion of the touch input surface 116 and/or a physical object is in close proximity to the surface.
- sensors 204 can detect when a portion of a user's hand 208 , such as one or more fingers, has come in contact with or is near to the touch input display surface 116 .
- Additional sensors can be embedded in the touch input display surface 116 and can include for example, accelerometers, pressure sensors, temperature sensors, image scanners, barcode scanners, etc., to detect multiple simultaneous inputs.
- the IR radiation reflected from the objects may be reflected from a user's hands, fingers, reflective ink patterns on the objects, metal designs on the objects or any other suitable reflector. Fingers reflect enough of the near IR to detect that a finger or hand 208 is located at a particular location on or adjacent the touchable surface 116 . A higher resolution of IR sensors may be used to scan objects in order to achieve higher resolution.
- Sensors 204 can be included (e.g., embedded) in a plurality of locations. The density of sensors 204 can be sufficient such that contact across the entirety of the touch input surface 116 can be detected. Sensors 204 may be configured to sample the surface of touch input display surface 206 at specified intervals, such as, for example, 1 ms, 5 ms, etc. for detected contact and/or near contact. The sensor data received from sensors 204 changes between sampling intervals as detected objects move on the touch surface; detected objects are no longer within range of detection; and when new objects come in range of detection. For example, the gesture manager 108 can determine that contact was first detected at a first location and then contact was subsequently moved to other locations. In response, the gesture manager 108 may determine when a gesture is received and what type of gesture is received.
- FIG. 2 provides just one example of a touch input system.
- the backlight may not comprise any IR sources and the surface 116 may include a frontlight which comprises at least one IR source.
- the touchable surface 116 of the system is a surface of the frontlight.
- the frontlight may comprise a light guide, so that IR radiation emitted from IR source travels through the light guide and is directed towards touchable surface and any objects in contact with or adjacent to it.
- both the backlight and frontlight may comprise IR sources.
- there is no backlight and the frontlight comprises both IR sources and visible light sources.
- the system may not comprise a frontlight or a backlight, but instead the IR sources may be integrated within the touch panel.
- the touch input system 200 may comprise an OLED display which comprises IR OLED emitters and IR-sensitive organic photosensors (which may comprise reverse-biased OLEDs).
- a display may not be included.
- the touch system 200 may be configured to not display images. For example, this may be the case when the touch input device is separate from a display.
- Other examples include a touchpad, a gesture pad, and similar non-display devices and components.
- a gesture may not be recognized when the gesture is not performed touching the surface 116 .
- a gesture may be recognized when performed above the surface 116 .
- the IR source of the touch input system may be turned on only if the touchable surface 116 is touched. Alternatively, the IR source may be turned on regardless of whether the touchable surface 116 is touched, and detection of whether actual contact between the touchable surface 116 and the object occurred is processed along with the output of the IR sensor.
- Actual contact between the touchable surface 116 and the object may be detected by any suitable means, including, for example, by a vibration sensor or microphone coupled to the touch panel 216 .
- sensors to detect contact includes pressure-based mechanisms, micro-machined accelerometers, piezoelectric devices, capacitive sensors, resistive sensors, inductive sensors, laser vibrometers, and LED vibrometers.
- the gesture manager 108 may be configured to recognize many different types of gestures. Some of the gestures may be context dependent, be specific to an application 104 , and/or be used within many different types of applications 104 . For example, gestures may be used to interact with a spreadsheet 106 that is associated with a spreadsheet application. Gestures may be received in many different locations relating to touch input device/display 116 . For example, a gesture may be received within a display of spreadsheet, within a gesture box, and/or at some other location on a display. Exemplary gestures and interactions with a spreadsheet are described below.
- gestures may be used in association with spreadsheet documents for manipulating information contained in spreadsheet documents and navigating spreadsheet documents, but also, gestures may be used for making structural and display changes to spreadsheet documents. As described below, gestures may be used for adding, deleting and moving columns and rows; for revealing or hiding columns and rows; for applying spreadsheet functionality; and for moving spreadsheet objects. As should be understood, these are but a few examples of some of the uses of gestures applied to spreadsheet documents as will be more fully described below.
- FIGS. 3A and 3B show a pull-apart gesture to insert one or more columns within a spreadsheet 106 .
- FIG. 3A shows a spreadsheet 106 before pulling apart columns 302 to insert one or more columns 302
- FIG. 3B shows the same spreadsheet 106 while performing a column insertion operation via a pull-apart gesture 310 .
- a pull-apart gesture 310 may be detected when a user touches a touchscreen 116 with two of his fingers and pulls them apart as indicated by gesture 310 .
- the user may select a first column 302 in a spreadsheet 106 , for example, by touching a column header 312 associated with the first column 302 ; select an adjacent column 302 , for example, by touching the column header 312 associated with the adjacent column 302 ; and may utilize the pull-apart gesture 310 along an X-axis 306 to insert a column 302 between the first and second columns 302 .
- one or more columns 302 may be inserted.
- one or more rows 304 may be inserted into a spreadsheet 106 via selecting adjacent rows 304 , and utilizing the pull-apart gesture 310 along a Y-axis 308 . Depending on how far apart the user moves his fingers, one or more rows 304 may be inserted. A live preview of the column/row insertion operation may be displayed such that the user may be able to determine a number of columns 302 or rows 304 that will be inserted upon releasing the touch.
- the pull-apart gesture 310 may be utilized to perform a column or row revelation operation, for example, when a column 302 or row 304 may be hidden between two columns 302 or rows 304 .
- the user may select a first column 302 in a spreadsheet 106 , for example, by touching a column header 312 associated with the first column 302 ; select an adjacent column 302 , for example, by touching the column header 312 associated with the adjacent column 302 ; and may utilize the pull-apart gesture 310 along an X-axis 306 to reveal one or more hidden column 314 between the first and second columns 302 .
- one or more hidden rows may be revealed into a spreadsheet 106 via selecting adjacent rows 304 , and utilizing the pull-apart gesture 310 along a Y-axis 308 .
- one or more rows 304 may be revealed.
- a live preview of the column/row revelation operation may be displayed such that the user may be able to determine how far apart to move the columns 302 or rows 304 to reveal the desired hidden column(s) 314 or rows.
- FIGS. 4A and 4B show a touch-and-slide gesture to insert one or more columns 302 within a spreadsheet 106 .
- FIG. 4A shows a spreadsheet 106 before one or more columns 302 are inserted via a touch-and-slide gesture 410
- FIG. 4B shows the same spreadsheet 106 while performing a column insertion operation via a touch-and-slide gesture 410 .
- a tap-and-slide gesture 410 may be detected when a user touches a touchscreen 116 on a displayed selectable UI control, such as an insert widget 402 , with a finger, and slides the insert widget 402 as indicated by gesture 410 .
- the selectable UI control (e.g., insert widget 402 ) may be displayed between adjoining columns 302 .
- the selectable UI control (e.g., insert widget 402 ) may be displayed upon receiving an indication of a selection of a column 302 .
- a user may select a column 302 via touching a column header 312 , and accordingly, an insert widget 402 may be displayed between the selected column 302 and an adjacent column 302 .
- the user may select the insert widget 402 (e.g., via touching the widget), and then slide the insert widget 402 (i.e., touch-and-slide gesture 410 ) along the X-axis 306 to insert a column 302 .
- one or more columns 302 may be inserted.
- one or more rows 304 may be inserted into a spreadsheet 106 via selecting a row header 412 , and utilizing the touch-and-slide gesture 410 to select and slide a displayed selectable UI control (e.g., insert widget 402 ) along the Y-axis 308 .
- a displayed selectable UI control e.g., insert widget 402
- a live preview of the column/row insertion operation may be displayed such that the user may be able to determine a number of columns 302 or rows 304 that will be inserted upon releasing the touch.
- FIGS. 5A and 5B show a drag-and-drop away gesture to delete a column 302 within a spreadsheet 106 .
- FIG. 5A shows a spreadsheet 106 before a column 302 is deleted via a drag-and-drop away gesture 510
- FIG. 5B shows the same spreadsheet 106 while performing a column deletion operation via a drag-and-drop away gesture 510 .
- a drag-and-drop away gesture 510 may be detected when a user selects a column 302 , for example, by touching a touchscreen 116 on an associated column header 312 with a finger, and drags the column header 312 in an upwards or downwards direction and away from the spreadsheet 106 as indicated by gesture 510 .
- the associated column 302 may be deleted.
- one or more rows 304 may be deleted from a spreadsheet 106 via selecting a row 304 , for example, by the user selecting a row header 412 of a row he wishes to delete, and utilizing the drag-and-drop away gesture 510 to drag the row header 412 leftwards (in a left-to-right reading scenario) and away from the other row headers 412 .
- the associated row 304 may be deleted.
- a live preview of the column/row deletion operation may be displayed such that the user may be able to determine how far he needs to drag the column header 312 or row header 412 to delete the associated column 302 or row 304 upon releasing the touch.
- FIGS. 6A and 6B show a touch-hold-pull gesture to reveal a hidden column 314 within a spreadsheet 106 .
- FIG. 6A shows a spreadsheet 106 before a hidden column 302 is shown via a touch-hold-pull gesture 610
- FIG. 6B shows the same spreadsheet 106 while performing a hidden column revelation operation via a touch-hold-pull gesture 610 .
- a touch-hold-pull gesture 610 may be detected when a user selects a first column 302 , for example, by touching a touchscreen 116 on the column header 312 associated with the first column with a finger on one hand 208 , and holding his finger on the column header 312 ; then selects a second column 302 , for example, by touching on the column header 312 associated with the second column with a finger on his other hand 208 ; and pulls the second column header 312 in a direction along the X-axis 306 away from the first column 302 as indicated by gesture 610 .
- a hidden column 314 beneath the second column 302 may be revealed.
- the width of the hidden column 314 and the number of hidden columns 314 (if there are multiple hidden columns 314 ) revealed may be determined by how far the user pulls the second column 302 away from the first column 302 .
- one or more hidden rows may be revealed in a spreadsheet 106 via selecting a row 304 , for example, by a user selecting a first row header 412 via touching the first row header 412 with a finger on one hand, and holding his finger on the first row header 412 ; then selecting a second row 306 , for example, by touching the second row header 412 with a finger on the user's other hand; then pulling the second column header 412 in a direction along the Y-axis 308 away from the first row 304 .
- the user releases the row headers 412 a hidden row beneath the second row 304 may be revealed.
- the height of the hidden row and the number of hidden rows (if there are multiple hidden rows) revealed may be determined by how far the user pulls the second row 304 away from the first row 304 .
- a live preview of the hidden column 314 /row revelation operation may be displayed such that the user may be able to determine how far he needs to drag the column header 312 or row header 412 to reveal the hidden column(s) 314 or rows.
- FIGS. 7A and 7B show a push-together gesture to hide one or more columns within a spreadsheet 106 .
- FIG. 7A shows a spreadsheet 106 before pushing together columns 302 to hide one or more columns 302
- FIG. 7B shows the same spreadsheet 106 while performing a column hiding operation via a push-together gesture 710 .
- a push-together gesture 710 may be detected when a user touches a touchscreen 116 with two of his fingers and pushes them together as indicated by gesture 710 .
- the user may select a first column 302 in a spreadsheet 106 , for example, by touching a column header 312 associated with the first column 302 ; select a second column 302 , for example, by touching the column header 312 associated with the second column 302 ; and may utilize the push-together gesture 710 along the X-axis 306 to hide one or more columns 302 between the first and second columns 302 . Depending on how far apart the user moves his fingers, one or more columns 302 may be hidden.
- one or more rows 304 may be hidden in a spreadsheet 106 via selecting two non-adjacent rows 304 , and utilizing the push-together gesture 710 along the Y-axis 308 . Depending on how far apart the user moves his fingers, one or more rows 304 may be hidden. A live preview of the column/row hiding operation may be displayed such that the user may be able to determine a number of columns 302 or rows 304 that will be hidden upon releasing the touch.
- the push-together gesture 710 may be utilized in the same manner as described with respect to FIGS. 7A and 7B to perform a column 302 or row 304 deletion operation. Rather than hiding one or more columns 302 or rows 304 when two columns 302 or two rows 304 are pushed together, the one or more columns 302 or rows 304 may be deleted.
- FIGS. 8A and 8B show a drag-and-drop between gesture to move a column within a spreadsheet 106 .
- FIGS. 8A and 8B show a spreadsheet 106 while performing the column moving operation via a drag-and-drop between gesture 810 .
- a drag-and-drop between gesture 810 may be detected when a user selects a column 302 , for example, by touching a touchscreen 116 on an associated column header 312 with a finger, and drags the column header 312 away from its current position in the spreadsheet 106 , and into a new position between two columns 302 as indicated by gesture 810 .
- the associated column 302 may be moved to the position between the two columns associated with the two other column headers 312 .
- one or more rows 304 may be moved within a spreadsheet 106 via selecting a row 304 , for example, by a user selecting a row header 412 of a row he wishes to move, and utilizing the drag-and-drop between gesture 810 to drag the row header 412 away from its current location and drag the row header 412 between two other row headers 412 .
- the associated row 304 may be moved between the two rows 304 associated with the two other row headers 412 .
- a live preview of the column/row move operation may be displayed such that the user may be able to see where the selected column 302 or row 304 may be placed upon releasing the touch.
- FIGS. 9A , 9 B, and 9 C show a multi-finger or multi-touch swipe gesture to navigate between individual pages or sheets (herein referred to as sheets 902 , 904 ) in a spreadsheet 106 document.
- FIG. 9A shows a first sheet 902 of a spreadsheet 106 displayed before navigating to a next sheet 904 of the spreadsheet 106 via a multi-finger swipe gesture 910
- FIG. 9B shows the first sheet 902 of the spreadsheet 106 being panned out of display while a second sheet 904 is being panned into the display via a multi-finger swipe gesture 910
- FIG. 9C shows the second sheet 904 of the spreadsheet 106 after it has been panned into view.
- a multi-finger swipe gesture 910 may be detected when a user touches a touchscreen 116 with multiple fingers, and swipes a spreadsheet 106 in a lateral direction along the X-axis 306 as indicated by gesture 910 .
- the speed of the movement of the multi-finger swipe gesture 910 may be used in determining a speed of the panning of the spreadsheet 106 . For example, a sudden movement in a direction may increase the speed of the panning action.
- a multi-finger swipe gesture 910 may perform a pan to a single next sheet 904 , or alternatively, according to another embodiment, a user may pan through multiple sheets 902 , 904 with a single multi-finger swipe gesture 910 , depending on the speed of the gesture.
- FIGS. 10A and 10B show a touch-and-throw gesture for extending a selected functionality from one portion of a spreadsheet document to another portion of the document.
- the touch-and-throw gesture may be used to extend a selection of one or more cells to a last non-blank cell in the same column 302 or row 304 as the selected cell(s), or if the next cell is blank, to extend the selection to a next nonblank cell within a spreadsheet 106 .
- FIG. 10A shows a spreadsheet 106 where a cell is selected, a selection handle 1002 is displayed around the selected cell, and a user touches the selection handle 1002 and “throws” it over other cells in the column 302 via a touch-and-throw gesture 1010 .
- FIG. 10B shows the same spreadsheet 106 after performing a selection extension operation via a touch-and-throw gesture 1010 .
- a touch-and-throw gesture 1010 may be detected when a user selects a selection handle 1002 , for example, by touching a touchscreen 116 on a selection handle 1002 displayed around one or more cells, and “throws” the selection handle 1002 in a direction along the Y-axis 308 as indicated by gesture 1010 . Accordingly, the selection may be extended to a last nonblank cell in the same column 302 as the selected cell(s), or if the next cell is blank, the selection may be extended to a next nonblank cell.
- a selection of one or more cells in a row 304 may be extended via selecting a selection handle 1002 displayed around the selected cell(s), and utilizing the touch-and-throw gesture 1010 to “throw” the selection handle 1002 in a direction along the X-axis 306 . Accordingly, the selection may be extended to a last nonblank cell in the same row 304 as the selected cell(s), or if the next cell is blank, the selection may be extended to a next nonblank cell.
- a selection may be extended to a contiguous block of data contained in cells in multiple columns 302 and multiple rows 304 via selecting a selection handle 1002 displayed around a selected cell or cells, and utilizing the touch-and-throw gesture 1010 to “throw” the selection handle 1002 in a diagonal direction.
- FIG. 10C shows a spreadsheet 106 where a cell is selected, a selection handle 1002 is displayed around the selected cell, and a user touches the selection handle 1002 and “throws” it at an angle over other cells in a block of data, wherein the block of data comprises multiple columns 302 and rows 304 via a touch-and-throw gesture 1010 .
- FIG. 10C shows a spreadsheet 106 where a cell is selected, a selection handle 1002 is displayed around the selected cell, and a user touches the selection handle 1002 and “throws” it at an angle over other cells in a block of data, wherein the block of data comprises multiple columns 302 and rows 304 via a touch-and-throw gesture 1010
- a touch-and-throw gesture 1010 may be detected when a user selects a selection handle 1002 , for example, by touching a touchscreen 116 on a selection handle 1002 displayed around one or more cells, and “throws” the selection handle 1002 in an angular direction (along both the X-axis 306 and the Y-axis 308 ) as indicated by gesture 1010 . Accordingly, the selection may be extended to select a contiguous block of rectangular data as illustrated in FIG. 10D .
- FIGS. 11A and 11B show a touch-and-pull gesture to sort data.
- FIG. 11A shows a spreadsheet 106 as a user selects a column header 312 and pulls it downward
- FIG. 11B shows the same spreadsheet 106 after performing a sort operation via a touch-and-pull gesture 1110 .
- a UI element 1102 may be displayed, providing an indication of a sort operation.
- a touch-and-pull gesture 1110 may be detected when a column header 312 is selected, for example, by touching a touchscreen 116 where a column header 312 is displayed, and pulled downward along the Y-axis 308 as indicated by gesture 1110 . Accordingly, the data in the column 302 may be sorted as illustrated in FIG. 11B .
- sorting of data in a row 304 may be performed via utilizing the touch-and-pull gesture 1110 .
- a row header 412 may be selected and pulled in a direction along the X-axis 306 . Accordingly, the data in the row associated with the selected row header 412 may be sorted.
- a visual indicator may be displayed, wherein the visual indicator may inform the user as to how far he may need to drag the header 312 , 412 , and then release it, before the sort is invoked. If the distance does not reach a predetermined threshold before release, the sort may not be invoked.
- FIGS. 12A and 12B show a pinch-and-turn gesture to rotate an element.
- FIG. 12A shows a spreadsheet 106 as a user selects an element 1202 and rotates it
- FIG. 12B shows the same spreadsheet 106 after performing a rotate operation via a pinch-and-turn gesture 1210 .
- a pinch-and-turn gesture 1210 may be detected when an element 1202 (e.g., text, picture, etc.) is selected, for example, by using two or more fingers to touch a touchscreen 116 where an element 1202 is displayed, and turning it in a circular motion as indicated by gesture 1210 . Accordingly, the element 1202 may be rotated as illustrated in FIG. 12B .
- an element 1202 e.g., text, picture, etc.
- FIG. 12B shows a pinch-and-turn gesture to rotate an element 1202 in various types of applications 104 .
- Embodiments may comprise discontiguous selection via a select-pin-select gesture. For example, where data may not be displayed in adjacent cells, a user may make a first selection of data with his finger, pin the selection, and then select a next selection of data. Pinning may be accomplished via various ways, which may include selection of a UI element that may be displayed next to the first selection, for example, by touching the UI element with another finger, by pressing and holding the first selection, etc. After the first selection is pinned, the user may be able to select a next set of data.
- FIGS. 13A and 13B show a multi-finger tear-and-pull gesture to “tear” objects out of a grid and to float on top of the grid or move to a canvas.
- FIG. 13A shows a spreadsheet 106 as a user selects data from the spreadsheet 106 via a multi-touch selection and tears/pulls the data from the spreadsheet 106
- FIG. 13B shows the same spreadsheet 106 after performing the “tear” operation via a tear-and-pull gesture 1310 .
- a tear-and-pull gesture 1310 may be detected when data or an object is selected, for example, by using two or more fingers to touch a touchscreen 116 where the data or object is displayed, and pulling the data or object from its current position as indicated by gesture 1310 . Accordingly, the data or object may be “torn out” of the grid and may float on top of the grid or moved to a canvas as illustrated in FIG. 13B . Although displayed in a spreadsheet, it is to be understood that utilizing a multi-finger tear-and-pull gesture 1310 to remove data as a separate element may be performed in various types of applications 104 .
- the method 1400 starts at OPERATION 1405 , and proceeds to OPERATION 1410 , where a gesture is received.
- the gesture may be any of the gestures described herein or other recognized gestures, such as a pulling apart gesture 310 , a touch-and-slide gesture 410 , a drag-and-drop gesture 510 , a tap-hold-pull gesture 610 , a push-together gesture 710 , a drag-and-drop between gesture 810 , a multi-finger swipe gesture 910 , a touch-and-throw gesture 1010 , a touch-and-pull gesture 1110 , a pinch-and-turn gesture 1210 , a multi-finger tear-and-pull gesture 1310 , a select-pin-select gesture, and the like may be received.
- a pulling apart gesture 310 a touch-and-slide gesture 410 , a drag-and-drop gesture 510 , a tap-hold-pull gesture 610 , a push-together gesture 710 , a drag-and-drop between gesture 810 , a multi-finger swipe
- the user places at least a portion of their hand 208 (e.g. one or more fingers) on the touch surface 116 . Additionally, according to some embodiments, the user may place his hand 208 near the surface of the touch surface 116 but not on the touch surface 116 .
- their hand 208 e.g. one or more fingers
- the method 1400 may proceed to OPERATION 1415 , where an action to be performed may be determined.
- the action relates to interacting with a spreadsheet 106 and comprises actions such as inserting, deleting, moving, hiding, revealing, panning, sorting, selecting, rotating, removing, and the like. While the actions described relate to interaction with spreadsheets 106 , other applications 104 may be utilized with the gestures described.
- the determined action may be performed.
- the action relates to manipulating data in a spreadsheet 106 .
- a column 302 or row 304 may be deleted, moved, inserted, sorted, or selected, or some other operation may be performed on data, and the like.
- the method 1400 may proceed to OPERATION 1425 , where the display may be updated.
- the display may be updated in response to the action performed per the gesture information that is received.
- the method 1400 may end at OPERATION 1430 .
- the embodiments and functionalities described herein may operate via a multitude of computing systems including, without limitation, desktop computer systems, wired and wireless computing systems, mobile computing systems (e.g., mobile telephones, netbooks, tablet or slate type computers, notebook computers, and laptop computers), hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, and mainframe computers.
- desktop computer systems e.g., desktop computer systems, wired and wireless computing systems, mobile computing systems (e.g., mobile telephones, netbooks, tablet or slate type computers, notebook computers, and laptop computers), hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, and mainframe computers.
- mobile computing systems e.g., mobile telephones, netbooks, tablet or slate type computers, notebook computers, and laptop computers
- hand-held devices e.g., multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, and mainframe computers.
- multiprocessor systems e.g
- embodiments and functionalities described herein may operate over distributed systems (e.g., cloud-based computing systems), where application functionality, memory, data storage and retrieval and various processing functions may be operated remotely from each other over a distributed computing network, such as the Internet or an intranet.
- a distributed computing network such as the Internet or an intranet.
- User interfaces and information of various types may be displayed via on-board computing device displays or via remote display units associated with one or more computing devices. For example user interfaces and information of various types may be displayed and interacted with on a wall surface onto which user interfaces and information of various types are projected.
- Interaction with the multitude of computing systems with which embodiments of the invention may be practiced include, keystroke entry, touch screen entry, voice or other audio entry, gesture entry where an associated computing device is equipped with detection (e.g., camera) functionality for capturing and interpreting user gestures for controlling the functionality of the computing device, and the like.
- detection e.g., camera
- FIGS. 15-17 and the associated descriptions provide a discussion of a variety of operating environments in which embodiments of the invention may be practiced.
- the devices and systems illustrated and discussed with respect to FIGS. 15-17 are for purposes of example and illustration and are not limiting of a vast number of computing device configurations that may be utilized for practicing embodiments of the invention, described herein.
- FIG. 15 is a block diagram illustrating physical components (i.e., hardware) of a computing device 1500 with which embodiments of the invention may be practiced.
- the computing device components described below may be suitable for the computing devices described above including the personal computer 102 a , the tablet computer 102 b , and the smart phone 102 c .
- the computing device 1500 may include at least one processing unit 1502 and a system memory 1504 .
- the system memory 1504 may comprise, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories.
- the system memory 1504 may include an operating system 1505 and one or more program modules 1506 suitable for running software applications 104 such as the spreadsheet application.
- the operating system 1505 may be suitable for controlling the operation of the computing device 1500 .
- embodiments of the invention may be practiced in conjunction with a graphics library, other operating systems, or any other application program and is not limited to any particular application or system.
- This basic configuration is illustrated in FIG. 15 by those components within a dashed line 1508 .
- the computing device 1500 may have additional features or functionality.
- the computing device 1500 may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated in FIG. 15 by a removable storage device 1509 and a non-removable storage device 1510 .
- program modules and data files may be stored in the system memory 1504 . While executing on the processing unit 1502 , the software applications 104 may perform processes including, but not limited to, one or more of the stages of the method 1400 illustrated in FIG. 14 .
- Other program modules that may be used in accordance with embodiments of the present invention may include electronic mail and contacts applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided application programs, etc.
- embodiments of the invention may be practiced in an electrical circuit comprising discrete electronic elements, packaged or integrated electronic chips containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic elements or microprocessors.
- embodiments of the invention may be practiced via a system-on-a-chip (SOC) where each or many of the components illustrated in FIG. 15 may be integrated onto a single integrated circuit.
- SOC system-on-a-chip
- Such an SOC device may include one or more processing units, graphics units, communications units, system virtualization units and various application functionality all of which are integrated (or “burned”) onto the chip substrate as a single integrated circuit.
- the functionality, described herein, with respect to the software applications 104 may be operated via application-specific logic integrated with other components of the computing device 1500 on the single integrated circuit (chip).
- Embodiments of the invention may also be practiced using other technologies capable of performing logical operations such as, for example, AND, OR, and NOT, including but not limited to mechanical, optical, fluidic, and quantum technologies.
- embodiments of the invention may be practiced within a general purpose computer or in any other circuits or systems.
- the computing device 1500 may also have one or more input device(s) 1512 such as a keyboard, a mouse, a pen, a sound input device, a touch input device, etc.
- the output device(s) 1514 such as a display, speakers, a printer, etc. may also be included.
- the aforementioned devices are examples and others may be used.
- the computing device 1500 may include one or more communication connections 1516 allowing communications with other computing devices 1518 . Examples of suitable communication connections 1516 include, but are not limited to, RF transmitter, receiver, and/or transceiver circuitry; universal serial bus (USB), parallel, and/or serial ports.
- USB universal serial bus
- Computer readable media may include computer storage media.
- Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, or program modules.
- the system memory 1504 , the removable storage device 1509 , and the non-removable storage device 1510 are all computer storage media examples (i.e., memory storage.)
- Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other article of manufacture which can be used to store information and which can be accessed by the computing device 1500 . Any such computer storage media may be part of the computing device 1500 .
- FIGS. 16A and 16B illustrate a mobile computing device 1600 , for example, a mobile telephone, a smart phone, a tablet personal computer, a laptop computer, and the like, with which embodiments of the invention may be practiced.
- a mobile computing device 1600 for implementing the embodiments is illustrated.
- the mobile computing device 1600 is a handheld computer having both input elements and output elements.
- the mobile computing device 1600 typically includes a display 1605 and one or more input buttons 1610 that allow the user to enter information into the mobile computing device 1600 .
- the display 1605 of the mobile computing device 1600 may also function as an input device (e.g., a touch screen display 116 ). If included, an optional side input element 1615 allows further user input.
- the side input element 1615 may be a rotary switch, a button, or any other type of manual input element.
- mobile computing device 1600 may incorporate more or less input elements.
- the display 1605 may not be a touch screen in some embodiments.
- the mobile computing device 1600 is a portable phone system, such as a cellular phone.
- the mobile computing device 1600 may also include an optional keypad 1635 .
- Optional keypad 1635 may be a physical keypad or a “soft” keypad generated on the touch screen display.
- the output elements include the display 1605 for showing a graphical user interface (GUI), a visual indicator 1620 (e.g., a light emitting diode), and/or an audio transducer 1625 (e.g., a speaker).
- GUI graphical user interface
- the mobile computing device 1600 incorporates a vibration transducer for providing the user with tactile feedback.
- the mobile computing device 1600 incorporates input and/or output ports, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., a HDMI port) for sending signals to or receiving signals from an external device.
- FIG. 16B is a block diagram illustrating the architecture of one embodiment of a mobile computing device. That is, the mobile computing device 1600 can incorporate a system (i.e., an architecture) 1602 to implement some embodiments.
- the system 1602 is implemented as a “smart phone” capable of running one or more applications (e.g., browser, e-mail, calendaring, contact managers, messaging clients, games, and media clients/players).
- the system 1602 is integrated as a computing device, such as an integrated personal digital assistant (PDA) and wireless phone.
- PDA personal digital assistant
- One or more application programs 104 may be loaded into the memory 1662 and run on or in association with the operating system 1664 .
- Examples of the application programs include phone dialer programs, e-mail programs, personal information management (PIM) programs, word processing programs, spreadsheet programs, Internet browser programs, messaging programs, and so forth.
- the system 1602 also includes a non-volatile storage area 1668 within the memory 1662 .
- the non-volatile storage area 1668 may be used to store persistent information that should not be lost if the system 1602 is powered down.
- the application programs 104 may use and store information in the non-volatile storage area 1668 , such as e-mail or other messages used by an e-mail application, and the like.
- a synchronization application (not shown) also resides on the system 1602 and is programmed to interact with a corresponding synchronization application resident on a host computer to keep the information stored in the non-volatile storage area 1668 synchronized with corresponding information stored at the host computer.
- other applications may be loaded into the memory 1662 and run on the mobile computing device 1600 , including software applications 1520 described herein.
- the system 1602 has a power supply 1670 , which may be implemented as one or more batteries.
- the power supply 1670 might further include an external power source, such as an AC adapter or a powered docking cradle that supplements or recharges the batteries.
- the system 1602 may also include a radio 1672 that performs the function of transmitting and receiving radio frequency communications.
- the radio 1672 facilitates wireless connectivity between the system 1602 and the “outside world,” via a communications carrier or service provider. Transmissions to and from the radio 1672 are conducted under control of the operating system 1664 . In other words, communications received by the radio 1672 may be disseminated to the application programs 104 via the operating system 1664 , and vice versa.
- the visual indicator 1620 may be used to provide visual notifications, and/or an audio interface 1674 may be used for producing audible notifications via the audio transducer 1625 .
- the visual indicator 1620 is a light emitting diode (LED) and the audio transducer 1625 is a speaker.
- LED light emitting diode
- the LED may be programmed to remain on indefinitely until the user takes action to indicate the powered-on status of the device.
- the audio interface 1674 is used to provide audible signals to and receive audible signals from the user.
- the audio interface 1674 may also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation.
- the microphone may also serve as an audio sensor to facilitate control of notifications, as will be described below.
- the system 1602 may further include a video interface 1676 that enables an operation of an on-board camera 1630 to record still images, video stream, and the like.
- a mobile computing device 1600 implementing the system 1602 may have additional features or functionality.
- the mobile computing device 1600 may also include additional data storage devices (removable and/or non-removable) such as, magnetic disks, optical disks, or tape.
- additional storage is illustrated in FIG. 16B by the non-volatile storage area 1668 .
- Data/information generated or captured by the mobile computing device 1600 and stored via the system 1602 may be stored locally on the mobile computing device 1600 , as described above, or the data may be stored on any number of storage media that may be accessed by the device via the radio 1672 or via a wired connection between the mobile computing device 1600 and a separate computing device associated with the mobile computing device 1600 , for example, a server computer in a distributed computing network, such as the Internet.
- a server computer in a distributed computing network such as the Internet.
- data/information may be accessed via the mobile computing device 1600 via the radio 1672 or via a distributed computing network 112 .
- data/information may be readily transferred between computing devices for storage and use according to well-known data/information transfer and storage means, including electronic mail and collaborative data/information sharing systems.
- FIG. 17 illustrates one embodiment of the architecture of a system for providing quick function gesture functionality to one or more client devices, as described above.
- Content developed, interacted with, or edited in association with an application 104 such as a spreadsheet application, may be stored in different communication channels or other storage types.
- various documents may be stored using a directory service 1722 , a web portal 1724 , a mailbox service 1726 , an instant messaging store 1728 , or a social networking site 1730 .
- the application 104 may use any of these types of systems or the like for enabling data utilization, as described herein.
- a server 110 may provide the spreadsheet application 104 to clients.
- the server 110 may be a web server providing the spreadsheet application 104 over the web.
- the server 110 may provide the spreadsheet application 104 over the web to clients through a network 112 .
- the client computing device may be implemented as the computing device 1500 and embodied in a personal computer 102 A, a tablet computer 102 B, and/or a mobile computing device 102 C (e.g., a smart phone). Any of these embodiments of the client computing device 102 A-C may obtain content from the store 1716 .
- Embodiments of the present invention are described above with reference to block diagrams and/or operational illustrations of methods, systems, and computer program products according to embodiments of the invention.
- the functions/acts noted in the blocks may occur out of the order as shown in any flowchart.
- two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved.
Abstract
Description
- With the advent of spreadsheet applications, computer and computer software users have become accustomed to processing and manipulating enormous amounts of data and using spreadsheet functions to perform many simple to very complex calculations and organizational functions with their data. Typical spreadsheet solutions provide a grid made up of rows and columns for receiving and manipulating data. Generally, users interact with spreadsheets through input devices, such as mice, touchscreens, motion-detection devices, graphical user interfaces, and keyboards. Sometimes this interaction can be frustrating. For example, when utilizing a spreadsheet application on a touch device, the user interface (UI) may be designed as a mouse-focused UI. Selection of and interaction with data and functionality controls via touch may be the same as or similar to selection and interaction via a mouse, wherein differences, in some cases, may include providing larger functionality controls for touch input, or that a user's finger is used instead of a mouse cursor.
- Oftentimes, it is believed that touch or motion-sensing devices may not be suited for productivity work, and may be more suited for consumption of data. Current UI interactions with a spreadsheet application on touch or motion-sensing devices may not take advantage of using natural gestures to interact with the application. Thus, an improvement and technical advantage will be gained by providing a variety of natural gestures for interacting with an application user interface, such as a spreadsheet application user interface and associated documents.
- It is with respect to these and other considerations that the present invention has been made. Although relatively specific problems have been discussed, it should be understood that the embodiments disclosed herein should not be limited to solving the specific problems identified in the background.
- This summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description section. This summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.
- Embodiments of the present invention solve the above and other problems by providing natural gestures for interacting with spreadsheet documents. According to embodiments, a user may interact with a spreadsheet document using a variety of touch-based gestures. Gestures applied to a spreadsheet document may be used for making structural and display changes to the document. For example, gestures may be used on the spreadsheet document for inserting new columns and rows; for revealing hidden columns and rows; for deleting columns and rows; for extending functionalities across columns and rows; for moving objects displayed in a spreadsheet document; and for navigating various parts of a spreadsheet document.
- The details of one or more embodiments are set forth in the accompanying drawings and description below. Other features and advantages will be apparent from a reading of the following detailed description and a review of the associated drawings. It is to be understood that the following detailed description is explanatory only and is not restrictive of the invention as claimed.
- Further features, aspects, and advantages of the present disclosure will become better understood by reference to the following detailed description, appended claims, and accompanying figures, wherein elements are not to scale so as to more clearly show the details, wherein like reference numbers indicate like elements throughout the several views, and wherein:
-
FIG. 1 illustrates a system architecture in which embodiments of the present invention may be practiced; -
FIG. 2 illustrates a touch input system in which embodiments of the present invention may be practiced; -
FIGS. 3A-3D illustrate a pulling-apart gesture; -
FIGS. 4A-4B illustrate a touch-and-slide gesture; -
FIGS. 5A-5B illustrate a drag-and-drop gesture; -
FIGS. 6A-6B illustrate a tap-hold-pull gesture; -
FIGS. 7A-7B illustrate a push-together gesture; -
FIGS. 8A-8B illustrate a drag-and-drop between gesture; -
FIGS. 9A-9C illustrate a multi-finger swipe gesture; -
FIGS. 10A-10D illustrate a touch-and-throw gesture; -
FIGS. 11A-11B illustrate a touch-and-pull gesture; -
FIGS. 12A-12B illustrate a pinch-and-turn gesture; -
FIGS. 13A-13B illustrate a multi-finger tear-and-pull gesture; -
FIG. 14 shows one embodiment of a method for providing natural quick function gestures; -
FIG. 15 is a block diagram illustrating one embodiment of the physical components of a computing device with which embodiments of the invention may be practiced; -
FIGS. 16A and 16B are simplified block diagrams of a mobile computing device with which embodiments of the present invention may be practiced; and -
FIG. 17 is a simplified block diagram of a distributed computing system in which embodiments of the present invention may be practiced. - Various embodiments are described more fully below with reference to the accompanying drawings, which form a part hereof, and which show specific exemplary embodiments. However, embodiments may be implemented in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the embodiments to those skilled in the art. Embodiments may be practiced as methods, systems, or devices. Accordingly, embodiments may take the form of a hardware implementation, an entirely software implementation or an implementation combining software and hardware aspects. The following detailed description is, therefore, not to be taken in a limiting sense.
- Embodiments of natural quick function gestures are described herein and are illustrated in the accompanying figures. Natural quick function gestures may be provided in the context of an application on any device, service, or general endpoint capable of running the application.
FIG. 1 illustrates a system architecture in which embodiments of the present invention may be practiced. Thesystem 100 includes one or more computing devices such as a personal computer (e.g., a desktop or laptop computer) 102A, atablet computer 102B, asmart phone 102C, and a large-scale display 102D. Each of the computing devices 102 may run anapplication 104 for displaying and editing data. As illustrated inFIG. 1 , anapplication 104 may include a spreadsheet application for displaying and editing aspreadsheet 106. As should be appreciated, the computing devices 102 are for purposes of illustration only and are not exhaustive of the various types of computing devices that may be used in accordance with embodiments of the present invention. For example, while the large-scale display 102D is illustrated as a display unit with an associatedcomputing device 102A, the large-form display device 102D may operate as an integrated display/computing device that may be used for any suitable use from large-scale display uses, for example, in a conference room, to personal computing, as desired. - According to embodiments, each of the computing devices 102 illustrated in
FIG. 1 may receive input through a variety of suitable means including touch or gesture input, keyboard input, mouse input, voice command input, electronic inking input, and the like. According to one embodiment, each computing device 102 may comprise an operating system configured to include support for touch and gesture input. According to another embodiment, agesture manager 108 illustrated in association with each of the computing devices and which will be described in greater detail below, may be utilized to process some/all of the touch and gesture input that is received from touch and gesture input. - Referring still to
FIG. 1 , theapplications 104 illustrated in association with each of the computing devices 102 are illustrative ofapplications 104 having sufficient computer executable instructions for enabling embodiments of the present invention as described herein. Anexample application 104 includes a spreadsheet application, such as Excel® manufactured by Microsoft Corporation of Redmond, Wash. As should be appreciated, this is but one example spreadsheet application, and anyother applications 104 suitable for enabling embodiments described herein may be used. - According to embodiments, application functionality associated with each of the computing devices 102 may operate according to one or more variations. For example, each
application 104 may be a fully functional “thick” client application having all application functionality including the ability to send and receive data to and from other applications 102 operating on other computing devices in the collaborative work session. According to another embodiment, the computing devices 102 may communicate via a distributedcomputing network 112, for example, the Internet. An Internet-based or “cloud-based”server computer 110 may be operated on one or moreweb services applications 114 for providing a collaborative work session. According to this embodiment, each of the devices 102 may operate as thin client systems, and collaborative communication between the devices may be operated at the direction of theweb services application 114 operated at a remote site. According to this embodiment, anapplication 104 may operate at theremote server 110 wherein each of the devices 102 may access and receive the functionality described herein via a remote service from the server based application, for example, via theweb services application 114. - As illustrated in
FIG. 1 , thesystem 100 may comprise agesture manager 108 configured to detect gestures that are received by a computing device 102. Generally, agesture manager 108 may be configured to sense when a gesture is received that is related to performing an operation in conjunction with anapplication program 104, such as a spreadsheet application. A gesture may include user interaction with data displayed on or functionalities associated with a computing device 102, wherein the user interaction may originate from a bodily movement (e.g., movement of one or more fingers, hands, arms, etc.). A gesture may include bodily contact with a touch input device (e.g., a touchscreen 116), or may include bodily movement detected by a sensor, such as a camera, an accelerometer, a motion detector, etc. A touch input device may be operable to recognize a presence of two or more points of contact on a sensing surface, sometimes referred to as multi-touch input. A gesture may include user interaction with atouchscreen 116 using bodily movements to manipulate data and/or to select functionalities, for example, data displayed in and functionalities associated with anapplication 104, such as a spreadsheet application. As used herein, the term “gesture” includes touch input. As can be appreciated, gesturing may enable users to communicate with a computing device 102 and interact naturally. Different types of gestures may be received. For example, a tapping gesture, a pulling gesture, a dragging gesture, a pushing gesture, a throwing gesture, panning gesture, a tearing gesture, a pinch-and-turn gesture, and the like may be received. The gestures may be used to change a display of information, activate/deactivate functions, and/or perform some other type of operation associated with anapplication 104 or some other function and/or program. Additional details regarding the gestures will be provided below. -
FIG. 2 illustrates an exemplarytouch input system 200. Thetouch input system 200 as illustrated comprises atouch panel 216 that comprisesseveral sensors 204 integrated therein. According to one embodiment, thesensors 204 are Infrared (IR) sensors. Thetouch input system 200 is configured to detect objects that either in contact with atouchable surface 116 or are close to but not in actual contact with (“adjacent”)touchable surface 116. The objects that are sensed may be many different types of objects such as finger, hands, or other physical objects.Infrared sensors 204 are distributed throughouttouch panel 216 and are arranged parallel totouchable surface 116. One or more of theinfrared sensors 204 may detect infrared radiation reflected from objects, such ashand 208, as indicated by the arrow. Although the term “above” is used in this description, it should be understood that the orientation of the touch panel system is irrelevant. As shown inFIG. 2 , thetouchable surface 116 is horizontal, but in a different embodiment generated by rotatingsystem 200 clockwise by 90 degrees, thetouchable surface 116 could be vertical. In that embodiment, the objects from which reflected IR radiation is detected are to the side oftouchable surface 116. The term “above” is intended to be applicable to all such orientations. Thetouchable surface 116 may also be changed to other orientations. - The
touch panel 216 may comprisefilters 212 that absorb visible light and transmit infrared radiation and are located betweentouchable surface 116 andIR sensors 204 in order to shieldIR sensors 204 from visible light incident ontouchable surface 116 in the case whereIR sensors 204 are sensitive to a broader range of wavelengths of light other than purely infrared wavelengths. - The
touch panel 216 may comprise a display that is configured to display images that are viewable via thetouchable surface 116. For example, the displayed image may be images relating to anapplication 104, such as aspreadsheet 106. The display may be, for example, an LCD, an organic light emitting diode (OLED) display, a flexible display such as electronic paper, or any other suitable display in which an IR sensor can be integrated. - The
system 200 may comprise abacklight 216 for the display. Thebacklight 216 may comprise at least oneIR source 218 that is configured to illuminate objects in contact with or adjacent to thetouchable surface 116 with infrared radiation through thetouchable surface 116, as indicated by the arrows.IR sensors 204 are sensitive to radiation incident from above, so IR radiation traveling directly from thebacklight 216 to theIR sensors 204 is not detected. - The output of
sensors 204 may be processed by thegesture manager 108 and/or functionality included within an operating system or some other application to detect when a physical object (e.g., a hand, a finger, a stylus, etc.) has come into physical contact with a portion of thetouch input surface 116 and/or a physical object is in close proximity to the surface. For example,sensors 204 can detect when a portion of a user'shand 208, such as one or more fingers, has come in contact with or is near to the touchinput display surface 116. Additional sensors can be embedded in the touchinput display surface 116 and can include for example, accelerometers, pressure sensors, temperature sensors, image scanners, barcode scanners, etc., to detect multiple simultaneous inputs. - When the
sensors 204 are IR sensors, the IR radiation reflected from the objects may be reflected from a user's hands, fingers, reflective ink patterns on the objects, metal designs on the objects or any other suitable reflector. Fingers reflect enough of the near IR to detect that a finger orhand 208 is located at a particular location on or adjacent thetouchable surface 116. A higher resolution of IR sensors may be used to scan objects in order to achieve higher resolution. -
Sensors 204 can be included (e.g., embedded) in a plurality of locations. The density ofsensors 204 can be sufficient such that contact across the entirety of thetouch input surface 116 can be detected.Sensors 204 may be configured to sample the surface of touch input display surface 206 at specified intervals, such as, for example, 1 ms, 5 ms, etc. for detected contact and/or near contact. The sensor data received fromsensors 204 changes between sampling intervals as detected objects move on the touch surface; detected objects are no longer within range of detection; and when new objects come in range of detection. For example, thegesture manager 108 can determine that contact was first detected at a first location and then contact was subsequently moved to other locations. In response, thegesture manager 108 may determine when a gesture is received and what type of gesture is received. -
FIG. 2 provides just one example of a touch input system. In other exemplary touch systems, the backlight may not comprise any IR sources and thesurface 116 may include a frontlight which comprises at least one IR source. In such an example, thetouchable surface 116 of the system is a surface of the frontlight. The frontlight may comprise a light guide, so that IR radiation emitted from IR source travels through the light guide and is directed towards touchable surface and any objects in contact with or adjacent to it. In other touch panel systems, both the backlight and frontlight may comprise IR sources. In yet other touch panel systems, there is no backlight and the frontlight comprises both IR sources and visible light sources. In further examples, the system may not comprise a frontlight or a backlight, but instead the IR sources may be integrated within the touch panel. In an implementation, thetouch input system 200 may comprise an OLED display which comprises IR OLED emitters and IR-sensitive organic photosensors (which may comprise reverse-biased OLEDs). In some touch systems, a display may not be included. Even if thetouch system 200 comprises one or more components or elements of a display, the touch system may be configured to not display images. For example, this may be the case when the touch input device is separate from a display. Other examples include a touchpad, a gesture pad, and similar non-display devices and components. - For some applications, it may be desirable to detect an object only if it is in actual contact with the
touchable surface 116 of thetouch panel system 200. For example, according to one embodiment, a gesture may not be recognized when the gesture is not performed touching thesurface 116. Similarly, a gesture may be recognized when performed above thesurface 116. The IR source of the touch input system may be turned on only if thetouchable surface 116 is touched. Alternatively, the IR source may be turned on regardless of whether thetouchable surface 116 is touched, and detection of whether actual contact between thetouchable surface 116 and the object occurred is processed along with the output of the IR sensor. Actual contact between thetouchable surface 116 and the object may be detected by any suitable means, including, for example, by a vibration sensor or microphone coupled to thetouch panel 216. A non-exhaustive list of examples for sensors to detect contact includes pressure-based mechanisms, micro-machined accelerometers, piezoelectric devices, capacitive sensors, resistive sensors, inductive sensors, laser vibrometers, and LED vibrometers. - The
gesture manager 108 may be configured to recognize many different types of gestures. Some of the gestures may be context dependent, be specific to anapplication 104, and/or be used within many different types ofapplications 104. For example, gestures may be used to interact with aspreadsheet 106 that is associated with a spreadsheet application. Gestures may be received in many different locations relating to touch input device/display 116. For example, a gesture may be received within a display of spreadsheet, within a gesture box, and/or at some other location on a display. Exemplary gestures and interactions with a spreadsheet are described below. - As illustrated and described with reference to
FIGS. 3-14 below, gestures may be used in association with spreadsheet documents for manipulating information contained in spreadsheet documents and navigating spreadsheet documents, but also, gestures may be used for making structural and display changes to spreadsheet documents. As described below, gestures may be used for adding, deleting and moving columns and rows; for revealing or hiding columns and rows; for applying spreadsheet functionality; and for moving spreadsheet objects. As should be understood, these are but a few examples of some of the uses of gestures applied to spreadsheet documents as will be more fully described below. -
FIGS. 3A and 3B show a pull-apart gesture to insert one or more columns within aspreadsheet 106. As illustrated,FIG. 3A shows aspreadsheet 106 before pulling apartcolumns 302 to insert one ormore columns 302, andFIG. 3B shows thesame spreadsheet 106 while performing a column insertion operation via a pull-apartgesture 310. According to an embodiment, a pull-apartgesture 310 may be detected when a user touches atouchscreen 116 with two of his fingers and pulls them apart as indicated bygesture 310. To perform a column insertion operation via the pull-apartgesture 310, the user may select afirst column 302 in aspreadsheet 106, for example, by touching acolumn header 312 associated with thefirst column 302; select anadjacent column 302, for example, by touching thecolumn header 312 associated with theadjacent column 302; and may utilize the pull-apartgesture 310 along anX-axis 306 to insert acolumn 302 between the first andsecond columns 302. Depending on how far apart the user moves his fingers, one ormore columns 302 may be inserted. - According to an embodiment, one or
more rows 304 may be inserted into aspreadsheet 106 via selectingadjacent rows 304, and utilizing the pull-apartgesture 310 along a Y-axis 308. Depending on how far apart the user moves his fingers, one ormore rows 304 may be inserted. A live preview of the column/row insertion operation may be displayed such that the user may be able to determine a number ofcolumns 302 orrows 304 that will be inserted upon releasing the touch. - According to another embodiment, and as illustrated in
FIGS. 3C and 3D , the pull-apartgesture 310 may be utilized to perform a column or row revelation operation, for example, when acolumn 302 orrow 304 may be hidden between twocolumns 302 orrows 304. To perform a hidden column revelation operation via the pull-apartgesture 310, the user may select afirst column 302 in aspreadsheet 106, for example, by touching acolumn header 312 associated with thefirst column 302; select anadjacent column 302, for example, by touching thecolumn header 312 associated with theadjacent column 302; and may utilize the pull-apartgesture 310 along anX-axis 306 to reveal one or morehidden column 314 between the first andsecond columns 302. - According to an embodiment, one or more hidden rows may be revealed into a
spreadsheet 106 via selectingadjacent rows 304, and utilizing the pull-apartgesture 310 along a Y-axis 308. Depending on how far apart the user moves his fingers and depending on howmany rows 304 may be hidden, one ormore rows 304 may be revealed. A live preview of the column/row revelation operation may be displayed such that the user may be able to determine how far apart to move thecolumns 302 orrows 304 to reveal the desired hidden column(s) 314 or rows. -
FIGS. 4A and 4B show a touch-and-slide gesture to insert one ormore columns 302 within aspreadsheet 106.FIG. 4A shows aspreadsheet 106 before one ormore columns 302 are inserted via a touch-and-slide gesture 410, andFIG. 4B shows thesame spreadsheet 106 while performing a column insertion operation via a touch-and-slide gesture 410. According to an embodiment, a tap-and-slide gesture 410 may be detected when a user touches atouchscreen 116 on a displayed selectable UI control, such as aninsert widget 402, with a finger, and slides theinsert widget 402 as indicated bygesture 410. The selectable UI control (e.g., insert widget 402) may be displayed between adjoiningcolumns 302. The selectable UI control (e.g., insert widget 402) may be displayed upon receiving an indication of a selection of acolumn 302. For example, a user may select acolumn 302 via touching acolumn header 312, and accordingly, aninsert widget 402 may be displayed between the selectedcolumn 302 and anadjacent column 302. The user may select the insert widget 402 (e.g., via touching the widget), and then slide the insert widget 402 (i.e., touch-and-slide gesture 410) along theX-axis 306 to insert acolumn 302. Depending on how far the user slides thewidget 402, one ormore columns 302 may be inserted. - According to an embodiment, one or
more rows 304 may be inserted into aspreadsheet 106 via selecting arow header 412, and utilizing the touch-and-slide gesture 410 to select and slide a displayed selectable UI control (e.g., insert widget 402) along the Y-axis 308. Depending on how far the user slides his finger, one ormore rows 304 may be inserted. A live preview of the column/row insertion operation may be displayed such that the user may be able to determine a number ofcolumns 302 orrows 304 that will be inserted upon releasing the touch. -
FIGS. 5A and 5B show a drag-and-drop away gesture to delete acolumn 302 within aspreadsheet 106.FIG. 5A shows aspreadsheet 106 before acolumn 302 is deleted via a drag-and-drop awaygesture 510, andFIG. 5B shows thesame spreadsheet 106 while performing a column deletion operation via a drag-and-drop awaygesture 510. According to an embodiment, a drag-and-drop awaygesture 510 may be detected when a user selects acolumn 302, for example, by touching atouchscreen 116 on an associatedcolumn header 312 with a finger, and drags thecolumn header 312 in an upwards or downwards direction and away from thespreadsheet 106 as indicated bygesture 510. When the user releases thecolumn header 312 when it has been dragged away from theother column headers 312, the associatedcolumn 302 may be deleted. - According to an embodiment, one or
more rows 304 may be deleted from aspreadsheet 106 via selecting arow 304, for example, by the user selecting arow header 412 of a row he wishes to delete, and utilizing the drag-and-drop awaygesture 510 to drag therow header 412 leftwards (in a left-to-right reading scenario) and away from theother row headers 412. When the user releases therow header 412 when it has been dragged away from theother row headers 412, the associatedrow 304 may be deleted. A live preview of the column/row deletion operation may be displayed such that the user may be able to determine how far he needs to drag thecolumn header 312 orrow header 412 to delete the associatedcolumn 302 orrow 304 upon releasing the touch. -
FIGS. 6A and 6B show a touch-hold-pull gesture to reveal a hiddencolumn 314 within aspreadsheet 106.FIG. 6A shows aspreadsheet 106 before ahidden column 302 is shown via a touch-hold-pull gesture 610, andFIG. 6B shows thesame spreadsheet 106 while performing a hidden column revelation operation via a touch-hold-pull gesture 610. According to an embodiment, a touch-hold-pull gesture 610 may be detected when a user selects afirst column 302, for example, by touching atouchscreen 116 on thecolumn header 312 associated with the first column with a finger on onehand 208, and holding his finger on thecolumn header 312; then selects asecond column 302, for example, by touching on thecolumn header 312 associated with the second column with a finger on hisother hand 208; and pulls thesecond column header 312 in a direction along theX-axis 306 away from thefirst column 302 as indicated bygesture 610. When the user releases the column headers 312 a hiddencolumn 314 beneath thesecond column 302 may be revealed. The width of the hiddencolumn 314 and the number of hidden columns 314 (if there are multiple hidden columns 314) revealed may be determined by how far the user pulls thesecond column 302 away from thefirst column 302. - According to an embodiment, one or more hidden rows may be revealed in a
spreadsheet 106 via selecting arow 304, for example, by a user selecting afirst row header 412 via touching thefirst row header 412 with a finger on one hand, and holding his finger on thefirst row header 412; then selecting asecond row 306, for example, by touching thesecond row header 412 with a finger on the user's other hand; then pulling thesecond column header 412 in a direction along the Y-axis 308 away from thefirst row 304. When the user releases the row headers 412 a hidden row beneath thesecond row 304 may be revealed. The height of the hidden row and the number of hidden rows (if there are multiple hidden rows) revealed may be determined by how far the user pulls thesecond row 304 away from thefirst row 304. A live preview of the hiddencolumn 314/row revelation operation may be displayed such that the user may be able to determine how far he needs to drag thecolumn header 312 orrow header 412 to reveal the hidden column(s) 314 or rows. -
FIGS. 7A and 7B show a push-together gesture to hide one or more columns within aspreadsheet 106. As illustrated,FIG. 7A shows aspreadsheet 106 before pushing togethercolumns 302 to hide one ormore columns 302, andFIG. 7B shows thesame spreadsheet 106 while performing a column hiding operation via a push-together gesture 710. According to an embodiment, a push-together gesture 710 may be detected when a user touches atouchscreen 116 with two of his fingers and pushes them together as indicated bygesture 710. To perform a column hiding operation via the push-together gesture 710, the user may select afirst column 302 in aspreadsheet 106, for example, by touching acolumn header 312 associated with thefirst column 302; select asecond column 302, for example, by touching thecolumn header 312 associated with thesecond column 302; and may utilize the push-together gesture 710 along theX-axis 306 to hide one ormore columns 302 between the first andsecond columns 302. Depending on how far apart the user moves his fingers, one ormore columns 302 may be hidden. - According to an embodiment, one or
more rows 304 may be hidden in aspreadsheet 106 via selecting twonon-adjacent rows 304, and utilizing the push-together gesture 710 along the Y-axis 308. Depending on how far apart the user moves his fingers, one ormore rows 304 may be hidden. A live preview of the column/row hiding operation may be displayed such that the user may be able to determine a number ofcolumns 302 orrows 304 that will be hidden upon releasing the touch. - According to another embodiment, the push-
together gesture 710 may be utilized in the same manner as described with respect toFIGS. 7A and 7B to perform acolumn 302 orrow 304 deletion operation. Rather than hiding one ormore columns 302 orrows 304 when twocolumns 302 or tworows 304 are pushed together, the one ormore columns 302 orrows 304 may be deleted. -
FIGS. 8A and 8B show a drag-and-drop between gesture to move a column within aspreadsheet 106. As illustrated,FIGS. 8A and 8B show aspreadsheet 106 while performing the column moving operation via a drag-and-drop betweengesture 810. According to an embodiment, a drag-and-drop betweengesture 810 may be detected when a user selects acolumn 302, for example, by touching atouchscreen 116 on an associatedcolumn header 312 with a finger, and drags thecolumn header 312 away from its current position in thespreadsheet 106, and into a new position between twocolumns 302 as indicated bygesture 810. When the user releases thecolumn header 312 when it has been dragged between twoother column headers 312, the associatedcolumn 302 may be moved to the position between the two columns associated with the twoother column headers 312. - According to an embodiment, one or
more rows 304 may be moved within aspreadsheet 106 via selecting arow 304, for example, by a user selecting arow header 412 of a row he wishes to move, and utilizing the drag-and-drop betweengesture 810 to drag therow header 412 away from its current location and drag therow header 412 between twoother row headers 412. When the user releases therow header 412, the associatedrow 304 may be moved between the tworows 304 associated with the twoother row headers 412. A live preview of the column/row move operation may be displayed such that the user may be able to see where the selectedcolumn 302 orrow 304 may be placed upon releasing the touch. -
FIGS. 9A , 9B, and 9C show a multi-finger or multi-touch swipe gesture to navigate between individual pages or sheets (herein referred to assheets 902,904) in aspreadsheet 106 document. As illustrated,FIG. 9A shows afirst sheet 902 of aspreadsheet 106 displayed before navigating to anext sheet 904 of thespreadsheet 106 via amulti-finger swipe gesture 910,FIG. 9B shows thefirst sheet 902 of thespreadsheet 106 being panned out of display while asecond sheet 904 is being panned into the display via amulti-finger swipe gesture 910, andFIG. 9C shows thesecond sheet 904 of thespreadsheet 106 after it has been panned into view. According to an embodiment, amulti-finger swipe gesture 910 may be detected when a user touches atouchscreen 116 with multiple fingers, and swipes aspreadsheet 106 in a lateral direction along theX-axis 306 as indicated bygesture 910. - The speed of the movement of the
multi-finger swipe gesture 910 may be used in determining a speed of the panning of thespreadsheet 106. For example, a sudden movement in a direction may increase the speed of the panning action. According to one embodiment, amulti-finger swipe gesture 910 may perform a pan to a singlenext sheet 904, or alternatively, according to another embodiment, a user may pan throughmultiple sheets multi-finger swipe gesture 910, depending on the speed of the gesture. -
FIGS. 10A and 10B show a touch-and-throw gesture for extending a selected functionality from one portion of a spreadsheet document to another portion of the document. According to one embodiment, the touch-and-throw gesture may be used to extend a selection of one or more cells to a last non-blank cell in thesame column 302 orrow 304 as the selected cell(s), or if the next cell is blank, to extend the selection to a next nonblank cell within aspreadsheet 106.FIG. 10A shows aspreadsheet 106 where a cell is selected, aselection handle 1002 is displayed around the selected cell, and a user touches theselection handle 1002 and “throws” it over other cells in thecolumn 302 via a touch-and-throw gesture 1010.FIG. 10B shows thesame spreadsheet 106 after performing a selection extension operation via a touch-and-throw gesture 1010. According to an embodiment, a touch-and-throw gesture 1010 may be detected when a user selects aselection handle 1002, for example, by touching atouchscreen 116 on aselection handle 1002 displayed around one or more cells, and “throws” theselection handle 1002 in a direction along the Y-axis 308 as indicated bygesture 1010. Accordingly, the selection may be extended to a last nonblank cell in thesame column 302 as the selected cell(s), or if the next cell is blank, the selection may be extended to a next nonblank cell. - According to an embodiment, a selection of one or more cells in a
row 304 may be extended via selecting aselection handle 1002 displayed around the selected cell(s), and utilizing the touch-and-throw gesture 1010 to “throw” theselection handle 1002 in a direction along theX-axis 306. Accordingly, the selection may be extended to a last nonblank cell in thesame row 304 as the selected cell(s), or if the next cell is blank, the selection may be extended to a next nonblank cell. - According to another embodiment, a selection may be extended to a contiguous block of data contained in cells in
multiple columns 302 andmultiple rows 304 via selecting aselection handle 1002 displayed around a selected cell or cells, and utilizing the touch-and-throw gesture 1010 to “throw” theselection handle 1002 in a diagonal direction. For example,FIG. 10C shows aspreadsheet 106 where a cell is selected, aselection handle 1002 is displayed around the selected cell, and a user touches theselection handle 1002 and “throws” it at an angle over other cells in a block of data, wherein the block of data comprisesmultiple columns 302 androws 304 via a touch-and-throw gesture 1010.FIG. 10D shows thesame spreadsheet 106 after performing a selection extension operation via a touch-and-throw gesture 1010. According to an embodiment, a touch-and-throw gesture 1010 may be detected when a user selects aselection handle 1002, for example, by touching atouchscreen 116 on aselection handle 1002 displayed around one or more cells, and “throws” theselection handle 1002 in an angular direction (along both theX-axis 306 and the Y-axis 308) as indicated bygesture 1010. Accordingly, the selection may be extended to select a contiguous block of rectangular data as illustrated inFIG. 10D . -
FIGS. 11A and 11B show a touch-and-pull gesture to sort data. As illustrated,FIG. 11A shows aspreadsheet 106 as a user selects acolumn header 312 and pulls it downward, andFIG. 11B shows thesame spreadsheet 106 after performing a sort operation via a touch-and-pull gesture 1110. AUI element 1102 may be displayed, providing an indication of a sort operation. According to an embodiment, a touch-and-pull gesture 1110 may be detected when acolumn header 312 is selected, for example, by touching atouchscreen 116 where acolumn header 312 is displayed, and pulled downward along the Y-axis 308 as indicated bygesture 1110. Accordingly, the data in thecolumn 302 may be sorted as illustrated inFIG. 11B . - According to an embodiment, sorting of data in a
row 304 may be performed via utilizing the touch-and-pull gesture 1110. Arow header 412 may be selected and pulled in a direction along theX-axis 306. Accordingly, the data in the row associated with the selectedrow header 412 may be sorted. According to an embodiment, as thecolumn header 312 or arow header 412 is pulled, a visual indicator may be displayed, wherein the visual indicator may inform the user as to how far he may need to drag theheader -
FIGS. 12A and 12B show a pinch-and-turn gesture to rotate an element. As illustrated,FIG. 12A shows aspreadsheet 106 as a user selects anelement 1202 and rotates it, andFIG. 12B shows thesame spreadsheet 106 after performing a rotate operation via a pinch-and-turn gesture 1210. According to an embodiment, a pinch-and-turn gesture 1210 may be detected when an element 1202 (e.g., text, picture, etc.) is selected, for example, by using two or more fingers to touch atouchscreen 116 where anelement 1202 is displayed, and turning it in a circular motion as indicated bygesture 1210. Accordingly, theelement 1202 may be rotated as illustrated inFIG. 12B . Although displayed in a spreadsheet, it is to be understood that utilizing a pinch-and-turn gesture 1210 to rotate anelement 1202 may be performed in various types ofapplications 104. - Embodiments may comprise discontiguous selection via a select-pin-select gesture. For example, where data may not be displayed in adjacent cells, a user may make a first selection of data with his finger, pin the selection, and then select a next selection of data. Pinning may be accomplished via various ways, which may include selection of a UI element that may be displayed next to the first selection, for example, by touching the UI element with another finger, by pressing and holding the first selection, etc. After the first selection is pinned, the user may be able to select a next set of data.
-
FIGS. 13A and 13B show a multi-finger tear-and-pull gesture to “tear” objects out of a grid and to float on top of the grid or move to a canvas. As illustrated,FIG. 13A shows aspreadsheet 106 as a user selects data from thespreadsheet 106 via a multi-touch selection and tears/pulls the data from thespreadsheet 106, andFIG. 13B shows thesame spreadsheet 106 after performing the “tear” operation via a tear-and-pull gesture 1310. According to an embodiment, a tear-and-pull gesture 1310 may be detected when data or an object is selected, for example, by using two or more fingers to touch atouchscreen 116 where the data or object is displayed, and pulling the data or object from its current position as indicated bygesture 1310. Accordingly, the data or object may be “torn out” of the grid and may float on top of the grid or moved to a canvas as illustrated inFIG. 13B . Although displayed in a spreadsheet, it is to be understood that utilizing a multi-finger tear-and-pull gesture 1310 to remove data as a separate element may be performed in various types ofapplications 104. - With reference now to
FIG. 14 , a flow chart illustrating amethod 1400 for using gestures to interact with aspreadsheet 106 will be described. Themethod 1400 starts atOPERATION 1405, and proceeds toOPERATION 1410, where a gesture is received. The gesture may be any of the gestures described herein or other recognized gestures, such as a pulling apartgesture 310, a touch-and-slide gesture 410, a drag-and-drop gesture 510, a tap-hold-pull gesture 610, a push-together gesture 710, a drag-and-drop betweengesture 810, amulti-finger swipe gesture 910, a touch-and-throw gesture 1010, a touch-and-pull gesture 1110, a pinch-and-turn gesture 1210, a multi-finger tear-and-pull gesture 1310, a select-pin-select gesture, and the like may be received. According to one embodiment, the user places at least a portion of their hand 208 (e.g. one or more fingers) on thetouch surface 116. Additionally, according to some embodiments, the user may place hishand 208 near the surface of thetouch surface 116 but not on thetouch surface 116. - The
method 1400 may proceed toOPERATION 1415, where an action to be performed may be determined. According to an embodiment, the action relates to interacting with aspreadsheet 106 and comprises actions such as inserting, deleting, moving, hiding, revealing, panning, sorting, selecting, rotating, removing, and the like. While the actions described relate to interaction withspreadsheets 106,other applications 104 may be utilized with the gestures described. - Flowing to
OPERATION 1420, the determined action may be performed. Generally, the action relates to manipulating data in aspreadsheet 106. For example, acolumn 302 orrow 304 may be deleted, moved, inserted, sorted, or selected, or some other operation may be performed on data, and the like. - The
method 1400 may proceed toOPERATION 1425, where the display may be updated. The display may be updated in response to the action performed per the gesture information that is received. Themethod 1400 may end atOPERATION 1430. - The embodiments and functionalities described herein may operate via a multitude of computing systems including, without limitation, desktop computer systems, wired and wireless computing systems, mobile computing systems (e.g., mobile telephones, netbooks, tablet or slate type computers, notebook computers, and laptop computers), hand-held devices, multiprocessor systems, microprocessor-based or programmable consumer electronics, minicomputers, and mainframe computers.
- In addition, the embodiments and functionalities described herein may operate over distributed systems (e.g., cloud-based computing systems), where application functionality, memory, data storage and retrieval and various processing functions may be operated remotely from each other over a distributed computing network, such as the Internet or an intranet. User interfaces and information of various types may be displayed via on-board computing device displays or via remote display units associated with one or more computing devices. For example user interfaces and information of various types may be displayed and interacted with on a wall surface onto which user interfaces and information of various types are projected. Interaction with the multitude of computing systems with which embodiments of the invention may be practiced include, keystroke entry, touch screen entry, voice or other audio entry, gesture entry where an associated computing device is equipped with detection (e.g., camera) functionality for capturing and interpreting user gestures for controlling the functionality of the computing device, and the like.
-
FIGS. 15-17 and the associated descriptions provide a discussion of a variety of operating environments in which embodiments of the invention may be practiced. However, the devices and systems illustrated and discussed with respect toFIGS. 15-17 are for purposes of example and illustration and are not limiting of a vast number of computing device configurations that may be utilized for practicing embodiments of the invention, described herein. -
FIG. 15 is a block diagram illustrating physical components (i.e., hardware) of acomputing device 1500 with which embodiments of the invention may be practiced. The computing device components described below may be suitable for the computing devices described above including the personal computer 102 a, the tablet computer 102 b, and the smart phone 102 c. In a basic configuration, thecomputing device 1500 may include at least oneprocessing unit 1502 and asystem memory 1504. Depending on the configuration and type of computing device, thesystem memory 1504 may comprise, but is not limited to, volatile storage (e.g., random access memory), non-volatile storage (e.g., read-only memory), flash memory, or any combination of such memories. Thesystem memory 1504 may include anoperating system 1505 and one ormore program modules 1506 suitable for runningsoftware applications 104 such as the spreadsheet application. Theoperating system 1505, for example, may be suitable for controlling the operation of thecomputing device 1500. Furthermore, embodiments of the invention may be practiced in conjunction with a graphics library, other operating systems, or any other application program and is not limited to any particular application or system. This basic configuration is illustrated inFIG. 15 by those components within a dashedline 1508. Thecomputing device 1500 may have additional features or functionality. For example, thecomputing device 1500 may also include additional data storage devices (removable and/or non-removable) such as, for example, magnetic disks, optical disks, or tape. Such additional storage is illustrated inFIG. 15 by aremovable storage device 1509 and anon-removable storage device 1510. - As stated above, a number of program modules and data files may be stored in the
system memory 1504. While executing on theprocessing unit 1502, thesoftware applications 104 may perform processes including, but not limited to, one or more of the stages of themethod 1400 illustrated inFIG. 14 . Other program modules that may be used in accordance with embodiments of the present invention may include electronic mail and contacts applications, word processing applications, spreadsheet applications, database applications, slide presentation applications, drawing or computer-aided application programs, etc. - Furthermore, embodiments of the invention may be practiced in an electrical circuit comprising discrete electronic elements, packaged or integrated electronic chips containing logic gates, a circuit utilizing a microprocessor, or on a single chip containing electronic elements or microprocessors. For example, embodiments of the invention may be practiced via a system-on-a-chip (SOC) where each or many of the components illustrated in
FIG. 15 may be integrated onto a single integrated circuit. Such an SOC device may include one or more processing units, graphics units, communications units, system virtualization units and various application functionality all of which are integrated (or “burned”) onto the chip substrate as a single integrated circuit. When operating via an SOC, the functionality, described herein, with respect to thesoftware applications 104 may be operated via application-specific logic integrated with other components of thecomputing device 1500 on the single integrated circuit (chip). Embodiments of the invention may also be practiced using other technologies capable of performing logical operations such as, for example, AND, OR, and NOT, including but not limited to mechanical, optical, fluidic, and quantum technologies. In addition, embodiments of the invention may be practiced within a general purpose computer or in any other circuits or systems. - The
computing device 1500 may also have one or more input device(s) 1512 such as a keyboard, a mouse, a pen, a sound input device, a touch input device, etc. The output device(s) 1514 such as a display, speakers, a printer, etc. may also be included. The aforementioned devices are examples and others may be used. Thecomputing device 1500 may include one ormore communication connections 1516 allowing communications withother computing devices 1518. Examples ofsuitable communication connections 1516 include, but are not limited to, RF transmitter, receiver, and/or transceiver circuitry; universal serial bus (USB), parallel, and/or serial ports. - The term computer readable media as used herein may include computer storage media. Computer storage media may include volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storage of information, such as computer readable instructions, data structures, or program modules. The
system memory 1504, theremovable storage device 1509, and thenon-removable storage device 1510 are all computer storage media examples (i.e., memory storage.) Computer storage media may include RAM, ROM, electrically erasable read-only memory (EEPROM), flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other article of manufacture which can be used to store information and which can be accessed by thecomputing device 1500. Any such computer storage media may be part of thecomputing device 1500. -
FIGS. 16A and 16B illustrate amobile computing device 1600, for example, a mobile telephone, a smart phone, a tablet personal computer, a laptop computer, and the like, with which embodiments of the invention may be practiced. With reference toFIG. 16A , one embodiment of amobile computing device 1600 for implementing the embodiments is illustrated. In a basic configuration, themobile computing device 1600 is a handheld computer having both input elements and output elements. Themobile computing device 1600 typically includes adisplay 1605 and one ormore input buttons 1610 that allow the user to enter information into themobile computing device 1600. Thedisplay 1605 of themobile computing device 1600 may also function as an input device (e.g., a touch screen display 116). If included, an optionalside input element 1615 allows further user input. Theside input element 1615 may be a rotary switch, a button, or any other type of manual input element. In alternative embodiments,mobile computing device 1600 may incorporate more or less input elements. For example, thedisplay 1605 may not be a touch screen in some embodiments. In yet another alternative embodiment, themobile computing device 1600 is a portable phone system, such as a cellular phone. Themobile computing device 1600 may also include anoptional keypad 1635.Optional keypad 1635 may be a physical keypad or a “soft” keypad generated on the touch screen display. In various embodiments, the output elements include thedisplay 1605 for showing a graphical user interface (GUI), a visual indicator 1620 (e.g., a light emitting diode), and/or an audio transducer 1625 (e.g., a speaker). In some embodiments, themobile computing device 1600 incorporates a vibration transducer for providing the user with tactile feedback. In yet another embodiment, themobile computing device 1600 incorporates input and/or output ports, such as an audio input (e.g., a microphone jack), an audio output (e.g., a headphone jack), and a video output (e.g., a HDMI port) for sending signals to or receiving signals from an external device. -
FIG. 16B is a block diagram illustrating the architecture of one embodiment of a mobile computing device. That is, themobile computing device 1600 can incorporate a system (i.e., an architecture) 1602 to implement some embodiments. In one embodiment, thesystem 1602 is implemented as a “smart phone” capable of running one or more applications (e.g., browser, e-mail, calendaring, contact managers, messaging clients, games, and media clients/players). In some embodiments, thesystem 1602 is integrated as a computing device, such as an integrated personal digital assistant (PDA) and wireless phone. - One or
more application programs 104 may be loaded into thememory 1662 and run on or in association with theoperating system 1664. Examples of the application programs include phone dialer programs, e-mail programs, personal information management (PIM) programs, word processing programs, spreadsheet programs, Internet browser programs, messaging programs, and so forth. Thesystem 1602 also includes anon-volatile storage area 1668 within thememory 1662. Thenon-volatile storage area 1668 may be used to store persistent information that should not be lost if thesystem 1602 is powered down. Theapplication programs 104 may use and store information in thenon-volatile storage area 1668, such as e-mail or other messages used by an e-mail application, and the like. A synchronization application (not shown) also resides on thesystem 1602 and is programmed to interact with a corresponding synchronization application resident on a host computer to keep the information stored in thenon-volatile storage area 1668 synchronized with corresponding information stored at the host computer. As should be appreciated, other applications may be loaded into thememory 1662 and run on themobile computing device 1600, including software applications 1520 described herein. - The
system 1602 has apower supply 1670, which may be implemented as one or more batteries. Thepower supply 1670 might further include an external power source, such as an AC adapter or a powered docking cradle that supplements or recharges the batteries. - The
system 1602 may also include aradio 1672 that performs the function of transmitting and receiving radio frequency communications. Theradio 1672 facilitates wireless connectivity between thesystem 1602 and the “outside world,” via a communications carrier or service provider. Transmissions to and from theradio 1672 are conducted under control of theoperating system 1664. In other words, communications received by theradio 1672 may be disseminated to theapplication programs 104 via theoperating system 1664, and vice versa. - The
visual indicator 1620 may be used to provide visual notifications, and/or anaudio interface 1674 may be used for producing audible notifications via theaudio transducer 1625. In the illustrated embodiment, thevisual indicator 1620 is a light emitting diode (LED) and theaudio transducer 1625 is a speaker. These devices may be directly coupled to thepower supply 1670 so that when activated, they remain on for a duration dictated by the notification mechanism even though theprocessor 1660 and other components might shut down for conserving battery power. The LED may be programmed to remain on indefinitely until the user takes action to indicate the powered-on status of the device. Theaudio interface 1674 is used to provide audible signals to and receive audible signals from the user. For example, in addition to being coupled to theaudio transducer 1625, theaudio interface 1674 may also be coupled to a microphone to receive audible input, such as to facilitate a telephone conversation. In accordance with embodiments of the present invention, the microphone may also serve as an audio sensor to facilitate control of notifications, as will be described below. Thesystem 1602 may further include avideo interface 1676 that enables an operation of an on-board camera 1630 to record still images, video stream, and the like. - A
mobile computing device 1600 implementing thesystem 1602 may have additional features or functionality. For example, themobile computing device 1600 may also include additional data storage devices (removable and/or non-removable) such as, magnetic disks, optical disks, or tape. Such additional storage is illustrated inFIG. 16B by thenon-volatile storage area 1668. - Data/information generated or captured by the
mobile computing device 1600 and stored via thesystem 1602 may be stored locally on themobile computing device 1600, as described above, or the data may be stored on any number of storage media that may be accessed by the device via theradio 1672 or via a wired connection between themobile computing device 1600 and a separate computing device associated with themobile computing device 1600, for example, a server computer in a distributed computing network, such as the Internet. As should be appreciated such data/information may be accessed via themobile computing device 1600 via theradio 1672 or via a distributedcomputing network 112. Similarly, such data/information may be readily transferred between computing devices for storage and use according to well-known data/information transfer and storage means, including electronic mail and collaborative data/information sharing systems. -
FIG. 17 illustrates one embodiment of the architecture of a system for providing quick function gesture functionality to one or more client devices, as described above. Content developed, interacted with, or edited in association with anapplication 104, such as a spreadsheet application, may be stored in different communication channels or other storage types. For example, various documents may be stored using adirectory service 1722, aweb portal 1724, amailbox service 1726, aninstant messaging store 1728, or asocial networking site 1730. Theapplication 104 may use any of these types of systems or the like for enabling data utilization, as described herein. Aserver 110 may provide thespreadsheet application 104 to clients. As one example, theserver 110 may be a web server providing thespreadsheet application 104 over the web. Theserver 110 may provide thespreadsheet application 104 over the web to clients through anetwork 112. By way of example, the client computing device may be implemented as thecomputing device 1500 and embodied in apersonal computer 102A, atablet computer 102B, and/or amobile computing device 102C (e.g., a smart phone). Any of these embodiments of theclient computing device 102A-C may obtain content from thestore 1716. - Embodiments of the present invention, for example, are described above with reference to block diagrams and/or operational illustrations of methods, systems, and computer program products according to embodiments of the invention. The functions/acts noted in the blocks may occur out of the order as shown in any flowchart. For example, two blocks shown in succession may in fact be executed substantially concurrently or the blocks may sometimes be executed in the reverse order, depending upon the functionality/acts involved.
- One skilled in the relevant art may recognize, however, that the embodiments may be practiced without one or more of the specific details, or with other methods, resources, materials, etc. In other instances, well known structures, resources, or operations have not been shown or described in detail merely to avoid obscuring aspects of the embodiments.
- The description and illustration of one or more embodiments provided in this application are not intended to limit or restrict the scope of the invention as claimed in any way. The embodiments, examples, and details provided in this application are considered sufficient to convey possession and enable others to make and use the best mode of claimed invention. The claimed invention should not be construed as being limited to any embodiment, example, or detail provided in this application. Regardless of whether shown and described in combination or separately, the various features (both structural and methodological) are intended to be selectively included or omitted to produce an embodiment with a particular set of features. Having been provided with the description and illustration of the present application, one skilled in the art may envision variations, modifications, and alternate embodiments falling within the spirit of the broader aspects of the general inventive concept embodied in this application that do not depart from the broader scope of the claimed invention.
Claims (20)
Priority Applications (6)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/918,871 US20140372856A1 (en) | 2013-06-14 | 2013-06-14 | Natural Quick Functions Gestures |
PCT/US2014/041071 WO2014200798A1 (en) | 2013-06-14 | 2014-06-05 | Natural quick function gestures |
CN201480045377.7A CN105474163B (en) | 2013-06-14 | 2014-06-05 | Natural quick function gesture |
EP14736150.5A EP3008575B1 (en) | 2013-06-14 | 2014-06-05 | Natural quick function gestures |
US16/833,098 US11157691B2 (en) | 2013-06-14 | 2020-03-27 | Natural quick function gestures |
US17/446,581 US11699031B2 (en) | 2013-06-14 | 2021-08-31 | Natural quick function gestures |
Applications Claiming Priority (1)
Application Number | Priority Date | Filing Date | Title |
---|---|---|---|
US13/918,871 US20140372856A1 (en) | 2013-06-14 | 2013-06-14 | Natural Quick Functions Gestures |
Related Child Applications (1)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/833,098 Continuation US11157691B2 (en) | 2013-06-14 | 2020-03-27 | Natural quick function gestures |
Publications (1)
Publication Number | Publication Date |
---|---|
US20140372856A1 true US20140372856A1 (en) | 2014-12-18 |
Family
ID=51134337
Family Applications (3)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US13/918,871 Abandoned US20140372856A1 (en) | 2013-06-14 | 2013-06-14 | Natural Quick Functions Gestures |
US16/833,098 Active US11157691B2 (en) | 2013-06-14 | 2020-03-27 | Natural quick function gestures |
US17/446,581 Active US11699031B2 (en) | 2013-06-14 | 2021-08-31 | Natural quick function gestures |
Family Applications After (2)
Application Number | Title | Priority Date | Filing Date |
---|---|---|---|
US16/833,098 Active US11157691B2 (en) | 2013-06-14 | 2020-03-27 | Natural quick function gestures |
US17/446,581 Active US11699031B2 (en) | 2013-06-14 | 2021-08-31 | Natural quick function gestures |
Country Status (4)
Country | Link |
---|---|
US (3) | US20140372856A1 (en) |
EP (1) | EP3008575B1 (en) |
CN (1) | CN105474163B (en) |
WO (1) | WO2014200798A1 (en) |
Cited By (35)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20150301693A1 (en) * | 2014-04-17 | 2015-10-22 | Google Inc. | Methods, systems, and media for presenting related content |
US20160048325A1 (en) * | 2013-08-16 | 2016-02-18 | Edward Lau | Electronic device and gesture input method of item selection |
US9747270B2 (en) | 2011-01-07 | 2017-08-29 | Microsoft Technology Licensing, Llc | Natural input for spreadsheet actions |
US9996518B2 (en) * | 2016-08-08 | 2018-06-12 | International Business Machines Corporation | Inserting new elements in a tabular data structure |
US20180335898A1 (en) * | 2017-05-18 | 2018-11-22 | Edward Lau | Electronic device and item selection method using gesture input |
US20190050383A1 (en) * | 2017-08-11 | 2019-02-14 | Emro Co., Ltd. | Method of providing data management service capable of column grouping and service providing server for performing the same |
US10664652B2 (en) | 2013-06-15 | 2020-05-26 | Microsoft Technology Licensing, Llc | Seamless grid and canvas integration in a spreadsheet application |
US11112964B2 (en) | 2018-02-09 | 2021-09-07 | Apple Inc. | Media capture lock affordance for graphical user interface |
US11128792B2 (en) | 2018-09-28 | 2021-09-21 | Apple Inc. | Capturing and displaying images with multiple focal planes |
US11165949B2 (en) | 2016-06-12 | 2021-11-02 | Apple Inc. | User interface for capturing photos with different camera magnifications |
US11178335B2 (en) | 2018-05-07 | 2021-11-16 | Apple Inc. | Creative camera |
US20210357579A1 (en) * | 2020-05-01 | 2021-11-18 | Monday.com Ltd. | Digital processing systems and methods for automatic display of value types based on custom heading in collaborative work systems |
US11204692B2 (en) | 2017-06-04 | 2021-12-21 | Apple Inc. | User interface camera effects |
US11212449B1 (en) | 2020-09-25 | 2021-12-28 | Apple Inc. | User interfaces for media capture and management |
US11216153B2 (en) * | 2014-05-15 | 2022-01-04 | Sony Corporation | Information processing device, display control method, and program |
US11223771B2 (en) | 2019-05-06 | 2022-01-11 | Apple Inc. | User interfaces for capturing and managing visual media |
US11275499B2 (en) * | 2016-06-10 | 2022-03-15 | Apple Inc. | Device, method, and graphical user interface for changing a number of columns of an application region |
US11321857B2 (en) | 2018-09-28 | 2022-05-03 | Apple Inc. | Displaying and editing images with depth information |
US11320983B1 (en) * | 2018-04-25 | 2022-05-03 | David Graham Boyers | Methods and graphical user interfaces for positioning a selection, selecting, and editing, on a computing device running applications under a touch-based operating system |
US11330184B2 (en) | 2020-06-01 | 2022-05-10 | Apple Inc. | User interfaces for managing media |
US11350026B1 (en) | 2021-04-30 | 2022-05-31 | Apple Inc. | User interfaces for altering visual media |
US11468625B2 (en) | 2018-09-11 | 2022-10-11 | Apple Inc. | User interfaces for simulated depth effects |
US11490017B2 (en) | 2015-04-23 | 2022-11-01 | Apple Inc. | Digital viewfinder user interface for multiple cameras |
US11687216B2 (en) | 2021-01-14 | 2023-06-27 | Monday.com Ltd. | Digital processing systems and methods for dynamically updating documents with data from linked files in collaborative work systems |
US11698890B2 (en) | 2018-07-04 | 2023-07-11 | Monday.com Ltd. | System and method for generating a column-oriented data structure repository for columns of single data types |
US11706521B2 (en) | 2019-05-06 | 2023-07-18 | Apple Inc. | User interfaces for capturing and managing visual media |
US11714544B2 (en) | 2020-06-25 | 2023-08-01 | Microsoft Technology Licensing, Llc | Gesture definition for multi-screen devices |
US11722764B2 (en) | 2018-05-07 | 2023-08-08 | Apple Inc. | Creative camera |
US11741071B1 (en) | 2022-12-28 | 2023-08-29 | Monday.com Ltd. | Digital processing systems and methods for navigating and viewing displayed content |
US11770601B2 (en) | 2019-05-06 | 2023-09-26 | Apple Inc. | User interfaces for capturing and managing visual media |
US11778339B2 (en) | 2021-04-30 | 2023-10-03 | Apple Inc. | User interfaces for altering visual media |
US11829953B1 (en) | 2020-05-01 | 2023-11-28 | Monday.com Ltd. | Digital processing systems and methods for managing sprints using linked electronic boards |
US11886683B1 (en) | 2022-12-30 | 2024-01-30 | Monday.com Ltd | Digital processing systems and methods for presenting board graphics |
US11893381B1 (en) | 2023-02-21 | 2024-02-06 | Monday.com Ltd | Digital processing systems and methods for reducing file bundle sizes |
US11962889B2 (en) | 2023-03-14 | 2024-04-16 | Apple Inc. | User interface for camera effects |
Families Citing this family (6)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US20140372856A1 (en) | 2013-06-14 | 2014-12-18 | Microsoft Corporation | Natural Quick Functions Gestures |
CN107977134A (en) * | 2017-11-14 | 2018-05-01 | 浙江慧脑信息科技有限公司 | A kind of form shows control method |
CN109858000B (en) * | 2019-01-10 | 2021-09-07 | 广州视源电子科技股份有限公司 | Form processing method, device and system, storage medium and interactive intelligent panel |
CN109947328B (en) * | 2019-03-21 | 2020-12-08 | 珠海格力电器股份有限公司 | Multi-parameter display method and device and air conditioning unit monitoring equipment |
CN112799564A (en) * | 2020-07-14 | 2021-05-14 | 珠海金山办公软件有限公司 | Method, device, electronic equipment and medium for inserting control in table |
CN115577676B (en) * | 2022-12-07 | 2023-03-28 | 苏州万店掌网络科技有限公司 | Page table control method, device, equipment and storage medium |
Citations (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6323846B1 (en) * | 1998-01-26 | 2001-11-27 | University Of Delaware | Method and apparatus for integrating manual input |
US20040060001A1 (en) * | 1998-12-31 | 2004-03-25 | Microsoft Corporation | Extension of formulas and formatting in an electronic spreadsheet |
US20060055662A1 (en) * | 2004-09-13 | 2006-03-16 | Microsoft Corporation | Flick gesture |
US20080270886A1 (en) * | 2007-04-30 | 2008-10-30 | Google Inc. | Hiding Portions of Display Content |
US20100031167A1 (en) * | 2008-08-04 | 2010-02-04 | Alexander Roytman | Browser-based development tools and methods for developing the same |
US20110074699A1 (en) * | 2009-09-25 | 2011-03-31 | Jason Robert Marr | Device, Method, and Graphical User Interface for Scrolling a Multi-Section Document |
US20110163968A1 (en) * | 2010-01-06 | 2011-07-07 | Hogan Edward P A | Device, Method, and Graphical User Interface for Manipulating Tables Using Multi-Contact Gestures |
US20120013540A1 (en) * | 2010-07-13 | 2012-01-19 | Hogan Edward P A | Table editing systems with gesture-based insertion and deletion of columns and rows |
US20130174025A1 (en) * | 2011-12-29 | 2013-07-04 | Keng Fai Lee | Visual comparison of document versions |
US20130212470A1 (en) * | 2012-02-15 | 2013-08-15 | Apple Inc. | Device, Method, and Graphical User Interface for Sharing a Content Object in a Document |
US20130229373A1 (en) * | 2002-11-04 | 2013-09-05 | Neonode Inc. | Light-based finger gesture user interface |
US20130321308A1 (en) * | 2012-05-30 | 2013-12-05 | Samsung Electronics Co., Ltd | Method and apparatus for moving object in terminal having touch screen |
US20130321283A1 (en) * | 2012-05-29 | 2013-12-05 | Research In Motion Limited | Portable electronic device including touch-sensitive display and method of controlling same |
US20130339903A1 (en) * | 2012-06-19 | 2013-12-19 | Microsoft Corporation | UI Differentiation Between Delete and Clear |
US20140033093A1 (en) * | 2012-07-25 | 2014-01-30 | Microsoft Corporation | Manipulating tables with touch gestures |
US20140194162A1 (en) * | 2013-01-04 | 2014-07-10 | Apple Inc. | Modifying A Selection Based on Tapping |
US8854433B1 (en) * | 2012-02-03 | 2014-10-07 | Aquifi, Inc. | Method and system enabling natural user interface gestures with an electronic system |
US20150161095A1 (en) * | 2012-05-04 | 2015-06-11 | Zhuhai Kingsoft Office Software Co., Ltd | Method for inserting or deleting cells, rows or columns in spreadsheet and a device therefor |
Family Cites Families (34)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6057837A (en) * | 1997-07-15 | 2000-05-02 | Microsoft Corporation | On-screen indentification and manipulation of sources that an object depends upon |
US6848080B1 (en) | 1999-11-05 | 2005-01-25 | Microsoft Corporation | Language input architecture for converting one text form to another text form with tolerance to spelling, typographical, and conversion errors |
US7117435B1 (en) | 2000-06-21 | 2006-10-03 | Microsoft Corporation | Spreadsheet fields in text |
US20030009411A1 (en) * | 2001-07-03 | 2003-01-09 | Pranil Ram | Interactive grid-based graphical trading system for real time security trading |
US9405507B2 (en) * | 2003-10-31 | 2016-08-02 | Adobe Systems Incorporated | Tabular data sorting and display |
US20050268215A1 (en) | 2004-06-01 | 2005-12-01 | Microsoft Corporation | Method and apparatus for viewing and interacting with a spreadsheet from within a web browser |
CN1904879B (en) | 2005-07-27 | 2011-01-12 | 国际商业机器公司 | Electronic table system, method for obtaining snapshot/history information of electronic table file |
US7657571B2 (en) | 2005-08-30 | 2010-02-02 | Microsoft Corporation | Electronic data snapshot generator |
US20070061699A1 (en) | 2005-09-09 | 2007-03-15 | Microsoft Corporation | Named object view of electronic data report |
US7752536B2 (en) | 2005-09-09 | 2010-07-06 | Microsoft Corporation | Designating, setting and discovering parameters for spreadsheet documents |
US7908549B2 (en) | 2005-12-08 | 2011-03-15 | Microsoft Corporation | Spreadsheet calculation as part of workflow |
US7587661B2 (en) | 2006-01-19 | 2009-09-08 | Microsoft Corporation | Identifying design issues in electronic forms |
US20080016182A1 (en) * | 2006-07-11 | 2008-01-17 | Nokia Corporation | Dynamic device profile interfaces |
US8564544B2 (en) * | 2006-09-06 | 2013-10-22 | Apple Inc. | Touch screen device, method, and graphical user interface for customizing display of content category icons |
US10180714B1 (en) | 2008-04-24 | 2019-01-15 | Pixar | Two-handed multi-stroke marking menus for multi-touch devices |
US20100229090A1 (en) | 2009-03-05 | 2010-09-09 | Next Holdings Limited | Systems and Methods for Interacting With Touch Displays Using Single-Touch and Multi-Touch Gestures |
US8549432B2 (en) | 2009-05-29 | 2013-10-01 | Apple Inc. | Radial menus |
US8698845B2 (en) | 2010-01-06 | 2014-04-15 | Apple Inc. | Device, method, and graphical user interface with interactive popup views |
US20130212541A1 (en) | 2010-06-01 | 2013-08-15 | Nokia Corporation | Method, a device and a system for receiving user input |
CN101882108A (en) | 2010-06-30 | 2010-11-10 | 三一重机有限公司 | Embedded software automatic test system and method thereof |
US8825745B2 (en) | 2010-07-11 | 2014-09-02 | Microsoft Corporation | URL-facilitated access to spreadsheet elements |
US20120013539A1 (en) * | 2010-07-13 | 2012-01-19 | Hogan Edward P A | Systems with gesture-based editing of tables |
US9747270B2 (en) | 2011-01-07 | 2017-08-29 | Microsoft Technology Licensing, Llc | Natural input for spreadsheet actions |
US9003298B2 (en) * | 2010-09-21 | 2015-04-07 | Microsoft Corporation | Web page application controls |
US8817053B2 (en) | 2010-09-30 | 2014-08-26 | Apple Inc. | Methods and systems for opening a file |
US8863019B2 (en) * | 2011-03-29 | 2014-10-14 | International Business Machines Corporation | Modifying numeric data presentation on a display |
US9053083B2 (en) | 2011-11-04 | 2015-06-09 | Microsoft Technology Licensing, Llc | Interaction between web gadgets and spreadsheets |
US9171099B2 (en) | 2012-01-26 | 2015-10-27 | Microsoft Technology Licensing, Llc | System and method for providing calculation web services for online documents |
US8527909B1 (en) * | 2012-05-29 | 2013-09-03 | Sencha, Inc. | Manipulating data visualizations on a touch screen |
US9141259B2 (en) * | 2013-01-21 | 2015-09-22 | International Business Machines Corporation | Pressure navigation on a touch sensitive user interface |
US9785240B2 (en) * | 2013-03-18 | 2017-10-10 | Fuji Xerox Co., Ltd. | Systems and methods for content-aware selection |
US20140372856A1 (en) | 2013-06-14 | 2014-12-18 | Microsoft Corporation | Natural Quick Functions Gestures |
US10664652B2 (en) | 2013-06-15 | 2020-05-26 | Microsoft Technology Licensing, Llc | Seamless grid and canvas integration in a spreadsheet application |
US20160026616A1 (en) | 2014-07-28 | 2016-01-28 | Microsoft Corporation | Presenting dataset of spreadsheet in form based view |
-
2013
- 2013-06-14 US US13/918,871 patent/US20140372856A1/en not_active Abandoned
-
2014
- 2014-06-05 WO PCT/US2014/041071 patent/WO2014200798A1/en active Application Filing
- 2014-06-05 CN CN201480045377.7A patent/CN105474163B/en active Active
- 2014-06-05 EP EP14736150.5A patent/EP3008575B1/en active Active
-
2020
- 2020-03-27 US US16/833,098 patent/US11157691B2/en active Active
-
2021
- 2021-08-31 US US17/446,581 patent/US11699031B2/en active Active
Patent Citations (18)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US6323846B1 (en) * | 1998-01-26 | 2001-11-27 | University Of Delaware | Method and apparatus for integrating manual input |
US20040060001A1 (en) * | 1998-12-31 | 2004-03-25 | Microsoft Corporation | Extension of formulas and formatting in an electronic spreadsheet |
US20130229373A1 (en) * | 2002-11-04 | 2013-09-05 | Neonode Inc. | Light-based finger gesture user interface |
US20060055662A1 (en) * | 2004-09-13 | 2006-03-16 | Microsoft Corporation | Flick gesture |
US20080270886A1 (en) * | 2007-04-30 | 2008-10-30 | Google Inc. | Hiding Portions of Display Content |
US20100031167A1 (en) * | 2008-08-04 | 2010-02-04 | Alexander Roytman | Browser-based development tools and methods for developing the same |
US20110074699A1 (en) * | 2009-09-25 | 2011-03-31 | Jason Robert Marr | Device, Method, and Graphical User Interface for Scrolling a Multi-Section Document |
US20110163968A1 (en) * | 2010-01-06 | 2011-07-07 | Hogan Edward P A | Device, Method, and Graphical User Interface for Manipulating Tables Using Multi-Contact Gestures |
US20120013540A1 (en) * | 2010-07-13 | 2012-01-19 | Hogan Edward P A | Table editing systems with gesture-based insertion and deletion of columns and rows |
US20130174025A1 (en) * | 2011-12-29 | 2013-07-04 | Keng Fai Lee | Visual comparison of document versions |
US8854433B1 (en) * | 2012-02-03 | 2014-10-07 | Aquifi, Inc. | Method and system enabling natural user interface gestures with an electronic system |
US20130212470A1 (en) * | 2012-02-15 | 2013-08-15 | Apple Inc. | Device, Method, and Graphical User Interface for Sharing a Content Object in a Document |
US20150161095A1 (en) * | 2012-05-04 | 2015-06-11 | Zhuhai Kingsoft Office Software Co., Ltd | Method for inserting or deleting cells, rows or columns in spreadsheet and a device therefor |
US20130321283A1 (en) * | 2012-05-29 | 2013-12-05 | Research In Motion Limited | Portable electronic device including touch-sensitive display and method of controlling same |
US20130321308A1 (en) * | 2012-05-30 | 2013-12-05 | Samsung Electronics Co., Ltd | Method and apparatus for moving object in terminal having touch screen |
US20130339903A1 (en) * | 2012-06-19 | 2013-12-19 | Microsoft Corporation | UI Differentiation Between Delete and Clear |
US20140033093A1 (en) * | 2012-07-25 | 2014-01-30 | Microsoft Corporation | Manipulating tables with touch gestures |
US20140194162A1 (en) * | 2013-01-04 | 2014-07-10 | Apple Inc. | Modifying A Selection Based on Tapping |
Cited By (56)
Publication number | Priority date | Publication date | Assignee | Title |
---|---|---|---|---|
US9747270B2 (en) | 2011-01-07 | 2017-08-29 | Microsoft Technology Licensing, Llc | Natural input for spreadsheet actions |
US10732825B2 (en) | 2011-01-07 | 2020-08-04 | Microsoft Technology Licensing, Llc | Natural input for spreadsheet actions |
US10664652B2 (en) | 2013-06-15 | 2020-05-26 | Microsoft Technology Licensing, Llc | Seamless grid and canvas integration in a spreadsheet application |
US20160048325A1 (en) * | 2013-08-16 | 2016-02-18 | Edward Lau | Electronic device and gesture input method of item selection |
US9529530B2 (en) * | 2013-08-16 | 2016-12-27 | Edward Lau | Electronic device and gesture input method of item selection |
US20150301693A1 (en) * | 2014-04-17 | 2015-10-22 | Google Inc. | Methods, systems, and media for presenting related content |
US11216153B2 (en) * | 2014-05-15 | 2022-01-04 | Sony Corporation | Information processing device, display control method, and program |
US20220091728A1 (en) * | 2014-05-15 | 2022-03-24 | Sony Corporation | Information processing device, display control method, and program |
US11693530B2 (en) * | 2014-05-15 | 2023-07-04 | Sony Corporation | Information processing device, display control method, and program |
US11711614B2 (en) | 2015-04-23 | 2023-07-25 | Apple Inc. | Digital viewfinder user interface for multiple cameras |
US11490017B2 (en) | 2015-04-23 | 2022-11-01 | Apple Inc. | Digital viewfinder user interface for multiple cameras |
US11275499B2 (en) * | 2016-06-10 | 2022-03-15 | Apple Inc. | Device, method, and graphical user interface for changing a number of columns of an application region |
US11641517B2 (en) | 2016-06-12 | 2023-05-02 | Apple Inc. | User interface for camera effects |
US11165949B2 (en) | 2016-06-12 | 2021-11-02 | Apple Inc. | User interface for capturing photos with different camera magnifications |
US11245837B2 (en) | 2016-06-12 | 2022-02-08 | Apple Inc. | User interface for camera effects |
US9996518B2 (en) * | 2016-08-08 | 2018-06-12 | International Business Machines Corporation | Inserting new elements in a tabular data structure |
US10204094B2 (en) * | 2016-08-08 | 2019-02-12 | International Business Machines Corporation | Inserting new elements in a tabular data structure |
US10025768B2 (en) | 2016-08-08 | 2018-07-17 | International Business Machines Corporation | Inserting new elements in a tabular data structure |
US20180335898A1 (en) * | 2017-05-18 | 2018-11-22 | Edward Lau | Electronic device and item selection method using gesture input |
US11204692B2 (en) | 2017-06-04 | 2021-12-21 | Apple Inc. | User interface camera effects |
US11687224B2 (en) | 2017-06-04 | 2023-06-27 | Apple Inc. | User interface camera effects |
US20190050383A1 (en) * | 2017-08-11 | 2019-02-14 | Emro Co., Ltd. | Method of providing data management service capable of column grouping and service providing server for performing the same |
US11112964B2 (en) | 2018-02-09 | 2021-09-07 | Apple Inc. | Media capture lock affordance for graphical user interface |
US11320983B1 (en) * | 2018-04-25 | 2022-05-03 | David Graham Boyers | Methods and graphical user interfaces for positioning a selection, selecting, and editing, on a computing device running applications under a touch-based operating system |
US11722764B2 (en) | 2018-05-07 | 2023-08-08 | Apple Inc. | Creative camera |
US11178335B2 (en) | 2018-05-07 | 2021-11-16 | Apple Inc. | Creative camera |
US11698890B2 (en) | 2018-07-04 | 2023-07-11 | Monday.com Ltd. | System and method for generating a column-oriented data structure repository for columns of single data types |
US11468625B2 (en) | 2018-09-11 | 2022-10-11 | Apple Inc. | User interfaces for simulated depth effects |
US11669985B2 (en) | 2018-09-28 | 2023-06-06 | Apple Inc. | Displaying and editing images with depth information |
US11895391B2 (en) | 2018-09-28 | 2024-02-06 | Apple Inc. | Capturing and displaying images with multiple focal planes |
US11128792B2 (en) | 2018-09-28 | 2021-09-21 | Apple Inc. | Capturing and displaying images with multiple focal planes |
US11321857B2 (en) | 2018-09-28 | 2022-05-03 | Apple Inc. | Displaying and editing images with depth information |
US11770601B2 (en) | 2019-05-06 | 2023-09-26 | Apple Inc. | User interfaces for capturing and managing visual media |
US11706521B2 (en) | 2019-05-06 | 2023-07-18 | Apple Inc. | User interfaces for capturing and managing visual media |
US11223771B2 (en) | 2019-05-06 | 2022-01-11 | Apple Inc. | User interfaces for capturing and managing visual media |
US11829953B1 (en) | 2020-05-01 | 2023-11-28 | Monday.com Ltd. | Digital processing systems and methods for managing sprints using linked electronic boards |
US20210357579A1 (en) * | 2020-05-01 | 2021-11-18 | Monday.com Ltd. | Digital processing systems and methods for automatic display of value types based on custom heading in collaborative work systems |
US11954428B2 (en) | 2020-05-01 | 2024-04-09 | Monday.com Ltd. | Digital processing systems and methods for accessing another's display via social layer interactions in collaborative work systems |
US11687706B2 (en) * | 2020-05-01 | 2023-06-27 | Monday.com Ltd. | Digital processing systems and methods for automatic display of value types based on custom heading in collaborative work systems |
US11886804B2 (en) | 2020-05-01 | 2024-01-30 | Monday.com Ltd. | Digital processing systems and methods for self-configuring automation packages in collaborative work systems |
US11755827B2 (en) | 2020-05-01 | 2023-09-12 | Monday.com Ltd. | Digital processing systems and methods for stripping data from workflows to create generic templates in collaborative work systems |
US11617022B2 (en) | 2020-06-01 | 2023-03-28 | Apple Inc. | User interfaces for managing media |
US11330184B2 (en) | 2020-06-01 | 2022-05-10 | Apple Inc. | User interfaces for managing media |
US11714544B2 (en) | 2020-06-25 | 2023-08-01 | Microsoft Technology Licensing, Llc | Gesture definition for multi-screen devices |
US11212449B1 (en) | 2020-09-25 | 2021-12-28 | Apple Inc. | User interfaces for media capture and management |
US11687216B2 (en) | 2021-01-14 | 2023-06-27 | Monday.com Ltd. | Digital processing systems and methods for dynamically updating documents with data from linked files in collaborative work systems |
US11893213B2 (en) | 2021-01-14 | 2024-02-06 | Monday.com Ltd. | Digital processing systems and methods for embedded live application in-line in a word processing document in collaborative work systems |
US11418699B1 (en) | 2021-04-30 | 2022-08-16 | Apple Inc. | User interfaces for altering visual media |
US11778339B2 (en) | 2021-04-30 | 2023-10-03 | Apple Inc. | User interfaces for altering visual media |
US11350026B1 (en) | 2021-04-30 | 2022-05-31 | Apple Inc. | User interfaces for altering visual media |
US11416134B1 (en) | 2021-04-30 | 2022-08-16 | Apple Inc. | User interfaces for altering visual media |
US11539876B2 (en) | 2021-04-30 | 2022-12-27 | Apple Inc. | User interfaces for altering visual media |
US11741071B1 (en) | 2022-12-28 | 2023-08-29 | Monday.com Ltd. | Digital processing systems and methods for navigating and viewing displayed content |
US11886683B1 (en) | 2022-12-30 | 2024-01-30 | Monday.com Ltd | Digital processing systems and methods for presenting board graphics |
US11893381B1 (en) | 2023-02-21 | 2024-02-06 | Monday.com Ltd | Digital processing systems and methods for reducing file bundle sizes |
US11962889B2 (en) | 2023-03-14 | 2024-04-16 | Apple Inc. | User interface for camera effects |
Also Published As
Publication number | Publication date |
---|---|
CN105474163B (en) | 2019-01-22 |
US20200285806A1 (en) | 2020-09-10 |
WO2014200798A1 (en) | 2014-12-18 |
EP3008575A1 (en) | 2016-04-20 |
CN105474163A (en) | 2016-04-06 |
US11157691B2 (en) | 2021-10-26 |
US20210390252A1 (en) | 2021-12-16 |
WO2014200798A8 (en) | 2015-08-13 |
US11699031B2 (en) | 2023-07-11 |
EP3008575B1 (en) | 2019-07-24 |
Similar Documents
Publication | Publication Date | Title |
---|---|---|
US11157691B2 (en) | Natural quick function gestures | |
US10732825B2 (en) | Natural input for spreadsheet actions | |
US10366629B2 (en) | Problem solver steps user interface | |
US10558341B2 (en) | Unified system for bimanual interactions on flexible representations of content | |
US8525839B2 (en) | Device, method, and graphical user interface for providing digital content products | |
US9164673B2 (en) | Location-dependent drag and drop UI | |
US11256388B2 (en) | Merged experience of reading and editing with seamless transition | |
US10635291B2 (en) | Thumb and pen interaction on a mobile device | |
US20150052465A1 (en) | Feedback for Lasso Selection | |
US10572137B2 (en) | Intuitive document navigation with interactive content elements | |
US10209864B2 (en) | UI differentiation between delete and clear | |
US20150169532A1 (en) | Interaction with Spreadsheet Application Function Tokens | |
JP2015530687A (en) | User interface for content selection and extended content selection | |
CN108780443B (en) | Intuitive selection of digital stroke groups | |
US20190258393A1 (en) | Navigating long distances on navigable surfaces | |
KR20100041150A (en) | A method for controlling user interface using multitouch | |
US10459612B2 (en) | Select and move hint |
Legal Events
Date | Code | Title | Description |
---|---|---|---|
AS | Assignment |
Owner name: MICROSOFT CORPORATION, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:RADAKOVITZ, SAM C.;CAMPBELL, JOHN;OTERO, CARLOS AUGUSTO;AND OTHERS;REEL/FRAME:030619/0531 Effective date: 20130614 |
|
AS | Assignment |
Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:034747/0417 Effective date: 20141014 Owner name: MICROSOFT TECHNOLOGY LICENSING, LLC, WASHINGTON Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:039025/0454 Effective date: 20141014 |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: DOCKETED NEW CASE - READY FOR EXAMINATION |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: NON FINAL ACTION MAILED |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: RESPONSE TO NON-FINAL OFFICE ACTION ENTERED AND FORWARDED TO EXAMINER |
|
STPP | Information on status: patent application and granting procedure in general |
Free format text: FINAL REJECTION MAILED |
|
STCB | Information on status: application discontinuation |
Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION |