US20110066933A1 - Value-driven visualization primitives for spreadsheets, tabular data, and advanced spreadsheet visualization - Google Patents

Value-driven visualization primitives for spreadsheets, tabular data, and advanced spreadsheet visualization Download PDF

Info

Publication number
US20110066933A1
US20110066933A1 US12/875,115 US87511510A US2011066933A1 US 20110066933 A1 US20110066933 A1 US 20110066933A1 US 87511510 A US87511510 A US 87511510A US 2011066933 A1 US2011066933 A1 US 2011066933A1
Authority
US
United States
Prior art keywords
data
spreadsheet
visualization
cell
displayed
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Abandoned
Application number
US12/875,115
Inventor
Lester F. Ludwig
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NRI R&D Patent Licensing LLC
Original Assignee
Individual
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Individual filed Critical Individual
Priority to US12/875,115 priority Critical patent/US20110066933A1/en
Publication of US20110066933A1 publication Critical patent/US20110066933A1/en
Priority to US13/669,431 priority patent/US9665554B2/en
Priority to US13/670,411 priority patent/US20130132811A1/en
Priority to US13/670,346 priority patent/US8826113B2/en
Priority to US13/673,904 priority patent/US20130174004A1/en
Priority to US13/673,732 priority patent/US8826114B2/en
Priority to US15/601,892 priority patent/US20170262425A1/en
Assigned to NRI R&D PATENT LICENSING, LLC reassignment NRI R&D PATENT LICENSING, LLC ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: LUDWIG, LESTER F
Abandoned legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/166—Editing, e.g. inserting or deleting
    • G06F40/177—Editing, e.g. inserting or deleting of tables; using ruled lines
    • G06F40/18—Editing, e.g. inserting or deleting of tables; using ruled lines of spreadsheets
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F40/00—Handling natural language data
    • G06F40/10—Text processing
    • G06F40/103—Formatting, i.e. changing of presentation of documents

Definitions

  • the invention relates to data visualization and in particular its implementation by and application to conventional spreadsheets and displays of tabular data.
  • Interactive electronic spreadsheets such as the 1979 VisiCalc numerical spreadsheet (http://www.danbricklin.com/visicalc.htm), Supercalc numerical spreadsheet (1980), Microsoft MultiplanTM 91982), Lotus 1-2-3TM (1983), the AppleWorksTM spreadsheet module (1984), Microsoft ExcelTM (1985), OpenOffice.orgTM Calc, Google's Google Spreadsheets, etc., present a tabular presentation of data and calculation formula flow in arrangements that have proven to be very useful for data analysis.
  • Interactive electronic spreadsheets typically include plotting routines that create graphical representation of selected data points. These plots can be used to provide visual representations of data and mathematical functions. In a sense, these interactive electronic spreadsheet functions provide a form of data visualization based on data in the interactive electronic spreadsheet.
  • interactive electronic spreadsheets present numerical, textual, and symbolic data to be presented in a tabular array of cells, with the numerical, textual, and symbolic data within the cells provided in various font styles and colors, and with various background colors and border styles of the associated cell.
  • interactive electronic spreadsheet functions known as “conditional formatting” these font styles and colors, cell background colors, and cell border styles can be controlled by values of data according to hand-specified conditional tests performed on data within one or more specified cells.
  • hand-specified conditional tests are entered through interactive electronic spreadsheet GUIs (Graphical User Interfaces), VBA (Visual Basic for Applications), APIs (Application programmer interfaces), etc.
  • These interactive electronic spreadsheet functions provide another form of data visualization based on data in the interactive electronic spreadsheet.
  • the invention provides for use of numerical values directly from tabular data or as calculated by hidden mathematical function, compositions of mathematical functions, or traditional spreadsheet formulas to easily provide rich detailed control of useful parameterized visual effects rendered in the context of conventional interactive electronic spreadsheet and tabular data display.
  • the visual effects can include variation of background color or texture or border color, thickness, grouping scope of cells enveloping characters conveying data, as well as font color, type, embellishment, size, format, location, decimal places, or supplemental symbols symbolic or image element rendered within the cell of characters conveying data.
  • the invention provides for data-driven 3D plots rendered in the context of or projection from interactive electronic spreadsheet or tabular data.
  • the invention provides for these visualization operations to be generated by an algorithm directly or indirectly in communication with data or parts of a program rendering an interactive electronic spreadsheet or visually displayed data table.
  • the invention also provides for various optional additional functions, for example:
  • the invention provides for a method to visually enhance the display of tabular data of an electronic spreadsheet comprising cells arranged in rows and columns. At least one data value is used to compute at least one visual parameter, and each visual parameter is a number within a range of numbers and is determined according to a numerically calculated mathematical function. A visual effect of the at least one data value is controlled according to the at least one visual parameter, and the visual effect is applied to the display of the at least one data value to visually enhance the display of tabular data of the electronic spreadsheet.
  • the visual effects can be automatically varied over a range responsive to data values or formulas according to mathematical functions rather than conditional tests found in interactive electronic spreadsheet products.
  • the visual parameters of a data or cell presentation attribute may be varied over a pre-defined range (for example, between 0 and 1, between 0 and 100, between ⁇ 1 and +1, etc).
  • Various attributes of the font in which the tabular data is displayed can be controlled. For example, one or more of the font color, the font type or style, font embellishments, and the font size may be controlled by the visual parameters. If the tabular data is numeric data, one or more of the number format, the number of decimal places, and the method and degree of rounding may also be controlled by the visual parameters.
  • the background of the cell in which tabular data is displayed may also be visually enhanced by the visual parameters.
  • one or more of the background color and background texture may be controlled by the visual parameters.
  • border color and border line thickness may be controlled by the visual parameters.
  • the border may enclose a single spreadsheet cell displaying tabular data or may enclose a plurality of contiguous cells displaying tabular data.
  • a symbol may be appended to the displayed tabular data, or an image or a data symbol may be independently displayed instead of the tabular data in a spreadsheet cell.
  • FIG. 1 provides a demography of ways in which data visualization and spreadsheets can be integrated together.
  • FIG. 2A depicts a region of a spreadsheet or table of tabular data.
  • FIG. 2B depicts a cell and its immediate neighborhood within a spreadsheet or table of tabular data.
  • FIG. 3 depicts of data and cell presentation attributes serving as candidates for visualization parameters within a spreadsheet display.
  • FIGS. 4A-4E illustrate several application settings where the invention is well-suited to add value.
  • FIG. 5 illustrates an arrangement of selected more general aspects of the invention that are not restricted to spreadsheet visualization.
  • FIG. 6 illustrates an adaptation of the arrangement of FIG. 5 to an environment supporting collaboration features.
  • FIG. 7 depicts a topological interconnection of data flow paths linking various elements depicted in FIG. 5 .
  • FIGS. 8A-8C depict approaches for mapping a data value lying within a pre-defined range to a value within a pre-defined range for a parameterized data or cell presentation attribute.
  • FIG. 9 depicts a more general view and organization of pre-visualization operations provided for by the invention.
  • FIG. 10 depicts an embodiment wherein a selected metaphor is used to automatically generate parameter assignments and graphics rendering operations.
  • FIG. 11A illustrates a user interface that permits users to select the geometric element boundary of a captured or rendered tabular matrix of tabular matrix data elements or cells within a spreadsheet.
  • FIG. 11B illustrates a user interface that permits users to select array operations to be performed.
  • FIGS. 12A-12G depict transformations of graphics objects via array operations as provided for by the invention.
  • FIG. 13 depicts a spreadsheet architecture and information flow as it might be used in a data visualization setting.
  • FIGS. 14A-14D depict various types of architectural approaches for combining and integrating spreadsheet programs, visualization operations provided by the invention, and a data set used by each.
  • FIG. 15 depicts an embodiment of an augmentation of a spreadsheet program with additional spreadsheet features as provided for by the invention.
  • FIGS. 16A-16B depict ways in which one or more user interface input devices (such as a mouse, trackball, touchpad, etc.) can be managed so as to control graphical user interfaces of a plural software application in the context of the invention.
  • user interface input devices such as a mouse, trackball, touchpad, etc.
  • FIGS. 17A-17F illustrate the six independently adjustable degrees of freedom of touch from a single finger that can be simultaneously measured by the HDTP technology.
  • FIG. 18 suggests general ways in which two or more of these independently adjustable degrees of freedom can be adjusted at once.
  • FIG. 19 demonstrates a few two-finger multi-touch postures and/or gestures from the many that can be readily recognized by HTDP technology.
  • FIG. 20 shows how raw measurements of the six quantities of FIGS. 17A-17F , together with shape recognition for distinguishing contact with various parts of the hand and the touchpad, can be used to create a rich information flux of parameters, rates, and symbols.
  • FIG. 21 shows an approach for incorporating posture recognition, gesture recognition, state machines, and parsers to create an even richer human/machine tactile interface system capable of incorporating syntax and grammars.
  • FIG. 22 depicts a user interface input arrangement incorporating one or more HDTPs that provides user interface input event and quantity routing of the type described earlier in conjunction with FIGS. 16A-16B .
  • FIGS. 23A-23L depict a number of arrangements employing the HDTP technology suitable for use with visualization environments.
  • FIGS. 24A and 24B illustrate examples of scroll-wheel mice provided with an additional scroll-wheel.
  • FIGS. 25A-25C illustrate a single trackball incorporated into the back of a conventional computer mouse.
  • FIGS. 26A-26C illustrate two trackballs incorporated into the back of a conventional computer mouse.
  • FIG. 27 depicts an arrangement wherein the HDTP or alternatives such as advanced mice can interface with browser-based applications via a browser plug-in.
  • FIG. 28 depicts an implementation for rendering visualization in a browser.
  • FIG. 29 depicts a web-based implementation of a visualization environment leveraging browser arrangements such as that depicted in FIGS. 27-28 , their variations, alternatives, etc.
  • FIG. 30A depicts a client-side collaboration implementation wherein a general purpose collaboration tool can be used to share the visualization environment session running on one computer with one or more other computers.
  • FIG. 30B depicts a server-side collaboration implementation wherein a general purpose or specialized server-side application sharing environment is used to share a server-based interactive data visualization application.
  • FIG. 30C depicts a server-side collaboration implementation comprising a server-based multi-user interactive data visualization application.
  • FIG. 31 illustrates a general framework for data sonification wherein a parameterized metaphor is used to manage parameter assignment and sound rendering and a parameterized data indexing operation.
  • FIG. 32 depicts an arrangement wherein the hue of a color can be varied through natural colors of red through violet for the bulk of the [0,1] range and a range of artificial colors, such as the purple through magenta continuum, are appended for the remaining portion of the [0,1] range.
  • FIG. 33 illustrates an arrangement wherein line width, line dashing period, line dashing duty-cycle, and line dashing sub-duty-cycle can be determined by a uniform parameter in the range of [0,1].
  • FIG. 34 illustrates an arrangement wherein stipple line width, stipple gap, and stipple angle can be determined by a uniform parameter in the range of [0,1].
  • FIG. 35 depicts an arrangement wherein at least one border rendering option is to be determined by a uniform parameter in the range of [0,1].
  • the border can be the border of a spreadsheet cell as depicted, or can be the border of another visually rendered graphical object.
  • FIG. 36A depicts an arrangement wherein a boldness parameter is under the control of uniform parameters in the range of [0,1].
  • FIG. 36B depicts an arrangement wherein an italics angle parameter is under the control of uniform parameters in the range of [0,1].
  • FIG. 36C depicts an arrangement wherein an font family interpolation parameter is under the control of uniform parameters in the range of [0,1].
  • FIG. 37A depicts an arrangement wherein exemplary vertical and horizontal dilation/contract parameters are each respectively under the control of uniform parameters in the range of [0,1].
  • FIG. 37B depicts an arrangement wherein font underline line width and font underline separation gap parameters are each respectively under the control of uniform parameters in the range of [0,1].
  • FIG. 38 depicts an attribute control panel wherein there are pre-selected values and a value which can be customized.
  • FIG. 39 depicts an arrangement wherein a change in color of a font can be specified for a cell, a group of cells, or globally across an entire worksheet.
  • FIG. 40 depicts an interface that determines a font type of the specified cell(s).
  • FIG. 41 depicts an interface that determines the embellishment of a font in rendering some data element within the spreadsheet or data table.
  • FIG. 42 depicts an interface to control the size of a font to be displayed.
  • FIG. 43 depicts an wherein data in a cells in a selected column can have symbol appended to data within the cells of the column.
  • FIG. 44 depicts an example use of controlling the background texture or stipples of cells as a function of data values in a spreadsheet or other form of tabular data.
  • FIG. 45 depicts an interface that can be used to specify the background texture or stipples to be applied to a cell or a group of cells.
  • FIG. 46 depicts an interface that can be used to specify the position of text being displayed in a cell.
  • FIG. 47 depicts an interface that can be used to specify the rotation of text being displayed combined with the vertical or horizontal placement of the text.
  • FIG. 48 depicts a visualization wherein an attribute of a column area of data elements is varied as a group.
  • FIG. 49 depicts a visualization wherein the shading of a of a row area of data elements is varied as a group as a function of a data value and also wherein the font boldness of a column area of data elements is varied as a group as a function of another data value.
  • FIG. 50 illustrates a priority job queue model tabular-data visualization wherein the row can automatically reorder among themselves as underlying priority vales change.
  • FIG. 51A illustrates a basic map-based visualization example generated from previous data.
  • FIG. 51B illustrates the visualization of FIG. 51A augmented with additional spatially located symbols or parameterized glyphs.
  • FIG. 52 depicts a 2D-surface representing a 3D data plot of at least 3-dimensional data rendered above a planar array of associated tabular data or spreadsheet, these separated by a vertical gap within a 3D visual field as provided for by the invention.
  • FIG. 53 depicts with dashed lines a few ways in which the observation viewpoint can be moved with respect to these abstract objects.
  • FIG. 54 depicts a translation and rotation of an exemplary 3D data plot within the 3D visual field.
  • FIG. 55A depicts an arrangement wherein data plotted in a 2D-surface representing a 3D data plot within the 3D visual field can directly echo data displayed in a planar array of associated tabular data or spreadsheet and/or can originate from another set of tabular data or spreadsheet region and/or can originate from other data.
  • FIG. 55B depicts an arrangement wherein processing of data via mathematical transformations, statistical processing, signal processing, etc. is made prior to creation of the 2D-surface representing at least 3-dimensional data.
  • FIG. 56 illustrates a height measuring visual in its use with 2D-surface representing at least 3-dimensional data and useful for representing at least 4-dimensional data.
  • FIG. 57 depicts a slicing function here used to provide a (planar-slice) level set curve.
  • FIG. 58A illustrates two 2D-surfaces, each representing at least 3-dimensional data.
  • FIG. 58B illustrates the interactively created intersection of the two exemplary 2D-surfaces and the resultant intersection curve.
  • FIG. 59 depicts an arrangement wherein a curve can be generated by tabular data and be suspended over tabular data used to create it or otherwise associated with the curve, and can be visually linked to a geometric rendering of tabular data values.
  • FIG. 60 depicts an arrangement wherein a curve can be given a thickness and or color under the control of data values.
  • FIG. 61 depicts an arrangement wherein the thickness of a curve can be varied as a function of one or more data values.
  • FIG. 62 depicts a curve intersection tool that can be used as a numerical solution operator for plotted, interpolated, and processed data.
  • FIG. 1 provides an example of ways in which data visualization and spreadsheets can be integrated.
  • the present invention provides several additional data visualization capabilities to traditional spreadsheets.
  • spreadsheets present data in a tabular form leveraging row, column, and sheet organization.
  • Traditional spreadsheets typically include plotting routines that create graphical representation of selected data points. These plots can be used to provide visual representations of data and mathematical functions. These spreadsheet functions provide a base-level form of data visualization based on data in the spreadsheet, as represented in rows A and B of FIG. 1 .
  • the third dimension can be used to render color-coded line and surface plots, and the resulting 3D graphics can be viewed from various virtual observation points for inspection. This is represented in row H of FIG. 1 .
  • Another aspect of the invention focuses on adapting and expanding this capability.
  • Traditional spreadsheets also present numerical, textual, and symbolic data in a tabular array of cells, with the numerical, textual, and/or symbolic data within the cells provided in various font styles and colors, and with various background colors and border styles of the associated cell.
  • Using spreadsheet functions known as “conditional formatting” these font styles and colors, cell background colors, and cell border styles can be controlled by values of data according to hand-specified (through spreadsheet GUI, VBA, APIs, etc.) conditional tests performed on data within one or more specified cells.
  • These spreadsheet functions provide another form of data visualization based on data in the spreadsheet.
  • One aspect of the invention focuses on adapting and expanding this capability. This is represented in row I of FIG. 1 .
  • FIG. 2A illustrates a region 200 of a traditional spreadsheet or table of tabular data.
  • Such spreadsheet or tabular data can be created by desktop applications such as Microsoft Excel or online applications such as Google Spreadsheet.
  • Aspects of the present invention can be implemented via an add-on package of existing software or additional plug-in script on any online applications.
  • the minimum block that makes up this whole region is referred to as a cell, and cells can be merged to compose a cell with bigger area expanding multiple rows, columns, or both.
  • Region 200 of an example spreadsheet contains four columns 281 - 284 and five rows 291 - 295 .
  • Cell 230 is confined within one row 293 and on column 283 and accordingly is defined by a left border 213 , a right border 214 , an upper border 222 , and a lower border 223 .
  • Each of the four columns 281 - 284 represents a different categorical subset of data (for example, an item, region, revenue area, inventory level, etc.).
  • another exemplary cell 240 in the left-most column comprises the height of three rows 293 , 294 , 295 .
  • the border of cell 230 is bolded with respect to the other cells depicted so as to call this cell out from the others. This can be rendered for any of a number of reasons, such as signifying the selection of the cell via a pointing device of a user interface, selection resulting from a query, the signifying of special status or conditions, etc.
  • the data displayed in cell 230 is a decimal number formatted as a national financial currency and with a thousands-separating comma.
  • the data displayed in cell 260 is an unformatted non-decimal number without a thousands-separating comma.
  • the data displayed in cell 250 comprises a text string
  • the data displayed in cell 240 comprises an alphabetic text string and an unformatted non-decimal number. In the latter, the unformatted non-decimal number can in fact be part of the preceding text string, or can be a calculated, retrieved, or can be a fixed number composed with a calculated, retrieved, or fixed text string.
  • the data displayed in the foreground of a cell can contain numbers, text, symbols, or a combination of these, and this data or portions of it can be further “emphasized” with attributes.
  • attributes include choice of the font family, font size, bolding, italicizing, stretching, color, transparency with respect to the cell background, position within the cell, various types of animations (modulations of aforementioned attributes, blinking, shimmering, vibrating, rotating, etc.)
  • Each individual font attribute can be fixed or determined by data values via conditional formatting.
  • the data displayed in the foreground of a cell can be fixed, replicated, or result from processing of other data by (no displayed) formulas and/or functions.
  • FIG. 2B depicts additional detail of cell 230 and its immediate neighborhood.
  • the dividing border lines 213 , 214 , 222 , 223 of FIG. 2A can be segmented into a plurality of sections—for example dividing border line 213 of FIG. 2A can be segmented into at least adjacent sections 213 a , 213 b , 213 c ; similarly dividing border line 214 of FIG. 2A can be segmented into at least adjacent sections 214 a , 214 b , 214 c , dividing border line 222 of FIG.
  • the exemplary cell 230 is visually defined as the geometric area enclosed by upper, lower, left, and right dividing border lines 213 b , 214 b , 222 b , and 223 b —these border line sections can be accented via thickening, darkening, coloring, etc.
  • Such border line section accenting can be fixed or determined by data values via conditional formatting.
  • the background 245 of each cell can be used to display a background color, shading pattern, image, etc. which can be fixed or determined by data values via conditional formatting.
  • the background can vary from cell to cell.
  • the displayed data in this particular case is, again, a decimal 241 number formatted as a national financial currency via a signifying currency symbol 242 and further employing a thousands-separating comma 243 .
  • FIG. 3 depicts of data and cell presentation attributes serving as candidates for visualization parameters within a spreadsheet display. Some of these attributes can be controllable under conditional formatting in traditional spreadsheets while others may not be controllable under conditional formatting in traditional spreadsheets.
  • a data or cell presentation attribute provided a pre-defined range of variability and is controlled by a parameter that can vary over a pre-defined range (for example, between 0 and 1, between 0 and 100, between ⁇ 1 and +1, etc). This parameter-controlled variability of a data or cell presentation attribute allows that presentation attribute to be used as a visualization parameter.
  • the invention provides new visualization capabilities to spreadsheet presentation, spreadsheet formatting, spreadsheet data handling, and spreadsheet interactive use in ways well-suited for use as a numeric-intensive data visualization tool.
  • Many of these spreadsheet features can be implemented with simple vector graphics rendering operations or easily managed mixtures of vector and raster graphics, permitting:
  • the invention provides for a value of a data element, or result of a formula calculated from one or more data elements in a spreadsheet or data table, to be used to determine spreadsheet presentation attributes, such as one or more of the following examples:
  • the invention also provides for one or more of the following capabilities:
  • the invention provides for the control of data and cell presentation attributes through use of a uniform parameterization framework.
  • pre-visualization operations such as scaling, translation, filtering, array (matrix, tensor) operations, nonlinear warping, etc. to be employed in a modular, cascadable fashion independent of the particular choice of data and cell presentation attributes.
  • the invention further provides for pre-visualization operations to themselves have parameters that can be adjusted in real time and/or be stored in files for recall.
  • the invention further provides for a network of pre-visualization operations to be stored in files for recall.
  • the invention additionally provides for advanced user interface devices, particularly those providing large numbers of simultaneously-adjustable interactive control parameters, to be used to control the viewing, presentation, and creation of the visualization as well as controlling the underlying data source such as databases, statistical packages, simulations, etc.
  • FIGS. 4 a - 4 e illustrate several example application settings where the invention is well-suited to add value.
  • a spreadsheet is initially used as a tool of choice or convenience 401 , and at some point some simple data visualizations are incorporated 402 .
  • Such capabilities can be within the range provided by available spreadsheet software products 400 .
  • additional degrees of visualization are incorporated 403 , potentially involving a need for a rich suite of visualization capabilities to create data visualization of substantial richness 404 .
  • FIG. 4A a spreadsheet is initially used as a tool of choice or convenience 401 , and at some point some simple data visualizations are incorporated 402 .
  • Such capabilities can be within the range provided by available spreadsheet software products 400 .
  • additional degrees of visualization are incorporated 403 , potentially involving a need for a rich suite of visualization capabilities to create data visualization of substantial richness 404 .
  • a spreadsheet is chosen as a vehicle or venue for visualization as it matches one dominating user needs (for example, original data already in or readily imported into a spreadsheet, quick assembly, tabular nature of data, tabular nature of formula replication, etc.).
  • a spreadsheet is chosen as a vehicle or venue for visualization as it is comfortable or intuitive for a particular audience.
  • a spreadsheet is chosen as a vehicle or venue for visualization as other aspects of the invention allow for it to be rendered in a web-based application.
  • a spreadsheet is chosen as a vehicle or venue for visualization as other aspects of the invention allow for it to be rendered in a collaborative context.
  • the spreadsheet visualization aspects of the present invention are a special case of other more general aspects of the invention including:
  • FIG. 5 illustrates an arrangement of these more general aspects of the invention. Implicit in FIG. 5 are more general aspects of the invention, not restricted to spreadsheet visualization, that support visual rendering in a browser window and as a web application. As shown in the figure, graphical user interfaces (GUIs) are provided for including one or more relating to data sources and one or more relating to visualization processes. Additionally, the invention also provides for multifunction GUIs that provide roles involving both data source matters and visualization processes. The overall collection of user interfaces provides the following capabilities:
  • FIG. 6 illustrates an adaptation of the arrangement of FIG. 5 to a collaboration environment supporting collaboration features such as one or more of file exchange, real-time or stored display sharing, real-time control sharing, real-time or stored collaborative annotation, and archives of collaboration sessions and transactions.
  • the invention provides for uniform parameterizations of selected or all visualization presentation parameters. This allows pre-visualization operations, such as scaling, translation, filtering, array (matrix, tensor) operations, nonlinear warping, etc. to be employed in a modular, cascadable fashion independent of the particular choice of data and cell presentation attributes.
  • pre-visualization operations such as scaling, translation, filtering, array (matrix, tensor) operations, nonlinear warping, etc.
  • the invention provides for the use of data flow paths to link arbitrary data sources with arbitrary data destinations via arbitrary topologies. This allows the selection and/or fusion of data sources, their interconnection with selected signal processing, statistical processing, pre-visualization operations, and visualization parameters (such as, among other examples, the cell and data presentation parameters of the spreadsheet visualization described earlier and in more detail to follow).
  • FIG. 7 depicts an exemplary topological interconnection of data flow paths linking various elements depicted in FIG. 5 .
  • functions such as data re-indexing, statistical processing, and signal processing can be provided as the data sources depicted in FIG. 5 or as the pre-visualization functions depicted in FIG. 5 .
  • numerical simulations for example (but not limited to) those rendered by the computer running a spreadsheet program, a high-performance or other computer, can serve as the data sources depicted in FIG. 5 .
  • Certain pre-visualization functions for example linear predictors, can in an embodiment be regarded as a numerical simulation.
  • calculations in the spreadsheet can provide a form of numerical simulation prior to visualization.
  • the invention provides for some or all of the data flow paths (such as depicted in the example of FIG. 7 ) to be specified in any convenient way, for example graphically via an interactive GUI or via a character-based language (interconnection, specification, and/or data-flow, etc.).
  • a GUI can permit the rendering of a graphic similar to that of FIG. 7 .
  • a GUI can permit creation and customization of instances of functional blocks such as the ones depicted in FIG. 7 from a library, menu, and/or graphical pallet.
  • a GUI can be used to create and link these customized instances of functional blocks, via link-by-link “drawing,” with a data path topology such as the ones depicted in FIG. 7 .
  • Various types of user interfaces may be used to create configurations such as that of FIG. 7 and set the various parameters within it.
  • the selection of function blocks and the general connectivity among them can be specified using a drawing tool and a palette of function blocks, and clicking on each function block would cause dialog windows to appear that can be used for setting parameters.
  • the parameters can be set by means of typed-in values, sliders, mouse manipulation, or the advanced high-dimension user interface devices to be described later in the specification. More generally, a wide range of approaches, styles, and forms of configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • FIGS. 8A-8C depict an approach for mapping a data value lying within a pre-defined range to a value within a pre-defined range for a parameterized data or cell presentation attribute.
  • the input data range must be at least scaled and shifted to match the pre-defined range for a parameterized presentation attribute.
  • This (linear or affine transformation) arrangement is depicted in FIG. 8A .
  • a library of fixed or adjustable nonlinearities are provided which are such that the input and output of the nonlinearity both match the pre-defined range for a parameterized presentation attribute. This arrangement is depicted in FIG. 8B .
  • the warping effect is provided with additional flexibility by allowing pre-scaling and pre-shifting prior to applying a selected nonlinearity and subjecting the outcome of the nonlinear warping to post-scaling and/or post-shifting operations in order to match the resulting range to the pre-defined range for a parameterized presentation attribute.
  • This arrangement is depicted in FIG. 8C .
  • Features and parameters of the above operations can be selected via user interface dialog windows. General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • FIG. 9 depicts a more general view and organization of pre-visualization operations provided for by the invention.
  • available pre-visualization operations include:
  • the invention also provides for the inclusion of statistical operations and statistical processing functions and for the linking to external programs to perform other types of pre-visualization operations.
  • Features and parameters of these can be selected via user interface dialog windows.
  • General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • the invention also provides for external programs to be added to the collection of available pre-visualization operations.
  • the invention additionally provides for the inclusion and use of visual metaphors to simplify visualization setup and user interaction for data exploration.
  • FIG. 10 depicts an exemplary embodiment wherein a selected metaphor is used to automatically generate parameter assignments and graphics rendering operations.
  • the invention provides for metaphors to control other aspects of the visualization and pre-visualization operations.
  • the invention provides for a metaphor to base its operations on characteristics of a data set being visualized, previously visualized, and anticipated to be visualized.
  • the invention additionally provides for metaphors to be selected and controlled by user interaction, data values, or other means.
  • Features and parameters of the above operations can be selected via user interface dialog windows. General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • the invention also provides for array (vector, matrix, tensor) operations such as (vector, matrix, tensor) linear combinations, (vector, matrix, tensor) multiplication, scalar multiplication, finding (matrix, tensor) determinants, finding (matrix, tensor) inverses and psuedoinverses, row reduction, factorization, change of basis, or calculation of an eigensystem (eigenvalues, eigenvectors, eigentensors).
  • array (vector, matrix, tensor) operations such as (vector, matrix, tensor) linear combinations, (vector, matrix, tensor) multiplication, scalar multiplication, finding (matrix, tensor) determinants, finding (matrix, tensor) inverses and psuedoinverses, row reduction, factorization, change of basis, or calculation of an eigensystem (eigenvalues, eigenve
  • FIG. 11A illustrates an example of a user interface that permits users to select the geometric element boundary of a captured or rendered tabular matrix data elements or cells within a spreadsheet. Other features and parameters of the above operations can be selected via user interface dialog windows. General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • FIG. 11B illustrates an exemplary user interface that permits users to select array operations to be performed on selected array data, for example as may be selected by the tool of FIG. 11A .
  • Other features and parameters of the above operations can be selected via user interface dialog windows.
  • General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • FIG. 12A illustrates a figure being resized, rotated, and shifted through array-based scaling, rotating, and translating linear transformations.
  • FIG. 12 c illustrates processes of figure being rescaled vertically or horizontally.
  • FIG. 12D illustrates a transformation process acting on 2D graphics objects such as a rectangle-bounded horizontal line array and a graphic.
  • the invention also provides for embodiments to extend these and related systems and methods to transform 3D-graphics via scalings, rotations, stretch, etc.
  • FIG. 12E illustrates a matrix and/or tensor transformation process and how it changes a 3D graphics object into a 2D object via dimension dropping.
  • FIG. 12F illustrates a 3D graphics object rendered of a 2D surface created by a vertical line rotated around the y-axis.
  • FIG. 12G illustrates a 3D graphics object rendered by perspective projection defined by a point.
  • the invention also provides for 3D graphics generation from 3D vector draw-lists provided by other programs, spreadsheet data, and/or the user.
  • the invention also provides for 3D graphics generation from equations provided by the user.
  • the invention further provides for pre-visualization operations to themselves have parameters that can be adjusted in real time and/or be stored in files for recall.
  • the invention further provides for a network of pre-visualization operations to be stored in files for recall.
  • the invention provides implementation of at least some of the invention's spreadsheet presentation features through use of APIs of existing spreadsheet products.
  • the invention provides implementation of at least some of the invention's spreadsheet presentation features with relatively small augmentation and modification of existing product software by product manufacturers.
  • FIG. 13 depicts an example of spreadsheet architecture (right side) and an information flow as it might be used in a data visualization setting.
  • One of or some combination of entered data, measured data, and/or computed data can be stored as stored data or stored in a queriable data base. In some situations live data feeds can also provide data.
  • One or some combination of these data sources can be used to provide a data set used by the spreadsheet program.
  • the spreadsheet program internally comprises at least a GUI component, a calculation compound, and a visual rendering component.
  • FIGS. 14A-14D depicts various types of exemplary relationships and information flows among a system or method for rendering a conventional spreadsheet or table of tabular data, a system or method for rendering the visualization operations of the invention, and a data set used by each.
  • Visualization and pre-visualization operations can be introduced between the data and the spreadsheet program as depicted in FIG. 14A .
  • visualization and pre-visualization operations can obtain data from the spreadsheet program, process data, and return the generated visual operation back to the spreadsheet program as depicted in FIG. 14B .
  • An example of this is using an API or the use of VBA in conjunction with Microsoft Excel.
  • Visualization and pre-visualization operations can also be implemented as parallel operations to the spreadsheet program.
  • Visualization and pre-visualization operations can be configured separately as an independent operation(s) with direct access to data (i.e., able to obtain data and send the result to the data without having to go through the spreadsheet program), as shown in FIG. 11C .
  • One advantage of this is that relevant categories of data can be extracted to generate complicated visualizations without the data being arranged into table form.
  • Visualization and pre-visualization operations can be part of a spreadsheet program without direct access to the data, as depicted in FIG. 14D .
  • FIG. 15 depicts an exemplary embodiment of an augmentation of a spreadsheet program with additional spreadsheet features as provided for by the invention.
  • a data set is subjected to data operations so as to produce modified data.
  • the data operations can be controlled by the spreadsheet GUI and spreadsheet calculation as well as by the visualization GUI and Visualization calculations.
  • FIGS. 16A-16B depict ways in which one or more user interface input devices (such as a mouse, trackball, touchpad, etc.) can be managed to control graphical user interfaces of a plural software application in the context of the invention.
  • the windowing system directs the input events from a single user interface input device to various applications, for example, a Database GUI, a Simulation GUI, a Spreadsheet GUI, and a Visualization GUI.
  • FIG. 16B depicts an alternative implementation wherein a Visualization Environment Input Router/Parser accepts input events from one or more user interface input devices and directs these events to each of the various applications.
  • the Visualization Environment Input Router/Parser can direct events from multiple user interface input devices to different inputs to the same application.
  • the Visualization Environment Input Router/Parser can split events originating from an individual user interface device and separately route these to inputs of separate applications.
  • a Visualization Environment Input Router/Parser works together with the Window System to route input events.
  • HDTP High Dimensional Touch Pad
  • a tactile sensor array pressure, proximity, etc.
  • real-time image and mathematical processing to provide a powerful user input device with both a higher number of interactive simultaneously-adjustable parameters and a rich range of syntactic and metaphorical capabilities well-suited to use with interactive visualization.
  • the HDTP technology can be readily implemented as a touchscreen through use of, for example, inexpensive transparent capacitive proximity-sensor arrays.
  • the present invention provides for the incorporation and use of the HDTP and other metaphor-rich user interface input devices into visualization environments. Attention is now directed to incorporation of the HDTP as part of a visualization environment user interface and then brief attention is directed to alternative use of a few alternative metaphor-rich enhanced user interface technology approaches.
  • FIGS. 17A-17F illustrate the six independently adjustable degrees of freedom of touch from a single finger that can be simultaneously measured by the HDTP technology.
  • the depiction in these figures is from the side of the touchpad.
  • FIGS. 17A-17C show actions of positional change (amounting to applied pressure in the case of FIG. 17C ) while FIGS. 17D-17F show actions of angular change.
  • Each of these can be used to control a user interface parameter, allowing the touch of a single fingertip to control up to six simultaneously-adjustable quantities in an interactive user interface.
  • FIG. 18 suggests general ways in which two or more of these six independently adjustable degrees of freedom 1811 - 1816 can be adjusted at once for a finger 1800 in contact with an HDTP sensor surface 1801 . More advanced implementations of the HTDP provide for multi-touch capabilities far more sophisticated than those popularized by the Apple iPhone, NYU, and others.
  • FIG. 19 demonstrates a few exemplary two-finger multi-touch postures and/or gestures from the hundreds that can be readily recognized by HTDP technology.
  • the two fingers may be spread apart somewhat, as in the cases 1901 - 1903 and 1931 - 1933 , or may be brought together, as in the cases 1911 - 1913 and 1941 - 1943 .
  • the separation of the two fingers adds an additional parameter that can be controlled by the user, and the yaw (pivot) parameter may be varied independently from the newly introduced finger separation parameter.
  • the hand can pivot clockwise 1902 , 1912 , or counter-clockwise 1903 , 1913 from the corresponding neutral positions 1901 , 1911 .
  • the pitch of the pair of fingers can be raised 1932 , 1942 or lowered 1933 , 1943 with respect to the corresponding neutral positions 1931 , 1941 , while independently from these motions the two fingers are spread apart 1931 - 1933 or brought together 1941 - 1943 . Note the finger spread can be varied considerably in this two-finger posture.
  • the pair of fingers can be moved as a group through all six degrees of freedom (left-right, forward-back, downward pressure, roll, pitch, yaw), and comfortably allow for two differences between the fingers (differences in downward pressure, and one angle of finger separation spread) and even three differences between the fingers (differences in downward pressure and two coordinates of separation when fingers curl to form independently controlled “x” and “y” components).
  • two-finger postures considered above can readily provide a nine-parameter set relating to the pair of fingers as a separate composite object adjustable within an ergonomically comfortable range.
  • One example nine-parameter set the two-finger postures comprises:
  • HTDP technology can also be configured to recognize and measure postures and/or gestures involving three or more fingers, various parts of the hand, the entire hand, multiple hands, etc.
  • multifinger contact can be used to provide control of up to three additional independently adjustable parameters for each additional finger.
  • FIG. 20 shows an example of how raw measurements of the six quantities of FIGS. 17A-17F , together with shape recognition for distinguishing contact with various parts of the hand and the touchpad, can be used to create a rich information flux of parameters, rates, and symbols.
  • FIG. 21 shows an exemplary approach for incorporating posture recognition, gesture recognition, state machines, and parsers to create an even richer human/machine tactile interface system capable of incorporating syntax and grammars.
  • FIG. 22 depicts an exemplary user interface input arrangement incorporating one or more HDTPs that provides a user interface input event and quantity routing of the type described earlier in conjunction with FIGS. 16 a - 16 b.
  • FIGS. 23A-23L depict a number of arrangements employing the HDTP technology suitable for use with visualization environments.
  • FIG. 23A illustrates an HDTP as a peripheral that can be used with a desktop computer (shown) or laptop (not shown).
  • FIG. 23B depicts an HDTP integrated into a laptop in place of the traditional touchpad pointing device.
  • the HDTP tactile sensor can be a stand-alone component or can be integrated over a display to form a touchscreen.
  • FIG. 23C depicts an HDTP integrated into a desktop computer display to form a touchscreen.
  • FIG. 23D shows the HDTP integrated into a laptop computer display to form a touchscreen.
  • FIG. 23E depicts an HDTP integrated into a cellphone, smartphone, PDA, or other hand-held consumer device.
  • FIG. 23F shows an HDTP integrated into a test instrument, portable service-tracking device, portable service-entry device, field instrument, or other hand-held industrial device.
  • the HDTP tactile sensor can be a stand-alone component or can be integrated over a display to form a touchscreen.
  • FIG. 23G depicts an HDTP touchscreen configuration that can be used in a tablet computer, wall-mount computer monitor, digital television, video conferencing screen, kiosk, etc.
  • FIGS. 23A , 23 C, 23 D, and 23 G, or other sufficiently large tactile sensor implementation of the HDTP more than one hand can be used and individually recognized as such.
  • FIGS. 23H-23K depict various integrations of an HDTP into the back of a conventional computer mouse.
  • the HDTP tactile sensor can be a stand-alone component or can be integrated over a display to form a touchscreen. More than two touchpads can be included, such as the example of FIG. 23L as taught in U.S. Pat. No. 7,557,797.
  • the types of human-machine geometric interaction between the hand and the HDTP facilitate many useful applications within a visualization environment.
  • a few of these include control of visualization observation viewpoint location, orientation of the visualization, and controlling fixed or selectable ensembles of one or more of viewing parameters, visualization rendering parameters, pre-visualization operations parameters, data selection parameters, simulation control parameters, etc.
  • the 6D orientation of a finger can be naturally associated with visualization observation viewpoint location and orientation, location and orientation of the visualization graphics, etc.
  • the 6D orientation of a finger can be naturally associated with a vector field orientation for introducing synthetic measurements in a numerical simulation.
  • at least some aspects of the 6D orientation of a finger can be naturally associated with the orientation of a robotically positioned sensor providing actual measurement data.
  • the 6D orientation of a finger can be naturally associated with an object location and orientation in a numerical simulation.
  • the large number of interactive parameters can be abstractly associated with viewing parameters, visualization rendering parameters, pre-visualization operations parameters, data selection parameters, numeric simulation control parameters, etc.
  • the x and y parameters provided by the HDTP can be used for focus selection and the remaining parameters can be used to control parameters within a selected GUI.
  • the x and y parameters provided by the HDTP can be regarded as specifying a position within an underlying base plane and the roll and pitch angles can be regarded as specifying a position within a superimposed parallel plane.
  • the yaw angle can be regarded as the rotational angle between the base and superimposed planes.
  • the finger pressure can be employed to determine the distance between the base and superimposed planes.
  • the base and superimposed planes are not fixed as being parallel but rather intersect at an angle associated with the yaw angle of the finger.
  • either or both of the two planes can be used to represent an index or indexed data, a position, pair of parameters, etc. of a viewing aspect, visualization rendering aspect, pre-visualization operations, data selection, numeric simulation control, etc.
  • a conventional user interface input device such as a conventional computer mouse, trackball, touchpad, etc.
  • the present invention provides for the use of other user interface input arrangements and devices as alternatives to or in conjunction with one or more HDTPs.
  • the scroll-wheel of a scroll-wheel mouse is used to provide a third simultaneously adjustable user interface parameter.
  • a second or yet more additional scroll-wheels can be added to a conventional scroll-wheel mouse.
  • the resultant collection of scroll-wheels can be relatively positioned in parallel, oriented at orthogonal angles to support a coordinate-metaphor, positioned on the sides of the mouse body, etc.
  • FIGS. 24A and 24B illustrate examples 2410 , 2420 of a conventional scroll-wheel mouse 2401 , 2402 with a traditional up-down scrollwheel 2421 and at least standard mouse buttons 2411 , 2412 provided with an added left-right scroll-wheel 2422 as taught in U.S. Pat. No. 7,557,797.
  • Such an arrangement can employ a connecting cable, or the device can be wireless.
  • a connecting cable or the device can be wireless.
  • FIGS. 24A and 24B as well as the ones described in the next paragraph, plus variations and adaptations, will be referred to as an “advanced mouse” or in the plural “advanced mice.”
  • FIGS. 25A-25C illustrate examples 2510 , 2520 , 2530 wherein a single trackball 2511 , 2521 , 2531 is incorporated into the back of a conventional computer mouse 2510 , 2520 , 2530 with at least standard mouse buttons 2512 - 2513 , 2522 - 2523 , 2532 - 2533 as taught in U.S. Pat. No. 7,557,797.
  • FIGS. 26A-26C illustrate examples where two trackballs 2605 a , 2605 b are incorporated into the back of a conventional computer mouse as taught in U.S. Pat. No. 7,557,797.
  • the trackballs in the arrangements of FIGS. 25A-25C and FIGS. 26A-26C can be the conventional two degree of freedom type (roll left-right, roll away-towards) or can provide three to six degrees of freedom as taught in U.S. Pat. No. 7,557,797; U.S. patent application Ser. No. 10/806,694.
  • Useful exemplary advanced mouse arrangements include the trackball 2605 a - 2605 b , touchpad 2605 c , scrollwheel 2668 mouse of FIG. 26C , the trackball/touchpad mouse of FIG. 26 d , and the multiple slider configuration of FIG. 26E , each as taught in U.S. Pat. No. 7,557,797. Each of these arrangements can employ a connecting cable, or the advanced mouse device can be wireless.
  • the additional parameters provided by the HDTP and the above alternatives are more than the usual number supported by conventional window systems (for example as described in conjunction with FIG. 16A ).
  • additional parameter handling for example such as in the HDTP arrangement of FIG. 22 or its equivalents for the alternative advanced mice as described above, additional arrangements must be made beyond the conventional operating system pointer device handling.
  • the invention provides for the HDTP or alternatives such as the advanced mice described above to interface with a browser via a browser plug-in.
  • This arrangement can be used to capture the additional user interface input parameters and pass these on to an application interfacing to the browser.
  • An example of such an arrangement is depicted in FIG. 27 .
  • the browser can interface with local or web-based applications that drive the visualization and/or control the data source(s), process the data, etc.
  • the browser can be provided with client-side software such as JAVA Script, AJAX, etc. Such an arrangement is particularly useful when combined with browser-based rendering of visualizations, as described in the next section.
  • the invention provides for HDTP parameters to be separated into groups which are individually directed to pointer device interfaces on multiple computers.
  • the invention provides for visualizations to be rendered in a browser. This allows for implementations wherein the browser is used as a viewer.
  • the browser can interface with local or web-based applications that drive the visualization.
  • An exemplary arrangement is depicted in FIG. 28 .
  • the browser can be provided with Simple Vector Graphics (“SVG”) utilities (natively or via an SVG plug-in) to render basic 2D vector and raster graphics.
  • SVG Simple Vector Graphics
  • the browser can be provided with a 3D graphics capability, for example via the Cortona 3D browser plug-in.
  • SVG Simple Vector Graphics
  • FIG. 28 also provides for other plug-ins (represented by the dashed-line box) such as the user interface input device plug-in described earlier.
  • FIG. 29 depicts an exemplary web-based implementation of a visualization environment leveraging browser arrangements such as that depicted in FIGS. 27-28 , their variations, alternatives, etc.
  • the client environment can employ such a browser to access a visualization server environment or visualization-related (databases, live data feeds, simulations, statistical or signal processing, etc.) server environment.
  • the client environment browser can connect over the internet with two or more visualization server environments and/or visualization-related server environments and/or local applications.
  • a visualization or visualization-related server environment can connect over the internet with two or more browsers in the client environment.
  • Other variations and alternative implementations are also possible as is clear to one skilled in the art.
  • a server environment can include web server foundation software such as Apache, IIS, etc.
  • a server environment can also include server-side scripting and dynamic software such as CGI, JAVA/JAVA Script, Python, PHP, Perl, JSP, ASP, etc.
  • CGI web server foundation software
  • JAVA/JAVA Script Python
  • PHP PHP
  • Perl PHP
  • JSP JSP
  • ASP ASP
  • the additional interactively-controlled parameters provided by the HDTP provide more than the usual number supported by conventional browser systems and browser networking environments. This may be addressed in a number of ways.
  • an HDTP interfaces with a browser both in a traditional way and additionally via a browser plug-in.
  • Such an arrangement may be used to capture the additional user interface input parameters and pass these on to an application interfacing to the browser.
  • An example of such an arrangement is depicted in FIG. 30D .
  • an HDTP interfaces with a browser in a traditional way and directs additional GUI parameters though other network channels.
  • Such an arrangement may be used to capture the additional user interface input parameters and pass these on to an application interfacing to the browser.
  • An example of such an arrangement is depicted in FIG. 30E .
  • an HDTP interfaces all parameters to the browser directly. Such an arrangement may be used to capture the additional user interface input parameters and pass these on to an application interfacing to the browser. An example of such an arrangement is depicted in FIG. 30F .
  • the browser may interface with local or web-based applications that drive the visualization and/or control the data source(s), process the data, etc.
  • the browser may be provided with client-side software such as JAVA Script, AJAX, etc.
  • the browser may provide also be configured advanced graphics to be rendered within the browser display environment, allowing the browser to be used as a viewer for data visualizations, advanced animations, etc., leveraging the additional multiple parameter capabilities of the HDTP.
  • the browser may interface with local or web-based applications that drive the advanced graphics.
  • the browser may be provided with Simple Vector Graphics (“SVG”) utilities (natively or via an SVG plug-in) so as to render basic 2D vector and raster graphics.
  • SVG Simple Vector Graphics
  • the browser may be provided with a 3D graphics capability, for example via the Cortona 3D browser plug-in.
  • the invention provides for collaboration implementations and the use of collaboration tools for collaborative use of visualization features of the invention and for the creation of a collaborative visualization environment.
  • FIG. 30A shows a client-side collaboration implementation wherein a general purpose collaboration tool can be used to share the visualization environment session running on one computer with one or more other computers.
  • the visualization environment can be a stand-alone visualization application or can comprise a browser implementation employing, for example, any of the various arrangements described in conjunction with FIGS. 27-29 .
  • the general purpose collaboration tool employed in the arrangement of FIG. 30 can comprise application sharing software (such as Microsoft's NetMeetingTM, IBM/Lotus' SameTimeTM, SM , HP's Shared-X, etc.) or display sharing software.
  • application sharing software such as Microsoft's NetMeetingTM, IBM/Lotus' SameTimeTM, SM , HP's Shared-X, etc.
  • display sharing software such as Microsoft's NetMeetingTM, IBM/Lotus' SameTimeTM, SM , HP's Shared-X, etc.
  • audio and/or video conferencing can also be used to facilitate interpersonal communication among collaborating session participants.
  • FIG. 30B shows server-side collaboration implementation wherein a general purpose or specialized server-side application sharing environment is used to share a server-based interactive data visualization application.
  • FIG. 30C shows a server-side collaboration implementation comprising a server-based multi-user interactive data visualization application.
  • the invention further provides for interfacing with established and advanced data sonification utilities.
  • Data sonification has received considerable attention and analysis but for the most part is fairly simplistic and often contributes little practical value.
  • More advanced data sonification techniques such as the multi-channel data sonification system described in U.S. Patent Application 61/268,856 employed in the environmental GIS (Geographic Information System) process monitoring and modeling system of U.S. Patent Application 61/268,873, provides new opportunities for practical use of data sonification.
  • FIG. 31 illustrates a general framework for data sonification where a parameterized metaphor is used to manage parameter assignment and sound rendering and a parameterized data indexing operation.
  • Data sonification may be applied to the same data used to generate visualization.
  • the data directed to data sonification may be selected by interacting with a rendered visualization via a user interface input device.
  • Data sonification may be provided by a multichannel data sonification system such as the one described in U.S. Patent Application 61/268,856.
  • Data sonification output may be shared using an audio channel of a real-time collaboration system, may be transmitted from a web-based application using an audio channel as provided by a high-fidelity VoIP system, may be shared using an audio channel as provided by a high-fidelity VoIP system, may be produced local to the user computer under the control of a web-based application, or may be produced local to the user computer under the control of a web-based application employing MIDI protocol.
  • the invention provides for visualization rendering parameters to be uniform over a common variational range. This permits uniform handling of visualization rendering parameters and numerical operation compositions. Any of these can be set, varied, or modulated as selected or as advantageous to represent data or information derived from data.
  • the invention provides for visualization rendering uniform parameters in the range of [0,1].
  • the uniform parameter range of [0,1] is used in the examples below, but it is understood that other choices for the uniform range are also possible and are provided for by the invention.
  • the invention provides for at least one color selection option to be determined by a uniform parameter in the range of [0,1].
  • FIG. 32 depicts an exemplary embodiment wherein the hue of a color can be varied through natural colors of red through violet for the bulk of the [0,1] range and a range of artificial colors, such as the purple through magenta continuum, are appended for the remaining portion of the [0,1] range.
  • the invention also provides for other color models, such as RGB, HSB, LUV, LAB, grayscale, etc. to be used with one to three uniform parameters, each in the range of [0,1].
  • Such uniform parameter colors can be assigned one or more of text, symbols, borders, backgrounds, gridlines, glyphs, lines, components of lines, components of stipples, components of gradients, surfaces, and other geometric primitives employed in the visualization of data.
  • Other variations and alternative arrangements are possible as is clear to one skilled in the art.
  • a uniform parameter range of [0,1] is used in the examples above, but it is understood that other choices for the uniform range are also possible and are provided for by the invention.
  • the invention provides for at least one line rendering option to be determined by a uniform parameter in the range of [0,1].
  • FIG. 33 illustrates an example wherein line width is determined by a uniform parameter in the range of [0,1].
  • FIG. 33 also illustrates an example wherein line dashing period is determined by a uniform parameter in the range of [0,1].
  • FIG. 33 further illustrates an example wherein line dashing duty-cycle is determined by a uniform parameter in the range of [0,1].
  • FIG. 33 also illustrates an example wherein line dashing sub-duty-cycle is determined by a uniform parameter in the range of [0,1].
  • Other variations and alternative arrangements are possible as is clear to one skilled in the art.
  • the invention provides for line color to be determined by a uniform parameter color model as described above and for line dashing to include a plurality of colors.
  • the invention provides for at least stipple pattern rendering option to be determined by a uniform parameter in the range of [0,1].
  • FIG. 34 illustrates an example wherein stipple line width is determined by a uniform parameter in the range of [0,1].
  • FIG. 34 also illustrates an example wherein stipple gap is determined by a uniform parameter in the range of [0,1].
  • a period and duty cycle model can be used instead and controlled via respectively associated uniform parameters varying over the range of [0,1].
  • FIG. 34 further illustrates an example wherein stipple angle is determined by a uniform parameter in the range of [0,1].
  • stipple angle is determined by a uniform parameter in the range of [0,1].
  • the invention provides for at least one of stipple foreground and background colors to be determined by a uniform parameter color model as described above.
  • the invention provides for a stipple pattern to comprise more than two colors, for the inclusion of line dashing in a stipple pattern, and for line dashing to include a plurality of colors.
  • the invention provides for at least one border rendering option to be determined by a uniform parameter in the range of [0,1].
  • the border rendering option can comprise the border of a spreadsheet cell as depicted in FIG. 35 , or can comprise the border of another visually rendered graphical object.
  • all four sides of the cell border are uniform.
  • Such a border to have a common width W 0 controlled by a uniform parameter in the range of [0,1] and to have a common color controlled by one or more uniform parameters in the range of [0,1].
  • the cell border can be rendered as a gradient among two or more selected colors, wherein the colors and/or gradient characteristic can be respectfully determined by an associated uniform parameter in the range of [0,1].
  • each such border segment provides for each such border segment to have an associated width (W 1 -W 4 ), each respectively controlled by an associated uniform parameter in the range of [0,1].
  • the invention further provides for each such border segment to have a common color controlled by one or more uniform parameters in the range of [0,1].
  • a border segment can be rendered as a gradient among two or more selected colors, wherein the colors and/or gradient characteristic can be respectfully determined by an associated uniform parameter in the range of [0,1].
  • the invention further provides for such a cell (interior) background to be filled with a color controlled by one or more uniform parameters in the range of [0,1].
  • a cell background can be rendered as a gradient among two or more selected colors, wherein the colors and/or gradient characteristic can be respectfully determined by an associated uniform parameter in the range of [0,1].
  • the invention also provides for the four sides of a cell border to be segmented in other ways, for example as in the two complete cells in the left side of FIG. 35 .
  • the invention provides for font morphing and font interpolation under the control of uniform parameters in the range of [0,1]. Other choices for the uniform range are also possible and are provided for by the invention.
  • FIG. 36A depicts a boldness parameter under the control of uniform parameters in the range of [0,1].
  • the variation in boldness can be implemented so that it is governed by the optical properties of de Groot Interpolation Theory.
  • a minimum and maximum degree of boldness are assigned to the extremes of the parameter range.
  • FIG. 36B depicts an italics angle parameter under the control of uniform parameters in the range of [0,1].
  • a minimum and maximum degree of italics angle is assigned to extremes of the parameter range.
  • FIG. 36C depicts an font family interpolation parameter under the control of uniform parameters in the range of [0,1].
  • One font family is assigned to the minimum extreme of the parameter range and a second font family is assigned to the maximum extreme of the parameter range.
  • FIG. 37A depicts vertical and horizontal dilation/contraction parameters, each respectively under the control of uniform parameters in the range of [0,1].
  • a minimum and maximum degree of italics angle are assigned to the extremes of the parameter range.
  • FIG. 37B shows font underline line width and font underline separation gap parameters, each respectively under the control of uniform parameters in the range of [0,1].
  • a minimum and maximum degree of italics angle are assigned to the extremes of the parameter range.
  • Configurations, features and parameters for related operations can be, for example, selected via traditional types of user interface dialog windows.
  • General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • a typical attribute control panel contains a few pre-selected values and a value which can be customized usually by selecting a value from a drop-down list.
  • users can specify if the changes in attributes are to be applied globally or are to be applied to a single cell or a group of cells in a spreadsheet.
  • the change to be made can also be determined by the value of a data element or by the result of a formula calculated from one or more data elements.
  • FIG. 39 illustrates an example of how change in color of a font can be specified for a cell, a group of cells, or globally across an entire worksheet. Users can select the color of a font simply by clicking on a region in the gray level or Hue color chart or by moving the slider that determines the gray level or parameters in Hue. A formula that determines such value can also be included. For example, the ratio of the data in a cell to the range of the data in a category can determine the value of gray level accordingly.
  • FIG. 40 depicts an interface for determining a font type of the specified cell(s).
  • Such interface can have a pre-selected list or a list to be customized or a combination of both, and can be implemented in a wide variety of ways.
  • FIG. 41 depicts an example of an interface for determining the embellishment of a font in rendering some data element within the spreadsheet or data table.
  • the data element can be displayed in bold face, italics or underlined.
  • the percentage of bolding or the thickness of the underline can be varied.
  • a data element can be emphasized by being displayed in bold face, or a title of a book or quotation can be displayed in italics, or a link to a website or a reference to another cell or region can be underlined.
  • FIG. 42 depicts an example interface to control the size of a font to be displayed.
  • Font size can be incremental with step sizes or continuous, which can be adjusted by controlling a slider, as long as the value is positive and less than reasonable size compared to the area of the text displayed.
  • the invention provides for a user interface to determine addition of an appending symbol.
  • the symbol(s) can be different types, numbers, or sizes of graphical objects.
  • the factor that determines can depend on the significance or the ratio of the element to the category or the value of another category in the dataset.
  • the symbols appended to the right end of the chart represent different categories of expenses. If the symbol appended is configured to represent quantity, the number of symbols to be appended will depend on the ratio of the value of the cell to the unit or base value. If the symbol appended is configured to represent significance, the size of symbols to be appended will depend on the significance of the value of the cell in that category.
  • FIG. 43 depicts an example wherein data in a cells in a selected column can have symbol appended to numbers or other data within the cells of the column.
  • the factor that determines the type of data can be obtained from the result of a hidden formula, from associated data such as the data from another column in the same row, or simply selected by the user. The number of decimals can be determined according to such factors. Also symbols can be attached according to the type of data. If the data in a cell represents currency, a currency symbol can be attached, and if the data represents percentage, a percentage sign will be attached. Further if the country is specified in another column, the corresponding country's currency sign can be attached.
  • the invention provides for the varying the color and opacity/transparency of the background color of a cell or group of cells as a function of data values.
  • Specifying background color and opacity/transparency can be done similarly with the case for color of the font is determined, as illustrated in FIG. 39 .
  • a user can affect the control the background color and opacity/transparency with the sliders or by simply clicking on a region of color or grayscale chart.
  • the background color control can also be incorporate auto-scaling.
  • the color can be determined depending on the ratio of the value of the cell to the values in the column.
  • the invention provides for the background texture or stipples of a cell or group of cells can also be varied as a function of data values.
  • FIG. 44 depicts an example use of controlling the background texture or stipples of cells as a function of data values in a spreadsheet or other form of tabular data.
  • FIG. 45 depicts an exemplary interface that can be used to specify the background texture or stipples to be applied to a cell or a group of cells.
  • the stipples of a cell can be made up of dots, shapes, or grid lines.
  • the size, thickness, density, or opacity of the background can be adjusted with sliders as well. Stipples can be used alone or in combination with the background color.
  • FIG. 46 depicts an exemplary interface that can be used to specify the position of text being displayed in a cell. Users can move sliders by clicking and dragging on the icon and moving it vertically or horizontally to specify vertical or horizontal position of text displayed in a cell. Users can specify the point of vertical and horizontal centering by clicking on a region in the provided box and selecting how texts are aligned horizontally or vertically relative to the point specified. In an embodiment, values representing the left-right or up-down ratio can be simply entered in text boxes by the users.
  • FIG. 47 depicts an exemplary interface that can be used to specify the rotation of text, symbols, or other information displayed combined with the vertical or horizontal placement of the text.
  • the invention provides for a user interface to select of the color of at least one border of a cell, the thickness of at least one border of a cell enveloping a font, the thickness of at least one border outlining of a plurality of contiguous cells, and for a rendered image element.
  • the invention provides for data of a specified category to be extracted from cells and matched with hidden formula to obtain data used in the visualization.
  • Data from another associated group of cells can be used for another attribute in the visualization. For example, data from one category can be used to determine the type of object displayed, another category can be used to determine the location of the object displayed, another category can be used to determine the number of objects displayed, and data from another category can be used to determine the size of the object(s) displayed.
  • Existing spreadsheet applications offer various types of charts, but the placing of the objects are limited to aligning them in certain orders or directions.
  • the invention provides for visualizations to be co-rendered with the spreadsheet program.
  • the invention provides for visualizations displayed on a specified region of a spreadsheet.
  • the invention provides for the result of visualizations to be exported to outside of the spreadsheet program.
  • FIG. 48 depicts an exemplary visualization wherein an attribute of a column area of data elements is varied as a group.
  • a set of collected data containing the state, main industry, population, the increase in population, number in current work force, unemployment rate, and the increase in unemployment rate can be used for different purposes.
  • an attribute of that column can be varied. Such emphasis can be done by varying the font boldness, font size, or font color.
  • 49 depicts an exemplary visualization wherein the shading of a row area of data elements is varied as a group as a function of a data value and also wherein the font boldness of a column area of data elements is varied as a group as a function of another data value.
  • the invention provides for a reordering of tabular rows and/or columns as a function of data values. Unlike a traditional spreadsheet data sort, such a reordering operates as a “live” function, responsive to interactive changes of the underlying data values.
  • FIG. 50 illustrates a model of a priority queue wherein the rows can automatically reorder among themselves as underlying priority sales change. Jobs to be done can have different attributes, such as location or types of jobs along with numbers that indicate the rating of priority. A user would expect jobs with a higher priority rating to be displayed in a higher position of the priority queue tree listing and to reorder as calculated priority values change.
  • FIG. 51A illustrates a basic map-based visualization example generated from previous data.
  • FIG. 51B illustrates the visualization of FIG. 51A augmented with additional spatially located symbols or parameterized glyphs.
  • the invention provides for interactive tabular spreadsheet metaphor for presentation of visualization renderings.
  • a cell may be selected and the data used to create the enclosed visualization can be captured (as structured for the enclosed visualization) to be used in a subsequent visualization or calculation.
  • a cell may be selected and the pre-visualization dataflow and processing steps used to create the enclosed visualization can be captured (as structured for the enclosed visualization) to be used in a subsequent visualization or calculation.
  • a cell may be selected and the visualization format employed by the enclosed visualization can be captured (as structured for the enclosed visualization) to be used in a subsequent visualization or calculation.
  • Two cells can be selected and the data used to create the enclosed visualizations can be captured (as structured for the enclosed visualization) and provided to a mathematical operation (for example, adding, subtracting, multiplying, convolving, etc.) to form a new dataset to be used in a subsequent visualization or calculation.
  • a mathematical operation for example, adding, subtracting, multiplying, convolving, etc.
  • multiple cells can be selected and the data used to create the enclosed visualizations can be captured (as structured for the enclosed visualization) and merged to form a new dataset to be used in a subsequent visualization or calculation.
  • Embodiments of the invention can also include one or more of the following optional features:
  • the invention provides for the inclusion of useful novel 3D graphics visualization functions for multidimensional data representation, data query, and numerical solution tools for simultaneous interactive numerical equations. These can be incorporated in various ways, as described below, to expand spreadsheet—based visualization capabilities yet further.
  • a first set of the useful novel 3D graphics visualization functions provided for by the invention pertain to multidimensional data visualizations based on 2D-surfaces embedded in a 3D visual field.
  • FIG. 52 data or spreadsheet, these separated by a vertical gap within a 3D visual field as provided for by the invention.
  • the invention also provides for interactively shifting the observation point with respect to these objects for more detailed feature, theme, or trend inspection.
  • the viewpoint can be changed according to the 6 degrees of freedom (three translations, three angles) of rigid motion.
  • FIG. 53 depicts with dashed lines a few ways in which the observation viewpoint can be moved with respect to these abstract objects.
  • the viewpoint is changed under the control of an HDTP or can be changed under the control of an advanced mouse.
  • the invention also provides for moving the 2D-surface representing a 3D data plot within the 3D visual field.
  • the 2D-surface can be moved according to the 6 degrees of freedom (three translations, three angles) of rigid motion.
  • FIG. 54 depicts a translation and rotation of a 3D data plot within the 3D visual field.
  • the 3D data plot can be moved under the control of an HDTP or the 3D data plot can be moved under the control of an advanced mouse.
  • the 2D-surface representing a 3D data plot within the 3D visual field can be virtually illuminated by one or more lighting sources.
  • the one or more lighting sources can be moved according to the 6 degrees of freedom (three translations, three angles) of rigid motion.
  • the one or more lighting sources can also permit control of the color and intensity of virtual light emitted.
  • a lighting source can be moved and controlled under the control of an HDTP or under the control of an advanced mouse.
  • the data plotted in the 2D-surface representing a 3D data plot within the 3D visual field can directly echo the data displayed in the exemplary planar array of associated tabular data or spreadsheet and/or can originate from another set of tabular data or spreadsheet region and/or can originate from other data.
  • FIG. 55A The invention also provides for processing of the data via mathematical transformations, statistical processing, signal processing, etc. prior to creation of the 2D-surface representing at least 3-dimensional data.
  • FIG. 55B the 2D-surface representing at least 3-dimensional data can plot filtered or averaged versions of tabular data, spreadsheet data, or other data.
  • the filtering or averaging can be controlled by an interactive parameter.
  • the invention provides for 2D-surface representing of at least 3-dimensional data to be rendered along with a height measuring visual so that the local color of the surface need not be a function of the vertical value being plotted. This frees up the local 2D-surface color to be used to represent an additional dimension of data.
  • FIG. 56 illustrates an exemplary height measuring visual in its use with 2D-surface representing at least 3-dimensional data and useful for representing at least 4-dimensional data.
  • the invention provides for a 2D-surface representing at least 3-dimensional data to be rendered with a parameterized locally varying surface texture that can be used to represent an additional dimension of data, with numerical values rendered on the surface that can be used to represent an additional dimension of data, with symbols rendered on the surface that can be used to represent an additional dimension of data, or with parameterized glyphs rendered on the surface that can be used to represent (an) additional dimension(s) of data.
  • FIG. 57 depicts an exemplary slicing function, here used to provide a (planar-slice) level set curve.
  • Other uses, implementations, and applications for slicing functions are also possible.
  • FIG. 58A illustrates two exemplary 2D-surfaces, each representing at least 3-dimensional data. Each of these can be interactively moved and rotated as described earlier.
  • FIG. 58B illustrates the interactively created intersection of the two exemplary 2D-surfaces and the resultant intersection curve.
  • a 2D-surface intersection tool allows the numerical values of selected points on a curve formed by the intersecting surfaces to be numerically calculated and visually displayed. The numerical values can also be used for other purposes and other subsequent calculations.
  • the sample points can be selected according to the details of the intersection.
  • An adaptive sampling method can be employed.
  • the resultant intersection data can be interpolated and re-sampled according to another sampling strategy. More than two surfaces can be intersected simultaneously, and various pair wise and group intersection data can be captured and displayed.
  • a second set of the useful novel 3D graphics visualization functions provided for by the invention pertain to multidimensional data visualizations based on 1D-curves embedded in a 3D visual field.
  • the invention provides for a curve to be generated by tabular data and to be suspended over tabular data used to create it or otherwise associated with the curve.
  • the invention provides for the curve to be visually linked to a geometric rendering of tabular data. These features are illustrated by FIG. 59 .
  • the invention provides for the curve to be given a thickness and or color under the control of one or more data values.
  • the invention provides for the color of the curve to be varied as a function of tabular (or other) data values.
  • FIG. 60 The invention provides for the thickness of the curve to be varied as a function of tabular (or other) data values. This feature is illustrated in FIG. 61 .
  • Such curve plotting utilities can be provided with functions, features, and operations like that of the 2D-surface utilities described above.
  • the invention also provides for interactively shifting the observation point with respect to these objects for more detailed feature, theme, or trend inspection.
  • the viewpoint can be changed according to the 6 degrees of freedom (three translations, three angles) of rigid motion.
  • the viewpoint is changed under the control of an HDTP or an advanced mouse.
  • the invention also provides for moving a curve within the 3D visual field.
  • the curve can be moved according to the 6 degrees of freedom (three translations, three angles) of rigid motion.
  • the 2D-surface can be moved under the control of an HDTP or an advanced mouse.
  • a curve of any thickness can be treated as if it comprises a surface that can be virtually illuminated by one or more lighting sources.
  • the one or more lighting sources can be moved according to the 6 degrees of freedom (three translations, three angles) of rigid motion.
  • the one or more lighting sources can also permit control of the color and intensity of virtual light emitted.
  • a lighting source can be moved and/or controlled under the control of an HDTP or an advanced mouse.
  • the data plotted in a curve can directly echo the data displayed in the exemplary planar array of associated tabular data or spreadsheet and/or can originate from another set of tabular data or spreadsheet region and/or can originate from other data as suggested earlier in FIG. 55A .
  • the invention also provides for processing of the data via mathematical transformations, statistical processing, signal processing, etc. prior to creation of the curve as was suggested earlier in FIG. 55B .
  • a curve can plot filtered or averaged versions of tabular data, spreadsheet data, or other data. In an embodiment, the filtering or averaging can be controlled by an interactive parameter.
  • the invention provides for a curve to be rendered along with a height measuring visual, and also provides for a locally variable parameterized texture to be imposed on the curve and for the texture to be varied as a function of tabular or other data.
  • the invention provides for the curve to be rendered with numerical values displayed adjacently at specific points on the curve and that these can be used to represent an additional dimension of data.
  • the invention also provides for the curve to be rendered with symbols displayed adjacently at specific points on the curve and that these can be used to represent an additional dimension of data.
  • the invention further provides for the curve to be rendered with parameterized glyphs displayed adjacently at specific points on the curve and that these can be used to represent (an) additional dimension(s) of data.
  • the invention provides for a curve intersection tool that can be used as a numerical solution operator for plotted, interpolated, and processed data.
  • FIG. 62 illustrates two exemplary curves, each representing at least 2-dimensional data. Each of these can be interactively moved and rotated as described earlier, interactively creating an intersection of the two exemplary curves and at least one resultant intersection point.
  • a 2D-surface intersection tool allows the numerical values of intersection points to be numerically calculated and visually signified. The numerical values can also be used for other purposes and other subsequent calculations.
  • more than two curves can be intersected simultaneously, and various pair-wise and group intersection data can be captured and displayed.
  • intersection data can be captured and displayed.
  • more than two curves and at least one surface can be intersected simultaneously, and various pair-wise and group intersection data can be captured and displayed.

Abstract

A method for visually enhancing the display of tabular data of an electronic spreadsheet is disclosed. The method includes obtaining data presented in at least one cell of the spreadsheet, computing at least one visual parameter based on the obtained data, using the at least one visual parameter to control a visual effect of the displayed data, and applying the at least one visual effect to the displayed data to visually enhance the displayed data. Each visual parameter is a number within a range of numbers and is determined according to a numerically calculated mathematical function. The visual enhancements include varying the background color or texture or border color, thickness, grouping scope of cells enveloping characters conveying data, as well as font color, type, embellishment, size, format, location, decimal places, rounding or supplemental symbols symbolic or image element rendered within the cell of characters conveying data.

Description

    CROSS-REFERENCE TO RELATED APPLICATIONS
  • This application claims benefit of priority of U.S. provisional application Ser. No. 61/239,349 filed on Sep. 2, 2009, incorporated herein by reference.
    • BACKGROUND OF THE INVENTION
  • 1. Field of the Invention
  • The invention relates to data visualization and in particular its implementation by and application to conventional spreadsheets and displays of tabular data.
  • 2. Background of the Invention
  • Interactive electronic spreadsheets such as the 1979 VisiCalc numerical spreadsheet (http://www.danbricklin.com/visicalc.htm), Supercalc numerical spreadsheet (1980), Microsoft Multiplan™ 91982), Lotus 1-2-3™ (1983), the AppleWorks™ spreadsheet module (1984), Microsoft Excel™ (1985), OpenOffice.org™ Calc, Google's Google Spreadsheets, etc., present a tabular presentation of data and calculation formula flow in arrangements that have proven to be very useful for data analysis. Interactive electronic spreadsheets typically include plotting routines that create graphical representation of selected data points. These plots can be used to provide visual representations of data and mathematical functions. In a sense, these interactive electronic spreadsheet functions provide a form of data visualization based on data in the interactive electronic spreadsheet.
  • Further, interactive electronic spreadsheets present numerical, textual, and symbolic data to be presented in a tabular array of cells, with the numerical, textual, and symbolic data within the cells provided in various font styles and colors, and with various background colors and border styles of the associated cell. Using interactive electronic spreadsheet functions known as “conditional formatting” these font styles and colors, cell background colors, and cell border styles can be controlled by values of data according to hand-specified conditional tests performed on data within one or more specified cells. Such hand-specified conditional tests are entered through interactive electronic spreadsheet GUIs (Graphical User Interfaces), VBA (Visual Basic for Applications), APIs (Application programmer interfaces), etc. These interactive electronic spreadsheet functions provide another form of data visualization based on data in the interactive electronic spreadsheet.
  • These visual data representation successes of the interactive electronic spreadsheet paradigm suggest additional opportunities for providing and combining additional data visualization capabilities with the well-established functionality and embedded deployment of interactive electronic spreadsheet software. The present invention provides several such additional data visualization capabilities to traditional interactive electronic spreadsheets. The visual effects provided by the invention can be automatically varied over a range responsive to values of data or formulas according to mathematical functions rather than conditional tests found in interactive electronic spreadsheet products.
  • Unlike the logical-test structure inherent in “conditional formatting”, the invention provides for use of numerical values directly from tabular data or as calculated by hidden mathematical function, compositions of mathematical functions, or traditional spreadsheet formulas to easily provide rich detailed control of useful parameterized visual effects rendered in the context of conventional interactive electronic spreadsheet and tabular data display.
  • The visual effects can include variation of background color or texture or border color, thickness, grouping scope of cells enveloping characters conveying data, as well as font color, type, embellishment, size, format, location, decimal places, or supplemental symbols symbolic or image element rendered within the cell of characters conveying data. Additionally, the invention provides for data-driven 3D plots rendered in the context of or projection from interactive electronic spreadsheet or tabular data. The invention provides for these visualization operations to be generated by an algorithm directly or indirectly in communication with data or parts of a program rendering an interactive electronic spreadsheet or visually displayed data table. The invention also provides for various optional additional functions, for example:
      • Data-driven automatic “live” sorting of rows and columns;
      • Incorporation of externally-provided stored data and live data feeds;
      • Real-time response to live input data, interactive data entry, or visualization manipulation via interactive user interfaces;
      • Collaboration capabilities;
      • Web-access capabilities;
      • Meaningful data sonification capabilities, including those involving multichannel timbre modulation.
  • Because many aspects of the invention are described in terms of, or pertain to interactive electronic spreadsheet programs, the shorthand “spreadsheet” will be understood to mean interactive electronic spreadsheet programs.
    • SUMMARY OF THE INVENTION
  • The invention provides for a method to visually enhance the display of tabular data of an electronic spreadsheet comprising cells arranged in rows and columns. At least one data value is used to compute at least one visual parameter, and each visual parameter is a number within a range of numbers and is determined according to a numerically calculated mathematical function. A visual effect of the at least one data value is controlled according to the at least one visual parameter, and the visual effect is applied to the display of the at least one data value to visually enhance the display of tabular data of the electronic spreadsheet.
  • Various aspects and features of the invention are described below. These aspects and features may be found either singly or in combination in the invention, and the descriptions of the various aspects and features are intended to be illustrative and not to limit the scope of the invention.
  • The visual effects can be automatically varied over a range responsive to data values or formulas according to mathematical functions rather than conditional tests found in interactive electronic spreadsheet products. The visual parameters of a data or cell presentation attribute may be varied over a pre-defined range (for example, between 0 and 1, between 0 and 100, between −1 and +1, etc).
  • Various attributes of the font in which the tabular data is displayed can be controlled. For example, one or more of the font color, the font type or style, font embellishments, and the font size may be controlled by the visual parameters. If the tabular data is numeric data, one or more of the number format, the number of decimal places, and the method and degree of rounding may also be controlled by the visual parameters.
  • The background of the cell in which tabular data is displayed may also be visually enhanced by the visual parameters. For example, one or more of the background color and background texture may be controlled by the visual parameters.
  • Further, one or more of border color and border line thickness may be controlled by the visual parameters. The border may enclose a single spreadsheet cell displaying tabular data or may enclose a plurality of contiguous cells displaying tabular data.
  • According to the visual parameters, a symbol may be appended to the displayed tabular data, or an image or a data symbol may be independently displayed instead of the tabular data in a spreadsheet cell.
  • While each of the above aspects and features of the invention may be applied to individual spreadsheet cells, these aspects and features may be applied to rows and columns containing tabular data.
  • Other aspects and features of the invention include:
      • visual parameters are generated by an algorithm in communication with the spreadsheet program, or from stored data;
      • storage and recall of sessions.
      • the support of data-driven automatic “live” rows and columns sorting;
      • live data feeds;
      • externally provided data;
      • data sonification;
      • web access;
      • collaboration with other users;
      • 3D capabilities; and
      • at least one user interface that providing the selection of function blocks and the general connectivity among them to be specified using a drawing tool and a palette of function blocks, wherein clicking on each function blocks would cause dialog windows to be displayed for setting parameters by means of typed-in values, sliders, mouse manipulation, or advanced high-dimension user interface devices.
    BRIEF DESCRIPTION OF THE DRAWINGS
  • The above and other aspects, features and advantages of the present invention will become more apparent upon consideration of the following description of preferred embodiments taken in conjunction with the accompanying drawing figures. The accompanying figures are examples of the various aspects and features of the present invention and are not limiting either individually or in combination.
  • FIG. 1 provides a demography of ways in which data visualization and spreadsheets can be integrated together.
  • FIG. 2A depicts a region of a spreadsheet or table of tabular data.
  • FIG. 2B depicts a cell and its immediate neighborhood within a spreadsheet or table of tabular data.
  • FIG. 3 depicts of data and cell presentation attributes serving as candidates for visualization parameters within a spreadsheet display.
  • FIGS. 4A-4E illustrate several application settings where the invention is well-suited to add value.
  • FIG. 5 illustrates an arrangement of selected more general aspects of the invention that are not restricted to spreadsheet visualization.
  • FIG. 6 illustrates an adaptation of the arrangement of FIG. 5 to an environment supporting collaboration features.
  • FIG. 7 depicts a topological interconnection of data flow paths linking various elements depicted in FIG. 5.
  • FIGS. 8A-8C depict approaches for mapping a data value lying within a pre-defined range to a value within a pre-defined range for a parameterized data or cell presentation attribute.
  • FIG. 9 depicts a more general view and organization of pre-visualization operations provided for by the invention.
  • FIG. 10 depicts an embodiment wherein a selected metaphor is used to automatically generate parameter assignments and graphics rendering operations.
  • FIG. 11A illustrates a user interface that permits users to select the geometric element boundary of a captured or rendered tabular matrix of tabular matrix data elements or cells within a spreadsheet. FIG. 11B illustrates a user interface that permits users to select array operations to be performed.
  • FIGS. 12A-12G depict transformations of graphics objects via array operations as provided for by the invention.
  • FIG. 13 depicts a spreadsheet architecture and information flow as it might be used in a data visualization setting.
  • FIGS. 14A-14D depict various types of architectural approaches for combining and integrating spreadsheet programs, visualization operations provided by the invention, and a data set used by each.
  • FIG. 15 depicts an embodiment of an augmentation of a spreadsheet program with additional spreadsheet features as provided for by the invention.
  • FIGS. 16A-16B depict ways in which one or more user interface input devices (such as a mouse, trackball, touchpad, etc.) can be managed so as to control graphical user interfaces of a plural software application in the context of the invention.
  • FIGS. 17A-17F illustrate the six independently adjustable degrees of freedom of touch from a single finger that can be simultaneously measured by the HDTP technology.
  • FIG. 18 suggests general ways in which two or more of these independently adjustable degrees of freedom can be adjusted at once.
  • FIG. 19 demonstrates a few two-finger multi-touch postures and/or gestures from the many that can be readily recognized by HTDP technology.
  • FIG. 20 shows how raw measurements of the six quantities of FIGS. 17A-17F, together with shape recognition for distinguishing contact with various parts of the hand and the touchpad, can be used to create a rich information flux of parameters, rates, and symbols.
  • FIG. 21 shows an approach for incorporating posture recognition, gesture recognition, state machines, and parsers to create an even richer human/machine tactile interface system capable of incorporating syntax and grammars.
  • FIG. 22 depicts a user interface input arrangement incorporating one or more HDTPs that provides user interface input event and quantity routing of the type described earlier in conjunction with FIGS. 16A-16B.
  • FIGS. 23A-23L depict a number of arrangements employing the HDTP technology suitable for use with visualization environments.
  • FIGS. 24A and 24B illustrate examples of scroll-wheel mice provided with an additional scroll-wheel.
  • FIGS. 25A-25C illustrate a single trackball incorporated into the back of a conventional computer mouse.
  • FIGS. 26A-26C illustrate two trackballs incorporated into the back of a conventional computer mouse.
  • FIG. 27 depicts an arrangement wherein the HDTP or alternatives such as advanced mice can interface with browser-based applications via a browser plug-in.
  • FIG. 28 depicts an implementation for rendering visualization in a browser.
  • FIG. 29 depicts a web-based implementation of a visualization environment leveraging browser arrangements such as that depicted in FIGS. 27-28, their variations, alternatives, etc.
  • FIG. 30A depicts a client-side collaboration implementation wherein a general purpose collaboration tool can be used to share the visualization environment session running on one computer with one or more other computers.
  • FIG. 30B depicts a server-side collaboration implementation wherein a general purpose or specialized server-side application sharing environment is used to share a server-based interactive data visualization application.
  • FIG. 30C depicts a server-side collaboration implementation comprising a server-based multi-user interactive data visualization application.
  • FIG. 31 illustrates a general framework for data sonification wherein a parameterized metaphor is used to manage parameter assignment and sound rendering and a parameterized data indexing operation.
  • FIG. 32 depicts an arrangement wherein the hue of a color can be varied through natural colors of red through violet for the bulk of the [0,1] range and a range of artificial colors, such as the purple through magenta continuum, are appended for the remaining portion of the [0,1] range.
  • FIG. 33 illustrates an arrangement wherein line width, line dashing period, line dashing duty-cycle, and line dashing sub-duty-cycle can be determined by a uniform parameter in the range of [0,1].
  • FIG. 34 illustrates an arrangement wherein stipple line width, stipple gap, and stipple angle can be determined by a uniform parameter in the range of [0,1].
  • FIG. 35 depicts an arrangement wherein at least one border rendering option is to be determined by a uniform parameter in the range of [0,1]. The border can be the border of a spreadsheet cell as depicted, or can be the border of another visually rendered graphical object.
  • FIG. 36A depicts an arrangement wherein a boldness parameter is under the control of uniform parameters in the range of [0,1]. FIG. 36B depicts an arrangement wherein an italics angle parameter is under the control of uniform parameters in the range of [0,1]. FIG. 36C depicts an arrangement wherein an font family interpolation parameter is under the control of uniform parameters in the range of [0,1].
  • FIG. 37A depicts an arrangement wherein exemplary vertical and horizontal dilation/contract parameters are each respectively under the control of uniform parameters in the range of [0,1]. FIG. 37B depicts an arrangement wherein font underline line width and font underline separation gap parameters are each respectively under the control of uniform parameters in the range of [0,1].
  • FIG. 38 depicts an attribute control panel wherein there are pre-selected values and a value which can be customized.
  • FIG. 39 depicts an arrangement wherein a change in color of a font can be specified for a cell, a group of cells, or globally across an entire worksheet.
  • FIG. 40 depicts an interface that determines a font type of the specified cell(s).
  • FIG. 41 depicts an interface that determines the embellishment of a font in rendering some data element within the spreadsheet or data table.
  • FIG. 42 depicts an interface to control the size of a font to be displayed.
  • FIG. 43 depicts an wherein data in a cells in a selected column can have symbol appended to data within the cells of the column.
  • FIG. 44 depicts an example use of controlling the background texture or stipples of cells as a function of data values in a spreadsheet or other form of tabular data.
  • FIG. 45 depicts an interface that can be used to specify the background texture or stipples to be applied to a cell or a group of cells.
  • FIG. 46 depicts an interface that can be used to specify the position of text being displayed in a cell.
  • FIG. 47 depicts an interface that can be used to specify the rotation of text being displayed combined with the vertical or horizontal placement of the text.
  • FIG. 48 depicts a visualization wherein an attribute of a column area of data elements is varied as a group.
  • FIG. 49 depicts a visualization wherein the shading of a of a row area of data elements is varied as a group as a function of a data value and also wherein the font boldness of a column area of data elements is varied as a group as a function of another data value.
  • FIG. 50 illustrates a priority job queue model tabular-data visualization wherein the row can automatically reorder among themselves as underlying priority vales change.
  • FIG. 51A illustrates a basic map-based visualization example generated from previous data. FIG. 51B illustrates the visualization of FIG. 51A augmented with additional spatially located symbols or parameterized glyphs.
  • FIG. 52 depicts a 2D-surface representing a 3D data plot of at least 3-dimensional data rendered above a planar array of associated tabular data or spreadsheet, these separated by a vertical gap within a 3D visual field as provided for by the invention.
  • FIG. 53 depicts with dashed lines a few ways in which the observation viewpoint can be moved with respect to these abstract objects.
  • FIG. 54 depicts a translation and rotation of an exemplary 3D data plot within the 3D visual field.
  • FIG. 55A depicts an arrangement wherein data plotted in a 2D-surface representing a 3D data plot within the 3D visual field can directly echo data displayed in a planar array of associated tabular data or spreadsheet and/or can originate from another set of tabular data or spreadsheet region and/or can originate from other data.
  • FIG. 55B depicts an arrangement wherein processing of data via mathematical transformations, statistical processing, signal processing, etc. is made prior to creation of the 2D-surface representing at least 3-dimensional data.
  • FIG. 56 illustrates a height measuring visual in its use with 2D-surface representing at least 3-dimensional data and useful for representing at least 4-dimensional data.
  • FIG. 57 depicts a slicing function here used to provide a (planar-slice) level set curve.
  • FIG. 58A illustrates two 2D-surfaces, each representing at least 3-dimensional data. FIG. 58B illustrates the interactively created intersection of the two exemplary 2D-surfaces and the resultant intersection curve.
  • FIG. 59 depicts an arrangement wherein a curve can be generated by tabular data and be suspended over tabular data used to create it or otherwise associated with the curve, and can be visually linked to a geometric rendering of tabular data values.
  • FIG. 60 depicts an arrangement wherein a curve can be given a thickness and or color under the control of data values.
  • FIG. 61 depicts an arrangement wherein the thickness of a curve can be varied as a function of one or more data values.
  • FIG. 62 depicts a curve intersection tool that can be used as a numerical solution operator for plotted, interpolated, and processed data.
    •  DETAILED DESCRIPTION
  • In the following description, reference is made to the accompanying drawing figures which form a part hereof, and which show by way of illustration specific embodiments of the invention. It is to be understood by those of ordinary skill in this technological field that other embodiments can be utilized, and structural, electrical, as well as procedural changes can be made without departing from the scope of the present invention. The aspects and features described herein may be used singly or in combination unless specifically stated otherwise.
  • The visual data representation successes of spreadsheets suggest additional opportunities for providing and combining additional data visualization capabilities with the well-established functionality and embedded deployment of spreadsheet software. FIG. 1 provides an example of ways in which data visualization and spreadsheets can be integrated. The present invention provides several additional data visualization capabilities to traditional spreadsheets.
  • First, traditional spreadsheets present data in a tabular form leveraging row, column, and sheet organization. Traditional spreadsheets typically include plotting routines that create graphical representation of selected data points. These plots can be used to provide visual representations of data and mathematical functions. These spreadsheet functions provide a base-level form of data visualization based on data in the spreadsheet, as represented in rows A and B of FIG. 1.
  • Additional subsequent work has been done to employ data visualization characterizing internals of spreadsheet structure such as information flow, cell formula dependences, and semantic classification. These do not provide visualization of data per se, but can be very useful in understanding the data handling of a given spreadsheet. This is represented in rows C and D of FIG. 1.
  • Further subsequent work has been done to supplement basic spreadsheet plotting utilities providing richer data plotting capabilities (for example, utilizing 3D graphics). These capabilities enhance the possible data visualization based on data in the spreadsheet. This is represented in row E of FIG. 1.
  • Yet other subsequent work has been done adopting the interactive format and metaphor of a spreadsheet for use in a tabular presentation of complex data visualization renderings. This is represented in row F of FIG. 1. One aspect of the invention focuses on adapting and significantly expanding this capability. This is represented in row G of FIG. 1.
  • Still other subsequent work has been done superimposing a third (height) dimension atop the 2-dimensional tabular data layout of an interactive electronic spreadsheet. The third dimension can be used to render color-coded line and surface plots, and the resulting 3D graphics can be viewed from various virtual observation points for inspection. This is represented in row H of FIG. 1. Another aspect of the invention focuses on adapting and expanding this capability.
  • Traditional spreadsheets also present numerical, textual, and symbolic data in a tabular array of cells, with the numerical, textual, and/or symbolic data within the cells provided in various font styles and colors, and with various background colors and border styles of the associated cell. Using spreadsheet functions known as “conditional formatting” these font styles and colors, cell background colors, and cell border styles can be controlled by values of data according to hand-specified (through spreadsheet GUI, VBA, APIs, etc.) conditional tests performed on data within one or more specified cells. These spreadsheet functions provide another form of data visualization based on data in the spreadsheet. One aspect of the invention focuses on adapting and expanding this capability. This is represented in row I of FIG. 1.
  • FIG. 2A illustrates a region 200 of a traditional spreadsheet or table of tabular data. Such spreadsheet or tabular data can be created by desktop applications such as Microsoft Excel or online applications such as Google Spreadsheet. Aspects of the present invention can be implemented via an add-on package of existing software or additional plug-in script on any online applications. The minimum block that makes up this whole region is referred to as a cell, and cells can be merged to compose a cell with bigger area expanding multiple rows, columns, or both. Region 200 of an example spreadsheet contains four columns 281-284 and five rows 291-295. Cell 230 is confined within one row 293 and on column 283 and accordingly is defined by a left border 213, a right border 214, an upper border 222, and a lower border 223. Each of the four columns 281-284 represents a different categorical subset of data (for example, an item, region, revenue area, inventory level, etc.). In contrast to exemplary cell 230, another exemplary cell 240 in the left-most column comprises the height of three rows 293, 294, 295.
  • The border of cell 230 is bolded with respect to the other cells depicted so as to call this cell out from the others. This can be rendered for any of a number of reasons, such as signifying the selection of the cell via a pointing device of a user interface, selection resulting from a query, the signifying of special status or conditions, etc. The data displayed in cell 230 is a decimal number formatted as a national financial currency and with a thousands-separating comma. In contrast, the data displayed in cell 260 is an unformatted non-decimal number without a thousands-separating comma. Also in contrast, the data displayed in cell 250 comprises a text string, while the data displayed in cell 240 comprises an alphabetic text string and an unformatted non-decimal number. In the latter, the unformatted non-decimal number can in fact be part of the preceding text string, or can be a calculated, retrieved, or can be a fixed number composed with a calculated, retrieved, or fixed text string.
  • In general the data displayed in the foreground of a cell can contain numbers, text, symbols, or a combination of these, and this data or portions of it can be further “emphasized” with attributes. Such attributes include choice of the font family, font size, bolding, italicizing, stretching, color, transparency with respect to the cell background, position within the cell, various types of animations (modulations of aforementioned attributes, blinking, shimmering, vibrating, rotating, etc.) Each individual font attribute can be fixed or determined by data values via conditional formatting. The data displayed in the foreground of a cell can be fixed, replicated, or result from processing of other data by (no displayed) formulas and/or functions.
  • FIG. 2B depicts additional detail of cell 230 and its immediate neighborhood. In order to render the bolded border of the cell, the dividing border lines 213, 214, 222, 223 of FIG. 2A can be segmented into a plurality of sections—for example dividing border line 213 of FIG. 2A can be segmented into at least adjacent sections 213 a, 213 b, 213 c; similarly dividing border line 214 of FIG. 2A can be segmented into at least adjacent sections 214 a, 214 b, 214 c, dividing border line 222 of FIG. 2A can be segmented into at least adjacent sections 222 a, 222 b, 222 c, and dividing border line 223 of FIG. 2A can be segmented into at least adjacent sections 223 a, 223 b, 223 c. The exemplary cell 230 is visually defined as the geometric area enclosed by upper, lower, left, and right dividing border lines 213 b, 214 b, 222 b, and 223 b—these border line sections can be accented via thickening, darkening, coloring, etc. so as to visually contrast with the other sections 213 a, 213 c, 214 b, 214 c, 222 a, 222 c, 223 a, 223 c. Such border line section accenting can be fixed or determined by data values via conditional formatting.
  • The background 245 of each cell can be used to display a background color, shading pattern, image, etc. which can be fixed or determined by data values via conditional formatting. The background can vary from cell to cell.
  • The displayed data in this particular case is, again, a decimal 241 number formatted as a national financial currency via a signifying currency symbol 242 and further employing a thousands-separating comma 243.
  • FIG. 3 depicts of data and cell presentation attributes serving as candidates for visualization parameters within a spreadsheet display. Some of these attributes can be controllable under conditional formatting in traditional spreadsheets while others may not be controllable under conditional formatting in traditional spreadsheets. In one aspect of the invention, a data or cell presentation attribute provided a pre-defined range of variability and is controlled by a parameter that can vary over a pre-defined range (for example, between 0 and 1, between 0 and 100, between −1 and +1, etc). This parameter-controlled variability of a data or cell presentation attribute allows that presentation attribute to be used as a visualization parameter.
  • For those data and cell presentation attributes that have been controllable under conditional formatting in traditional spreadsheets, the parameter-controlled variability provided for by the invention is fundamentally different in a number of ways. A few of these include:
      • Conditional formatting provides a particular custom-specified result for satisfaction of a custom-specified rule;
      • For each individual result desired, a corresponding hand-specified rule must be entered; for example in order to span 64 color steps, 64 hand-specified rules must be hand-entered;
      • In the case of colors, shades, fonts, stipples, etc., conditional formatting provided (or extended by VBA, APIs, etc.) in traditional spreadsheets offers only a small discrete set of choices. In the case of colors, Microsoft Excel for example provides a fixed pre-selected collection of 56 colors, some of which are duplicates. Even if Microsoft Excel provided 64 colors, this would only permit 4 steps per primary color in an RGB color model or a very limited color wheel in a HSB color model. Additionally, the fixed collection of pre-selected colors will always have a significant portion of colors that will be unusable as foreground or background colors as they will not be able to visibly stand out, respectively, against a corresponding range of background or foreground colors. Thus there are typically considerably less than the full collection of pre-selected colors to work with in a given color-control data presentation visualization if numeric or text information is co-displayed.
        Clearly the present invention provides a far superior approach with greater capability, practicality, and ease of use.
  • Among other capabilities, the invention provides new visualization capabilities to spreadsheet presentation, spreadsheet formatting, spreadsheet data handling, and spreadsheet interactive use in ways well-suited for use as a numeric-intensive data visualization tool. Many of these spreadsheet features can be implemented with simple vector graphics rendering operations or easily managed mixtures of vector and raster graphics, permitting:
      • Implementation of a number of the invention's spreadsheet presentation features through use of APIs of existing spreadsheet products;
      • Implementation of many of the invention's spreadsheet presentation features with relatively small augmentation and modification of existing product software by product manufacturers;
      • Implementation of most of the invention's spreadsheet presentation features with well-established graphics utilities such as SVG (http://www.w3.org/TR/SVG11/) and the graphics utilities of various computer operating systems provided by companies such as Microsoft and Apple;
      • Via these and/or other graphics utilities, implementation of most of the invention's spreadsheet presentation features in browser-rendered web applications, thus making the invention's spreadsheet presentation features available in a web page;
      • Via web and/or other implementation approaches, implementation of the invention's presentation features in a collaborative viewing environment.
  • The invention provides for a value of a data element, or result of a formula calculated from one or more data elements in a spreadsheet or data table, to be used to determine spreadsheet presentation attributes, such as one or more of the following examples:
  • Attributes of a Font and/or Symbol:
      • the color of a font employed in rendering some data element within the spreadsheet or data table;
      • the opacity of a font employed in rendering some data element within the spreadsheet or data table;
      • the selection of a font employed in rendering some data element within the spreadsheet or data table;
      • the embellishment of a font (bolding, italicizing, underlining, etc.) employed in rendering some data element within the spreadsheet or data table;
      • the size of a font employed in rendering some data element within the spreadsheet or data table;
  • Attributes of Appended Symbols:
      • the condition for the addition of an appending symbol or string employed in rendering some data element within the spreadsheet or data table;
      • where to append symbol or string
      • color(s) of symbol or string
      • opacity of symbol or string
      • other parameters of the symbol or contents of string;
  • Attributes of Numerical Quantization of Cell Data:
      • the number of decimal places employed in rendering some data element within the spreadsheet or data table;
      • the method of numerical rounding employed in rendering some data element within the spreadsheet or data table;
      • the degree of numerical rounding employed in rendering some data element within the spreadsheet or data table;
      • the selection of a symbolic data element rendered within the spreadsheet or data table;
      • the selection of an image element rendered within the spreadsheet or data table;
  • Attributes of a Cell Border:
      • the color of at least one border of a cell enveloping a font employed in rendering some data element within the spreadsheet or data table;
      • the thickness/width of at least one border wall of a cell enveloping a font employed in rendering some data element within the spreadsheet or data table;
      • the thickness/width of at least one border outlining of a plurality of contiguous cells, where each cell envelopes a font employed in rendering of data elements within the spreadsheet or data table;
  • Attributes of Cell Interior:
      • the background color of a cell enveloping some data element within the spreadsheet or data table;
      • the background texture of a cell enveloping some data element within the spreadsheet or data table;
      • the location of data element within a cell enveloping that data element within the spreadsheet or data table;
  • Attributes of Rows & Columns:
      • an attribute of a column employed in rendering a plurality of data elements within the spreadsheet or data table;
      • an attribute of a row employed in rendering a plurality of data elements within the spreadsheet or data table.
      • an attribute of a row-by-column area employed in rendering a plurality of data elements within the spreadsheet or data table.
  • The invention also provides for one or more of the following capabilities:
      • reordering of tabular rows and/or columns as a function of data values;
      • superimposing a third (height) dimension atop the 2-dimensional tabular data layout of an interactive electronic spreadsheet.
      • adopting the interactive format and metaphor of an interactive electronic spreadsheet for use in a tabular presentation of complex data visualization renderings;
  • Additionally, the invention provides for the control of data and cell presentation attributes through use of a uniform parameterization framework. This allows pre-visualization operations, such as scaling, translation, filtering, array (matrix, tensor) operations, nonlinear warping, etc. to be employed in a modular, cascadable fashion independent of the particular choice of data and cell presentation attributes. The invention further provides for pre-visualization operations to themselves have parameters that can be adjusted in real time and/or be stored in files for recall. The invention further provides for a network of pre-visualization operations to be stored in files for recall.
  • The invention additionally provides for advanced user interface devices, particularly those providing large numbers of simultaneously-adjustable interactive control parameters, to be used to control the viewing, presentation, and creation of the visualization as well as controlling the underlying data source such as databases, statistical packages, simulations, etc.
  • Further, via web and other implementation approaches, the implementation of the invention's presentation features in a collaborative interactive use environment.
  • The invention thus can serve a variety of callings. FIGS. 4 a-4 e illustrate several example application settings where the invention is well-suited to add value. In one scenario, depicted in FIG. 4A, a spreadsheet is initially used as a tool of choice or convenience 401, and at some point some simple data visualizations are incorporated 402. Such capabilities can be within the range provided by available spreadsheet software products 400. As study of the data or development of the model intensifies, additional degrees of visualization are incorporated 403, potentially involving a need for a rich suite of visualization capabilities to create data visualization of substantial richness 404. In FIG. 4B, a spreadsheet is chosen as a vehicle or venue for visualization as it matches one dominating user needs (for example, original data already in or readily imported into a spreadsheet, quick assembly, tabular nature of data, tabular nature of formula replication, etc.). In FIG. 4C, a spreadsheet is chosen as a vehicle or venue for visualization as it is comfortable or intuitive for a particular audience. In FIG. 4D, a spreadsheet is chosen as a vehicle or venue for visualization as other aspects of the invention allow for it to be rendered in a web-based application. In FIG. 4E, a spreadsheet is chosen as a vehicle or venue for visualization as other aspects of the invention allow for it to be rendered in a collaborative context.
  • The spreadsheet visualization aspects of the present invention are a special case of other more general aspects of the invention including:
      • the use of arbitrary or integrated data sources (such as static databases, dynamic databases, streaming databases, live sensing data streams, numerical simulations, signal processing, statistical processing, linear and nonlinear transformations, etc.);
      • uniform parameterizations of selected or all visualization presentation parameters;
      • the support for real-time updates to integrated data sources (such as static databases, dynamic databases, streaming databases, live sensing data streams, numerical simulations, signal processing, statistical processing, linear and nonlinear transformations, etc.);
      • the use of data flow paths to link arbitrary data sources with arbitrary data destinations via arbitrary topologies (graphically, via an interconnection, specification, and/or data-flow language, etc.);
      • the providing of shared GUI environments for controlling two or more of visualization rendering, pre-visualization operations, and data sources.
  • FIG. 5 illustrates an arrangement of these more general aspects of the invention. Implicit in FIG. 5 are more general aspects of the invention, not restricted to spreadsheet visualization, that support visual rendering in a browser window and as a web application. As shown in the figure, graphical user interfaces (GUIs) are provided for including one or more relating to data sources and one or more relating to visualization processes. Additionally, the invention also provides for multifunction GUIs that provide roles involving both data source matters and visualization processes. The overall collection of user interfaces provides the following capabilities:
      • Setup of a fixed data analysis configuration;
      • Setup of fixed data visualization configuration;
      • Setup of fixed data visualization presentation;
      • Setup of an interactive data analysis configuration;
      • Setup of an interactive data analysis session;
      • Setup of an interactive data visualization configuration;
      • Setup of an interactive visualization session;
      • Interactive control of data source selection and usage;
      • Interactive control of data analysis sessions;
      • Interactive control of visualization sessions;
      • Storage and recall of configurations;
      • Storage and recall of data;
      • Storage and recall of visualizations;
      • Storage and recall of sessions.
  • Yet additional aspects of the invention, not restricted to spreadsheet visualization, also include (real-time and non-real-time) collaboration capabilities. FIG. 6 illustrates an adaptation of the arrangement of FIG. 5 to a collaboration environment supporting collaboration features such as one or more of file exchange, real-time or stored display sharing, real-time control sharing, real-time or stored collaborative annotation, and archives of collaboration sessions and transactions.
    • Uniform Parameterizations of Cell and Data Presentation Attributes
  • The invention provides for uniform parameterizations of selected or all visualization presentation parameters. This allows pre-visualization operations, such as scaling, translation, filtering, array (matrix, tensor) operations, nonlinear warping, etc. to be employed in a modular, cascadable fashion independent of the particular choice of data and cell presentation attributes.
    • Data Flow Paths to Implement Arbitrary Interconnection Topologies
  • The invention provides for the use of data flow paths to link arbitrary data sources with arbitrary data destinations via arbitrary topologies. This allows the selection and/or fusion of data sources, their interconnection with selected signal processing, statistical processing, pre-visualization operations, and visualization parameters (such as, among other examples, the cell and data presentation parameters of the spreadsheet visualization described earlier and in more detail to follow).
  • FIG. 7 depicts an exemplary topological interconnection of data flow paths linking various elements depicted in FIG. 5. In various embodiments, functions such as data re-indexing, statistical processing, and signal processing can be provided as the data sources depicted in FIG. 5 or as the pre-visualization functions depicted in FIG. 5. Similarly, numerical simulations, for example (but not limited to) those rendered by the computer running a spreadsheet program, a high-performance or other computer, can serve as the data sources depicted in FIG. 5. Certain pre-visualization functions, for example linear predictors, can in an embodiment be regarded as a numerical simulation. Additionally, in a spreadsheet visualization setting, calculations in the spreadsheet can provide a form of numerical simulation prior to visualization.
  • The invention provides for some or all of the data flow paths (such as depicted in the example of FIG. 7) to be specified in any convenient way, for example graphically via an interactive GUI or via a character-based language (interconnection, specification, and/or data-flow, etc.). In an exemplary embodiment, a GUI can permit the rendering of a graphic similar to that of FIG. 7. A GUI can permit creation and customization of instances of functional blocks such as the ones depicted in FIG. 7 from a library, menu, and/or graphical pallet. In an exemplary embodiment, a GUI can be used to create and link these customized instances of functional blocks, via link-by-link “drawing,” with a data path topology such as the ones depicted in FIG. 7.
  • Various types of user interfaces may be used to create configurations such as that of FIG. 7 and set the various parameters within it. As one example approach, the selection of function blocks and the general connectivity among them can be specified using a drawing tool and a palette of function blocks, and clicking on each function block would cause dialog windows to appear that can be used for setting parameters. The parameters can be set by means of typed-in values, sliders, mouse manipulation, or the advanced high-dimension user interface devices to be described later in the specification. More generally, a wide range of approaches, styles, and forms of configuration-specification, feature-selection, and parameter-setting GUIs are known in the art. A few example user interfaces or aspects of them will be mentioned to showcase different aspects, but a comprehensive treatment of user interface approaches is not necessary for enablement as general forms of configuration specification GUIs and parameter setting GUIs are known in the art. A number of example configuration-specification, feature-selection, and parameter-setting GUIs will be the subject of a later companion patent application.
    • Pre-Visualization Operations
  • Attention is now directed to consideration of pre-visualization operations. FIGS. 8A-8C depict an approach for mapping a data value lying within a pre-defined range to a value within a pre-defined range for a parameterized data or cell presentation attribute. In most cases the input data range must be at least scaled and shifted to match the pre-defined range for a parameterized presentation attribute. This (linear or affine transformation) arrangement is depicted in FIG. 8A. In some circumstances it can also be desirable to warp the data range with nonlinearity. In one embodiment a library of fixed or adjustable nonlinearities are provided which are such that the input and output of the nonlinearity both match the pre-defined range for a parameterized presentation attribute. This arrangement is depicted in FIG. 8B. In another embodiment the warping effect is provided with additional flexibility by allowing pre-scaling and pre-shifting prior to applying a selected nonlinearity and subjecting the outcome of the nonlinear warping to post-scaling and/or post-shifting operations in order to match the resulting range to the pre-defined range for a parameterized presentation attribute. This arrangement is depicted in FIG. 8C. Features and parameters of the above operations can be selected via user interface dialog windows. General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • FIG. 9 depicts a more general view and organization of pre-visualization operations provided for by the invention. In this example, available pre-visualization operations include:
      • Data indexing/re-indexing, data sorting, data suppression, and similar types of data operations;
      • Normalization, shifting (translation), and other types of linear and affine transformations;
      • Linear filtering, convolution, linear prediction, and other types of signal processing operations;
      • Warping, clipping, nonlinear transformations, nonlinear prediction, and other nonlinear transformations.
      • The invention provides for other types of pre-visualization operations as well.
        Features and parameters of the above operations can be selected via user interface dialog windows. General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • The invention also provides for the inclusion of statistical operations and statistical processing functions and for the linking to external programs to perform other types of pre-visualization operations. Features and parameters of these can be selected via user interface dialog windows. General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • The invention also provides for external programs to be added to the collection of available pre-visualization operations.
  • The invention additionally provides for the inclusion and use of visual metaphors to simplify visualization setup and user interaction for data exploration. As an example, FIG. 10 depicts an exemplary embodiment wherein a selected metaphor is used to automatically generate parameter assignments and graphics rendering operations. The invention provides for metaphors to control other aspects of the visualization and pre-visualization operations. The invention provides for a metaphor to base its operations on characteristics of a data set being visualized, previously visualized, and anticipated to be visualized. The invention additionally provides for metaphors to be selected and controlled by user interaction, data values, or other means. Features and parameters of the above operations can be selected via user interface dialog windows. General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art. The invention also provides for array (vector, matrix, tensor) operations such as (vector, matrix, tensor) linear combinations, (vector, matrix, tensor) multiplication, scalar multiplication, finding (matrix, tensor) determinants, finding (matrix, tensor) inverses and psuedoinverses, row reduction, factorization, change of basis, or calculation of an eigensystem (eigenvalues, eigenvectors, eigentensors).
  • Users can select the region of the spreadsheet to be stored in an array and give a name to the array. Different programs can require different formatting of arrays, thus, users can also choose the separating characters, such as a comma and a space or tab. The key of the array can be numbers from 0 to (the number of entries—1) or the values of any column user selects. Such array created can be exported to an external program. Similarly, data of one or multiple arrays can be implemented to be imported from an external program and the values of elements are displayed in appropriate numbers of rows and columns. FIG. 11A illustrates an example of a user interface that permits users to select the geometric element boundary of a captured or rendered tabular matrix data elements or cells within a spreadsheet. Other features and parameters of the above operations can be selected via user interface dialog windows. General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • In the case of spreadsheet visualization, spreadsheets natively provide certain types of array calculations with relative ease while other standard matrix and array operations, such as matrix/tensor multiplication, eigensystem calculations, convolutions, etc., have no native support within the spreadsheet paradigm. The invention provides for a matrix boundary tool for use in capturing and labeling matrix data and rendering matrix operation results. For example, users can specify the group of cells as the elements of one matrix or multiple matrices. FIG. 11B illustrates an exemplary user interface that permits users to select array operations to be performed on selected array data, for example as may be selected by the tool of FIG. 11A. Other features and parameters of the above operations can be selected via user interface dialog windows. General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • The invention can additionally transform a graphics object via matrix operations. As examples, FIG. 12A illustrates a figure being resized, rotated, and shifted through array-based scaling, rotating, and translating linear transformations. FIG. 12B illustrates processes of a figure being rotated around x-axis, y-axis, line of y=x, and line of y=−x. and FIG. 12 c illustrates processes of figure being rescaled vertically or horizontally. FIG. 12D illustrates a transformation process acting on 2D graphics objects such as a rectangle-bounded horizontal line array and a graphic. The invention also provides for embodiments to extend these and related systems and methods to transform 3D-graphics via scalings, rotations, stretch, etc.
  • The invention provides for dimensional transformations among points, 1D, 2D, and 3D graphics objects. FIG. 12E illustrates a matrix and/or tensor transformation process and how it changes a 3D graphics object into a 2D object via dimension dropping. FIG. 12F illustrates a 3D graphics object rendered of a 2D surface created by a vertical line rotated around the y-axis. FIG. 12G illustrates a 3D graphics object rendered by perspective projection defined by a point.
  • The invention also provides for 3D graphics generation from 3D vector draw-lists provided by other programs, spreadsheet data, and/or the user. In an embodiment, the invention also provides for 3D graphics generation from equations provided by the user.
  • The invention further provides for pre-visualization operations to themselves have parameters that can be adjusted in real time and/or be stored in files for recall.
  • The invention further provides for a network of pre-visualization operations to be stored in files for recall.
    • Spreadsheet Visualization Integration Architectures
  • The invention provides implementation of at least some of the invention's spreadsheet presentation features through use of APIs of existing spreadsheet products. The invention provides implementation of at least some of the invention's spreadsheet presentation features with relatively small augmentation and modification of existing product software by product manufacturers.
  • FIG. 13 depicts an example of spreadsheet architecture (right side) and an information flow as it might be used in a data visualization setting. One of or some combination of entered data, measured data, and/or computed data can be stored as stored data or stored in a queriable data base. In some situations live data feeds can also provide data. One or some combination of these data sources can be used to provide a data set used by the spreadsheet program. The spreadsheet program internally comprises at least a GUI component, a calculation compound, and a visual rendering component.
  • FIGS. 14A-14D depicts various types of exemplary relationships and information flows among a system or method for rendering a conventional spreadsheet or table of tabular data, a system or method for rendering the visualization operations of the invention, and a data set used by each. Visualization and pre-visualization operations can be introduced between the data and the spreadsheet program as depicted in FIG. 14A. In another embodiment, visualization and pre-visualization operations can obtain data from the spreadsheet program, process data, and return the generated visual operation back to the spreadsheet program as depicted in FIG. 14B. An example of this is using an API or the use of VBA in conjunction with Microsoft Excel.
  • Visualization and pre-visualization operations can also be implemented as parallel operations to the spreadsheet program. Visualization and pre-visualization operations can be configured separately as an independent operation(s) with direct access to data (i.e., able to obtain data and send the result to the data without having to go through the spreadsheet program), as shown in FIG. 11C. One advantage of this is that relevant categories of data can be extracted to generate complicated visualizations without the data being arranged into table form. Visualization and pre-visualization operations can be part of a spreadsheet program without direct access to the data, as depicted in FIG. 14D.
    • Spreadsheet Visualization User Interface Architectures: Traditional and High-Dimension User Interface Devices (HDTP and Advanced Mice)
  • Next, user interface architectures for general visualization and spreadsheet visualization environments are considered. FIG. 15 depicts an exemplary embodiment of an augmentation of a spreadsheet program with additional spreadsheet features as provided for by the invention. A data set is subjected to data operations so as to produce modified data. The data operations can be controlled by the spreadsheet GUI and spreadsheet calculation as well as by the visualization GUI and Visualization calculations.
  • FIGS. 16A-16B depict ways in which one or more user interface input devices (such as a mouse, trackball, touchpad, etc.) can be managed to control graphical user interfaces of a plural software application in the context of the invention. FIG. 16A, the windowing system directs the input events from a single user interface input device to various applications, for example, a Database GUI, a Simulation GUI, a Spreadsheet GUI, and a Visualization GUI. FIG. 16B depicts an alternative implementation wherein a Visualization Environment Input Router/Parser accepts input events from one or more user interface input devices and directs these events to each of the various applications. The Visualization Environment Input Router/Parser can direct events from multiple user interface input devices to different inputs to the same application. In another embodiment the Visualization Environment Input Router/Parser can split events originating from an individual user interface device and separately route these to inputs of separate applications. In another embodiment, a Visualization Environment Input Router/Parser works together with the Window System to route input events.
  • The above arrangements are general and apply to use of conventional mice, trackballs, touchpads, etc. However, visualization and CAD workstations have often been provided with more sophisticated user input devices that provide a higher number of interactive simultaneously-adjustable parameters. Classic examples of this are knob-boxes (as used in HP and SGI workstations), the DataGlove (VPL, General Reality), the SpaceBall (Logitech3Dconnexion, Labtec, HP/Compaq), etc., although few of these have survived product cycles to remain in active use or with wide availability. More recently enhanced touch-based interfaces have attracted a great deal of attention, mostly for their multi-touch and gesture recognition capabilities. However, some enhanced touch-based interfaces such as the HDTP (“High Dimensional Touch Pad,” U.S. Pat. No. 6,570,078; U.S. patent application Ser. Nos. 11/761,978 U.S. Ser. No. 12/418,605, and U.S. Ser. No. 12/502,203 among others) employ a tactile sensor array (pressure, proximity, etc.) and real-time image and mathematical processing to provide a powerful user input device with both a higher number of interactive simultaneously-adjustable parameters and a rich range of syntactic and metaphorical capabilities well-suited to use with interactive visualization. Additionally, the HDTP technology can be readily implemented as a touchscreen through use of, for example, inexpensive transparent capacitive proximity-sensor arrays. The present invention provides for the incorporation and use of the HDTP and other metaphor-rich user interface input devices into visualization environments. Attention is now directed to incorporation of the HDTP as part of a visualization environment user interface and then brief attention is directed to alternative use of a few alternative metaphor-rich enhanced user interface technology approaches.
  • FIGS. 17A-17F illustrate the six independently adjustable degrees of freedom of touch from a single finger that can be simultaneously measured by the HDTP technology. The depiction in these figures is from the side of the touchpad. FIGS. 17A-17C show actions of positional change (amounting to applied pressure in the case of FIG. 17C) while FIGS. 17D-17F show actions of angular change. Each of these can be used to control a user interface parameter, allowing the touch of a single fingertip to control up to six simultaneously-adjustable quantities in an interactive user interface. FIG. 18 suggests general ways in which two or more of these six independently adjustable degrees of freedom 1811-1816 can be adjusted at once for a finger 1800 in contact with an HDTP sensor surface 1801. More advanced implementations of the HTDP provide for multi-touch capabilities far more sophisticated than those popularized by the Apple iPhone, NYU, and others.
  • FIG. 19 demonstrates a few exemplary two-finger multi-touch postures and/or gestures from the hundreds that can be readily recognized by HTDP technology. The two fingers may be spread apart somewhat, as in the cases 1901-1903 and 1931-1933, or may be brought together, as in the cases 1911-1913 and 1941-1943. The separation of the two fingers adds an additional parameter that can be controlled by the user, and the yaw (pivot) parameter may be varied independently from the newly introduced finger separation parameter. For example the hand can pivot clockwise 1902, 1912, or counter-clockwise 1903, 1913 from the corresponding neutral positions 1901, 1911. Similarly, the pitch of the pair of fingers can be raised 1932, 1942 or lowered 1933, 1943 with respect to the corresponding neutral positions 1931, 1941, while independently from these motions the two fingers are spread apart 1931-1933 or brought together 1941-1943. Note the finger spread can be varied considerably in this two-finger posture.
  • The pair of fingers can be moved as a group through all six degrees of freedom (left-right, forward-back, downward pressure, roll, pitch, yaw), and comfortably allow for two differences between the fingers (differences in downward pressure, and one angle of finger separation spread) and even three differences between the fingers (differences in downward pressure and two coordinates of separation when fingers curl to form independently controlled “x” and “y” components). Thus two-finger postures considered above can readily provide a nine-parameter set relating to the pair of fingers as a separate composite object adjustable within an ergonomically comfortable range. One example nine-parameter set the two-finger postures comprises:
      • composite (group) average x position;
      • inter-finger differential x position;
      • composite (group) average y position;
      • inter-finger differential y position;
      • composite (group) average pressure;
      • inter-finger differential pressure;
      • composite (group) roll;
      • composite (group) pitch;
      • composite (group) yaw.
  • HTDP technology can also be configured to recognize and measure postures and/or gestures involving three or more fingers, various parts of the hand, the entire hand, multiple hands, etc. In general, multifinger contact can be used to provide control of up to three additional independently adjustable parameters for each additional finger.
  • FIG. 20 shows an example of how raw measurements of the six quantities of FIGS. 17A-17F, together with shape recognition for distinguishing contact with various parts of the hand and the touchpad, can be used to create a rich information flux of parameters, rates, and symbols. FIG. 21 shows an exemplary approach for incorporating posture recognition, gesture recognition, state machines, and parsers to create an even richer human/machine tactile interface system capable of incorporating syntax and grammars.
  • FIG. 22 depicts an exemplary user interface input arrangement incorporating one or more HDTPs that provides a user interface input event and quantity routing of the type described earlier in conjunction with FIGS. 16 a-16 b.
  • FIGS. 23A-23L depict a number of arrangements employing the HDTP technology suitable for use with visualization environments. FIG. 23A illustrates an HDTP as a peripheral that can be used with a desktop computer (shown) or laptop (not shown). FIG. 23B depicts an HDTP integrated into a laptop in place of the traditional touchpad pointing device. In FIGS. 23A-23B the HDTP tactile sensor can be a stand-alone component or can be integrated over a display to form a touchscreen. FIG. 23C depicts an HDTP integrated into a desktop computer display to form a touchscreen. FIG. 23D shows the HDTP integrated into a laptop computer display to form a touchscreen. FIG. 23E depicts an HDTP integrated into a cellphone, smartphone, PDA, or other hand-held consumer device. FIG. 23F shows an HDTP integrated into a test instrument, portable service-tracking device, portable service-entry device, field instrument, or other hand-held industrial device. In FIGS. 23E-23F the HDTP tactile sensor can be a stand-alone component or can be integrated over a display to form a touchscreen. FIG. 23G depicts an HDTP touchscreen configuration that can be used in a tablet computer, wall-mount computer monitor, digital television, video conferencing screen, kiosk, etc. In at least the arrangements of FIGS. 23A, 23C, 23D, and 23G, or other sufficiently large tactile sensor implementation of the HDTP, more than one hand can be used and individually recognized as such.
  • FIGS. 23H-23K depict various integrations of an HDTP into the back of a conventional computer mouse. In FIGS. 23H-23K the HDTP tactile sensor can be a stand-alone component or can be integrated over a display to form a touchscreen. More than two touchpads can be included, such as the example of FIG. 23L as taught in U.S. Pat. No. 7,557,797.
  • The types of human-machine geometric interaction between the hand and the HDTP facilitate many useful applications within a visualization environment. A few of these include control of visualization observation viewpoint location, orientation of the visualization, and controlling fixed or selectable ensembles of one or more of viewing parameters, visualization rendering parameters, pre-visualization operations parameters, data selection parameters, simulation control parameters, etc. As one example, the 6D orientation of a finger can be naturally associated with visualization observation viewpoint location and orientation, location and orientation of the visualization graphics, etc. As another example, the 6D orientation of a finger can be naturally associated with a vector field orientation for introducing synthetic measurements in a numerical simulation. As another example, at least some aspects of the 6D orientation of a finger can be naturally associated with the orientation of a robotically positioned sensor providing actual measurement data. As another example, the 6D orientation of a finger can be naturally associated with an object location and orientation in a numerical simulation. As another example, the large number of interactive parameters can be abstractly associated with viewing parameters, visualization rendering parameters, pre-visualization operations parameters, data selection parameters, numeric simulation control parameters, etc.
  • In another example, the x and y parameters provided by the HDTP can be used for focus selection and the remaining parameters can be used to control parameters within a selected GUI.
  • In another example, the x and y parameters provided by the HDTP can be regarded as specifying a position within an underlying base plane and the roll and pitch angles can be regarded as specifying a position within a superimposed parallel plane. In a first exemplary extension of the previous two-plane example, the yaw angle can be regarded as the rotational angle between the base and superimposed planes. In a second exemplary extension of the previous two-plane example, the finger pressure can be employed to determine the distance between the base and superimposed planes. In a variation of the previous two-plane example, the base and superimposed planes are not fixed as being parallel but rather intersect at an angle associated with the yaw angle of the finger. In each of these examples, either or both of the two planes can be used to represent an index or indexed data, a position, pair of parameters, etc. of a viewing aspect, visualization rendering aspect, pre-visualization operations, data selection, numeric simulation control, etc.
  • A large number of other examples are possible as is appreciated by one skilled in the art.
  • One use of the HDTP in the above examples is simply to supply more than the usual two user interface parameters provided by a conventional user interface input device such as a conventional computer mouse, trackball, touchpad, etc. The present invention provides for the use of other user interface input arrangements and devices as alternatives to or in conjunction with one or more HDTPs.
  • In a simple example, the scroll-wheel of a scroll-wheel mouse is used to provide a third simultaneously adjustable user interface parameter. In another example, a second or yet more additional scroll-wheels can be added to a conventional scroll-wheel mouse. The resultant collection of scroll-wheels can be relatively positioned in parallel, oriented at orthogonal angles to support a coordinate-metaphor, positioned on the sides of the mouse body, etc. FIGS. 24A and 24B illustrate examples 2410, 2420 of a conventional scroll- wheel mouse 2401, 2402 with a traditional up-down scrollwheel 2421 and at least standard mouse buttons 2411, 2412 provided with an added left-right scroll-wheel 2422 as taught in U.S. Pat. No. 7,557,797. Such an arrangement can employ a connecting cable, or the device can be wireless. Each of the arrangements of FIGS. 24A and 24B as well as the ones described in the next paragraph, plus variations and adaptations, will be referred to as an “advanced mouse” or in the plural “advanced mice.”
  • In another example of an advanced mouse, one or more trackballs can be added to a conventional computer mouse. FIGS. 25A-25C illustrate examples 2510, 2520, 2530 wherein a single trackball 2511, 2521, 2531 is incorporated into the back of a conventional computer mouse 2510, 2520, 2530 with at least standard mouse buttons 2512-2513, 2522-2523, 2532-2533 as taught in U.S. Pat. No. 7,557,797. FIGS. 26A-26C illustrate examples where two trackballs 2605 a, 2605 b are incorporated into the back of a conventional computer mouse as taught in U.S. Pat. No. 7,557,797. The trackballs in the arrangements of FIGS. 25A-25C and FIGS. 26A-26C can be the conventional two degree of freedom type (roll left-right, roll away-towards) or can provide three to six degrees of freedom as taught in U.S. Pat. No. 7,557,797; U.S. patent application Ser. No. 10/806,694. Useful exemplary advanced mouse arrangements include the trackball 2605 a-2605 b, touchpad 2605 c, scrollwheel 2668 mouse of FIG. 26C, the trackball/touchpad mouse of FIG. 26 d, and the multiple slider configuration of FIG. 26E, each as taught in U.S. Pat. No. 7,557,797. Each of these arrangements can employ a connecting cable, or the advanced mouse device can be wireless.
  • The additional parameters provided by the HDTP and the above alternatives are more than the usual number supported by conventional window systems (for example as described in conjunction with FIG. 16A). To implement additional parameter handling, for example such as in the HDTP arrangement of FIG. 22 or its equivalents for the alternative advanced mice as described above, additional arrangements must be made beyond the conventional operating system pointer device handling.
  • In an additional approach, the invention provides for the HDTP or alternatives such as the advanced mice described above to interface with a browser via a browser plug-in. This arrangement can be used to capture the additional user interface input parameters and pass these on to an application interfacing to the browser. An example of such an arrangement is depicted in FIG. 27. The browser can interface with local or web-based applications that drive the visualization and/or control the data source(s), process the data, etc. The browser can be provided with client-side software such as JAVA Script, AJAX, etc. Such an arrangement is particularly useful when combined with browser-based rendering of visualizations, as described in the next section.
  • The invention provides for HDTP parameters to be separated into groups which are individually directed to pointer device interfaces on multiple computers.
    • Browser-Rendered Implementations
  • The invention provides for visualizations to be rendered in a browser. This allows for implementations wherein the browser is used as a viewer. The browser can interface with local or web-based applications that drive the visualization. An exemplary arrangement is depicted in FIG. 28. In an embodiment, the browser can be provided with Simple Vector Graphics (“SVG”) utilities (natively or via an SVG plug-in) to render basic 2D vector and raster graphics. In another embodiment, the browser can be provided with a 3D graphics capability, for example via the Cortona 3D browser plug-in. These embodiments or alternatives can be provided with client-side software such as JAVA Script, AJAX, etc. The example of FIG. 28 also provides for other plug-ins (represented by the dashed-line box) such as the user interface input device plug-in described earlier.
    • Web-Based Implementations
  • The invention provides for web-based implementations of the visualization environment. FIG. 29 depicts an exemplary web-based implementation of a visualization environment leveraging browser arrangements such as that depicted in FIGS. 27-28, their variations, alternatives, etc. As depicted in the arrangement of FIG. 29, the client environment can employ such a browser to access a visualization server environment or visualization-related (databases, live data feeds, simulations, statistical or signal processing, etc.) server environment. In an embodiment, the client environment browser can connect over the internet with two or more visualization server environments and/or visualization-related server environments and/or local applications. In an embodiment, a visualization or visualization-related server environment can connect over the internet with two or more browsers in the client environment. Other variations and alternative implementations are also possible as is clear to one skilled in the art.
  • A server environment can include web server foundation software such as Apache, IIS, etc. A server environment can also include server-side scripting and dynamic software such as CGI, JAVA/JAVA Script, Python, PHP, Perl, JSP, ASP, etc. Other variations and alternative implementations are also possible as is clear to one skilled in the art.
  • The additional interactively-controlled parameters provided by the HDTP provide more than the usual number supported by conventional browser systems and browser networking environments. This may be addressed in a number of ways.
  • In a first approach, an HDTP interfaces with a browser both in a traditional way and additionally via a browser plug-in. Such an arrangement may be used to capture the additional user interface input parameters and pass these on to an application interfacing to the browser. An example of such an arrangement is depicted in FIG. 30D.
  • In a second approach, an HDTP interfaces with a browser in a traditional way and directs additional GUI parameters though other network channels. Such an arrangement may be used to capture the additional user interface input parameters and pass these on to an application interfacing to the browser. An example of such an arrangement is depicted in FIG. 30E.
  • In a third approach, an HDTP interfaces all parameters to the browser directly. Such an arrangement may be used to capture the additional user interface input parameters and pass these on to an application interfacing to the browser. An example of such an arrangement is depicted in FIG. 30F.
  • The browser may interface with local or web-based applications that drive the visualization and/or control the data source(s), process the data, etc. The browser may be provided with client-side software such as JAVA Script, AJAX, etc. The browser may provide also be configured advanced graphics to be rendered within the browser display environment, allowing the browser to be used as a viewer for data visualizations, advanced animations, etc., leveraging the additional multiple parameter capabilities of the HDTP. The browser may interface with local or web-based applications that drive the advanced graphics. In an embodiment, the browser may be provided with Simple Vector Graphics (“SVG”) utilities (natively or via an SVG plug-in) so as to render basic 2D vector and raster graphics. In another embodiment, the browser may be provided with a 3D graphics capability, for example via the Cortona 3D browser plug-in.
    • Collaboration Implementations
  • The invention provides for collaboration implementations and the use of collaboration tools for collaborative use of visualization features of the invention and for the creation of a collaborative visualization environment.
  • FIG. 30A shows a client-side collaboration implementation wherein a general purpose collaboration tool can be used to share the visualization environment session running on one computer with one or more other computers. The visualization environment can be a stand-alone visualization application or can comprise a browser implementation employing, for example, any of the various arrangements described in conjunction with FIGS. 27-29.
  • In the case of real-time collaboration, the general purpose collaboration tool employed in the arrangement of FIG. 30 can comprise application sharing software (such as Microsoft's NetMeeting™, IBM/Lotus' SameTime™,SM, HP's Shared-X, etc.) or display sharing software. In the case of real-time collaboration, audio and/or video conferencing can also be used to facilitate interpersonal communication among collaborating session participants.
  • FIG. 30B shows server-side collaboration implementation wherein a general purpose or specialized server-side application sharing environment is used to share a server-based interactive data visualization application.
  • FIG. 30C shows a server-side collaboration implementation comprising a server-based multi-user interactive data visualization application.
  • Interfacing with Sonification
  • The invention further provides for interfacing with established and advanced data sonification utilities. Data sonification has received considerable attention and analysis but for the most part is fairly simplistic and often contributes little practical value. More advanced data sonification techniques, such as the multi-channel data sonification system described in U.S. Patent Application 61/268,856 employed in the environmental GIS (Geographic Information System) process monitoring and modeling system of U.S. Patent Application 61/268,873, provides new opportunities for practical use of data sonification.
  • FIG. 31 illustrates a general framework for data sonification where a parameterized metaphor is used to manage parameter assignment and sound rendering and a parameterized data indexing operation.
  • Data sonification may be applied to the same data used to generate visualization. The data directed to data sonification may be selected by interacting with a rendered visualization via a user interface input device. Data sonification may be provided by a multichannel data sonification system such as the one described in U.S. Patent Application 61/268,856. Data sonification output may be shared using an audio channel of a real-time collaboration system, may be transmitted from a web-based application using an audio channel as provided by a high-fidelity VoIP system, may be shared using an audio channel as provided by a high-fidelity VoIP system, may be produced local to the user computer under the control of a web-based application, or may be produced local to the user computer under the control of a web-based application employing MIDI protocol.
  • Some user interface considerations relating to configurations, features and parameters of data sonification operations are described in the aforementioned U.S. Patent Application 61/268,856. Beyond these, and more generally, configurations, features and parameters of data sonification operations can be selected via user interface dialog windows. General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
    • Uniform Parameterizations of Cell and Data Presentation Attributes
  • The invention provides for visualization rendering parameters to be uniform over a common variational range. This permits uniform handling of visualization rendering parameters and numerical operation compositions. Any of these can be set, varied, or modulated as selected or as advantageous to represent data or information derived from data. In an embodiment, the invention provides for visualization rendering uniform parameters in the range of [0,1]. The uniform parameter range of [0,1] is used in the examples below, but it is understood that other choices for the uniform range are also possible and are provided for by the invention.
  • Color Morphing
  • The invention provides for at least one color selection option to be determined by a uniform parameter in the range of [0,1]. FIG. 32 depicts an exemplary embodiment wherein the hue of a color can be varied through natural colors of red through violet for the bulk of the [0,1] range and a range of artificial colors, such as the purple through magenta continuum, are appended for the remaining portion of the [0,1] range. The invention also provides for other color models, such as RGB, HSB, LUV, LAB, grayscale, etc. to be used with one to three uniform parameters, each in the range of [0,1]. Such uniform parameter colors can be assigned one or more of text, symbols, borders, backgrounds, gridlines, glyphs, lines, components of lines, components of stipples, components of gradients, surfaces, and other geometric primitives employed in the visualization of data. Other variations and alternative arrangements are possible as is clear to one skilled in the art.
  • A uniform parameter range of [0,1] is used in the examples above, but it is understood that other choices for the uniform range are also possible and are provided for by the invention.
  • Line Morphing
  • The invention provides for at least one line rendering option to be determined by a uniform parameter in the range of [0,1]. FIG. 33 illustrates an example wherein line width is determined by a uniform parameter in the range of [0,1]. FIG. 33 also illustrates an example wherein line dashing period is determined by a uniform parameter in the range of [0,1]. FIG. 33 further illustrates an example wherein line dashing duty-cycle is determined by a uniform parameter in the range of [0,1]. FIG. 33 also illustrates an example wherein line dashing sub-duty-cycle is determined by a uniform parameter in the range of [0,1]. Other variations and alternative arrangements are possible as is clear to one skilled in the art.
  • The invention provides for line color to be determined by a uniform parameter color model as described above and for line dashing to include a plurality of colors.
  • Stipple Morphing
  • The invention provides for at least stipple pattern rendering option to be determined by a uniform parameter in the range of [0,1]. FIG. 34 illustrates an example wherein stipple line width is determined by a uniform parameter in the range of [0,1]. FIG. 34 also illustrates an example wherein stipple gap is determined by a uniform parameter in the range of [0,1]. A period and duty cycle model can be used instead and controlled via respectively associated uniform parameters varying over the range of [0,1].
  • FIG. 34 further illustrates an example wherein stipple angle is determined by a uniform parameter in the range of [0,1]. Other variations and alternative arrangements are possible as is clear to one skilled in the art.
  • The invention provides for at least one of stipple foreground and background colors to be determined by a uniform parameter color model as described above. The invention provides for a stipple pattern to comprise more than two colors, for the inclusion of line dashing in a stipple pattern, and for line dashing to include a plurality of colors.
  • Other variations and alternative arrangements are possible as is clear to one skilled in the art. A uniform parameter range of [0,1] is used in the examples above, but it is understood that other choices for the uniform range are also possible and are provided for by the invention.
  • Border Morphing
  • The invention provides for at least one border rendering option to be determined by a uniform parameter in the range of [0,1]. The border rendering option can comprise the border of a spreadsheet cell as depicted in FIG. 35, or can comprise the border of another visually rendered graphical object.
  • In the complete cell depicted in the upper right corner of FIG. 35, all four sides of the cell border are uniform. Such a border to have a common width W0 controlled by a uniform parameter in the range of [0,1] and to have a common color controlled by one or more uniform parameters in the range of [0,1]. The cell border can be rendered as a gradient among two or more selected colors, wherein the colors and/or gradient characteristic can be respectfully determined by an associated uniform parameter in the range of [0,1].
  • In the complete cell depicted in the lower right corner of FIG. 35, all four sides of the cell border are distinct. The invention provides for each such border segment to have an associated width (W1-W4), each respectively controlled by an associated uniform parameter in the range of [0,1]. The invention further provides for each such border segment to have a common color controlled by one or more uniform parameters in the range of [0,1]. A border segment can be rendered as a gradient among two or more selected colors, wherein the colors and/or gradient characteristic can be respectfully determined by an associated uniform parameter in the range of [0,1].
  • The invention further provides for such a cell (interior) background to be filled with a color controlled by one or more uniform parameters in the range of [0,1]. In an embodiment, the invention provides for stipple patterns to be used as a cell background. The cell background can be rendered as a gradient among two or more selected colors, wherein the colors and/or gradient characteristic can be respectfully determined by an associated uniform parameter in the range of [0,1].
  • The invention also provides for the four sides of a cell border to be segmented in other ways, for example as in the two complete cells in the left side of FIG. 35.
  • Other variations and alternative arrangements are possible as is clear to one skilled in the art. A uniform parameter range of [0,1] is used in the examples above, but it is understood that other choices for the uniform range are also possible and are provided for by the invention.
  • Font Morphing
  • The invention provides for font morphing and font interpolation under the control of uniform parameters in the range of [0,1]. Other choices for the uniform range are also possible and are provided for by the invention.
  • FIG. 36A depicts a boldness parameter under the control of uniform parameters in the range of [0,1]. In an embodiment, the variation in boldness can be implemented so that it is governed by the optical properties of de Groot Interpolation Theory. In an embodiment, a minimum and maximum degree of boldness are assigned to the extremes of the parameter range.
  • FIG. 36B depicts an italics angle parameter under the control of uniform parameters in the range of [0,1]. In an embodiment, a minimum and maximum degree of italics angle is assigned to extremes of the parameter range.
  • FIG. 36C depicts an font family interpolation parameter under the control of uniform parameters in the range of [0,1]. One font family is assigned to the minimum extreme of the parameter range and a second font family is assigned to the maximum extreme of the parameter range.
  • FIG. 37A depicts vertical and horizontal dilation/contraction parameters, each respectively under the control of uniform parameters in the range of [0,1]. In an embodiment, a minimum and maximum degree of italics angle are assigned to the extremes of the parameter range.
  • FIG. 37B shows font underline line width and font underline separation gap parameters, each respectively under the control of uniform parameters in the range of [0,1]. In an embodiment, a minimum and maximum degree of italics angle are assigned to the extremes of the parameter range.
  • Other variations and alternative arrangements are possible as is clear to one skilled in the art. A uniform parameter range of [0,1] is used in the examples above, but it is understood that other choices for the uniform range are also possible and are provided for by the invention.
    • User Interfaces for Adjusting Cell and Data Presentation Attributes
  • Configurations, features and parameters for related operations, such as data set selection, data ranges, offsets, scalings, warping functions, etc., can be, for example, selected via traditional types of user interface dialog windows. General forms configuration-specification, feature-selection, and parameter-setting GUIs are known in the art.
  • The invention provides for user interfaces to simplify operation with a friendly user interface for new or occasional users. A typical attribute control panel, as shown in FIG. 38, contains a few pre-selected values and a value which can be customized usually by selecting a value from a drop-down list.
  • Also, users can specify if the changes in attributes are to be applied globally or are to be applied to a single cell or a group of cells in a spreadsheet. The change to be made can also be determined by the value of a data element or by the result of a formula calculated from one or more data elements.
  • FIG. 39 illustrates an example of how change in color of a font can be specified for a cell, a group of cells, or globally across an entire worksheet. Users can select the color of a font simply by clicking on a region in the gray level or Hue color chart or by moving the slider that determines the gray level or parameters in Hue. A formula that determines such value can also be included. For example, the ratio of the data in a cell to the range of the data in a category can determine the value of gray level accordingly.
  • FIG. 40 depicts an interface for determining a font type of the specified cell(s). Such interface can have a pre-selected list or a list to be customized or a combination of both, and can be implemented in a wide variety of ways.
  • FIG. 41 depicts an example of an interface for determining the embellishment of a font in rendering some data element within the spreadsheet or data table. The data element can be displayed in bold face, italics or underlined. The percentage of bolding or the thickness of the underline can be varied. For example a data element can be emphasized by being displayed in bold face, or a title of a book or quotation can be displayed in italics, or a link to a website or a reference to another cell or region can be underlined.
  • FIG. 42 depicts an example interface to control the size of a font to be displayed. Several preset sizes are listed on the panel as well as a slider that enables users to control the font size gradually. Font size can be incremental with step sizes or continuous, which can be adjusted by controlling a slider, as long as the value is positive and less than reasonable size compared to the area of the text displayed.
  • The invention provides for a user interface to determine addition of an appending symbol. The symbol(s) can be different types, numbers, or sizes of graphical objects. The factor that determines can depend on the significance or the ratio of the element to the category or the value of another category in the dataset. In this example, the symbols appended to the right end of the chart represent different categories of expenses. If the symbol appended is configured to represent quantity, the number of symbols to be appended will depend on the ratio of the value of the cell to the unit or base value. If the symbol appended is configured to represent significance, the size of symbols to be appended will depend on the significance of the value of the cell in that category. FIG. 43 depicts an example wherein data in a cells in a selected column can have symbol appended to numbers or other data within the cells of the column.
  • Different formatting options can be applied to different types of data. The factor that determines the type of data can be obtained from the result of a hidden formula, from associated data such as the data from another column in the same row, or simply selected by the user. The number of decimals can be determined according to such factors. Also symbols can be attached according to the type of data. If the data in a cell represents currency, a currency symbol can be attached, and if the data represents percentage, a percentage sign will be attached. Further if the country is specified in another column, the corresponding country's currency sign can be attached.
  • The invention provides for the varying the color and opacity/transparency of the background color of a cell or group of cells as a function of data values. Specifying background color and opacity/transparency can be done similarly with the case for color of the font is determined, as illustrated in FIG. 39. For example, a user can affect the control the background color and opacity/transparency with the sliders or by simply clicking on a region of color or grayscale chart. The background color control can also be incorporate auto-scaling. Optionally, and the color can be determined depending on the ratio of the value of the cell to the values in the column.
  • Similarly, the invention provides for the background texture or stipples of a cell or group of cells can also be varied as a function of data values. FIG. 44 depicts an example use of controlling the background texture or stipples of cells as a function of data values in a spreadsheet or other form of tabular data. FIG. 45 depicts an exemplary interface that can be used to specify the background texture or stipples to be applied to a cell or a group of cells. The stipples of a cell can be made up of dots, shapes, or grid lines. The size, thickness, density, or opacity of the background can be adjusted with sliders as well. Stipples can be used alone or in combination with the background color.
  • The invention provides for the specifying the location of a displayed data element within a cell. FIG. 46 depicts an exemplary interface that can be used to specify the position of text being displayed in a cell. Users can move sliders by clicking and dragging on the icon and moving it vertically or horizontally to specify vertical or horizontal position of text displayed in a cell. Users can specify the point of vertical and horizontal centering by clicking on a region in the provided box and selecting how texts are aligned horizontally or vertically relative to the point specified. In an embodiment, values representing the left-right or up-down ratio can be simply entered in text boxes by the users.
  • The invention provides for the specifying the location of a displayed rotation of text, symbols, or other information within a cell. FIG. 47 depicts an exemplary interface that can be used to specify the rotation of text, symbols, or other information displayed combined with the vertical or horizontal placement of the text. By clicking and dragging the right tip of the slider handle, the user can control the text to be rotated counter-clockwise, and vice versa. Vertical or horizontal shift of the position of the text can be controlled by clicking and dragging the middle of the slider handle to the desired direction.
  • The invention provides for a user interface to select of the color of at least one border of a cell, the thickness of at least one border of a cell enveloping a font, the thickness of at least one border outlining of a plurality of contiguous cells, and for a rendered image element.
  • The invention provides for data of a specified category to be extracted from cells and matched with hidden formula to obtain data used in the visualization. Data from another associated group of cells can be used for another attribute in the visualization. For example, data from one category can be used to determine the type of object displayed, another category can be used to determine the location of the object displayed, another category can be used to determine the number of objects displayed, and data from another category can be used to determine the size of the object(s) displayed. Existing spreadsheet applications offer various types of charts, but the placing of the objects are limited to aligning them in certain orders or directions.
  • The invention provides for visualizations to be co-rendered with the spreadsheet program. The invention provides for visualizations displayed on a specified region of a spreadsheet. In an embodiment, the invention provides for the result of visualizations to be exported to outside of the spreadsheet program.
    • Spreadsheet Visualizations
  • The invention provides for visual aspects of groups of cells to be varied together as a function of a data value. For example, FIG. 48 depicts an exemplary visualization wherein an attribute of a column area of data elements is varied as a group. In this example a set of collected data containing the state, main industry, population, the increase in population, number in current work force, unemployment rate, and the increase in unemployment rate can be used for different purposes. Depending on what category the emphasis is on, an attribute of that column can be varied. Such emphasis can be done by varying the font boldness, font size, or font color. FIG. 49 depicts an exemplary visualization wherein the shading of a row area of data elements is varied as a group as a function of a data value and also wherein the font boldness of a column area of data elements is varied as a group as a function of another data value.
  • The invention provides for a reordering of tabular rows and/or columns as a function of data values. Unlike a traditional spreadsheet data sort, such a reordering operates as a “live” function, responsive to interactive changes of the underlying data values. FIG. 50 illustrates a model of a priority queue wherein the rows can automatically reorder among themselves as underlying priority sales change. Jobs to be done can have different attributes, such as location or types of jobs along with numbers that indicate the rating of priority. A user would expect jobs with a higher priority rating to be displayed in a higher position of the priority queue tree listing and to reorder as calculated priority values change.
  • In another aspect of the invention function or mapping utilities can be included in the program. FIG. 51A illustrates a basic map-based visualization example generated from previous data. FIG. 51B illustrates the visualization of FIG. 51A augmented with additional spatially located symbols or parameterized glyphs.
    • Interactive Tabular Spreadsheet Metaphor for Presentation of Visualization Renderings
  • The invention provides for interactive tabular spreadsheet metaphor for presentation of visualization renderings. A cell may be selected and the data used to create the enclosed visualization can be captured (as structured for the enclosed visualization) to be used in a subsequent visualization or calculation. A cell may be selected and the pre-visualization dataflow and processing steps used to create the enclosed visualization can be captured (as structured for the enclosed visualization) to be used in a subsequent visualization or calculation. A cell may be selected and the visualization format employed by the enclosed visualization can be captured (as structured for the enclosed visualization) to be used in a subsequent visualization or calculation.
  • Two cells can be selected and the data used to create the enclosed visualizations can be captured (as structured for the enclosed visualization) and provided to a mathematical operation (for example, adding, subtracting, multiplying, convolving, etc.) to form a new dataset to be used in a subsequent visualization or calculation. Similarly, multiple cells can be selected and the data used to create the enclosed visualizations can be captured (as structured for the enclosed visualization) and merged to form a new dataset to be used in a subsequent visualization or calculation. Embodiments of the invention can also include one or more of the following optional features:
      • At least part of the captured data can be re-sampled. In an embodiment, redundant data occurring in the merged dataset is removed;
      • Gaps in data occurring in the merged dataset are noted. In an embodiment, gaps in data occurring in the merged dataset can be filled in via interpolating functions;
      • At least one partial row and at least one partial column can be selected and an operation applied to the underlying data;
      • At least one partial row and at least one partial column can be selected and an operation applied to the individually enclosed visualizations;
      • At least one partial row and at least one partial column can be selected and at least one attribute and/or parameter of a common visualization operation can be adjusted and applied to each of the individually enclosed visualizations;
      • At least one partial row and at least one partial column can be selected and at least one attribute and/or parameter of a common pre-visualization operation can be adjusted and applied to each of the individually enclosed visualizations;
        • At least one partial row and at least one partial column can be selected and at least one attribute and/or parameter of a common data source selection can be adjusted and applied to each of the individually enclosed visualizations.
    Exemplary 3D Graphics Augmentations to Spreadsheet Visualizations
  • The invention provides for the inclusion of useful novel 3D graphics visualization functions for multidimensional data representation, data query, and numerical solution tools for simultaneous interactive numerical equations. These can be incorporated in various ways, as described below, to expand spreadsheet—based visualization capabilities yet further.
  • A first set of the useful novel 3D graphics visualization functions provided for by the invention pertain to multidimensional data visualizations based on 2D-surfaces embedded in a 3D visual field. FIG. 52 data or spreadsheet, these separated by a vertical gap within a 3D visual field as provided for by the invention. The invention also provides for interactively shifting the observation point with respect to these objects for more detailed feature, theme, or trend inspection. The viewpoint can be changed according to the 6 degrees of freedom (three translations, three angles) of rigid motion. FIG. 53 depicts with dashed lines a few ways in which the observation viewpoint can be moved with respect to these abstract objects. The viewpoint is changed under the control of an HDTP or can be changed under the control of an advanced mouse.
  • The invention also provides for moving the 2D-surface representing a 3D data plot within the 3D visual field. The 2D-surface can be moved according to the 6 degrees of freedom (three translations, three angles) of rigid motion. FIG. 54 depicts a translation and rotation of a 3D data plot within the 3D visual field. The 3D data plot can be moved under the control of an HDTP or the 3D data plot can be moved under the control of an advanced mouse.
  • The 2D-surface representing a 3D data plot within the 3D visual field can be virtually illuminated by one or more lighting sources. The one or more lighting sources can be moved according to the 6 degrees of freedom (three translations, three angles) of rigid motion. The one or more lighting sources can also permit control of the color and intensity of virtual light emitted. A lighting source can be moved and controlled under the control of an HDTP or under the control of an advanced mouse.
  • The data plotted in the 2D-surface representing a 3D data plot within the 3D visual field can directly echo the data displayed in the exemplary planar array of associated tabular data or spreadsheet and/or can originate from another set of tabular data or spreadsheet region and/or can originate from other data. This is suggested by FIG. 55A. The invention also provides for processing of the data via mathematical transformations, statistical processing, signal processing, etc. prior to creation of the 2D-surface representing at least 3-dimensional data. This is suggested by FIG. 55B. In an embodiment, the 2D-surface representing at least 3-dimensional data can plot filtered or averaged versions of tabular data, spreadsheet data, or other data. In an embodiment, and the filtering or averaging can be controlled by an interactive parameter.
  • The invention provides for 2D-surface representing of at least 3-dimensional data to be rendered along with a height measuring visual so that the local color of the surface need not be a function of the vertical value being plotted. This frees up the local 2D-surface color to be used to represent an additional dimension of data. FIG. 56 illustrates an exemplary height measuring visual in its use with 2D-surface representing at least 3-dimensional data and useful for representing at least 4-dimensional data.
  • The invention provides for a 2D-surface representing at least 3-dimensional data to be rendered with a parameterized locally varying surface texture that can be used to represent an additional dimension of data, with numerical values rendered on the surface that can be used to represent an additional dimension of data, with symbols rendered on the surface that can be used to represent an additional dimension of data, or with parameterized glyphs rendered on the surface that can be used to represent (an) additional dimension(s) of data.
  • The invention provides for a slicing function to provide level set data. FIG. 57 depicts an exemplary slicing function, here used to provide a (planar-slice) level set curve. Other uses, implementations, and applications for slicing functions are also possible.
  • The invention provides for a more general surface intersection tool that can be used as a numerical solution operator for plotted, interpolated, and processed data. FIG. 58A illustrates two exemplary 2D-surfaces, each representing at least 3-dimensional data. Each of these can be interactively moved and rotated as described earlier. FIG. 58B illustrates the interactively created intersection of the two exemplary 2D-surfaces and the resultant intersection curve. A 2D-surface intersection tool allows the numerical values of selected points on a curve formed by the intersecting surfaces to be numerically calculated and visually displayed. The numerical values can also be used for other purposes and other subsequent calculations.
  • The sample points can be selected according to the details of the intersection. An adaptive sampling method can be employed. The resultant intersection data can be interpolated and re-sampled according to another sampling strategy. More than two surfaces can be intersected simultaneously, and various pair wise and group intersection data can be captured and displayed.
  • A second set of the useful novel 3D graphics visualization functions provided for by the invention pertain to multidimensional data visualizations based on 1D-curves embedded in a 3D visual field.
  • The invention provides for a curve to be generated by tabular data and to be suspended over tabular data used to create it or otherwise associated with the curve. The invention provides for the curve to be visually linked to a geometric rendering of tabular data. These features are illustrated by FIG. 59. The invention provides for the curve to be given a thickness and or color under the control of one or more data values. The invention provides for the color of the curve to be varied as a function of tabular (or other) data values. These features are illustrated by FIG. 60. The invention provides for the thickness of the curve to be varied as a function of tabular (or other) data values. This feature is illustrated in FIG. 61.
  • Such curve plotting utilities can be provided with functions, features, and operations like that of the 2D-surface utilities described above. The invention also provides for interactively shifting the observation point with respect to these objects for more detailed feature, theme, or trend inspection. The viewpoint can be changed according to the 6 degrees of freedom (three translations, three angles) of rigid motion. The viewpoint is changed under the control of an HDTP or an advanced mouse. The invention also provides for moving a curve within the 3D visual field. The curve can be moved according to the 6 degrees of freedom (three translations, three angles) of rigid motion. The 2D-surface can be moved under the control of an HDTP or an advanced mouse.
  • A curve of any thickness can be treated as if it comprises a surface that can be virtually illuminated by one or more lighting sources. The one or more lighting sources can be moved according to the 6 degrees of freedom (three translations, three angles) of rigid motion. The one or more lighting sources can also permit control of the color and intensity of virtual light emitted. In an embodiment, a lighting source can be moved and/or controlled under the control of an HDTP or an advanced mouse.
  • The data plotted in a curve can directly echo the data displayed in the exemplary planar array of associated tabular data or spreadsheet and/or can originate from another set of tabular data or spreadsheet region and/or can originate from other data as suggested earlier in FIG. 55A. The invention also provides for processing of the data via mathematical transformations, statistical processing, signal processing, etc. prior to creation of the curve as was suggested earlier in FIG. 55B. A curve can plot filtered or averaged versions of tabular data, spreadsheet data, or other data. In an embodiment, the filtering or averaging can be controlled by an interactive parameter.
  • The invention provides for a curve to be rendered along with a height measuring visual, and also provides for a locally variable parameterized texture to be imposed on the curve and for the texture to be varied as a function of tabular or other data.
  • The invention provides for the curve to be rendered with numerical values displayed adjacently at specific points on the curve and that these can be used to represent an additional dimension of data. The invention also provides for the curve to be rendered with symbols displayed adjacently at specific points on the curve and that these can be used to represent an additional dimension of data. The invention further provides for the curve to be rendered with parameterized glyphs displayed adjacently at specific points on the curve and that these can be used to represent (an) additional dimension(s) of data.
  • The invention provides for a curve intersection tool that can be used as a numerical solution operator for plotted, interpolated, and processed data. FIG. 62 illustrates two exemplary curves, each representing at least 2-dimensional data. Each of these can be interactively moved and rotated as described earlier, interactively creating an intersection of the two exemplary curves and at least one resultant intersection point. In an embodiment, a 2D-surface intersection tool allows the numerical values of intersection points to be numerically calculated and visually signified. The numerical values can also be used for other purposes and other subsequent calculations.
  • In an embodiment, more than two curves can be intersected simultaneously, and various pair-wise and group intersection data can be captured and displayed.
  • For example, a curve and a surface can be intersected simultaneously, and intersection data can be captured and displayed. As another example, more than two curves and at least one surface can be intersected simultaneously, and various pair-wise and group intersection data can be captured and displayed.
  • While the invention has been described in detail with reference to disclosed embodiments, various modifications within the scope of the invention will be apparent to those of ordinary skill in this technological field. It is to be appreciated that features described with respect to one embodiment typically can be applied to other embodiments.
  • The invention can be embodied in other specific forms without departing from the spirit or essential characteristics thereof. The present embodiments are therefore to be considered in all respects as illustrative and not restrictive, the scope of the invention being indicated by the appended claims rather than by the foregoing description, and all changes which come within the meaning and range of equivalency of the claims are therefore intended to be embraced therein. Therefore, the invention properly is to be construed with reference to the claims.

Claims (10)

1. A method for visually enhancing the display of tabular data of an electronic spreadsheet comprising cells arranged in rows and columns, the method comprising:
obtaining at least one data value of the tabular data;
computing at least one visual parameter responsive to the at least one data value, wherein each visual parameter is a number within a range of numbers and is determined according to a numerically calculated mathematical function;
controlling a visual effect of the at least one data value according to the at least one visual parameter; and
applying the visual effect to the display of the at least one data value to visually enhance the display of tabular data of the electronic spreadsheet.
2. The method of claim 1 wherein the visual effect is a color of at least one of a font, a background, and a border of a spreadsheet cell in which the at least one data value is displayed.
3. The method of claim 1 wherein the visual effect is at least one of a font style of, a font size of, and a font embellishment of the at least one displayed data value.
4. The method of claim 1 wherein the visual effect is at least one of a format of, a degree of rounding of, a number of displayed decimal places of, and a rounding method of the at least one displayed data value.
5. The method of claim 1 wherein the visual effect is a thickness a border of a spreadsheet cell in which the at least one data value is displayed.
6. The method of claim 6 wherein the border encompasses a plurality of spreadsheet cells in which more than one data value is displayed.
7. The method of claim 1 wherein the visual effect is displaying a background texture of a spreadsheet cell in which the at least one data value is displayed.
8. The method of claim 1 wherein the visual effect is aligning the at least one data value within one or more spreadsheet cells in which the at least one data value is displayed.
9. The method of claim 1 wherein the visual effect is at least one of appending a symbol to the at least one data value displayed in a spreadsheet cell, displaying a symbol in a spreadsheet cell, and displaying an image in a spreadsheet cell.
10. The method of claim 1 wherein the visual effect is applied to the at least one of the row and the column in which the at least one data value is displayed.
US12/875,115 2009-09-02 2010-09-02 Value-driven visualization primitives for spreadsheets, tabular data, and advanced spreadsheet visualization Abandoned US20110066933A1 (en)

Priority Applications (7)

Application Number Priority Date Filing Date Title
US12/875,115 US20110066933A1 (en) 2009-09-02 2010-09-02 Value-driven visualization primitives for spreadsheets, tabular data, and advanced spreadsheet visualization
US13/669,431 US9665554B2 (en) 2009-09-02 2012-11-05 Value-driven visualization primitives for tabular data of spreadsheets
US13/670,411 US20130132811A1 (en) 2009-09-02 2012-11-06 Graphical Surface Rendering Data Visualization Tools and Primitives for Tabular Data and Spreadsheets
US13/670,346 US8826113B2 (en) 2009-09-02 2012-11-06 Surface-surface graphical intersection tools and primitives for data visualization, tabular data, and advanced spreadsheets
US13/673,904 US20130174004A1 (en) 2009-09-02 2012-11-09 Graphical 3D Curve Rendering Data Visualization Tools and Primitives for Tabular Data and Spreadsheets
US13/673,732 US8826114B2 (en) 2009-09-02 2012-11-09 Surface-curve graphical intersection tools and primitives for data visualization, tabular data, and advanced spreadsheets
US15/601,892 US20170262425A1 (en) 2009-09-02 2017-05-22 Interactive graphical surface-rendering data visualization tools for tabular data and spreadsheets

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US23934909P 2009-09-02 2009-09-02
US12/875,115 US20110066933A1 (en) 2009-09-02 2010-09-02 Value-driven visualization primitives for spreadsheets, tabular data, and advanced spreadsheet visualization

Related Child Applications (5)

Application Number Title Priority Date Filing Date
US13/669,431 Continuation US9665554B2 (en) 2009-09-02 2012-11-05 Value-driven visualization primitives for tabular data of spreadsheets
US13/670,346 Continuation US8826113B2 (en) 2009-09-02 2012-11-06 Surface-surface graphical intersection tools and primitives for data visualization, tabular data, and advanced spreadsheets
US13/670,411 Continuation US20130132811A1 (en) 2009-09-02 2012-11-06 Graphical Surface Rendering Data Visualization Tools and Primitives for Tabular Data and Spreadsheets
US13/673,904 Continuation US20130174004A1 (en) 2009-09-02 2012-11-09 Graphical 3D Curve Rendering Data Visualization Tools and Primitives for Tabular Data and Spreadsheets
US13/673,732 Continuation US8826114B2 (en) 2009-09-02 2012-11-09 Surface-curve graphical intersection tools and primitives for data visualization, tabular data, and advanced spreadsheets

Publications (1)

Publication Number Publication Date
US20110066933A1 true US20110066933A1 (en) 2011-03-17

Family

ID=43731675

Family Applications (7)

Application Number Title Priority Date Filing Date
US12/875,115 Abandoned US20110066933A1 (en) 2009-09-02 2010-09-02 Value-driven visualization primitives for spreadsheets, tabular data, and advanced spreadsheet visualization
US13/669,431 Expired - Fee Related US9665554B2 (en) 2009-09-02 2012-11-05 Value-driven visualization primitives for tabular data of spreadsheets
US13/670,346 Expired - Fee Related US8826113B2 (en) 2009-09-02 2012-11-06 Surface-surface graphical intersection tools and primitives for data visualization, tabular data, and advanced spreadsheets
US13/670,411 Abandoned US20130132811A1 (en) 2009-09-02 2012-11-06 Graphical Surface Rendering Data Visualization Tools and Primitives for Tabular Data and Spreadsheets
US13/673,904 Abandoned US20130174004A1 (en) 2009-09-02 2012-11-09 Graphical 3D Curve Rendering Data Visualization Tools and Primitives for Tabular Data and Spreadsheets
US13/673,732 Expired - Fee Related US8826114B2 (en) 2009-09-02 2012-11-09 Surface-curve graphical intersection tools and primitives for data visualization, tabular data, and advanced spreadsheets
US15/601,892 Abandoned US20170262425A1 (en) 2009-09-02 2017-05-22 Interactive graphical surface-rendering data visualization tools for tabular data and spreadsheets

Family Applications After (6)

Application Number Title Priority Date Filing Date
US13/669,431 Expired - Fee Related US9665554B2 (en) 2009-09-02 2012-11-05 Value-driven visualization primitives for tabular data of spreadsheets
US13/670,346 Expired - Fee Related US8826113B2 (en) 2009-09-02 2012-11-06 Surface-surface graphical intersection tools and primitives for data visualization, tabular data, and advanced spreadsheets
US13/670,411 Abandoned US20130132811A1 (en) 2009-09-02 2012-11-06 Graphical Surface Rendering Data Visualization Tools and Primitives for Tabular Data and Spreadsheets
US13/673,904 Abandoned US20130174004A1 (en) 2009-09-02 2012-11-09 Graphical 3D Curve Rendering Data Visualization Tools and Primitives for Tabular Data and Spreadsheets
US13/673,732 Expired - Fee Related US8826114B2 (en) 2009-09-02 2012-11-09 Surface-curve graphical intersection tools and primitives for data visualization, tabular data, and advanced spreadsheets
US15/601,892 Abandoned US20170262425A1 (en) 2009-09-02 2017-05-22 Interactive graphical surface-rendering data visualization tools for tabular data and spreadsheets

Country Status (1)

Country Link
US (7) US20110066933A1 (en)

Cited By (108)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110179360A1 (en) * 2010-01-19 2011-07-21 Livevol Inc. System and method for presenting option data using animated 3-dimensional graphical display
US20120254783A1 (en) * 2011-03-29 2012-10-04 International Business Machines Corporation Modifying numeric data presentation on a display
US8309833B2 (en) * 2010-06-17 2012-11-13 Ludwig Lester F Multi-channel data sonification in spatial sound fields with partitioned timbre spaces using modulation of timbre and rendered spatial location as sonification information carriers
US20140330898A1 (en) * 2010-11-08 2014-11-06 Sony Corporation Methods and systems for use in providing a remote user interface
EP2911100A1 (en) * 2014-02-20 2015-08-26 Palantir Technologies, Inc. Relationship visualizations
US9311289B1 (en) * 2013-08-16 2016-04-12 Google Inc. Spreadsheet document tab conditional formatting
US9317963B2 (en) 2012-08-10 2016-04-19 Microsoft Technology Licensing, Llc Generating scenes and tours in a spreadsheet application
US9367872B1 (en) 2014-12-22 2016-06-14 Palantir Technologies Inc. Systems and user interfaces for dynamic and interactive investigation of bad actor behavior based on automatic clustering of related data in various data structures
US9380431B1 (en) 2013-01-31 2016-06-28 Palantir Technologies, Inc. Use of teams in a mobile application
US9383911B2 (en) 2008-09-15 2016-07-05 Palantir Technologies, Inc. Modal-less interface enhancements
US9454281B2 (en) 2014-09-03 2016-09-27 Palantir Technologies Inc. System for providing dynamic linked panels in user interface
US9454785B1 (en) 2015-07-30 2016-09-27 Palantir Technologies Inc. Systems and user interfaces for holistic, data-driven investigation of bad actor behavior based on clustering and scoring of related data
US9501851B2 (en) 2014-10-03 2016-11-22 Palantir Technologies Inc. Time-series analysis system
US9514200B2 (en) 2013-10-18 2016-12-06 Palantir Technologies Inc. Systems and user interfaces for dynamic and interactive simultaneous querying of multiple data stores
US20170017617A1 (en) * 2014-05-27 2017-01-19 Hitachi, Ltd. Management system for managing information system
US9558352B1 (en) 2014-11-06 2017-01-31 Palantir Technologies Inc. Malicious software detection in a computing system
US9588954B2 (en) 2013-06-15 2017-03-07 Microsoft Technology Licensing, Llc Application/document collaboration in a multi-device environment
US9619557B2 (en) 2014-06-30 2017-04-11 Palantir Technologies, Inc. Systems and methods for key phrase characterization of documents
US9646396B2 (en) 2013-03-15 2017-05-09 Palantir Technologies Inc. Generating object time series and data objects
WO2017079138A1 (en) * 2015-11-02 2017-05-11 Microsoft Technology Licensing, Llc Images and additional data associated with cells in spreadsheets
US20170132189A1 (en) * 2015-11-10 2017-05-11 Fuji Xerox Co., Ltd. Information processing apparatus, information processing method, and non-transitory computer readable medium
US20170220633A1 (en) * 2016-02-01 2017-08-03 Splunk Inc. Context-Adaptive Selection Options in a Modular Visualization Framework
US9727560B2 (en) 2015-02-25 2017-08-08 Palantir Technologies Inc. Systems and methods for organizing and identifying documents via hierarchies and dimensions of tags
US9767172B2 (en) 2014-10-03 2017-09-19 Palantir Technologies Inc. Data aggregation and analysis system
US9817563B1 (en) 2014-12-29 2017-11-14 Palantir Technologies Inc. System and method of generating data points from one or more data stores of data items for chart creation and manipulation
US9823818B1 (en) 2015-12-29 2017-11-21 Palantir Technologies Inc. Systems and interactive user interfaces for automatic generation of temporal representation of data objects
US9852195B2 (en) 2013-03-15 2017-12-26 Palantir Technologies Inc. System and method for generating event visualizations
US9852205B2 (en) 2013-03-15 2017-12-26 Palantir Technologies Inc. Time-sensitive cube
US9857958B2 (en) 2014-04-28 2018-01-02 Palantir Technologies Inc. Systems and user interfaces for dynamic and interactive access of, investigation of, and analysis of data objects stored in one or more databases
US9870389B2 (en) 2014-12-29 2018-01-16 Palantir Technologies Inc. Interactive user interface for dynamic data analysis exploration and query processing
US9880987B2 (en) 2011-08-25 2018-01-30 Palantir Technologies, Inc. System and method for parameterizing documents for automatic workflow generation
US9891808B2 (en) 2015-03-16 2018-02-13 Palantir Technologies Inc. Interactive user interfaces for location-based data analysis
US9898528B2 (en) 2014-12-22 2018-02-20 Palantir Technologies Inc. Concept indexing among database of documents using machine learning techniques
US9898335B1 (en) 2012-10-22 2018-02-20 Palantir Technologies Inc. System and method for batch evaluation programs
US9953445B2 (en) 2013-05-07 2018-04-24 Palantir Technologies Inc. Interactive data object map
US9959265B1 (en) 2014-05-08 2018-05-01 Google Llc Populating values in a spreadsheet using semantic cues
US9965937B2 (en) 2013-03-15 2018-05-08 Palantir Technologies Inc. External malware data item clustering and analysis
US9984133B2 (en) 2014-10-16 2018-05-29 Palantir Technologies Inc. Schematic and database linking system
US9998485B2 (en) 2014-07-03 2018-06-12 Palantir Technologies, Inc. Network intrusion data item clustering and analysis
US10013413B2 (en) 2013-06-14 2018-07-03 Microsoft Technology Licensing, Llc Smart fill
CN108268193A (en) * 2017-01-04 2018-07-10 珠海金山办公软件有限公司 The display methods and device of the corresponding prompt message of function in electrical form
US10037383B2 (en) 2013-11-11 2018-07-31 Palantir Technologies, Inc. Simple web search
US10037314B2 (en) 2013-03-14 2018-07-31 Palantir Technologies, Inc. Mobile reports
CN108920145A (en) * 2018-05-08 2018-11-30 陕西法士特齿轮有限责任公司 A kind of UI table color lump moving method
US10180929B1 (en) 2014-06-30 2019-01-15 Palantir Technologies, Inc. Systems and methods for identifying key phrase clusters within documents
US10180977B2 (en) 2014-03-18 2019-01-15 Palantir Technologies Inc. Determining and extracting changed data from a data source
US10198515B1 (en) 2013-12-10 2019-02-05 Palantir Technologies Inc. System and method for aggregating data from a plurality of data sources
US10216801B2 (en) 2013-03-15 2019-02-26 Palantir Technologies Inc. Generating data clusters
US10229284B2 (en) 2007-02-21 2019-03-12 Palantir Technologies Inc. Providing unique views of data based on changes or rules
US10230746B2 (en) 2014-01-03 2019-03-12 Palantir Technologies Inc. System and method for evaluating network threats and usage
US10296617B1 (en) 2015-10-05 2019-05-21 Palantir Technologies Inc. Searches of highly structured data
US10318630B1 (en) 2016-11-21 2019-06-11 Palantir Technologies Inc. Analysis of large bodies of textual data
US10324609B2 (en) 2016-07-21 2019-06-18 Palantir Technologies Inc. System for providing dynamic linked panels in user interface
US10356032B2 (en) 2013-12-26 2019-07-16 Palantir Technologies Inc. System and method for detecting confidential information emails
US10372808B1 (en) 2012-12-12 2019-08-06 Google Llc Passing functional spreadsheet data by reference
US10409892B2 (en) 2011-01-26 2019-09-10 Microsoft Technology Licensing, Llc Formatting data by example
US10423582B2 (en) 2011-06-23 2019-09-24 Palantir Technologies, Inc. System and method for investigating large amounts of data
US10437612B1 (en) * 2015-12-30 2019-10-08 Palantir Technologies Inc. Composite graphical interface with shareable data-objects
US10444941B2 (en) 2015-08-17 2019-10-15 Palantir Technologies Inc. Interactive geospatial map
US10452678B2 (en) 2013-03-15 2019-10-22 Palantir Technologies Inc. Filter chains for exploring large data sets
US10474317B2 (en) * 2014-06-25 2019-11-12 Oracle International Corporation Dynamic node grouping in grid-based visualizations
US10484407B2 (en) 2015-08-06 2019-11-19 Palantir Technologies Inc. Systems, methods, user interfaces, and computer-readable media for investigating potential malicious communications
US10489391B1 (en) 2015-08-17 2019-11-26 Palantir Technologies Inc. Systems and methods for grouping and enriching data items accessed from one or more databases for presentation in a user interface
CN110515996A (en) * 2019-08-22 2019-11-29 河南理工大学 A kind of multichannel tropospheric delay resolved data integrates visible processing method
US10503824B2 (en) 2015-11-02 2019-12-10 Microsoft Technology Licensing, Llc Video on charts
US20200019603A1 (en) * 2018-07-13 2020-01-16 Microsoft Technology Licensing, Llc Systems, methods, and computer-readable media for improved table identification using a neural network
US10552994B2 (en) 2014-12-22 2020-02-04 Palantir Technologies Inc. Systems and interactive user interfaces for dynamic retrieval, analysis, and triage of data items
US10572487B1 (en) 2015-10-30 2020-02-25 Palantir Technologies Inc. Periodic database search manager for multiple data sources
US10678860B1 (en) 2015-12-17 2020-06-09 Palantir Technologies, Inc. Automatic generation of composite datasets based on hierarchical fields
US10684762B2 (en) * 2018-08-27 2020-06-16 Sap Se Analytics design system
US10699071B2 (en) 2013-08-08 2020-06-30 Palantir Technologies Inc. Systems and methods for template based custom document generation
US10698938B2 (en) 2016-03-18 2020-06-30 Palantir Technologies Inc. Systems and methods for organizing and identifying documents via hierarchies and dimensions of tags
US10706434B1 (en) 2015-09-01 2020-07-07 Palantir Technologies Inc. Methods and systems for determining location information
US10719188B2 (en) 2016-07-21 2020-07-21 Palantir Technologies Inc. Cached database and synchronization system for providing dynamic linked panels in user interface
US10795723B2 (en) 2014-03-04 2020-10-06 Palantir Technologies Inc. Mobile tasks
US10817513B2 (en) 2013-03-14 2020-10-27 Palantir Technologies Inc. Fair scheduling for mixed-query loads
US10824799B2 (en) 2014-06-30 2020-11-03 Microsoft Technology Licensing, Llc Summary data autofill
US10853380B1 (en) 2016-07-31 2020-12-01 Splunk Inc. Framework for displaying interactive visualizations of event data
US10853383B2 (en) 2016-07-31 2020-12-01 Splunk Inc. Interactive parallel coordinates visualizations
US10853382B2 (en) 2016-07-31 2020-12-01 Splunk Inc. Interactive punchcard visualizations
US10861202B1 (en) 2016-07-31 2020-12-08 Splunk Inc. Sankey graph visualization for machine data search and analysis system
US10885021B1 (en) 2018-05-02 2021-01-05 Palantir Technologies Inc. Interactive interpreter and graphical user interface
WO2021024040A1 (en) * 2019-08-08 2021-02-11 Mann, Roy Digital processing systems and methods for automatic relationship recognition in tables of collaborative work systems
US10929436B2 (en) 2014-07-03 2021-02-23 Palantir Technologies Inc. System and method for news events detection and visualization
US11030556B1 (en) 2019-11-18 2021-06-08 Monday.Com Digital processing systems and methods for dynamic object display of tabular information in collaborative work systems
US11037342B1 (en) 2016-07-31 2021-06-15 Splunk Inc. Visualization modules for use within a framework for displaying interactive visualizations of event data
US11086640B2 (en) * 2015-12-30 2021-08-10 Palantir Technologies Inc. Composite graphical interface with shareable data-objects
US20210248134A1 (en) * 2017-12-04 2021-08-12 Palantir Technologies Inc. Query-based time-series data display and processing system
US11138180B2 (en) 2011-09-02 2021-10-05 Palantir Technologies Inc. Transaction protocol for reading database values
US11144079B2 (en) 2013-02-11 2021-10-12 Graco Minnesota Inc. Remote monitoring for fluid applicator system
US11150917B2 (en) 2015-08-26 2021-10-19 Palantir Technologies Inc. System for data aggregation and analysis of data from a plurality of data sources
US11277361B2 (en) 2020-05-03 2022-03-15 Monday.com Ltd. Digital processing systems and methods for variable hang-time for social layer messages in collaborative work systems
US11277452B2 (en) 2020-05-01 2022-03-15 Monday.com Ltd. Digital processing systems and methods for multi-board mirroring of consolidated information in collaborative work systems
US11301623B2 (en) 2020-02-12 2022-04-12 Monday.com Ltd Digital processing systems and methods for hybrid scaling/snap zoom function in table views of collaborative work systems
US11307753B2 (en) 2019-11-18 2022-04-19 Monday.Com Systems and methods for automating tablature in collaborative work systems
US11392556B1 (en) 2021-01-14 2022-07-19 Monday.com Ltd. Digital processing systems and methods for draft and time slider for presentations in collaborative work systems
US11410129B2 (en) 2010-05-01 2022-08-09 Monday.com Ltd. Digital processing systems and methods for two-way syncing with third party applications in collaborative work systems
US11436359B2 (en) 2018-07-04 2022-09-06 Monday.com Ltd. System and method for managing permissions of users for a single data type column-oriented data structure
US11442707B2 (en) * 2016-12-03 2022-09-13 Thomas STACHURA Spreadsheet-based software application development
US11599369B1 (en) 2018-03-08 2023-03-07 Palantir Technologies Inc. Graphical user interface configuration system
US11698890B2 (en) 2018-07-04 2023-07-11 Monday.com Ltd. System and method for generating a column-oriented data structure repository for columns of single data types
US11726753B2 (en) 2016-12-03 2023-08-15 Thomas STACHURA Spreadsheet-based software application development
US11741071B1 (en) 2022-12-28 2023-08-29 Monday.com Ltd. Digital processing systems and methods for navigating and viewing displayed content
US11829953B1 (en) 2020-05-01 2023-11-28 Monday.com Ltd. Digital processing systems and methods for managing sprints using linked electronic boards
US11886683B1 (en) 2022-12-30 2024-01-30 Monday.com Ltd Digital processing systems and methods for presenting board graphics
US11893381B1 (en) 2023-02-21 2024-02-06 Monday.com Ltd Digital processing systems and methods for reducing file bundle sizes
US11934212B2 (en) 2013-02-11 2024-03-19 Graco Minnesota Inc. Paint sprayer distributed control and output volume monitoring architectures
US11954428B2 (en) 2021-04-29 2024-04-09 Monday.com Ltd. Digital processing systems and methods for accessing another's display via social layer interactions in collaborative work systems

Families Citing this family (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US9792017B1 (en) * 2011-07-12 2017-10-17 Domo, Inc. Automatic creation of drill paths
US20130219304A1 (en) * 2011-08-23 2013-08-22 Salesforce.Com, Inc. Systems and methods for a graphical user interface of a touchscreen display
US9164673B2 (en) * 2012-07-16 2015-10-20 Microsoft Technology Licensing, Llc Location-dependent drag and drop UI
US9360992B2 (en) 2013-07-29 2016-06-07 Microsoft Technology Licensing, Llc Three dimensional conditional formatting
US9418336B2 (en) 2013-08-02 2016-08-16 Microsoft Technology Licensing, Llc Automatic recognition and insights of data
EP3100178A1 (en) 2014-01-30 2016-12-07 Microsoft Technology Licensing, LLC Automatic insights for spreadsheets
USD822060S1 (en) * 2014-09-04 2018-07-03 Rockwell Collins, Inc. Avionics display with icon
CN105320396B (en) * 2014-10-31 2019-08-09 深圳市雅都软件股份有限公司 Planar graph single dimension local association Zoom display system and method
CN105321148B (en) * 2014-11-26 2019-08-09 深圳市雅都软件股份有限公司 Multizone is associated with the method and system of Zoom display K line chart or U.S.'s line chart
US9286286B1 (en) * 2015-01-03 2016-03-15 Chahid Kamel Ghaddar Method, apparatus, and computer program product for optimizing parameterized models using functional paradigm of spreadsheet software
US9760591B2 (en) 2015-05-14 2017-09-12 Walleye Software, LLC Dynamic code loading
US10290147B2 (en) 2015-08-11 2019-05-14 Microsoft Technology Licensing, Llc Using perspective to visualize data
US10198469B1 (en) 2017-08-24 2019-02-05 Deephaven Data Labs Llc Computer data system data source refreshing using an update propagation graph having a merged join listener
US10902194B2 (en) 2018-02-09 2021-01-26 Microsoft Technology Licensing, Llc Natively handling approximate values in spreadsheet applications
US10699070B2 (en) * 2018-03-05 2020-06-30 Sap Se Dynamic retrieval and rendering of user interface content
US10586362B2 (en) * 2018-06-18 2020-03-10 Microsoft Technology Licensing, Llc Interactive layout-aware construction of bespoke charts
CN109871524B (en) * 2019-02-21 2023-06-09 腾讯科技（深圳）有限公司 Chart generation method and device
US11238108B1 (en) * 2020-07-28 2022-02-01 Grey Wall Software Llc Custom data filtering systems and methods

Citations (91)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4748676A (en) * 1986-05-16 1988-05-31 Fuji Electric Co., Ltd. Pattern rotation angle detecting system
US4899137A (en) * 1987-09-26 1990-02-06 Aeg Olympia Aktiengesellschaft Arrangement for the input and processing of characters and/or graphic patterns
US5237647A (en) * 1989-09-15 1993-08-17 Massachusetts Institute Of Technology Computer aided drawing in three dimensions
US5270711A (en) * 1989-05-08 1993-12-14 U.S. Philips Corporation Touch sensor array systems and display systems incorporating such
US5292999A (en) * 1991-08-14 1994-03-08 Fernandes Co., Ltd. Electric stringed instrument having a device for sustaining the vibration of the string
US5341133A (en) * 1991-05-09 1994-08-23 The Rowland Institute For Science, Inc. Keyboard having touch sensor keys for conveying information electronically
US5347295A (en) * 1990-10-31 1994-09-13 Go Corporation Control of a computer through a position-sensed stylus
US5357048A (en) * 1992-10-08 1994-10-18 Sgroi John J MIDI sound designer with randomizer function
US5378850A (en) * 1992-01-14 1995-01-03 Fernandes Co., Ltd. Electric stringed instrument having an arrangement for adjusting the generation of magnetic feedback
US5386219A (en) * 1991-10-16 1995-01-31 International Business Machines Corp. Touch overlay for improved touch sensitivity
US5420936A (en) * 1992-10-16 1995-05-30 International Business Machines Corporation Method and apparatus for accessing touch screen desktop objects via fingerprint recognition
US5440072A (en) * 1992-09-25 1995-08-08 Willis; Raymon A. System for rejuvenating vintage organs and pianos
US5442168A (en) * 1991-10-15 1995-08-15 Interactive Light, Inc. Dynamically-activated optical instrument for producing control signals having a self-calibration means
US5459282A (en) * 1992-09-25 1995-10-17 Willis; Raymon A. System for rejuvenating vintage organs and pianos
US5471008A (en) * 1990-11-19 1995-11-28 Kabushiki Kaisha Kawai Gakki Seisakusho MIDI control apparatus
US5475214A (en) * 1991-10-15 1995-12-12 Interactive Light, Inc. Musical sound effects controller having a radiated emission space
US5550964A (en) * 1992-12-18 1996-08-27 Borland International, Inc. System and methods for intelligent analytical graphing
US5565641A (en) * 1994-03-28 1996-10-15 Gruenbaum; Leon Relativistic electronic musical instrument
US5581678A (en) * 1993-08-06 1996-12-03 Borland International, Inc. System and methods for automated graphing of spreadsheet information
US5585588A (en) * 1991-10-24 1996-12-17 Fernandes Co., Ltd. Electric stringed instrument having a device for sustaining the vibration of a string and an electromagnetic driver for the device
US5592572A (en) * 1993-11-05 1997-01-07 The United States Of America As Represented By The Department Of Health And Human Services Automated portrait/landscape mode detection on a binary image
US5592752A (en) * 1993-05-13 1997-01-14 Industrial Technology Research Institute Process and an apparatus for producing teas
US5659145A (en) * 1995-04-27 1997-08-19 Weil; Robert P. Foot operated audio signal controller with lighted visual reference
US5659466A (en) * 1994-11-02 1997-08-19 Advanced Micro Devices, Inc. Monolithic PC audio circuit with enhanced digital wavetable audio synthesizer
US5665927A (en) * 1993-06-30 1997-09-09 Casio Computer Co., Ltd. Method and apparatus for inputting musical data without requiring selection of a displayed icon
US5668338A (en) * 1994-11-02 1997-09-16 Advanced Micro Devices, Inc. Wavetable audio synthesizer with low frequency oscillators for tremolo and vibrato effects
US5675100A (en) * 1993-11-03 1997-10-07 Hewlett; Walter B. Method for encoding music printing information in a MIDI message
US5717939A (en) * 1991-11-18 1998-02-10 Compaq Computer Corporation Method and apparatus for entering and manipulating spreadsheet cell data
US5719561A (en) * 1995-10-25 1998-02-17 Gilbert R. Gonzales Tactile communication device and method
US5719347A (en) * 1994-12-07 1998-02-17 Yamaha Corporation Keyboard apparatus for electronic musical instrument with key depression detection unit
US5724985A (en) * 1995-08-02 1998-03-10 Pacesetter, Inc. User interface for an implantable medical device using an integrated digitizer display screen
US5741993A (en) * 1995-08-01 1998-04-21 Kabushiki Kaisha Kawai Gakki Seisakusho Electronic keyboard having a discrete pitch bender
US5748184A (en) * 1996-05-28 1998-05-05 International Business Machines Corporation Virtual pointing device for touchscreens
US5763806A (en) * 1996-07-15 1998-06-09 Willis; Raymon A. Method and apparatus for midifying vintage organs and pianos
US5786540A (en) * 1996-03-05 1998-07-28 Westlund; Robert L. Controller apparatus for music sequencer
US5801340A (en) * 1995-06-29 1998-09-01 Invotronics Manufacturing Proximity sensor
US5805137A (en) * 1991-11-26 1998-09-08 Itu Research, Inc. Touch sensitive input control device
US5824930A (en) * 1995-06-09 1998-10-20 Yamaha Corporation Keyboard musical instrument having key monitor exactly discriminating key motion
US5827989A (en) * 1997-06-23 1998-10-27 Microsoft Corporation System and method for representing a musical event and for converting the musical event into a series of discrete events
US5841428A (en) * 1993-11-05 1998-11-24 Intertactile Technologies Corporation Rotary circuit control devices with changeable graphics
US5850051A (en) * 1996-08-15 1998-12-15 Yamaha Corporation Method and apparatus for creating an automatic accompaniment pattern on the basis of analytic parameters
US5852251A (en) * 1997-06-25 1998-12-22 Industrial Technology Research Institute Method and apparatus for real-time dynamic midi control
US5889236A (en) * 1992-06-08 1999-03-30 Synaptics Incorporated Pressure sensitive scrollbar feature
US5932827A (en) * 1995-01-09 1999-08-03 Osborne; Gary T. Sustainer for a musical instrument
US5969283A (en) * 1998-06-17 1999-10-19 Looney Productions, Llc Music organizer and entertainment center
US5977466A (en) * 1996-07-11 1999-11-02 Yamaha Corporation Keyboard musical instrument equipped with small simple economical key touch generator
US5986224A (en) * 1995-04-19 1999-11-16 Elo Touchsystems, Inc. Acoustic condition sensor employing a plurality of mutually non-orthogonal waves
US6005545A (en) * 1995-01-17 1999-12-21 Sega Enterprise, Ltd. Image processing method and electronic device
US6037937A (en) * 1997-12-04 2000-03-14 Nortel Networks Corporation Navigation tool for graphical user interface
US6047073A (en) * 1994-11-02 2000-04-04 Advanced Micro Devices, Inc. Digital wavetable audio synthesizer with delay-based effects processing
US6051769A (en) * 1998-11-25 2000-04-18 Brown, Jr.; Donival Computerized reading display
US6100461A (en) * 1998-06-10 2000-08-08 Advanced Micro Devices, Inc. Wavetable cache using simplified looping
US6107997A (en) * 1996-06-27 2000-08-22 Ure; Michael J. Touch-sensitive keyboard/mouse and computing device using the same
US6138130A (en) * 1995-12-08 2000-10-24 Inventure Technologies, Inc. System and method for processing data in an electronic spreadsheet in accordance with a data type
US6140565A (en) * 1998-06-08 2000-10-31 Yamaha Corporation Method of visualizing music system by combination of scenery picture and player icons
US6204441B1 (en) * 1998-04-09 2001-03-20 Yamaha Corporation Method and apparatus for effectively displaying musical information with visual display
US6225975B1 (en) * 1997-03-18 2001-05-01 Alps Electric Co., Ltd. Control device for game machine
US6285358B1 (en) * 1993-04-30 2001-09-04 Microtouch Systems, Inc. Method of and apparatus for the elimination of the effects of inertial interference in force measurement systems, including touch-input computer and related displays employing touch force location measurement techniques
US6288317B1 (en) * 1998-05-29 2001-09-11 Raymon A. Willis Real time transmission of keyboard musical performance
US6310610B1 (en) * 1997-12-04 2001-10-30 Nortel Networks Limited Intelligent touch display
US6310279B1 (en) * 1997-12-27 2001-10-30 Yamaha Corporation Device and method for generating a picture and/or tone on the basis of detection of a physical event from performance information
US20010036299A1 (en) * 1998-05-15 2001-11-01 Andrew William Senior Combined fingerprint acquisition and control device
US6320112B1 (en) * 2000-05-19 2001-11-20 Martin Lotze Procedure and device for the automatic selection of musical and/or tonal compositions
US6323846B1 (en) * 1998-01-26 2001-11-27 University Of Delaware Method and apparatus for integrating manual input
US20010049695A1 (en) * 1998-10-20 2001-12-06 Ed H. Chi Visualization spreadsheet
US20020005108A1 (en) * 1998-05-15 2002-01-17 Ludwig Lester Frank Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting
US6360019B1 (en) * 1996-03-28 2002-03-19 Microsoft Corporation Table-based compression with embedded coding
US6363159B1 (en) * 1993-11-18 2002-03-26 Digimarc Corporation Consumer audio appliance responsive to watermark data
US6373475B1 (en) * 1997-04-25 2002-04-16 Michael Challis Converter for resistive touchscreens
US6392705B1 (en) * 1997-03-17 2002-05-21 Microsoft Corporation Multimedia compression system with additive temporal layers
US6392636B1 (en) * 1998-01-22 2002-05-21 Stmicroelectronics, Inc. Touchpad providing screen cursor/pointer movement control
US6408087B1 (en) * 1998-01-13 2002-06-18 Stmicroelectronics, Inc. Capacitive semiconductor user input device
US20020087275A1 (en) * 2000-07-31 2002-07-04 Junhyong Kim Visualization and manipulation of biomolecular relationships using graph operators
US20020093491A1 (en) * 1992-06-08 2002-07-18 David W. Gillespie Object position detector with edge motion feature and gesture recognition
US6703552B2 (en) * 2001-07-19 2004-03-09 Lippold Haken Continuous music keyboard
US20040060001A1 (en) * 1998-12-31 2004-03-25 Microsoft Corporation Extension of formulas and formatting in an electronic spreadsheet
US6793619B1 (en) * 1999-06-09 2004-09-21 Yaacov Blumental Computer-implemented method and system for giving a user an impression of tactile feedback
US20040251402A1 (en) * 2001-09-25 2004-12-16 Gerd Reime Circuit with an opto-electronic display unit
US6988241B1 (en) * 2000-10-16 2006-01-17 International Business Machines Corporation Client side, web-based spreadsheet
US7030860B1 (en) * 1999-10-08 2006-04-18 Synaptics Incorporated Flexible transparent touch sensing system for electronic devices
US20060252530A1 (en) * 2003-01-08 2006-11-09 Igt Mobile device for providing filtered casino information based on real time data
US20070044019A1 (en) * 2003-05-23 2007-02-22 Byung-Ro Moon Multi-campaign assignment apparatus considering overlapping recommendation problem
US20070050699A1 (en) * 2005-08-30 2007-03-01 Microsoft Corporation Customizable spreadsheet table styles
US20070063990A1 (en) * 2005-09-21 2007-03-22 Samsung Electronics Co., Ltd. Touch sensitive display device and driving apparatus thereof, and method of detecting a touch
US20070204212A1 (en) * 2006-02-27 2007-08-30 Microsoft Corporation Dynamic thresholds for conditional formats
US20080010616A1 (en) * 2006-07-06 2008-01-10 Cherif Atia Algreatly Spherical coordinates cursor, mouse, and method
US20080143690A1 (en) * 2006-12-15 2008-06-19 Lg.Philips Lcd Co., Ltd. Display device having multi-touch recognizing function and driving method thereof
US20080164076A1 (en) * 2007-01-04 2008-07-10 Timothy James Orsley Capacitive sensing and absolute position mapping in displacement type pointing devices
US20080243823A1 (en) * 2007-03-28 2008-10-02 Elumindata, Inc. System and method for automatically generating information within an eletronic document
US7546523B2 (en) * 2002-03-28 2009-06-09 International Business Machines Corporation Method in an electronic spreadsheet for displaying and/or hiding range of cells
US7712018B2 (en) * 2005-12-12 2010-05-04 Microsoft Corporation Selecting and formatting warped text

Family Cites Families (60)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5276785A (en) * 1990-08-02 1994-01-04 Xerox Corporation Moving viewpoint with respect to a target in a three-dimensional workspace
EP0574213B1 (en) 1992-06-08 1999-03-24 Synaptics, Inc. Object position detector
US5604854A (en) * 1994-04-22 1997-02-18 Borland International, Inc. System and methods for reformatting multi-dimensional spreadsheet information
US20020091728A1 (en) * 1998-08-05 2002-07-11 Henrik Kjaer Multidimensional electronic spreadsheet system and method
AU4478800A (en) * 1999-04-21 2000-11-02 Spss Inc. Computer method and apparatus for creating visible graphics by using a graph algebra
FR2806183B1 (en) * 1999-12-01 2006-09-01 Cartesis S A DEVICE AND METHOD FOR INSTANT CONSOLIDATION, ENRICHMENT AND "REPORTING" OR BACKGROUND OF INFORMATION IN A MULTIDIMENSIONAL DATABASE
US7611409B2 (en) 2001-09-20 2009-11-03 Igt Method and apparatus for registering a mobile device with a gaming machine
US20040027350A1 (en) * 2002-08-08 2004-02-12 Robert Kincaid Methods and system for simultaneous visualization and manipulation of multiple data types
US7010364B1 (en) * 2003-09-22 2006-03-07 The Mathworks, Inc. System and method for performing process visualization
US7961909B2 (en) 2006-03-08 2011-06-14 Electronic Scripting Products, Inc. Computer interface employing a manipulated object with absolute pose detection component and a display
US7620915B2 (en) 2004-02-13 2009-11-17 Ludwig Lester F Electronic document editing employing multiple cursors
FR2869723A1 (en) 2004-04-29 2005-11-04 Thomson Licensing Sa NON-CONTACT TRANSITION ELEMENT BETWEEN A WAVEGUIDE AND A MOCRORUBAN LINE
US20060055945A1 (en) * 2004-09-13 2006-03-16 Fazakerly William B Color-mapped data display
US7598949B2 (en) 2004-10-22 2009-10-06 New York University Multi-touch sensing light emitting diode display and method for using the same
US8151213B2 (en) * 2005-03-25 2012-04-03 International Business Machines Corporation System, method and program product for tabular data with dynamic visual cells
US8678901B1 (en) 2005-09-07 2014-03-25 Bally Gaming System gaming
US7864160B2 (en) 2005-10-05 2011-01-04 3M Innovative Properties Company Interleaved electrodes for touch sensing
US7627812B2 (en) * 2005-10-27 2009-12-01 Microsoft Corporation Variable formatting of cells
US8024666B2 (en) * 2006-06-30 2011-09-20 Business Objects Software Ltd. Apparatus and method for visualizing data
US8125455B2 (en) 2007-01-03 2012-02-28 Apple Inc. Full scale calibration measurement for multi-touch surfaces
US8144129B2 (en) 2007-01-05 2012-03-27 Apple Inc. Flexible touch sensing circuits
US8301993B2 (en) * 2007-02-08 2012-10-30 International Business Machines Corporation Mapping raw spreadsheet data into graphs on arbitrary subsets of the data
US8115753B2 (en) 2007-04-11 2012-02-14 Next Holdings Limited Touch screen system with hover and click input methods
US9317110B2 (en) 2007-05-29 2016-04-19 Cfph, Llc Game with hand motion control
US7936341B2 (en) 2007-05-30 2011-05-03 Microsoft Corporation Recognizing selection regions from multiple simultaneous inputs
US8935249B2 (en) * 2007-06-26 2015-01-13 Oracle Otc Subsidiary Llc Visualization of concepts within a collection of information
US7835999B2 (en) 2007-06-27 2010-11-16 Microsoft Corporation Recognizing input gestures using a multi-touch input device, calculated graphs, and a neural network with link weights
EP2206041A4 (en) * 2007-10-01 2011-02-16 Iconics Inc Visualization of process control data
US20090124348A1 (en) 2007-11-09 2009-05-14 Yoseloff Mark L Electronic dice control in gaming
JP2009140368A (en) 2007-12-07 2009-06-25 Sony Corp Input device, display device, input method, display method, and program
US9170649B2 (en) 2007-12-28 2015-10-27 Nokia Technologies Oy Audio and tactile feedback based on visual environment
US9019237B2 (en) 2008-04-06 2015-04-28 Lester F. Ludwig Multitouch parameter and gesture user interface employing an LED-array tactile sensor that can also operate as a display
US9870629B2 (en) 2008-06-20 2018-01-16 New Bis Safe Luxco S.À R.L Methods, apparatus and systems for data visualization and related applications
US8345014B2 (en) 2008-07-12 2013-01-01 Lester F. Ludwig Control of the operating system on a computing device via finger angle using a high dimensional touchpad (HDTP) touch user interface
US8169414B2 (en) 2008-07-12 2012-05-01 Lim Seung E Control of electronic games via finger angle using a high dimensional touchpad (HDTP) touch user interface
US8604364B2 (en) 2008-08-15 2013-12-10 Lester F. Ludwig Sensors, algorithms and applications for a high dimensional touchpad
US20100079385A1 (en) 2008-09-29 2010-04-01 Smart Technologies Ulc Method for calibrating an interactive input system and interactive input system executing the calibration method
US8529345B2 (en) 2008-10-02 2013-09-10 Igt Gaming system including a gaming table with mobile user input devices
US9104311B2 (en) 2008-10-09 2015-08-11 Lenovo (Singapore) Pte. Ltd. Slate computer with tactile home keys
JP4766101B2 (en) 2008-11-10 2011-09-07 ソニー株式会社 Tactile behavior recognition device, tactile behavior recognition method, information processing device, and computer program
US20100177118A1 (en) 2009-01-09 2010-07-15 Sytnikov Vadim Vasilievich Path creation utility for image editor
US8170346B2 (en) 2009-03-14 2012-05-01 Ludwig Lester F High-performance closed-form single-scan calculation of oblong-shape rotation angles from binary images of arbitrary size using running sums
US8274536B2 (en) 2009-03-16 2012-09-25 Apple Inc. Smart keyboard management for a multifunction device with a touch screen display
US8154529B2 (en) 2009-05-14 2012-04-10 Atmel Corporation Two-dimensional touch sensors
US8836648B2 (en) 2009-05-27 2014-09-16 Microsoft Corporation Touch pull-in gesture
US8217912B2 (en) 2009-06-17 2012-07-10 Broadcom Corporation Graphical authentication for a portable device and methods for use therewith
US20100328032A1 (en) 2009-06-24 2010-12-30 Broadcom Corporation Security for computing unit with femtocell ap functionality
US8179376B2 (en) 2009-08-27 2012-05-15 Research In Motion Limited Touch-sensitive display with capacitive and resistive touch sensors and method of control
US8313377B2 (en) 2009-10-14 2012-11-20 Sony Computer Entertainment America Llc Playing browser based games with alternative controls and interfaces
US20110285648A1 (en) 2010-01-22 2011-11-24 Lester Ludwig Use of fingerprint scanning sensor data to detect finger roll and pitch angles
US20110202934A1 (en) 2010-02-12 2011-08-18 Ludwig Lester F Window manger input focus control for high dimensional touchpad (htpd), advanced mice, and other multidimensional user interfaces
US20120056846A1 (en) 2010-03-01 2012-03-08 Lester F. Ludwig Touch-based user interfaces employing artificial neural networks for hdtp parameter and symbol derivation
US8686960B2 (en) 2010-04-23 2014-04-01 Lester F. Ludwig Piecewise-linear and piecewise-affine transformations for high dimensional touchpad (HDTP) output decoupling and corrections
US8754862B2 (en) 2010-07-11 2014-06-17 Lester F. Ludwig Sequential classification recognition of gesture primitives and window-based parameter smoothing for high dimensional touchpad (HDTP) user interfaces
US9950256B2 (en) 2010-08-05 2018-04-24 Nri R&D Patent Licensing, Llc High-dimensional touchpad game controller with multiple usage and networking modalities
US20120192119A1 (en) 2011-01-24 2012-07-26 Lester F. Ludwig Usb hid device abstraction for hdtp user interfaces
US8797288B2 (en) 2011-03-07 2014-08-05 Lester F. Ludwig Human user interfaces utilizing interruption of the execution of a first recognized gesture with the execution of a recognized second gesture
US20120280927A1 (en) 2011-05-04 2012-11-08 Ludwig Lester F Simple touch interface and hdtp grammars for rapid operation of physical computer aided design (cad) systems
US20130009896A1 (en) 2011-07-09 2013-01-10 Lester F. Ludwig 3d finger posture detection and gesture recognition on touch surfaces
US9052772B2 (en) 2011-08-10 2015-06-09 Lester F. Ludwig Heuristics for 3D and 6D touch gesture touch parameter calculations for high-dimensional touch parameter (HDTP) user interfaces

Patent Citations (98)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US4748676A (en) * 1986-05-16 1988-05-31 Fuji Electric Co., Ltd. Pattern rotation angle detecting system
US4899137A (en) * 1987-09-26 1990-02-06 Aeg Olympia Aktiengesellschaft Arrangement for the input and processing of characters and/or graphic patterns
US5270711A (en) * 1989-05-08 1993-12-14 U.S. Philips Corporation Touch sensor array systems and display systems incorporating such
US5237647A (en) * 1989-09-15 1993-08-17 Massachusetts Institute Of Technology Computer aided drawing in three dimensions
US5347295A (en) * 1990-10-31 1994-09-13 Go Corporation Control of a computer through a position-sensed stylus
US5471008A (en) * 1990-11-19 1995-11-28 Kabushiki Kaisha Kawai Gakki Seisakusho MIDI control apparatus
US5341133A (en) * 1991-05-09 1994-08-23 The Rowland Institute For Science, Inc. Keyboard having touch sensor keys for conveying information electronically
US5292999A (en) * 1991-08-14 1994-03-08 Fernandes Co., Ltd. Electric stringed instrument having a device for sustaining the vibration of the string
US5442168A (en) * 1991-10-15 1995-08-15 Interactive Light, Inc. Dynamically-activated optical instrument for producing control signals having a self-calibration means
US5475214A (en) * 1991-10-15 1995-12-12 Interactive Light, Inc. Musical sound effects controller having a radiated emission space
US5386219A (en) * 1991-10-16 1995-01-31 International Business Machines Corp. Touch overlay for improved touch sensitivity
US5585588A (en) * 1991-10-24 1996-12-17 Fernandes Co., Ltd. Electric stringed instrument having a device for sustaining the vibration of a string and an electromagnetic driver for the device
US5717939A (en) * 1991-11-18 1998-02-10 Compaq Computer Corporation Method and apparatus for entering and manipulating spreadsheet cell data
US5805137A (en) * 1991-11-26 1998-09-08 Itu Research, Inc. Touch sensitive input control device
US5378850A (en) * 1992-01-14 1995-01-03 Fernandes Co., Ltd. Electric stringed instrument having an arrangement for adjusting the generation of magnetic feedback
US20020093491A1 (en) * 1992-06-08 2002-07-18 David W. Gillespie Object position detector with edge motion feature and gesture recognition
US5889236A (en) * 1992-06-08 1999-03-30 Synaptics Incorporated Pressure sensitive scrollbar feature
US5440072A (en) * 1992-09-25 1995-08-08 Willis; Raymon A. System for rejuvenating vintage organs and pianos
US5459282A (en) * 1992-09-25 1995-10-17 Willis; Raymon A. System for rejuvenating vintage organs and pianos
US5357048A (en) * 1992-10-08 1994-10-18 Sgroi John J MIDI sound designer with randomizer function
US5420936A (en) * 1992-10-16 1995-05-30 International Business Machines Corporation Method and apparatus for accessing touch screen desktop objects via fingerprint recognition
US5550964A (en) * 1992-12-18 1996-08-27 Borland International, Inc. System and methods for intelligent analytical graphing
US6285358B1 (en) * 1993-04-30 2001-09-04 Microtouch Systems, Inc. Method of and apparatus for the elimination of the effects of inertial interference in force measurement systems, including touch-input computer and related displays employing touch force location measurement techniques
US5592752A (en) * 1993-05-13 1997-01-14 Industrial Technology Research Institute Process and an apparatus for producing teas
US5665927A (en) * 1993-06-30 1997-09-09 Casio Computer Co., Ltd. Method and apparatus for inputting musical data without requiring selection of a displayed icon
US5581678A (en) * 1993-08-06 1996-12-03 Borland International, Inc. System and methods for automated graphing of spreadsheet information
US5675100A (en) * 1993-11-03 1997-10-07 Hewlett; Walter B. Method for encoding music printing information in a MIDI message
US5592572A (en) * 1993-11-05 1997-01-07 The United States Of America As Represented By The Department Of Health And Human Services Automated portrait/landscape mode detection on a binary image
US5841428A (en) * 1993-11-05 1998-11-24 Intertactile Technologies Corporation Rotary circuit control devices with changeable graphics
US6404898B1 (en) * 1993-11-18 2002-06-11 Digimarc Corporation Method and system for encoding image and audio content
US6363159B1 (en) * 1993-11-18 2002-03-26 Digimarc Corporation Consumer audio appliance responsive to watermark data
US5565641A (en) * 1994-03-28 1996-10-15 Gruenbaum; Leon Relativistic electronic musical instrument
US6047073A (en) * 1994-11-02 2000-04-04 Advanced Micro Devices, Inc. Digital wavetable audio synthesizer with delay-based effects processing
US5659466A (en) * 1994-11-02 1997-08-19 Advanced Micro Devices, Inc. Monolithic PC audio circuit with enhanced digital wavetable audio synthesizer
US5668338A (en) * 1994-11-02 1997-09-16 Advanced Micro Devices, Inc. Wavetable audio synthesizer with low frequency oscillators for tremolo and vibrato effects
US5719347A (en) * 1994-12-07 1998-02-17 Yamaha Corporation Keyboard apparatus for electronic musical instrument with key depression detection unit
US5932827A (en) * 1995-01-09 1999-08-03 Osborne; Gary T. Sustainer for a musical instrument
US6005545A (en) * 1995-01-17 1999-12-21 Sega Enterprise, Ltd. Image processing method and electronic device
US5986224A (en) * 1995-04-19 1999-11-16 Elo Touchsystems, Inc. Acoustic condition sensor employing a plurality of mutually non-orthogonal waves
US5659145A (en) * 1995-04-27 1997-08-19 Weil; Robert P. Foot operated audio signal controller with lighted visual reference
US5824930A (en) * 1995-06-09 1998-10-20 Yamaha Corporation Keyboard musical instrument having key monitor exactly discriminating key motion
US5801340A (en) * 1995-06-29 1998-09-01 Invotronics Manufacturing Proximity sensor
US5741993A (en) * 1995-08-01 1998-04-21 Kabushiki Kaisha Kawai Gakki Seisakusho Electronic keyboard having a discrete pitch bender
US5724985A (en) * 1995-08-02 1998-03-10 Pacesetter, Inc. User interface for an implantable medical device using an integrated digitizer display screen
US5719561A (en) * 1995-10-25 1998-02-17 Gilbert R. Gonzales Tactile communication device and method
US6138130A (en) * 1995-12-08 2000-10-24 Inventure Technologies, Inc. System and method for processing data in an electronic spreadsheet in accordance with a data type
US5786540A (en) * 1996-03-05 1998-07-28 Westlund; Robert L. Controller apparatus for music sequencer
US6360019B1 (en) * 1996-03-28 2002-03-19 Microsoft Corporation Table-based compression with embedded coding
US5748184A (en) * 1996-05-28 1998-05-05 International Business Machines Corporation Virtual pointing device for touchscreens
US6107997A (en) * 1996-06-27 2000-08-22 Ure; Michael J. Touch-sensitive keyboard/mouse and computing device using the same
US5977466A (en) * 1996-07-11 1999-11-02 Yamaha Corporation Keyboard musical instrument equipped with small simple economical key touch generator
US5763806A (en) * 1996-07-15 1998-06-09 Willis; Raymon A. Method and apparatus for midifying vintage organs and pianos
US5850051A (en) * 1996-08-15 1998-12-15 Yamaha Corporation Method and apparatus for creating an automatic accompaniment pattern on the basis of analytic parameters
US6392705B1 (en) * 1997-03-17 2002-05-21 Microsoft Corporation Multimedia compression system with additive temporal layers
US6225975B1 (en) * 1997-03-18 2001-05-01 Alps Electric Co., Ltd. Control device for game machine
US6373475B1 (en) * 1997-04-25 2002-04-16 Michael Challis Converter for resistive touchscreens
US5827989A (en) * 1997-06-23 1998-10-27 Microsoft Corporation System and method for representing a musical event and for converting the musical event into a series of discrete events
US5852251A (en) * 1997-06-25 1998-12-22 Industrial Technology Research Institute Method and apparatus for real-time dynamic midi control
US6037937A (en) * 1997-12-04 2000-03-14 Nortel Networks Corporation Navigation tool for graphical user interface
US6310610B1 (en) * 1997-12-04 2001-10-30 Nortel Networks Limited Intelligent touch display
US6310279B1 (en) * 1997-12-27 2001-10-30 Yamaha Corporation Device and method for generating a picture and/or tone on the basis of detection of a physical event from performance information
US6408087B1 (en) * 1998-01-13 2002-06-18 Stmicroelectronics, Inc. Capacitive semiconductor user input device
US6392636B1 (en) * 1998-01-22 2002-05-21 Stmicroelectronics, Inc. Touchpad providing screen cursor/pointer movement control
US6323846B1 (en) * 1998-01-26 2001-11-27 University Of Delaware Method and apparatus for integrating manual input
US6204441B1 (en) * 1998-04-09 2001-03-20 Yamaha Corporation Method and apparatus for effectively displaying musical information with visual display
US6570078B2 (en) * 1998-05-15 2003-05-27 Lester Frank Ludwig Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting
US20070229477A1 (en) * 1998-05-15 2007-10-04 Ludwig Lester F High parameter-count touchpad controller
US20020005108A1 (en) * 1998-05-15 2002-01-17 Ludwig Lester Frank Tactile, visual, and array controllers for real-time control of music signal processing, mixing, video, and lighting
US20040118268A1 (en) * 1998-05-15 2004-06-24 Ludwig Lester F. Controlling and enhancing electronic musical instruments with video
US20010036299A1 (en) * 1998-05-15 2001-11-01 Andrew William Senior Combined fingerprint acquisition and control device
US6400836B2 (en) * 1998-05-15 2002-06-04 International Business Machines Corporation Combined fingerprint acquisition and control device
US20040074379A1 (en) * 1998-05-15 2004-04-22 Ludwig Lester F. Functional extensions of traditional music keyboards
US6288317B1 (en) * 1998-05-29 2001-09-11 Raymon A. Willis Real time transmission of keyboard musical performance
US6140565A (en) * 1998-06-08 2000-10-31 Yamaha Corporation Method of visualizing music system by combination of scenery picture and player icons
US6100461A (en) * 1998-06-10 2000-08-08 Advanced Micro Devices, Inc. Wavetable cache using simplified looping
US5969283A (en) * 1998-06-17 1999-10-19 Looney Productions, Llc Music organizer and entertainment center
US20010049695A1 (en) * 1998-10-20 2001-12-06 Ed H. Chi Visualization spreadsheet
US6051769A (en) * 1998-11-25 2000-04-18 Brown, Jr.; Donival Computerized reading display
US20040060001A1 (en) * 1998-12-31 2004-03-25 Microsoft Corporation Extension of formulas and formatting in an electronic spreadsheet
US6793619B1 (en) * 1999-06-09 2004-09-21 Yaacov Blumental Computer-implemented method and system for giving a user an impression of tactile feedback
US7030860B1 (en) * 1999-10-08 2006-04-18 Synaptics Incorporated Flexible transparent touch sensing system for electronic devices
US6320112B1 (en) * 2000-05-19 2001-11-20 Martin Lotze Procedure and device for the automatic selection of musical and/or tonal compositions
US20020087275A1 (en) * 2000-07-31 2002-07-04 Junhyong Kim Visualization and manipulation of biomolecular relationships using graph operators
US6988241B1 (en) * 2000-10-16 2006-01-17 International Business Machines Corporation Client side, web-based spreadsheet
US6703552B2 (en) * 2001-07-19 2004-03-09 Lippold Haken Continuous music keyboard
US20040251402A1 (en) * 2001-09-25 2004-12-16 Gerd Reime Circuit with an opto-electronic display unit
US7546523B2 (en) * 2002-03-28 2009-06-09 International Business Machines Corporation Method in an electronic spreadsheet for displaying and/or hiding range of cells
US20090187816A1 (en) * 2002-03-28 2009-07-23 Jean-Jacques Aureglia System and method in an electronic spreadsheet for displaying and/or hiding range of cells
US20060252530A1 (en) * 2003-01-08 2006-11-09 Igt Mobile device for providing filtered casino information based on real time data
US20070044019A1 (en) * 2003-05-23 2007-02-22 Byung-Ro Moon Multi-campaign assignment apparatus considering overlapping recommendation problem
US20070050699A1 (en) * 2005-08-30 2007-03-01 Microsoft Corporation Customizable spreadsheet table styles
US20070063990A1 (en) * 2005-09-21 2007-03-22 Samsung Electronics Co., Ltd. Touch sensitive display device and driving apparatus thereof, and method of detecting a touch
US7712018B2 (en) * 2005-12-12 2010-05-04 Microsoft Corporation Selecting and formatting warped text
US20070204212A1 (en) * 2006-02-27 2007-08-30 Microsoft Corporation Dynamic thresholds for conditional formats
US20080010616A1 (en) * 2006-07-06 2008-01-10 Cherif Atia Algreatly Spherical coordinates cursor, mouse, and method
US20080143690A1 (en) * 2006-12-15 2008-06-19 Lg.Philips Lcd Co., Ltd. Display device having multi-touch recognizing function and driving method thereof
US20080164076A1 (en) * 2007-01-04 2008-07-10 Timothy James Orsley Capacitive sensing and absolute position mapping in displacement type pointing devices
US20080243823A1 (en) * 2007-03-28 2008-10-02 Elumindata, Inc. System and method for automatically generating information within an eletronic document

Non-Patent Citations (6)

* Cited by examiner, † Cited by third party
Title
Ali El-Hajj et al., "The Use of Spreadsheet to Calculate the Convolution Sum of Two Finite Sequences", Tempus, 2004, pp 867-871 *
Chester et al. "Excel 97, Fourth Edition"; 1997, SYBEX, pp 1-19 (scanned copies) *
Chester et al. "Excel 97, Fourth Edition"; 1997, SYBEX, pp 1-5 (scanned copies) *
Chi et al., "Principles for Information Visualization Spreadsheets", ACM, 1998, pp 30-38 *
Fulton, Jennifier, "10 Minute Guide To Excel 97", 1997, QUE,pp 1-9 (scanned copies) *
Lu et al., "Piecewise Linear Image Coding Using Surface Triangulation and Geometric Compression", IEEE, 2000, pp 1-10 *

Cited By (236)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10719621B2 (en) 2007-02-21 2020-07-21 Palantir Technologies Inc. Providing unique views of data based on changes or rules
US10229284B2 (en) 2007-02-21 2019-03-12 Palantir Technologies Inc. Providing unique views of data based on changes or rules
US10248294B2 (en) 2008-09-15 2019-04-02 Palantir Technologies, Inc. Modal-less interface enhancements
US10747952B2 (en) 2008-09-15 2020-08-18 Palantir Technologies, Inc. Automatic creation and server push of multiple distinct drafts
US9383911B2 (en) 2008-09-15 2016-07-05 Palantir Technologies, Inc. Modal-less interface enhancements
US20110179360A1 (en) * 2010-01-19 2011-07-21 Livevol Inc. System and method for presenting option data using animated 3-dimensional graphical display
US8799783B2 (en) * 2010-01-19 2014-08-05 Livevol, Inc. System and method for presenting option data using animated 3-dimensional graphical display
US11410129B2 (en) 2010-05-01 2022-08-09 Monday.com Ltd. Digital processing systems and methods for two-way syncing with third party applications in collaborative work systems
US8309833B2 (en) * 2010-06-17 2012-11-13 Ludwig Lester F Multi-channel data sonification in spatial sound fields with partitioned timbre spaces using modulation of timbre and rendered spatial location as sonification information carriers
US20140330898A1 (en) * 2010-11-08 2014-11-06 Sony Corporation Methods and systems for use in providing a remote user interface
US11108848B2 (en) 2010-11-08 2021-08-31 Saturn Licensing Llc Methods and systems for use in providing a remote user interface
US10409892B2 (en) 2011-01-26 2019-09-10 Microsoft Technology Licensing, Llc Formatting data by example
US8863019B2 (en) * 2011-03-29 2014-10-14 International Business Machines Corporation Modifying numeric data presentation on a display
US20120254783A1 (en) * 2011-03-29 2012-10-04 International Business Machines Corporation Modifying numeric data presentation on a display
US10423582B2 (en) 2011-06-23 2019-09-24 Palantir Technologies, Inc. System and method for investigating large amounts of data
US11392550B2 (en) 2011-06-23 2022-07-19 Palantir Technologies Inc. System and method for investigating large amounts of data
US10706220B2 (en) 2011-08-25 2020-07-07 Palantir Technologies, Inc. System and method for parameterizing documents for automatic workflow generation
US9880987B2 (en) 2011-08-25 2018-01-30 Palantir Technologies, Inc. System and method for parameterizing documents for automatic workflow generation
US11138180B2 (en) 2011-09-02 2021-10-05 Palantir Technologies Inc. Transaction protocol for reading database values
US9881396B2 (en) 2012-08-10 2018-01-30 Microsoft Technology Licensing, Llc Displaying temporal information in a spreadsheet application
US9317963B2 (en) 2012-08-10 2016-04-19 Microsoft Technology Licensing, Llc Generating scenes and tours in a spreadsheet application
US9996953B2 (en) 2012-08-10 2018-06-12 Microsoft Technology Licensing, Llc Three-dimensional annotation facing
US10008015B2 (en) 2012-08-10 2018-06-26 Microsoft Technology Licensing, Llc Generating scenes and tours in a spreadsheet application
US9898335B1 (en) 2012-10-22 2018-02-20 Palantir Technologies Inc. System and method for batch evaluation programs
US11182204B2 (en) 2012-10-22 2021-11-23 Palantir Technologies Inc. System and method for batch evaluation programs
US10372808B1 (en) 2012-12-12 2019-08-06 Google Llc Passing functional spreadsheet data by reference
US11630948B1 (en) 2012-12-12 2023-04-18 Google Llc Passing functional spreadsheet data by reference
US10922482B1 (en) 2012-12-12 2021-02-16 Google Llc Passing functional spreadsheet data by reference
US10743133B2 (en) 2013-01-31 2020-08-11 Palantir Technologies Inc. Populating property values of event objects of an object-centric data model using image metadata
US9380431B1 (en) 2013-01-31 2016-06-28 Palantir Technologies, Inc. Use of teams in a mobile application
US10313833B2 (en) 2013-01-31 2019-06-04 Palantir Technologies Inc. Populating property values of event objects of an object-centric data model using image metadata
US11144079B2 (en) 2013-02-11 2021-10-12 Graco Minnesota Inc. Remote monitoring for fluid applicator system
US11630470B2 (en) 2013-02-11 2023-04-18 Graco Inc. Remote monitoring for fluid applicator system
US11698650B2 (en) 2013-02-11 2023-07-11 Graco Minnesota Inc. Remote monitoring for fluid applicator system
US11934212B2 (en) 2013-02-11 2024-03-19 Graco Minnesota Inc. Paint sprayer distributed control and output volume monitoring architectures
US11934211B2 (en) 2013-02-11 2024-03-19 Graco Minnesota Inc. Paint sprayer distributed control and output volume monitoring architectures
US11249498B2 (en) 2013-02-11 2022-02-15 Graco Minnesota Inc. Remote monitoring for fluid applicator system
US11372432B2 (en) 2013-02-11 2022-06-28 Graco Minnesota Inc. Remote monitoring for fluid applicator system
US11934210B2 (en) 2013-02-11 2024-03-19 Graco Minnesota Inc. Paint sprayer distributed control and output volume monitoring architectures
US11592850B2 (en) 2013-02-11 2023-02-28 Graco Minnesota Inc. Remote monitoring for fluid applicator system
US10817513B2 (en) 2013-03-14 2020-10-27 Palantir Technologies Inc. Fair scheduling for mixed-query loads
US10997363B2 (en) 2013-03-14 2021-05-04 Palantir Technologies Inc. Method of generating objects and links from mobile reports
US10037314B2 (en) 2013-03-14 2018-07-31 Palantir Technologies, Inc. Mobile reports
US9646396B2 (en) 2013-03-15 2017-05-09 Palantir Technologies Inc. Generating object time series and data objects
US10452678B2 (en) 2013-03-15 2019-10-22 Palantir Technologies Inc. Filter chains for exploring large data sets
US10453229B2 (en) 2013-03-15 2019-10-22 Palantir Technologies Inc. Generating object time series from data objects
US10216801B2 (en) 2013-03-15 2019-02-26 Palantir Technologies Inc. Generating data clusters
US10482097B2 (en) 2013-03-15 2019-11-19 Palantir Technologies Inc. System and method for generating event visualizations
US10264014B2 (en) 2013-03-15 2019-04-16 Palantir Technologies Inc. Systems and user interfaces for dynamic and interactive investigation based on automatic clustering of related data in various data structures
US9852205B2 (en) 2013-03-15 2017-12-26 Palantir Technologies Inc. Time-sensitive cube
US9852195B2 (en) 2013-03-15 2017-12-26 Palantir Technologies Inc. System and method for generating event visualizations
US9965937B2 (en) 2013-03-15 2018-05-08 Palantir Technologies Inc. External malware data item clustering and analysis
US10977279B2 (en) 2013-03-15 2021-04-13 Palantir Technologies Inc. Time-sensitive cube
US9779525B2 (en) 2013-03-15 2017-10-03 Palantir Technologies Inc. Generating object time series from data objects
US10360705B2 (en) 2013-05-07 2019-07-23 Palantir Technologies Inc. Interactive data object map
US9953445B2 (en) 2013-05-07 2018-04-24 Palantir Technologies Inc. Interactive data object map
US10013413B2 (en) 2013-06-14 2018-07-03 Microsoft Technology Licensing, Llc Smart fill
US10229101B2 (en) 2013-06-14 2019-03-12 Microsoft Technology Licensing, Llc Smart fill
US10120854B2 (en) 2013-06-15 2018-11-06 Microsoft Technology Licensing, Llc Application/document collaboration in a multi-device environment
US9588954B2 (en) 2013-06-15 2017-03-07 Microsoft Technology Licensing, Llc Application/document collaboration in a multi-device environment
US10699071B2 (en) 2013-08-08 2020-06-30 Palantir Technologies Inc. Systems and methods for template based custom document generation
US9311289B1 (en) * 2013-08-16 2016-04-12 Google Inc. Spreadsheet document tab conditional formatting
US9514200B2 (en) 2013-10-18 2016-12-06 Palantir Technologies Inc. Systems and user interfaces for dynamic and interactive simultaneous querying of multiple data stores
US10719527B2 (en) 2013-10-18 2020-07-21 Palantir Technologies Inc. Systems and user interfaces for dynamic and interactive simultaneous querying of multiple data stores
US10037383B2 (en) 2013-11-11 2018-07-31 Palantir Technologies, Inc. Simple web search
US11100174B2 (en) 2013-11-11 2021-08-24 Palantir Technologies Inc. Simple web search
US10198515B1 (en) 2013-12-10 2019-02-05 Palantir Technologies Inc. System and method for aggregating data from a plurality of data sources
US11138279B1 (en) 2013-12-10 2021-10-05 Palantir Technologies Inc. System and method for aggregating data from a plurality of data sources
US10356032B2 (en) 2013-12-26 2019-07-16 Palantir Technologies Inc. System and method for detecting confidential information emails
US10230746B2 (en) 2014-01-03 2019-03-12 Palantir Technologies Inc. System and method for evaluating network threats and usage
US10805321B2 (en) 2014-01-03 2020-10-13 Palantir Technologies Inc. System and method for evaluating network threats and usage
EP2911100A1 (en) * 2014-02-20 2015-08-26 Palantir Technologies, Inc. Relationship visualizations
US9483162B2 (en) 2014-02-20 2016-11-01 Palantir Technologies Inc. Relationship visualizations
US10402054B2 (en) 2014-02-20 2019-09-03 Palantir Technologies Inc. Relationship visualizations
US10795723B2 (en) 2014-03-04 2020-10-06 Palantir Technologies Inc. Mobile tasks
US10180977B2 (en) 2014-03-18 2019-01-15 Palantir Technologies Inc. Determining and extracting changed data from a data source
US10871887B2 (en) 2014-04-28 2020-12-22 Palantir Technologies Inc. Systems and user interfaces for dynamic and interactive access of, investigation of, and analysis of data objects stored in one or more databases
US9857958B2 (en) 2014-04-28 2018-01-02 Palantir Technologies Inc. Systems and user interfaces for dynamic and interactive access of, investigation of, and analysis of data objects stored in one or more databases
US10621281B2 (en) 2014-05-08 2020-04-14 Google Llc Populating values in a spreadsheet using semantic cues
US9959265B1 (en) 2014-05-08 2018-05-01 Google Llc Populating values in a spreadsheet using semantic cues
US10169303B2 (en) * 2014-05-27 2019-01-01 Hitachi, Ltd. Management system for managing information system
US20170017617A1 (en) * 2014-05-27 2017-01-19 Hitachi, Ltd. Management system for managing information system
US10474317B2 (en) * 2014-06-25 2019-11-12 Oracle International Corporation Dynamic node grouping in grid-based visualizations
US10180929B1 (en) 2014-06-30 2019-01-15 Palantir Technologies, Inc. Systems and methods for identifying key phrase clusters within documents
US11341178B2 (en) 2014-06-30 2022-05-24 Palantir Technologies Inc. Systems and methods for key phrase characterization of documents
US10824799B2 (en) 2014-06-30 2020-11-03 Microsoft Technology Licensing, Llc Summary data autofill
US10162887B2 (en) 2014-06-30 2018-12-25 Palantir Technologies Inc. Systems and methods for key phrase characterization of documents
US9619557B2 (en) 2014-06-30 2017-04-11 Palantir Technologies, Inc. Systems and methods for key phrase characterization of documents
US10929436B2 (en) 2014-07-03 2021-02-23 Palantir Technologies Inc. System and method for news events detection and visualization
US10798116B2 (en) 2014-07-03 2020-10-06 Palantir Technologies Inc. External malware data item clustering and analysis
US9998485B2 (en) 2014-07-03 2018-06-12 Palantir Technologies, Inc. Network intrusion data item clustering and analysis
US9454281B2 (en) 2014-09-03 2016-09-27 Palantir Technologies Inc. System for providing dynamic linked panels in user interface
US10866685B2 (en) 2014-09-03 2020-12-15 Palantir Technologies Inc. System for providing dynamic linked panels in user interface
US9880696B2 (en) 2014-09-03 2018-01-30 Palantir Technologies Inc. System for providing dynamic linked panels in user interface
US10664490B2 (en) 2014-10-03 2020-05-26 Palantir Technologies Inc. Data aggregation and analysis system
US10360702B2 (en) 2014-10-03 2019-07-23 Palantir Technologies Inc. Time-series analysis system
US9767172B2 (en) 2014-10-03 2017-09-19 Palantir Technologies Inc. Data aggregation and analysis system
US9501851B2 (en) 2014-10-03 2016-11-22 Palantir Technologies Inc. Time-series analysis system
US11004244B2 (en) 2014-10-03 2021-05-11 Palantir Technologies Inc. Time-series analysis system
US11275753B2 (en) 2014-10-16 2022-03-15 Palantir Technologies Inc. Schematic and database linking system
US9984133B2 (en) 2014-10-16 2018-05-29 Palantir Technologies Inc. Schematic and database linking system
US10728277B2 (en) 2014-11-06 2020-07-28 Palantir Technologies Inc. Malicious software detection in a computing system
US10135863B2 (en) 2014-11-06 2018-11-20 Palantir Technologies Inc. Malicious software detection in a computing system
US9558352B1 (en) 2014-11-06 2017-01-31 Palantir Technologies Inc. Malicious software detection in a computing system
US9898528B2 (en) 2014-12-22 2018-02-20 Palantir Technologies Inc. Concept indexing among database of documents using machine learning techniques
US9589299B2 (en) 2014-12-22 2017-03-07 Palantir Technologies Inc. Systems and user interfaces for dynamic and interactive investigation of bad actor behavior based on automatic clustering of related data in various data structures
US10552994B2 (en) 2014-12-22 2020-02-04 Palantir Technologies Inc. Systems and interactive user interfaces for dynamic retrieval, analysis, and triage of data items
US9367872B1 (en) 2014-12-22 2016-06-14 Palantir Technologies Inc. Systems and user interfaces for dynamic and interactive investigation of bad actor behavior based on automatic clustering of related data in various data structures
US10447712B2 (en) 2014-12-22 2019-10-15 Palantir Technologies Inc. Systems and user interfaces for dynamic and interactive investigation of bad actor behavior based on automatic clustering of related data in various data structures
US10157200B2 (en) 2014-12-29 2018-12-18 Palantir Technologies Inc. Interactive user interface for dynamic data analysis exploration and query processing
US9870389B2 (en) 2014-12-29 2018-01-16 Palantir Technologies Inc. Interactive user interface for dynamic data analysis exploration and query processing
US10552998B2 (en) 2014-12-29 2020-02-04 Palantir Technologies Inc. System and method of generating data points from one or more data stores of data items for chart creation and manipulation
US9817563B1 (en) 2014-12-29 2017-11-14 Palantir Technologies Inc. System and method of generating data points from one or more data stores of data items for chart creation and manipulation
US9727560B2 (en) 2015-02-25 2017-08-08 Palantir Technologies Inc. Systems and methods for organizing and identifying documents via hierarchies and dimensions of tags
US10474326B2 (en) 2015-02-25 2019-11-12 Palantir Technologies Inc. Systems and methods for organizing and identifying documents via hierarchies and dimensions of tags
US10459619B2 (en) 2015-03-16 2019-10-29 Palantir Technologies Inc. Interactive user interfaces for location-based data analysis
US9891808B2 (en) 2015-03-16 2018-02-13 Palantir Technologies Inc. Interactive user interfaces for location-based data analysis
US10223748B2 (en) 2015-07-30 2019-03-05 Palantir Technologies Inc. Systems and user interfaces for holistic, data-driven investigation of bad actor behavior based on clustering and scoring of related data
US9454785B1 (en) 2015-07-30 2016-09-27 Palantir Technologies Inc. Systems and user interfaces for holistic, data-driven investigation of bad actor behavior based on clustering and scoring of related data
US11501369B2 (en) 2015-07-30 2022-11-15 Palantir Technologies Inc. Systems and user interfaces for holistic, data-driven investigation of bad actor behavior based on clustering and scoring of related data
US10484407B2 (en) 2015-08-06 2019-11-19 Palantir Technologies Inc. Systems, methods, user interfaces, and computer-readable media for investigating potential malicious communications
US10444940B2 (en) 2015-08-17 2019-10-15 Palantir Technologies Inc. Interactive geospatial map
US10444941B2 (en) 2015-08-17 2019-10-15 Palantir Technologies Inc. Interactive geospatial map
US10489391B1 (en) 2015-08-17 2019-11-26 Palantir Technologies Inc. Systems and methods for grouping and enriching data items accessed from one or more databases for presentation in a user interface
US11150917B2 (en) 2015-08-26 2021-10-19 Palantir Technologies Inc. System for data aggregation and analysis of data from a plurality of data sources
US11934847B2 (en) 2015-08-26 2024-03-19 Palantir Technologies Inc. System for data aggregation and analysis of data from a plurality of data sources
US10706434B1 (en) 2015-09-01 2020-07-07 Palantir Technologies Inc. Methods and systems for determining location information
US10296617B1 (en) 2015-10-05 2019-05-21 Palantir Technologies Inc. Searches of highly structured data
US10572487B1 (en) 2015-10-30 2020-02-25 Palantir Technologies Inc. Periodic database search manager for multiple data sources
US10031906B2 (en) 2015-11-02 2018-07-24 Microsoft Technology Licensing, Llc Images and additional data associated with cells in spreadsheets
US10599764B2 (en) 2015-11-02 2020-03-24 Microsoft Technology Licensing, Llc Operations on images associated with cells in spreadsheets
US10713428B2 (en) 2015-11-02 2020-07-14 Microsoft Technology Licensing, Llc Images associated with cells in spreadsheets
WO2017079138A1 (en) * 2015-11-02 2017-05-11 Microsoft Technology Licensing, Llc Images and additional data associated with cells in spreadsheets
WO2017079032A1 (en) * 2015-11-02 2017-05-11 Microsoft Technology Licensing, Llc Operations on sound files associated with cells in spreadsheets
WO2017079345A1 (en) * 2015-11-02 2017-05-11 Microsoft Technology Licensing, Llc Images associated with cells in spreadsheets
WO2017079031A1 (en) * 2015-11-02 2017-05-11 Microsoft Technology Licensing, Llc Sound associated with cells in spreadsheets
WO2017079357A1 (en) * 2015-11-02 2017-05-11 Microsoft Technology Licensing, Llc Calculations on images within cells in spreadsheets
WO2017079128A1 (en) * 2015-11-02 2017-05-11 Microsoft Technology Licensing, Llc Operations on images associated with cells in spreadsheets
WO2017079358A1 (en) * 2015-11-02 2017-05-11 Microsoft Technology Licensing, Llc Calculations on sound associated with cells in spreadsheets
US11630947B2 (en) 2015-11-02 2023-04-18 Microsoft Technology Licensing, Llc Compound data objects
US11106865B2 (en) 2015-11-02 2021-08-31 Microsoft Technology Licensing, Llc Sound on charts
US10366157B2 (en) 2015-11-02 2019-07-30 Microsoft Technology Licensing, Llc Images on charts
US10503824B2 (en) 2015-11-02 2019-12-10 Microsoft Technology Licensing, Llc Video on charts
US11200372B2 (en) 2015-11-02 2021-12-14 Microsoft Technology Licensing, Llc Calculations on images within cells in spreadsheets
US10579724B2 (en) 2015-11-02 2020-03-03 Microsoft Technology Licensing, Llc Rich data types
US10997364B2 (en) 2015-11-02 2021-05-04 Microsoft Technology Licensing, Llc Operations on sound files associated with cells in spreadsheets
US11080474B2 (en) 2015-11-02 2021-08-03 Microsoft Technology Licensing, Llc Calculations on sound associated with cells in spreadsheets
US9934215B2 (en) 2015-11-02 2018-04-03 Microsoft Technology Licensing, Llc Generating sound files and transcriptions for use in spreadsheet applications
US10013409B2 (en) * 2015-11-10 2018-07-03 Fuji Xerox Co., Ltd. Information processing apparatus, information processing method, and non-transitory computer readable medium
JP2017091197A (en) * 2015-11-10 2017-05-25 富士ゼロックス株式会社 Information processing device and information processing program
US20170132189A1 (en) * 2015-11-10 2017-05-11 Fuji Xerox Co., Ltd. Information processing apparatus, information processing method, and non-transitory computer readable medium
US10678860B1 (en) 2015-12-17 2020-06-09 Palantir Technologies, Inc. Automatic generation of composite datasets based on hierarchical fields
US9823818B1 (en) 2015-12-29 2017-11-21 Palantir Technologies Inc. Systems and interactive user interfaces for automatic generation of temporal representation of data objects
US10540061B2 (en) 2015-12-29 2020-01-21 Palantir Technologies Inc. Systems and interactive user interfaces for automatic generation of temporal representation of data objects
US10437612B1 (en) * 2015-12-30 2019-10-08 Palantir Technologies Inc. Composite graphical interface with shareable data-objects
US11086640B2 (en) * 2015-12-30 2021-08-10 Palantir Technologies Inc. Composite graphical interface with shareable data-objects
US10997190B2 (en) * 2016-02-01 2021-05-04 Splunk Inc. Context-adaptive selection options in a modular visualization framework
US20170220633A1 (en) * 2016-02-01 2017-08-03 Splunk Inc. Context-Adaptive Selection Options in a Modular Visualization Framework
US10698938B2 (en) 2016-03-18 2020-06-30 Palantir Technologies Inc. Systems and methods for organizing and identifying documents via hierarchies and dimensions of tags
US10324609B2 (en) 2016-07-21 2019-06-18 Palantir Technologies Inc. System for providing dynamic linked panels in user interface
US10698594B2 (en) 2016-07-21 2020-06-30 Palantir Technologies Inc. System for providing dynamic linked panels in user interface
US10719188B2 (en) 2016-07-21 2020-07-21 Palantir Technologies Inc. Cached database and synchronization system for providing dynamic linked panels in user interface
US10853380B1 (en) 2016-07-31 2020-12-01 Splunk Inc. Framework for displaying interactive visualizations of event data
US10861202B1 (en) 2016-07-31 2020-12-08 Splunk Inc. Sankey graph visualization for machine data search and analysis system
US11037342B1 (en) 2016-07-31 2021-06-15 Splunk Inc. Visualization modules for use within a framework for displaying interactive visualizations of event data
US10853383B2 (en) 2016-07-31 2020-12-01 Splunk Inc. Interactive parallel coordinates visualizations
US10853382B2 (en) 2016-07-31 2020-12-01 Splunk Inc. Interactive punchcard visualizations
US10318630B1 (en) 2016-11-21 2019-06-11 Palantir Technologies Inc. Analysis of large bodies of textual data
US11893366B2 (en) 2016-12-03 2024-02-06 Thomas STACHURA Spreadsheet-based software application development
US11726753B2 (en) 2016-12-03 2023-08-15 Thomas STACHURA Spreadsheet-based software application development
US11442707B2 (en) * 2016-12-03 2022-09-13 Thomas STACHURA Spreadsheet-based software application development
CN108268193A (en) * 2017-01-04 2018-07-10 珠海金山办公软件有限公司 The display methods and device of the corresponding prompt message of function in electrical form
US11709852B2 (en) * 2017-12-04 2023-07-25 Palantir Technologies Inc. Query-based time-series data display and processing system
US20210248134A1 (en) * 2017-12-04 2021-08-12 Palantir Technologies Inc. Query-based time-series data display and processing system
US11599369B1 (en) 2018-03-08 2023-03-07 Palantir Technologies Inc. Graphical user interface configuration system
US10885021B1 (en) 2018-05-02 2021-01-05 Palantir Technologies Inc. Interactive interpreter and graphical user interface
CN108920145A (en) * 2018-05-08 2018-11-30 陕西法士特齿轮有限责任公司 A kind of UI table color lump moving method
US11698890B2 (en) 2018-07-04 2023-07-11 Monday.com Ltd. System and method for generating a column-oriented data structure repository for columns of single data types
US11436359B2 (en) 2018-07-04 2022-09-06 Monday.com Ltd. System and method for managing permissions of users for a single data type column-oriented data structure
US20200019603A1 (en) * 2018-07-13 2020-01-16 Microsoft Technology Licensing, Llc Systems, methods, and computer-readable media for improved table identification using a neural network
US11079922B2 (en) 2018-08-27 2021-08-03 Sap Se Analytics design system
US10684762B2 (en) * 2018-08-27 2020-06-16 Sap Se Analytics design system
WO2021024040A1 (en) * 2019-08-08 2021-02-11 Mann, Roy Digital processing systems and methods for automatic relationship recognition in tables of collaborative work systems
US11170163B2 (en) * 2019-08-08 2021-11-09 Monday.Com Digital processing systems and methods for automatic relationship recognition in tables of collaborative work systems
CN110515996A (en) * 2019-08-22 2019-11-29 河南理工大学 A kind of multichannel tropospheric delay resolved data integrates visible processing method
US11307753B2 (en) 2019-11-18 2022-04-19 Monday.Com Systems and methods for automating tablature in collaborative work systems
US11507738B2 (en) * 2019-11-18 2022-11-22 Monday.Com Digital processing systems and methods for automatic updates in collaborative work systems
US11175816B2 (en) 2019-11-18 2021-11-16 Monday.Com Digital processing systems and methods for automatic user time zone updates in collaborative work systems
US11775890B2 (en) 2019-11-18 2023-10-03 Monday.Com Digital processing systems and methods for map-based data organization in collaborative work systems
US11727323B2 (en) 2019-11-18 2023-08-15 Monday.Com Digital processing systems and methods for dual permission access in tables of collaborative work systems
US11030556B1 (en) 2019-11-18 2021-06-08 Monday.Com Digital processing systems and methods for dynamic object display of tabular information in collaborative work systems
US11361156B2 (en) * 2019-11-18 2022-06-14 Monday.Com Digital processing systems and methods for real-time status aggregation in collaborative work systems
US11526661B2 (en) 2019-11-18 2022-12-13 Monday.com Ltd. Digital processing systems and methods for integrated communications module in tables of collaborative work systems
US11301623B2 (en) 2020-02-12 2022-04-12 Monday.com Ltd Digital processing systems and methods for hybrid scaling/snap zoom function in table views of collaborative work systems
US11675972B2 (en) 2020-05-01 2023-06-13 Monday.com Ltd. Digital processing systems and methods for digital workflow system dispensing physical reward in collaborative work systems
US11347721B2 (en) 2020-05-01 2022-05-31 Monday.com Ltd. Digital processing systems and methods for automatic application of sub-board templates in collaborative work systems
US11367050B2 (en) 2020-05-01 2022-06-21 Monday.Com, Ltd. Digital processing systems and methods for customized chart generation based on table data selection in collaborative work systems
US11501256B2 (en) 2020-05-01 2022-11-15 Monday.com Ltd. Digital processing systems and methods for data visualization extrapolation engine for item extraction and mapping in collaborative work systems
US11501255B2 (en) 2020-05-01 2022-11-15 Monday.com Ltd. Digital processing systems and methods for virtual file-based electronic white board in collaborative work systems
US11755827B2 (en) 2020-05-01 2023-09-12 Monday.com Ltd. Digital processing systems and methods for stripping data from workflows to create generic templates in collaborative work systems
US11475408B2 (en) 2020-05-01 2022-10-18 Monday.com Ltd. Digital processing systems and methods for automation troubleshooting tool in collaborative work systems
US11277452B2 (en) 2020-05-01 2022-03-15 Monday.com Ltd. Digital processing systems and methods for multi-board mirroring of consolidated information in collaborative work systems
US11531966B2 (en) 2020-05-01 2022-12-20 Monday.com Ltd. Digital processing systems and methods for digital sound simulation system
US11537991B2 (en) 2020-05-01 2022-12-27 Monday.com Ltd. Digital processing systems and methods for pre-populating templates in a tablature system
US11587039B2 (en) 2020-05-01 2023-02-21 Monday.com Ltd. Digital processing systems and methods for communications triggering table entries in collaborative work systems
US11275742B2 (en) 2020-05-01 2022-03-15 Monday.com Ltd. Digital processing systems and methods for smart table filter with embedded boolean logic in collaborative work systems
US11354624B2 (en) 2020-05-01 2022-06-07 Monday.com Ltd. Digital processing systems and methods for dynamic customized user experience that changes over time in collaborative work systems
US11410128B2 (en) 2020-05-01 2022-08-09 Monday.com Ltd. Digital processing systems and methods for recommendation engine for automations in collaborative work systems
US11348070B2 (en) 2020-05-01 2022-05-31 Monday.com Ltd. Digital processing systems and methods for context based analysis during generation of sub-board templates in collaborative work systems
US11301813B2 (en) 2020-05-01 2022-04-12 Monday.com Ltd. Digital processing systems and methods for hierarchical table structure with conditional linking rules in collaborative work systems
US11397922B2 (en) 2020-05-01 2022-07-26 Monday.Com, Ltd. Digital processing systems and methods for multi-board automation triggers in collaborative work systems
US11687706B2 (en) 2020-05-01 2023-06-27 Monday.com Ltd. Digital processing systems and methods for automatic display of value types based on custom heading in collaborative work systems
US11907653B2 (en) 2020-05-01 2024-02-20 Monday.com Ltd. Digital processing systems and methods for network map visualizations of team interactions in collaborative work systems
US11282037B2 (en) 2020-05-01 2022-03-22 Monday.com Ltd. Digital processing systems and methods for graphical interface for aggregating and dissociating data from multiple tables in collaborative work systems
US11301812B2 (en) 2020-05-01 2022-04-12 Monday.com Ltd. Digital processing systems and methods for data visualization extrapolation engine for widget 360 in collaborative work systems
US11301811B2 (en) 2020-05-01 2022-04-12 Monday.com Ltd. Digital processing systems and methods for self-monitoring software recommending more efficient tool usage in collaborative work systems
US11416820B2 (en) 2020-05-01 2022-08-16 Monday.com Ltd. Digital processing systems and methods for third party blocks in automations in collaborative work systems
US11301814B2 (en) 2020-05-01 2022-04-12 Monday.com Ltd. Digital processing systems and methods for column automation recommendation engine in collaborative work systems
US11886804B2 (en) 2020-05-01 2024-01-30 Monday.com Ltd. Digital processing systems and methods for self-configuring automation packages in collaborative work systems
US11829953B1 (en) 2020-05-01 2023-11-28 Monday.com Ltd. Digital processing systems and methods for managing sprints using linked electronic boards
US11277361B2 (en) 2020-05-03 2022-03-15 Monday.com Ltd. Digital processing systems and methods for variable hang-time for social layer messages in collaborative work systems
US11449668B2 (en) 2021-01-14 2022-09-20 Monday.com Ltd. Digital processing systems and methods for embedding a functioning application in a word processing document in collaborative work systems
US11481288B2 (en) 2021-01-14 2022-10-25 Monday.com Ltd. Digital processing systems and methods for historical review of specific document edits in collaborative work systems
US11397847B1 (en) 2021-01-14 2022-07-26 Monday.com Ltd. Digital processing systems and methods for display pane scroll locking during collaborative document editing in collaborative work systems
US11726640B2 (en) 2021-01-14 2023-08-15 Monday.com Ltd. Digital processing systems and methods for granular permission system for electronic documents in collaborative work systems
US11475215B2 (en) 2021-01-14 2022-10-18 Monday.com Ltd. Digital processing systems and methods for dynamic work document updates using embedded in-line links in collaborative work systems
US11531452B2 (en) 2021-01-14 2022-12-20 Monday.com Ltd. Digital processing systems and methods for group-based document edit tracking in collaborative work systems
US11893213B2 (en) 2021-01-14 2024-02-06 Monday.com Ltd. Digital processing systems and methods for embedded live application in-line in a word processing document in collaborative work systems
US11392556B1 (en) 2021-01-14 2022-07-19 Monday.com Ltd. Digital processing systems and methods for draft and time slider for presentations in collaborative work systems
US11687216B2 (en) 2021-01-14 2023-06-27 Monday.com Ltd. Digital processing systems and methods for dynamically updating documents with data from linked files in collaborative work systems
US11928315B2 (en) 2021-01-14 2024-03-12 Monday.com Ltd. Digital processing systems and methods for tagging extraction engine for generating new documents in collaborative work systems
US11782582B2 (en) 2021-01-14 2023-10-10 Monday.com Ltd. Digital processing systems and methods for detectable codes in presentation enabling targeted feedback in collaborative work systems
US11954428B2 (en) 2021-04-29 2024-04-09 Monday.com Ltd. Digital processing systems and methods for accessing another's display via social layer interactions in collaborative work systems
US11741071B1 (en) 2022-12-28 2023-08-29 Monday.com Ltd. Digital processing systems and methods for navigating and viewing displayed content
US11886683B1 (en) 2022-12-30 2024-01-30 Monday.com Ltd Digital processing systems and methods for presenting board graphics
US11893381B1 (en) 2023-02-21 2024-02-06 Monday.com Ltd Digital processing systems and methods for reducing file bundle sizes

Also Published As

Publication number Publication date
US9665554B2 (en) 2017-05-30
US8826114B2 (en) 2014-09-02
US8826113B2 (en) 2014-09-02
US20130191712A1 (en) 2013-07-25
US20130167002A1 (en) 2013-06-27
US20130185619A1 (en) 2013-07-18
US20130132811A1 (en) 2013-05-23
US20170262425A1 (en) 2017-09-14
US20130174004A1 (en) 2013-07-04

Similar Documents

Publication Publication Date Title
US9665554B2 (en) Value-driven visualization primitives for tabular data of spreadsheets
US20110055722A1 (en) Data Visualization Environment with DataFlow Processing, Web, Collaboration, Advanced User Interfaces, and Spreadsheet Visualization
Satyanarayan et al. Critical reflections on visualization authoring systems
JP7354294B2 (en) System and method for providing responsive editing and display integrating hierarchical fluid components and dynamic layout
US20130091437A1 (en) Interactive data visulization utilizing hdtp touchpad hdtp touchscreens, advanced multitouch, or advanced mice
US10528236B2 (en) Creating a display pattern for multiple data-bound graphic objects
Méndez et al. iVoLVER: Interactive visual language for visualization extraction and reconstruction
Satyanarayan et al. Lyra: An interactive visualization design environment
US10067635B2 (en) Three dimensional conditional formatting
US7460123B1 (en) Dynamic control of graphic representations of data
Marriott et al. Just 5 questions: toward a design framework for immersive analytics
US20140208203A1 (en) Key-Frame based Authoring and Generation of Highly Resizable Document Layout
CN113011400A (en) Automatic identification and insight of data
US20050183009A1 (en) Method of entering a presentation into a computer
US10613725B2 (en) Fixing spaced relationships between graphic objects
US20190114817A1 (en) Generating multiple data-bound graphic objects
Bouzit et al. A design space for engineering graphical adaptive menus
Liu et al. Interactive extended reality techniques in information visualization
Romat et al. Expressive authoring of node-link diagrams with graphies
US20220044457A1 (en) Graph plotter and replicator tool
Edsall et al. Interactive maps for exploring spatial data
US11495328B2 (en) Arrowland: an online multiscale interactive tool for -omics data visualization
CN114003226A (en) Layered distributed graphic design system
Sievert et al. Extending ggplot2 for linked and animated web graphics
US11023110B2 (en) Creating an axis for data-bound objects

Legal Events

Date Code Title Description
STCB Information on status: application discontinuation

Free format text: ABANDONED -- FAILURE TO RESPOND TO AN OFFICE ACTION

AS Assignment

Owner name: NRI R&D PATENT LICENSING, LLC, TEXAS

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:LUDWIG, LESTER F;REEL/FRAME:042745/0063

Effective date: 20170608