WO2001040918A1

WO2001040918A1 - Method for identifying a data region of a document

Info

Publication number: WO2001040918A1
Application number: PCT/US2000/032676
Authority: WO
Inventors: Boris Dogramadgi; Dimitry Krasnopolsky; Vlad Loukiantchenko; Igor Makarov; Ilya Shpilberg; Alex Vaindiner; Illan Poreh
Original assignee: Anysoft, Ltd.
Priority date: 1999-11-30
Filing date: 2000-11-30
Publication date: 2001-06-07
Also published as: EP1242862A1; CA2392988A1; IL133230A0; AU1938301A

Abstract

There is provided a unique method for accurately identifying and locating a data region (60) of a document or base area (70) that has changed its visual representation. Optionally, data from the data region is acquired for further processing.

Description

TITLE

Method for identifying a data region of a document

RELATED APPLICATION(S)/CLAIM OF PRIORITY

This application claims the benefit of priority of Israeli patent application serial number 133230, which was filed on November 1 1, 1999 and entitled "Method for identifying regions in documents on the screen." Moreover, this application relates to United States Patent 5,889,518, United States Patent 5,903,269 and United States Patent 6,040,832, each of which is entitled, "Apparatus for and method of acquiring, processing and routing data contained in a gui window."

The entire disclosure contained in each of the forgoing patents and patent applications is incorporated by reference as if set forth at length herein.

FIELD OF THE INVENTION

The present invention relates generally to data processing and more particularly, to an apparatus, method and article of manufacture for defining a data region of a document.

BACKGROUND

There are several well-known methods for identifying and extracting data from selected regions of document images. In each of these methods, data regions of a document image are typically selected in order to acquire meaningful data located therein. In one method, template forms are utilized. Here, predefined regions within a given document are identified by matching the document to a database of template forms. The template forms consist of constant data. This constant data is stripped from the document and the data contained in the data regions acquired.

In another method, marks are set in certain places of the document in order to properly identify the coordinates of data regions. Marks are utilized as footholders. A footholder is any graphical object having some persistent properties, i.e. coordinates, text, child window identification number (ID), etc. A footholder can be recognized in run-time according to its properties. A graphical object is any object displayed on the screen, for example, windows and their elements, such as, borders and captions, strings of text, parts of strings, such as words, bitmaps, etc. Each graphical object is characterized by a surrounding geometrical region. The geometrical region does not have to be rectangular in shape. This method of data region identification is generally hard coded in document recognition systems. Data is further acquired from the data region.

Unfortunately, the existing methods, only provide for the possibility for identifying data regions in and acquiring data from document having a fixed structure of regions. Such fixed regions must have the same coordinates and the same relative position in the whole document. Therefore, in the cases where the document view or sizes of elements comprising the document is significantly changed, the prior art methods fail to provide satisfactory results. This is the case when two applications are executing in the computer. The first application draws graphical data (document) in window on the screen. The second application is "aware" of screen presentation of graphical objects of the first applications. The first application can change fonts on the document when window is resized, or document view may be changed due to scrolling. In this case coordinates of regions in document does not provide the possibility to accurately identify regions.

SUMMARY

The present invention is a solution to the forgoing unresolved problems and deficiencies of the prior art. Specifically, one object of the present invention is to identify data regions in documents that can change their visual representation. This is solved in such a way that at least one region with informative data on the base area, for example, a full screen, is selected. Then at least one footholder for identification of the region is selected, so that the position of the region is related with the position of the footholder by an algorithm which does not depend on the footholder's and the region's coordinates on the base area. Algorithm for automated identification of the region, which is based on the footholder's position, is constructed. Automated identification of the region on the base area is executed and data from the region is acquired. Therefore, in accordance with one aspect of the present invention, there is generally provided an apparatus, method and product for selecting at least one footholder having a fixed relative position with respect to a data region to provide a footholder location identifier; providing a logical coordinate system related to the location of the footholder, the coordinate system being in the document and resizing therewith and surrounding the data region; locating the data region in relationship to the logical coordinate system; and transforming the preceding steps into a set of commands thereby creating an algorithm for locating the data region.

In accordance with a second aspect of the present invention, there is provided an apparatus, method and product for executing, preferably automatically, the previously created algorithm for locating the data region.

In accordance with a third aspect of the present invention, there is provided an apparatus, method and product for acquiring the data within the data region.

These and other aspects, features and advantages of the present invention will become better understood with regard to the following description, appended claims, and accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWING(S)

Turning briefly to the drawings: Figure 1 is a block diagram of one embodiment of a data processing system constructed in accordance with the teachings herein.

Figure 2 is a block diagram of one memory layout of the data processing system of Figure 1.

Figure 3 is a flow diagram of an exemplary embodiment of the overall data region identification process.

Figure 4 illustrates a document with data to be processed according to the teachings herein.

Figure 5 illustrates the document of Figure 4 wherein several footholders are selected..

Figure 6 illustrates the document of Figure 5 wherein a logical reference grid based on the footholders is also shown.

DETAILED DESCRIPTION

Referring more specifically to the drawings, for illustrative purposes the present invention is embodied in the system configuration, method of operation and product or computer-readable medium, such as floppy disks, conventional hard disks, CD-ROMS, Flash ROMS, nonvolatile ROM, RAM and any other equivalent computer memory device, generally shown in Figures 1 - 6. It will be appreciated that the system, method of operation and product may vary as to the details of its configuration and operation without departing from the basic concepts disclosed herein.

1. Hardware Configuration

Referring to Figure 1 , there is shown a block diagram of one embodiment of a data processing system constructed in accordance with the teachings of the present invention. As shown, the system 1 comprises: memory 10, a central processing unit (CPU) 12, an input device 16, such as a keyboard and mouse; and a display device 18, such as a screen. Optionally, the system may also include a secondary storage device 14, such as a hard disk drive. A representative system 1 is a stationary or portable computer designed according to widely understood principles, e.g., a desktop computer, a laptop computer, a hand-held computer, or any other known or future device having at least the necessary physical and processing capabilities that are required to implement aspects of the present invention.

Referring to Figure 2, associated with memory 10 and executable by the CPU 12 are one or more applications, specifically, applications 34, 36, 38 and a client application 40 embodying the teachings of the present invention. These applications execute under a window-based

operating system, preferably a multitasking operating system 32 such as Microsoft® Windows

2000® or Windows NT®. Applications 34, 36 and 38 craw data on the screen 18 by making one or more calls 42 to interface functions of operating system 32. In doing so, applications 34, 36 and 38 expose all properties of their graphical objects, such as texts, coordinates of surrounding geometrical regions, etc., to the client application 40. This is accomplished either through Object Linking and Embedding (OLE) interfaces provided by the particular application itself or the help of other methods such as Microsoft's Active Accessibility or Optical Character Recognition methods (OCR).

Client application 40 gains information 44 about graphical objects, which are drawn on the screen 18. Information about the graphical objects drawn by the applications 34, 36 and 38 are not essential to the present description and, accordingly, will not be discussed further.

Client application 40 identifies data regions of a document during run-time even if the visual representation of the document has changed.

2. Method of Operation

A. Overview

In order to acquire data from a desired data region of a document, the client application 40 must automatically identify the data region during run-time. In order for the client application 40 to do so, a user, whether virtual or real, must first identify the data region during design time by performing/executing a set of operations. These operations, when combined, create an algorithm for accurately identifying the data region. Preferably, the algorithm is stored in memory to be used by the client application 40 during run-time. Note: as used herein, "design time" refers to the time spent for manual operations for region identification and creation of algorithm in accordance with this operations. "Run-time" refers to the period of time during which a client application executes the algorithm of identification that was constructed in design time.

Thereafter, client application 40, in executing the algorithm during run-time, always identifies the previously identified data region, in order to, for example, acquire the data contained within the identified data region. Therefore, as an example, even if a window containing the data region is manipulated, i.e., it is scrolled, closed, minimized, opened or restored, and/or the screen resolution changes, client application 40 will still be able to identify the data region.

More specifically and referring to Figure 3A and 3B in particular, at 50, a user would select one or more footholders that surround the desired data region. This action of selection corresponds to one or more operations which provide the first code segment of the data region identification algorithm. At 52, a logical coordinate system is constructed based on the footholders. This action of constructing a logical coordinate system corresponds to one or more operations which provide a second code segment comprising the data region identification algorithm. At 54, the forgoing code segments are combined to create the data region identification algorithm. During runtime, at 56, the previously constructed data region identification algorithm executes and at 58, the data region is identified. B. Implementation of the Present Invention

What follows is one application of the present invention, described with reference to Figures 4-6. Again, the following description is merely illustrative and should not be deemed as unnecessarily limiting.

Figure 4 illustrates a portion of a display device, such as a computer screen, whereupon a document containing data is displayed to the user. A user, wishing to define a particular data region of the displayed document, must select one or more footholders surrounding the data region.

Thus, turning to Figure 5, a user, desirous of identifying the data region 60 containing the data "Ben@usa.net", selects the surrounding text and/or text strings "site!", "address" and "Click to continue...", and sets them as footholders 62, 64 and 66, respectively.

Referring to Figure 6, next, the user causes a logical coordinate system to be constructed based on the relative position of the footholders 62, 64 and 66 to the region 60. In one embodiment, the logical coordinate system is a logical reference grid having enumerated grid lines along the sides of the region 60. It should be noted that the actual placement of the grid lines depends on multiple variants. However, the grid lines should serve to localize the region 60. Thus, as depicted, the user sets the grid lines 70, 72 and 74 along the sides of the footholders 62, 64 and 66.

As earlier stated, the grid lines are enumerated with coordinate values in order to uniquely identify the region 60 in the logical coordinate system. For example, horizontal grid lines 72 and 74 are numbered "1" and "2", respectively, and vertical grid line 70 is numbered "1". Thus, the region 60 is now determined by its logical coordinates, i.e. logical coordinates

[(1,2), (1)] ((horizontal coordinates " 1 " and "2"), (left vertical coordinate "1")]. Since, there is

no bounding footholder positioned right of the region 60, (hence, no right vertical coordinate), as shown in Figure 6, the region 60 spreads until it reaches the right border of the document/window.

The manual choosing of footholders and the manual creating of the logical reference grid based on the boundaries of the selected footholders are considered a set of commands, which are transformed into a data region identification algorithm 54 (See Figure 3 A). Of particular importance to the present solution is the fact that the data region identification algorithm 54 does not depend on the screen coordinates of the selected footholders and the identified data region 60. Instead, the algorithm 54 relates the identified region 60 with the positions of footholders 62, 64 and 66. Hence, although the screen coordinates of the identified region 60 may change due to window scrolling, resizing and/or other events, the logical coordinates of the identified region 60 will remain constant due to the fixed relative positions of the footholders to the region 60. In other words, the logical coordinate system moves with and changes with the window. See Figure 7.

Multiple algorithms can correspond to manual operations. In the present example the following method is chosen. Client application 40 (Figure 2) gets information about all graphical objects drawn by applications 34, 36 and 38, as was mentioned hereinabove. Footholders with words "site!" 62 (Figure 5), "address" 64 and "Click to continue..." 66 are identified inside the text on the screen. Algorithms of searching predefined words in text are known in the art. The coordinate grid in screen coordinates is formed based on the boundaries of these footholders. The grid is constiucted automatically from its prototype, which was constructed in design-time . Region 60 (Figure 6) is determined by its logical coordinates on the grid (logical coordinates ("1","2"), ("1 ")). In run-time screen coordinates are known for surrounding regions of footholders 62, 64 and 66. Thus, logical coordinates of region 60 can be recalculated to screen coordinates. Scrolling, resizing and many other events on the screen can change screen coordinates of the regions. In the preferred embodiment of the present invention it is considered that logical coordinates of identified regions with information are not changed due to these operations. After each of these events screen coordinates of regions must be recalculated.

Having now described a preferred embodiment of the invention, it should be apparent to those skilled in the art that the foregoing is illustrative only and not limiting, having been presented by way of example only. All the features disclosed in this specification (including any accompanying claims, abstract, and drawings) may be replaced by alternative features serving the same purpose, and equivalents or similar purpose, unless expressly stated otherwise. Therefore, numerous other embodiments of the modifications thereof are contemplated as falling within the scope of the present invention as defined by the appended claims and equivalents thereto.

For example, other means of identifying data regions are possible. Part of manual operations, which where described above can be done automatically (programmatically). For example after manual identification of region 60 it is possible to choose automatically nearest to this region footholders and construct grid lines on the basis of these footholders. Chosen region 60 can be automatically related to the grid. The resulting identification algorithm is constructed in the same manner as it was described previously.

Claims

CLAIMSWhat is claimed is:

1. A method for locating a first data region in a document, said data region having data, said method comprising: selecting at least one footholder having a fixed relative position with respect to said first data region to provide a footholder location identifier; providing a logical coordinate system related to said footholder location; said coordinate system being in said document and resizing therewith and surrounding said first data region; and locating said first data region in relationship to said logical coordinate system.

2. A method as in claim 1 further comprising transforming said preceding steps into a set of operations thereby creating an algorithm for automatically locating said first data region.

3. A method as in claim 1 wherein said logical coordinate system is a logical reference grid having enumerated grid lines for identifying said first data region in logical coordinates.

4. A method as in claim 2 wherein said logical coordinate system is a logical reference grid having enumerated grid lines for identifying said first data region in logical coordinates.

5. A method as in claim 2, further comprising: executing said algorithm thereby identifying said first data region.

6. A method as in claim 5 further comprising acquiring said data of said first data region.

7. A method as in claim 5 further comprising identifying a second data region in said first data region by iteratively performing the same steps used to identify said first data region and wherein said at least one footholder of said second data region is selected from within the boundaries of said first data region.

8. An apparatus for locating a first data region in a document, said data region having data, said apparatus comprising: a central processing unit; a display device associated with said central processing unit for displaying said document; an application associated with said central processing unit for: marking at least one footholder having a fixed relative position with respect to said first data region to provide a footholder location identifier; providing a logical coordinate system related to said footholder location; said coordinate system being in said document and resizing therewith and surrounding said first data region; and locating said first data region in relationship to said logical coordinate system.

9. An apparatus as in claim 8 further comprising an input device for selecting at least one footholder.

10. An apparatus as in claim 8 wherein said logical coordinate system is a logical reference grid having enumerated grid lines for identifying said first data region in logical coordinates.

1 1. An apparatus as in claim 8 wherein said application also transforms said preceding steps into a set of operations thereby creating an algorithm for automatically locating said first data region.

12. An apparatus as in claim 1 1 wherein said application also executes said algorithm thereby identifying said first data region.

13. An apparatus as in claim 12 wherein said application also acquires said data of said first data region.

14. An apparatus as in claim 1 wherein said application also identifies a second data region in said first data region by iteratively performing the same steps used to identify said first data region and wherein said at least one footholder of said second data region is selected from within the boundaries of said first data region.

15. A computer-readable medium having computer-executable instructions for performing a method for locating a first data region in a document, said first data region having data, said method comprising: marking at least one footholder having a fixed relative position with respect to said first data region to provide a footholder location identifier; providing a logical coordinate system related to said footholder location; said coordinate system being in said document and resizing therewith and surrounding said first data region; and locating said first data region in relationship to said logical coordinate system.

16. A computer-readable medium as in claim 15 wherein said logical coordinate system is a logical reference grid having enumerated grid lines for identifying said first data region in logical coordinates.

17. A computer-readable medium as in claim 15 wherein said method further comprises transforming said preceding steps into a set of operations thereby creating an algorithm for automatically locating said first data region.

18. A computer-readable medium as in claim 17 wherein said method further comprises executing said algorithm thereby identifying said first data region.

19. A computer-readable medium as in claim 18 wherein said method further comprises acquiring said data of said first data region.

20. A computer-readable medium as in claim 15 wherein said method further further comprises identifying a second data region in said first data region by iteratively performing the same steps used to identify said first data region and wherein said at least one footholder of said second data region is selected from within the boundaries of said first data region.