EP0052996A1

EP0052996A1 - Image information synthesizing apparatus

Info

Publication number: EP0052996A1
Application number: EP81305430A
Authority: EP
Inventors: Takumi Mizokawa
Original assignee: Kobe Steel Ltd
Current assignee: Kobe Steel Ltd
Priority date: 1980-11-20
Filing date: 1981-11-17
Publication date: 1982-06-02
Also published as: JPS5794784A; US4530009A

Abstract

A display terminal equipment for information processing systems comprising an image storage unit and an image synthesizing unit. Information to be displayed is divided into a fixed background picture such as graphic patterns, tables and flowsheets produced by a video camera and stored in the image storage unit, and a foreground picture mainly made up of numeric characters and is supplied from a computer. The background and foreground pictures are mixed by the image synthesizing unit and displayed on a CRT screen.

Description

This invention relates to an image information synthesizing apparatus used as a display terminal for information processors such as computers.
Information processing utilizing computers has been expanding in scope and has also increased its versatility and comprehensiveness. It is now often applied to process control in addition to office jobs and scientific calculations.
For example, in process control of various chemical plants and the like, the computer may control, monitor and record data for the process. In this situation, it is most important for information to be transferred smoothly and surely between the computer and the operator, that is, there should be good man-machine communication.
Between the operator and the computer, information is passed through terminal equipment referred to as a console, which typically includes a CRT (cathode ray tube) display because of the following advantages.

1. Much information can be displayed on the CRT screen at a time.
2. Colour display is possible.
3. Graphic patterns, tables and graphic panels can be displayed in addition to literal and numeric characters.
4. Quick display response is achieved without noise.
5. Various graphic patterns can be displayed alternately on the same CRT screen.

The CRT display of the console is used to display alphabetic and numeric characters. There is a trend towards using the CRT screen for displaying a combination of graphic patterns, tables, process flowsheets, graphic panels, Kanji, and the like in order to achieve more comprehensive communication between the computer and the operator. it is also sometimes necessary to provide acoustic information in addition to the graphically displayed information mentioned above.
The conventional terminal equipment, however, only displays information retrieved from the computer, and in order to satisfy the above-mentioned diverse requirements it is necessary to devote a considerably amount of manpower to develop suitable output programs, a large storage capacity for storing the software and a vast support software package are required, and also there is an overhead problem of the computer.
It is aesirable to create a screen format in the local language or one that can be understood by the user (e.g.. French, Portuguese, Spanish, Arabic, Chinese, Russian, etc.). To do this, it is necessary that the screen format is amended to meet each user's language, resulting in a further increase of software for displaying information.
Prior Art located are Japanese Patent Publication No SHO 49-21970 which corresponds to Swedish Patent 349.165 EPC application No 79300383.1 which is published as 4,197 to RCA corp., and IBM Technical Disclosure Bulletin Vol. 22 No 3 Aug. 1979, PP 1200 & 1201, "Extended Function on Character Display".
The present invention provides image information synthesizing apparatus for an information processor which produces an output video signal of a foreground picture in accordance with an output program; .said image information synthesizing apparatus comprising an image storage apparatus including means for storing a video signal of at least one background picture which has been produced by a picture signal generating apparatus, and means for outputting a video signal of the background picture specified by a background picture selector signal; an image signal synthesizing apparatus connected to said image storage apparatus and including means for connection to the information processor including means for adding the video signal of the selected background picture outputted from said image storage apparatus to the video signal of a foreground picture from the information processor which varies in accordance with the result of information processing by said information processor and means for outputting a combined video signal of the combined background and foreground pictures; and a display apparatus for displaying a picture in response to said combined video signal supplied from said image signal synthesizing apparatus.
The information processor is preferably a computer and the picture signal generating apparatus is preferably a video camera.
The background picture may be for example the outline of a table (with parts written in the users language) and the foreground picture numbers or letters inserted in the table. In another example the background picture may be a process diagram of, say, a flow process, and the foreground picture may be figures or arrows added to the process diagram. Thus the background picture is stored in the image storage unit and retrieved therefrom instead of being produced by an output program in the computer, thereby facilitating the production of picture information such as the user oriented language, graphic patterns, tables, process flowcharts, and the like, and providing a smooth communication between the operator and the computer as well as reducing the necessary capacity of the computer.
The image storage unit may store an audio signal as well as the video signal, thereby allowing a video output and audio output at the same time. The apparatus may include means to allow the production of a hard copy of the screen display.
There may be provided a storage, unit for storing the video signal from the image signal synthesizing apparatus so that the operation of the information processor can be checked and analysed.
The features of the invention will become apparent from the following detailed description and the accompanying drawings of preferred embodiments of the invention applied to process control.

FIGURE 1 is a block diagram showing the configuration of an image information synthesizing terminal apparatus embodying the present invention;
FIGURE 2a and 2b are plan views each illustrating an example of the background picture panel;
FIGURE 3 is a block diagram showing a variation of the system of FIGURE 1;
FIGURE 4 is a block diagram showing the addition of a hard copy unit to the system of FIGURE 1 or 3; and
FIGURE 5 is a block diagram showing another embodiment of part of the invention wherein the operational history may be recorded and reproduced.

An operation screen (CRT) is divided into a foreground picture which varies during the display and a generally fixed background picture. The foreground picture is displayed in accordance with data processed by a computer, whereas the background picture is displayed in accordance with information retrieved from an image storage unit such as a video disk file operating substantially independently of the computer, and these two pieces of display information are superimposed on the CRT screen. FIGURE 1 is a block diagram of an embodiment of this invention, showing a computer 1 for providing process data 2, an image information synthesizing terminal apparatus which will be described in the following, and a video camera 4 for transforming a background picture 6 drawn on a background picture panel 5 into a video signal.
The image information synthesizing terminal apparatus 3 comprises a video disk file 7 as an image storage unit for storing the video signal of the background picture 6, an image synthesizing unit 8 for adding the video signal of the background picture 6 stored in the video disk file 7 to the video signal of the foreground picture supplied from the computer 1, and a CRT display unit 9 for displaying the combined video signal from the image synthesizing unit 8. The image information synthesizing terminal apparatus may further comprise a keyboard so that the necessary image information can be selected out of video memory disk file by operating the keyboard.
The video disk file 7 stores the video signal produced by shooting a background picture with the video camera 4, the background picture 6 having been drawn on the background picture panel 5 in accordance with one or more output program of the computer 1, and sends out to the image synthesizing unit 8 the video signal of a background picture specified by the background picture selection signal issued by the computer in accordance with the output program.
The image synthesizing unit 8 is a video mixer circuit known in the art, which adds the video signal of a foreground picture supplied from the computer 1 to the video signal of a background picture supplied from the video disk file 7 in accordance with the background picture picture selection signal. The resultant combined output video signal is supplied to the CRT display unit 9, on which the foreground picture 10 is superimposed on the background picture 6. If necessary more than two pieces of graphic information can be superimposed to form a background picture.
Furthermore, such a graphic information can be displayed on a CRT screen with a desired display co-ordinate, desired size and desired brightness so that such information occupies only a limited but desired area of the CRT screen.
Graphic information stored in a file can be shared by different CRT screens by way of the above mentioned technique.
Graphic information stored in a file can be displayed with adjusted brightness in concordance with data of the computer.
The basic operations of the system shown in FIGURE 1 will now be described.
Background picture panel production:

The background picture panel 5 is produced by conventional drawing procedures. That is, items of the background picture 6, such as a table shown in FIGURE 2a or a process flowsheet shown in FIGURE 2b, used in the output program of the computer 1 is drawn by hand or produced by sticking on a sheet of drawing paper or the like.

In this case, any type of characters including alphabetic and numeric characters, Arabic letters, Russian letters, etc. in any size which can be shot by the video camera 4 can be used, and a colour video camera 4 may be used for providing a colourful background picture 6.
Background picture recording on video disk file:

The background picture panel 5 as produced by the above-mentioned process is shot by the video camera 4, and then stored in the video disk file 7.

Since a shot of short duration is required, the video disk file 7 can store numerous background pictures.
Foreground information production:

The computer 1 receives information such as process data 2 which is to be displayed as a foreground picture 10, and edits the data in accordance with the specification of the foreground picture 10 so that the foreground picture will coincide and properly relate to the background picture.

Operation screen display request:

When the operator enters an operation screen display request to the computer 1, it edits and outputs the continuously varying foreground information and also issues the background picture selection signal corresponding to the background information needed for the requested operation screen.

Superimposed display of operation screen:

Through the process as described in "Operation screen display request", the video signal of the continuously varying foreground picture 10 supplied from the computer 1 and the video signal of the background picture 6 stored in the video disk file 7 in accordance with the background picture selection signal issued by the computer 1 are added bu the image synthesizing unit 8 to form a combined video signal. Then, a display of numeric characters, e.g., "1.0", "2.8", etc., are displayed as the foreground picture 10 in the correct position with respect to the table forming the background picture 6 on the screen of the CRT display unit 9, as illustrated in FIGURE 1.

It can be seen from the above desciption that the part of the output program of the computer normally reserved for the background picture 6 is replaced by the video signal of the background picture 6 stored in the video disk file 7.
Operation screen modification:

In order to modify the background picture 6 for the operation screen, a modification is made to the background picture 6 drawn on the background picture panel 5 using an eraser, and then it is shot by the video camera 4 so as to update the video image data in the video disk file 7.

Modification of the foreground picture 10 for the operation screen requires a change of the output program of the computer 1. However, the foreground picture 10 is mainly made up of numeric characters such as for process data, and the output program for such display information can be modified easily.
Language alteration on operation screen:

There is occasionally a requirement for producing an operation screen in a.specific language, for example, for the heading of the table, while leaving remaining portions as the existing work.

In such cases, only a background picture panel in the specific language needs to be produced.
FIGURE 3 shows an embodiment of the invention wherein the operator is guided both visually and audibly.
In the embodiment of FIGURE 3, an audio message 11 to be passed to the operator of the process control system is transformed into an electrical signal by a microphone and an associated amplifier (which may be incorporated within the video camera 4, or separate), and the message 11 is stored together with the background picture 6 in a video disk file 7, so that the video disk file 7 outputs the video signal of the background picture 6 and the audio message 11 in response to the background picture selection signal and the message selection signal issued by the computer 1, thereby directing the operator both visually and audibly.
Instructions to the operator both visually and audibly as described above will significantly enhance the man-machine communication.
FIGURE 4 shows a modified embodiment-based on FIGURE 1 or 3, wherein there is provided a hard copy unit 12 which produces a hard copy of the display on the screen of the CRT display unit 9 in response to the video signal from the image synthesizing unit 8, whereby the display on the CRT display unit 9 can be preserved in the form of a hard copy.
FIGURE 5 shows a modified embodiment based on FIGURE 1, 3 or 4, wherein a video disk file 7 is arranged to have a storage area, in addition to an area 7a for storing the video signal of a background picture 6, where the operational history of the operation screen which has been supplied from the image synthesizing unit 8 and displayed actually on the CRT display unit 9 is stored in time-series fashion.
In this arrangement, an arbitrary screen can be reproduced on the CRT display unit 9 by the record selection signal issued by the computer 1. In this case, the audible message can also be recorded and reproduced.
Basic embodiments of the present invention have been described, however, the invention is not limited to the above embodiments. For example, in a system where several terminals use an operation screen of the same specification, a single video disk file 7 can be shared by the terminals, thereby allowing the production of only a single video disk file 7.
Otherwise, if it is difficult to share a single video disk file between the terminals,.the video disk file 7 may be copied and distributed to other locations, thereby allowing the original production and modification of a video disk file only once.
Moreover, the background alone may be changed leaving the foreground unchanged, thereby providing a possibility of animation of the background picture.
Furthermore, in FIGURES 1, 3 and 5, it is possible that the background picture selection signal, the message selection signal and the record selection signal may be generated by the image synthesizing terminal apparatus 3 itself, not by the computer 2.
As can be seen from the above detailed description, according to the present invention, a background picture is provided by a video signal stored in the image storage unit instead of an output program of a computer for producing the background picture. Accordingly,--the operation screen including a user-oriented language, graphic patterns, tables, process flowsheets, and the like can be produced and modified promptly and inexpensively by storing the background picture in the image storage unit. In addition, the background picture which has been complicated and difficult to produce can be produced independently of the information processor such as the computer and without the need of any knowledge on the information processor, thereby allowing the standardization of software for producing the operation screen.
According to the present invention, by use of an image storage unit for storing an audio signal as well as the video signal, it becomes possible to inform the operator both visually and audibly.
Moreover, by arrangement of the image storage unit to store the video signal of the operation screen, the operational history can be recorded and reproduced, and further a hard copy can be obtained.

Claims

1. Image information synthesizing apparatus for an information processor which produces an output video signal of a foreground picture in accordance with an output program; said image information synthesizing apparatus comprising an image storage apparatus including means for storing a video signal of at least one background picture which has been produced by a picture signal generating apparatus, and means for outputting a video signal of the background picture specified by a background picture selector signal; an image signal synthesizing apparatus connected to said image storage apparatus and including means for connection to the information processor including means for adding the video signal of the selected background picture outputted from said image storage apparatus to the video signal of a foreground picture from the information processor which varies in accordance with the result of information processing by said information processor, and means for outputting a combined video signal of the combined background and foreground pictures; and a display apparatus for displaying a picture in response to said combined video signal supplied from said image signal synthesizing apparatus.

2. Image information synthesizing apparatus as claimed in claim 1, wherein said image storage apparatus includes means to store an audio signal, so as to allow visual display and audible addressing at the same time.

3. Image information synthesizing apparatus as claimed in claim 1 or 2, including hard copy apparatus connected to receive said combined video signal to produce a hard copy of the picture displayed.

4. Image information synthesizing apparatus as claimed in any of claims 1, 2 and 3, wherein said image storage apparatus is provided with a storage area for storing a video signal from said image signal synthesizing apparatus so that previously displayed information may be reproduced in response to a command signal from said information processor.

5. Image information synthesizing apparatus as claimed in any of claims 1 to 4 in which said background selector signal is produced by said computer.

6. Image information synthesizing apparatus as claimed in any of claims 1 to 4 in which the image information synthesizing apparatus includes means to produce said background picture selector signal.

7. Image information synthesizing apparatus as claimed in any of claims 1 to 6 in which a succession of background pictures signals are passed to the image signals synthesizing apparatus to provide an animated background picture.

8. An image information synthesizing terminal equipment for an information processing system such as a computer which performs display in accordance with an output program and has a program area for selecting an unvaried background picture, said image information synthesizing equipment comprising an image storage apparatus for storing a video signal of said background picture which has been produced corresponding to said output program by a picture signal generating apparatus such as a video camera for transforming a picture into a video signal, and for outputting a video signal of the background picture specified by said selecting program; an image signal synthesizing apparatus for adding a video signal of a background picture outputted from said image storage apparatus to a video signal of a foreground picture which varies in accordance with the result of information processing by said information processing apparatus, and for outputting a resultant video signal; and a . display apparatus for displaying a picture in response to a video signal supplied from said image signal synthesizing apparatus, whereby a background picture is displayed by use of said image storage apparatus instead of a background picture output programme.