EP1076221A2

EP1076221A2 - A robot with gauging system for determining three-dimensional measurement data

Info

Publication number: EP1076221A2
Application number: EP00117126A
Authority: EP
Inventors: William D. Long; Charles C. Kingston
Original assignee: Perceptron Inc
Current assignee: Perceptron Inc
Priority date: 1999-08-12
Filing date: 2000-08-10
Publication date: 2001-02-14
Anticipated expiration: 2020-08-10
Also published as: CA2315508A1; ES2340462T3; DE60044170D1; CA2315508C; US6166811A; EP1076221A3; EP1076221B1

Abstract

A robot-based gauging system (10) is provided for determining three-dimensional measurement data of the surface of an object (12). The robot-based gauging system includes a robot (24) having at least one movable member (22) and a robot controller for controlling the movement of the movable member (22) in relation to the surface of an object (12). A non-contact sensor (20) is coupled to the movable member (22) of the robot (24) for capturing image data representative of the surface of the object (12). A position reporting device reports position data representative of the position of the non-contact sensor. A synch signal generator for generating and transmitting a synch signal is electrically connected to each of the non-contact sensor and the position reporting device, such that the non-contact sensor reports image data in response to the synch signal and the position reporting device reports position data in response to the synch signal.

Description

Background and Summary of the Invention

The present invention relates generally to non-contact gauging systems and, more particularly, a robot-based gauging system and method for determining three-dimensional measurement data of an object.
Demand for higher quality has pressed manufacturers of mass produced articles, such as automotive vehicles, to employ automated manufacturing techniques that were unheard of when assembly line manufacturing was first conceived. Today, robotic equipment is used to assemble, weld, finish, gauge and test manufactured articles with a much higher degree of quality and precision than has been heretofore possible. Computer-aided manufacturing techniques allow designers to graphically conceptualize and design a new product on a computer workstation and the automated manufacturing process ensures that the design is faithfully carried out precisely according to specification. Machine vision is a key part of today's manufacturing environment. Machine vision systems are used with robotics and computer-aided design systems to ensure quality is achieved at the lowest practical cost.
In a typical manufacturing environment, there may be a plurality of different non-contact sensors, such as optical sensors, positioned at various predetermined locations within the workpiece manufacturing, gauging or testing station. In order to obtain three-dimensional measurement data using a two-dimensional non-contact sensor, the sensor may be moved in relation to the workpiece. Therefore, it is important to know the exact location and orientation of the moving item (either the sensor or the workpiece) each time the sensor acquires image data of the workpiece. This tends to be fairly straightforward for accurate motion devices (e.g., a CMM), since the reported position reflects the actual position of the device. The motion controller of an accurate motion device may include various compensation factors that get applied to the motion control to ensure the commanded position matches the actual position of the device. However, the high cost associated with these types of motion devices is prohibitive to them being used in a typical manufacturing environment.
As an alternative to these more expensive motion devices, it is possible to use a conventional industrial robot as the motion device in the gauging system. By affixing the non-contact sensor to the industrial robot, the robot can be used to move the sensor in relation to the workpiece. Industrial robots are well suited to perform complex motion sequences which are customized for the workpiece. In addition, many standard industrial tools are available which allow the programming of the motion sequences to be performed off-line and subsequently downloaded to the robot controller.
The problem with a robot-based gauging system is that the conventional industrial robot is not an inherently accurate motion device. Robot controllers generally utilize ideal kinematic models when instructing robot motion. The physical characteristics of the robot arm vary slightly from the ideal model, and thus the actual movement of the robot arm varies slightly from the commanded motion sequence. Furthermore, these physical characteristics are likely to change as the robot is subject to wear and thermal changes. Most robot controllers are not equipped with any additional means for translating and/or correcting the reported position into an actual position of the robot arm. As a result, conventional industrial robots have not heretofore been used in non-contact gauging systems and other highly accurate manufacturing workstation applications.
To this end, the robot-based gauging system of the present invention accurately determines three-dimensional measurement data for a workpiece through the use of a synchronized scanning process without the need for an expensive motion device, such as a CMM.
The gauging system includes a robot having at least one movable member and a robot controller for controlling the movement of the movable member in relation to the surface of an object. A non-contact sensor is coupled to the movable member of the robot for capturing image data representative of the surface of the object. A position reporting device is used to report position data representative of the position of the non-contact sensor. In addition, a synch signal generator is used to generate and transmit a synch signal to each of the non-contact sensor and the position reporting device, wherein the non-contact sensor reports image data in response to the synch signal and the position reporting device reports position data in response to the synch signal.
The gauging system further includes a vision system adapted to retrieve image data from the non-contact sensor and position data from the position reporting device. As the non-contact sensor scans the surface of the object, there is a time latency between when the sensor strobes its laser to when processed image data is available to the vision system. In other words, the image data reported by the sensor lags in time behind the position data reported by the position reporting device. Therefore, the vision system must be able to synchronize the image data with the position data. A memory storage space is used to store position data so that the vision system can synchronize the image data with the position data. In this way, the robot-based gauging system of the present invention is able to quickly and accurately determine three-dimensional measurement data for the object.
For a more complete understanding of the invention, its objects and advantages, reference may be had to the following specification and to the accompanying drawings.

Brief Description of the Drawings

Figure 1 is a perspective view of a robot-based gauging station incorporated into an automotive door assembly line in accordance with the present invention;
Figure 2 illustrates the primary components of a robot-based gauging system in accordance with the present invention;
Figure 3 is a block diagram showing a first preferred embodiment of the robot-based gauging system of the present invention;
Figure 4 illustrates the memory configuration used to synchronize the image data with the position data in the robot-based gauging system of the present invention; and
Figure 5 is a block diagram showing a second preferred embodiment of a robot-based gauging system in accordance with the present invention.

Description of the Preferred Embodiment

An exemplary robot-based gauging system of the type commonly employed in vehicle assembly lines or other automotive applications is shown in Figure 1. A robot-based gauging system 10 is often used for ensuring that each vehicle body component is assembled within predefined tolerances. For example, the robot-based gauging system 10 may measure the door of a vehicle body 12. Although the invention is not limited to automotive applications, an exemplary use for the robot-based gauging system 10 would be in an automotive assembly plant.
A single non-contact sensor 20 is mounted on a robot arm 22 of a robot 24. The sensor 20 in the present embodiment is a contour sensor which is only capable of two dimensional (2D) measurements. During operation, the sensor 20 emits a planar structured light pattern. By illuminating the workpiece, a characteristic reflected light pattern is achieved which can be detected by the sensor 20. The contour sensor can measure (through triangulation) in two dimensions a plane on the reflecting workpiece. For further details regarding a suitable structured light sensor, reference may be had to U.S. Patent No. 4,645,348 to Dewar et al., assigned to the assignee of the present invention.
Communication cables 28 connect the sensor 20 and the controller of the robot 24 to a machine vision computer 30 which includes a CRT (cathode ray tube) display 32. A printer 34 is optionally provided with a typical machine vision computer.
In order to generate a three-dimensional representation of the workpiece, the robot arm 22 is movable to a plurality of positions for measuring the periphery of a workpiece (e.g., a door). In contrast to the static approach that requires the sensor to stop and settle for each image acquisition, the sensor 20 can continuously acquire data as the robot arm 22 traverses the surface of a workpiece. In other words, three-dimensional measurement data for the workpiece may be acquired by "scanning" or continuously moving the two-dimensional measurement sensor in relation to the workpiece. In addition, the gauging system 10 is also able to report the three-dimensional measurement data with respect to a predetermined reference frame or coordinate system associated, for example, with the vehicle body 12 to be measured, or with respect to an external reference frame associated with the gauging station.
Figure 2 illustrates the basic components associated with the robot-based gauging system 10. A non-contact sensor 20 is affixed to the end of a robot arm 22 which in turn is connected to a robot controller 38. In operation, the robot controller 38 is operative to control the movement of the robot arm 22 in relation to the surface of an object to be gauged by the system, and the non-contact sensor 20 is operative to capture image data representative of a portion of the surface of the object. A position reporting device 24 is used to report position data representative of the position of the non-contact sensor 20. The non-contact sensor 20 and the position reporting device 24 are each in turn electrically connected to a vision system 40 residing on the machine vision computer 30.
As will be further described below, the position of the sensor 20 can be reported either directly or indirectly by the position reporting device 26. In the direct approach, the actual position of the sensor 20 is reported (e.g., by a photogrammetry system) to the vision system 40, whereas in the indirect approach, the position of the sensor 20 is deduced from the position of the robot arm 22 which is reported (e.g., by the robot controller) to the vision system 40. Lastly, the vision system 40 synchronizes the image data with the position data, thereby determining three-dimensional measurement data for the workpiece.
More specifically, the vision system 40 includes a synchronization module 42 and a timing signal generator 44. To synchronize the image data received from the sensor 20 with the position data received from the position reporting device 24, the vision system 40 uses a timing signal or a hardware latch signal which is generated by the timing signal generator 44. The timing signal is then simultaneously transmitted to the sensor 20 and the position reporting device 24. In response to the timing signal, the sensor 20 records image data for the workpiece and the position reporting device 24 records current position data for the non-contact sensor 20. Image data and position data can then be requested by and delivered to the vision system 40. Thus, the vision system 40 acquires the measurement data needed to construct a contour line representative of the surface of the workpiece. It should be noted that the vision system does not rely on the sensor motion being at a constant velocity in relation to the workpiece. On the contrary, it is envisioned that the velocity of the sensor may vary as it scans the workpiece.
A first preferred embodiment of a gauging system which utilizes the indirect approach to acquire position data is shown in Figure 3. The indirect approach deduces the position of the sensor from the position of the robot arm 22. To do so, the position data is reported by the robot controller 38 to the vision system 40. However, as previously described, the actual position of the robot arm varies from the commanded position data as reported by the robot controller 38. Thus, the vision system 40 further incorporates a kinematic correction module 50 which applies a real time kinematic correction factor to the reported position data received from the robot controller 38. In a preferred embodiment, the DynaCal Robot Cell Calibration System developed by Dynalog, Inc. of Bloomfield Hills, Michigan may be adapted to serve as the kinematic correction module 50.
One skilled in the art will readily recognize that the kinematic correction module 50 incorporates a procedure that determines physical deviations between the actual robot and its corresponding ideal model. These deviations are stored as robot-specific parameters. In operation, the kinematic correction module 50 receives the reported position data from the robot controller 38. Since the robot controller 38 is unaware of the robot's physical deviations, the reported position data is based on the ideal model. The kinematic correction module 50 translates the reported position data into actual position data by using the robot-specific parameters. Next, the actual position data of the robot arm is transformed to position data for the sensor. The sensor position data is then provided to the synchronization module 42.
The non-contact sensor 20 is preferably a Tricam non-contact sensor which is manufactured by Perceptron, Inc. of Plymouth, Michigan. While the following description is provided with reference to the Tricam non-contact sensor, it is readily understood that the explanation is applicable to other non-contact sensors. In the case of the Tricam sensor, the timing signal generator 44 is a software module that resides on a circuit board which can easily be incorporated into the machine vision computer 30. Moreover, the timing signal generator 44 is associated with the sensor 20, such that the hardware latch signal is generated internal to the sensor 20. In operation, the sensor generates a hardware latch signal each time it acquires image data. The sensor 20 is further configured to simultaneously transmit the hardware latch signal to the robot controller 38. It is also envisioned that the Tricam non-contact sensor may support an external input that allows for image acquisition to be initiated by an hardware latch signal which is generated external to the sensor 20. Thus, the timing signal generator 44 may be associated with some external computing device (e.g., the robot controller).
In order to "scan" the sensor in relation to the workpiece, image data must be captured at a rate which allows for the continuous motion of the sensor. In a continuous operation mode, the Tricam non-contact sensor is limited by the frame rate of its camera which operates at a frequency of 30 Hz. However, there is a time latency between when the sensor strobes its laser to when processed image data is available to the vision system 40. As a result, the image data reported by the sensor 20 lags in time behind the position data reported by the robot controller 38 by two frame times (e.g., 2/30 second). Accordingly, the image data received from the sensor 20 requires synchronization with the position data received from the robot controller 38.
The present invention synchronizes the image data with the position data as shown in Figure 4. A memory space internal to the vision system is used to store and process position data received from the robot controller 38. The memory space is further defined into an image memory space for storing image data and at least three positional memory spaces for storing position data. A first timing signal causes the vision system to retrieve position data(t₀) from the robot controller which is in turn stored in a first memory space 62. In response to a second timing signal, the vision system copies the position data(t₀) in the first memory space 62 to a second memory space 64 and then retrieves the next available position data(t₁) into the first memory space 62. Similarly, a third timing signal causes the position data(t₀) in the second memory space 64 to be copied to a third memory space 66, the position data(t₁) in the first memory space 62 to be copied to the second memory space 64, and new position data(t₂) to be retrieved into the first memory space 62. In other words, the vision system 40 is able to store the three most recent sets of position data received from the robot controller 38
At this point, image data(t₀) corresponding to the initial position data(t₀) is available from the sensor to the vision system. In response to the third timing signal, this image data is read into the image memory space 68 of the vision system. By linking this image data with the position data stored in the third memory space 66, the vision system is able to accurately construct measurement data for the workpiece. For each additional timing signal, this synchronization process is repeated by the vision system.
As previously discussed, the operation of Tricam non-contact sensor vanes slightly from the description provided in relation to Figure 4. In the case of the Tricam sensor, a timing signal tells the vision system to retrieve the current image data from the sensor and to retrieve the current position data from the robot controller. The first instance of this timing signal does not occur until after the first set of image data has been processed and is available to the vision system. Since there are two frame times between the laser strobe and this first timing signal, the available position data no longer matches the available image data. In other words, the position data which is retreived in response to the first timing signal corresponds to the image data retrieved in response to the third timing signal. As a result, the first two sets of image data are discarded, and synchronization begins with the third set of image data.
Rather than determining the position of the sensor from the information provided by the robot controller, it is also envisioned that the position of the sensor is directly reported to the vision system 40 by an independent measurement system as shown in Figure 5. While the following description is provided with reference to a photogrammetry system 80, it is readily understood that other measurement devices may be suitably used in conjunction with the present invention.
In this alternative embodiment of the present invention, one or more photogrammetric cameras are positioned at convenient vantage points in the gauging station. Each photogrammetric camera operates under conventional photogrammetry techniques. That is, the photogrammetric camera emits a burst of quasi-monochromatic light and its camera detects the reflection of this light. For example, retroreflective markers may be placed on the surface of the non-contact sensor 20. When the markers are illuminated by infrared light from a camera, light is reflected by the markers, as bright light spots, back to a motion capture unit within the camera. The light spots are analyzed, such that their position and sizes are calculated by the camera, thereby determining the position of the markers. It is envisioned that the photogrammtric camera may operate using other conventional operational techniques, including but not limited to tracking of luminous targets.
In any event, as long as the sensor is within the field of view of the photogrammetric cameras, the photogrammetry system will provide an accurate determination of the position of the sensor. In this way, the actual position of the sensor is reported to the vision system 40. It should be noted that since the photogrammetry system 80 reports the actual position of the sensor, there is no need for the kinematic correction module in this embodiment. This alternative embodiment of the robot-based gauging system 10 can otherwise operate as previously discussed in relation to Figures 1-4.
From the foregoing, it will be appreciated that the present invention provides a significant advance in the art of robot-based gauging systems. The invention accurately determines three-dimensional measurement data for a workpiece through the use of a synchronized scanning process and without the need for expensive sensor positioning devices, such as CMMs.
While the invention has been described in its presently preferred form, it will be understood that the invention is capable of modification without departing from the spirit of the invention as set forth in the appended claims.

Claims

An apparatus for determining three-dimensional measurement data for the surface of an object (12), comprising:

a movable member (22);

a controller (38) for causing said movable member (22) to move in relation to the surface of the object (12);

a non-contact sensor (20) disposed on said movable member (22) for collecting image data representative of a portion of the surface of the object (12);

a position reporting device (26; 80) for reporting position data representative of the position of said non-contact sensor (20);
characterized by:

a timing signal generator (44) for periodically generating and transmitting a synch signal, said position reporting device (26; 80) electrically connected to said timing signal generator (44) and reporting position data in response to each of said synch signals, and said non-contact sensor (20) being electrically connected to said timing signal generator (44) and collecting image data in response to each of said synch signals; and

a data processor (40) adapted to retrieve image data from said non-contact sensor (20) and position data from said position reporting device (26; 80), and being operable to synchronize the image data with the position data, thereby generating a three-dimensional representation of the object (12).
The apparatus of claim 1, characterized in that said movable member (22) scans the surface of the object (12), such that the image data reported by said non-contact sensor (20) lags in time behind the position data reported by said position reporting device (26; 80).
The apparatus of claim 1 or 2, characterized by a memory storage space (62, 64, 66) for storing position data from said position reporting device (26; 80), said data processor (40) connected to said memory storage space (62, 64, 66) for synchronizing the image data with the corresponding position data.
The apparatus of any of claim 1 to 3, characterized in that said non-contact sensor (20) is a two-dimensional structural light sensor (20).
The apparatus of any of claims 1 to 4, characterized in that said non-contact sensor (20) emits structured light in a predefined planar configuration and having an optical receiver for receiving reflected light emitted by said non-contact sensor (20), where the reflected light being indicative of the surface of the object (12).
The apparatus of any of claims 1 to 5, characterized by a kinematic correction module (50) for applying a real-time correction to the position data received from the position reporting device (26) based on a kinematic model of the movable member (22).
The apparatus of any of claims 1 to 5, characterized in that said position reporting device (80) is further defined as photogrammetric measurement system (80).
A robot-based gauging system for determining three-dimensional measurement data of an object with respect to a reference frame, said gauging system comprising an apparatus of any of claims 1 to 7, characterized in that

said non-contact sensor (20) is mounted to a movable member (22) of said robot for movement of the sensor (20) relative to the object (12),

said controller is configured as a robot controller (38) for controlling the movement of said movable member (22);

said timing signal generator (44) is electrically connected to said robot controller (38) for periodically generating and transmitting a synch signal, said robot controller (38) reporting a set of said position data in response to each of said synch signals, thereby generating a plurality of position data sets;

said non-contact sensor (20) is responsive to each of said synch signals for capturing a set of image data representative of a portion of said object (12), thereby generating a corresponding plurality of image data sets representative of a plurality of portions of the object (12); and

said data processor is configured as a vision system (40) adapted to retrieve said plurality of image data sets from said non-contact sensor (20) and to synchronize each image data set with a corresponding position data set, thereby generating a three-dimensional representation of the object (12).
The gauging system of claim 8, characterized by a coordinate transformation system for transforming each position data set to a sensor reference frame, where the sensor reference frame is associated with said non-contact sensor (20).
The gauging system of claim 8 or 9, characterized in that said robot controller (38) is configured for reporting position data representative of the position of said movable member (22).
The gauging system of claim 8 or 9, characterized in that a photogrammetric measurement system (80) is provided for reporting position data representative of the position of said non-contact sensor (20) by having a calibration field of observation and being positionable at vantage point such that said non-contact sensor (20) is within the calibration field, said photogrammetric measurement system (80) electrically connects to said timing signal generator (44) and reporting a set of position data in response to each of said synch signals, thereby generating a plurality of position data sets.