CA2090660A1

CA2090660A1 - Vehicle lane position detection system

Info

Publication number: CA2090660A1
Application number: CA002090660A
Authority: CA
Inventors: Bimal P. Mathur; H. Taichi Wang
Original assignee: Rockwell International Corp
Current assignee: Iteris Inc
Priority date: 1992-08-12
Filing date: 1993-03-01
Publication date: 1994-02-13
Also published as: US5351044A; JPH06109433A; DE69302975T2; EP0586857B1; DE69302975D1; EP0586857A1

Abstract

ABSTRACT OF THE DISCLOSURE

A low cost, real time vehicle lane position detection system is provided for determining and maintaining the position of the vehicle on a highway. The system comprises an image sensor mounted on the front of an automotive vehicle and an integrated processor for performing real-time lane mark detection. The sensor/processor system identifies highway lane marks on the detector image plane by using a nonlinear resistive network for detecting A line detection algorithm, such as the Hough transform, is used to determine the lane marks from the outliers on the image plane. Because the expected lane position can be determined in advance, an added degree of signal-to-noise discrimination is achieved by providing feedback to the processor for outlier detection. The position of the vehicle in the lane is determined and tracked from the position of the detected lane marks on the image plane given the sensor position and optical geometry.

Description

~o9~o VEHICLE LANE POSITION OETECTION SYSTEM

TECHNICAL FIELD
The present inven~on relates to moving vehicle senso~ systems and, in particular, to a low cost, real time high7vay lane position detection system fo~ automodve veh*les. -BACKGROUND OF THE INVENTION
S Reducing conges~on on the highways has been a goal for many years. One possible solution is to ma~e existing highways more efficient through automation. To be safe and ef~ective, however, automated highways require means f~ positioning vehicles within lanes as well as maintaining optimum distance bet~veen vehicles. There~ore, fully automated highway systems require sensor aDd data processing systems ~o detect and control the positions moving vehicles.
Positioning vehicles on an au~omated highway, such as the proposed ~telligent Yehicie Highway System (IVHS), is complicated by the elutter of unwanted in~ormadon from the environment that is con~nually received by the sensor system. P~visions must be made fo~
system calibration, changing weather, vehicles entering and exiting ehe highway, and numerous other obstacles dlat might be encountered. Various systems have been proposed for automated highways, including those employing artive sensors such as mm wave radar, laser radar, or sonar, and passive systems such as stereo vision for measuring distance between vehicles. The hlown systems, however, have high cost fac~s and/or technical problems th2t have not been overcome. For e~ample, a wide field of view is needed for lane detection, and a highly ~esolved image with many pixels cu~rently cannot be pr~cessed in real time. GiYen the fo~going constrain2s and the desire to develop automat~d highways, there is a need for a safe, effective, low cost, real ~me system for sensing and controlling the position of automotive vehicles in lanes of present highways and automa~ed highways of the future.

" '~ ' ' ,,,"' ' . ' "", ~' ' ' ~ , '' ' ' ' ' ' ~ "'' ' ';' ~ ' `'';"
`'' . ' ' ' ; ' ' ' . ' : , '.
. ' ' . '. ' .

2~066~) SUMMARY OF THE INVENTION
The present invention comprises a vehicle lane position detection system for use on present roads as well as automated highways of the future. The lane position detec~ion system comprises an image sens~r mounted on the front of an automotive vebicle and a computer 5 processor for per~orming real-time lane mark de,~ection, tracking, and warning. The sensor/processor system detects the location of highway lane marks on ~he detector imaging pl~ne by using a nonlinear resistive net~vork to detect pixels in the image that have a hi~her output (i.e., outliers) compared to sulrounding pixels. Line detection algorithms7 such as the Hou~h transf(trm, are used to detennine the lane position ~om the outliers on the image plane.
10 Berause the desired IMe posidoll can be estimated in advance, an added degree of signal-to-noise discrimina~on may be achieved by providing feedback to the processor. l'he posidon of the vehicle in the lane is then determined ~om the position of the detected lane marks on the image plane.
A plincipal object of the invention is to control and maintain the position of an 15 automo~ve vehicle in a lane of a highway. A featuIe of the invention is an image senso~ and processor system mounted on a vehicle for det~ng and detelmining ~he position of highway lane marks. An advantage of the invendon is a low cost, real time sensor system that deteImines the position of 2 moving vehicle within a lane of a highway. IJse of the invention may be extended to controlling the position of a vehicle within a lane of an automated highway.

20 BRIEF DESCRIPTION OF T~3[E DRAWINGS
For a more complete understanding of ~he pTesent inv~ntion and for filr~er advantages thereof, the following Detailed Desc~iption of the P~efe~d lEmbodiment makes reference to the accompanying Drawings, in which:
FIGURli 1 is a schematic diagram of automobiles using a lane position detecsion 25 sys~em of the present invendon on a highway;
FIGURE 2 is a schelmatic diagr~m of a nonlinear resistive networlc utilized by the system of dle present invention to detect disc~ntinui~es and pixels in the sensor image that have a higher output ti.e., outliers) compared to su~Dunding pixels; and . : . .

~9~ o FIGURE 3 is a block diagram illustrating major functions and flow of information in the lane position detecdon system of the presene inven~on.

I)ETAILED DESCRIPTION OF l'HE PREFERREI3 EMBODIl~qENT
The present invention comprises a vehicle lane position detection system that ca~ be 5 used on present highways and is designed to be part of a comprehensive automated highway system. A significallt problem to be overcome in this aIea is the very wide fleld of vieYv needed for sensing the position of a vehicle in a lane. With a wide field of view, highly resolved images ha~ing many pixels are undesirable because they cannot be processed in ~eal ~me given the cu~ent st~e of technology. Therefore, a vast amount of unwanted in~ormation and noise~
10 resulting from changes in the weather and variations in lighting conditions, for example, must be sPpara~ed from the cri~cal lane position informa~on.
The system of the prvsent inYention comprises an imaging system, such as a came~a having an imagiilg alTay for lane mark detection, and a microprocesso~. The imagng ~Tay typically comprises aptics and an integrated chip that are moun~ed on the front of an automo~ve 1~ vehicle for lane mark detec~lon. As illus~rated in Figu~e 1, an integrated detector 12 can be mounted cen~ally on ~e front (e.g., on the hood~ o~ a vehicle I 1. Detect~ 12 is designed with a field of view large enough to detect lane marks on both sides of vehicle 11. With only one detector, a microp~ocessor can be integrated with detector 12. In an alterna~ve embodiment, integrated detectors 13 and I5 can be mounted on either one or both sides of vehicle 14. With 20 one detector (such as detector 15) mounted on ~he side of vehicle 14, the field of view includes the lane marks on only one side of the highway. With dual detectors, both de~ectors 13 and 15 can be sonnected to and served by a single mic~oprosessor 16.
De~ector 12 includes imaging optics, an integrated imaging ar~ay for lane mark detection, and circuit~y for biasing and clock genera~on. Detecta¢ 12 provides output ~ both a 25 smoothed image and a c~esponding outlier map. The output may compnse analog o~ di~tal signals. For an analog output, dle outlier map may ~e digitized using a commercially availaUe image acquisition board. The digitized image may then be ~ansfeIred ~m a frame buffer to random access memory (RAM) associated with the microprocessor, where computations such as lane finding and decision making take place. In a uni~ied system, all detec~or and 30 micloprocessor cireuitly is integrated on a single boarcl inside the ima~ng came~a.

- , , : .
: : ~ ., ~ .

2~9~6~0 Referring to Figure 2, a nonlinear resistive network 20 used for outlier detection includes an image plane comprising a gr~d of resistive elements (illustrated as resistor 21 connected in series with switch 23), a transconductance amplifier 24 (which includes resistive element Rd), a switch 25, and an difference comparator 26 connected between each node i and 5 itS associated detector input, such as sensor element 22. Sensor element 22 comprises one of a p~ ity of sensor inputs, as from an imaging a~ray, for example. Network 20 is the subject of co-pending IJ.S. Pat. Appl9n Ser. No. 9M,76$ ~llecl 06/26/~2, and is further described by J.G. Harris, 3.C. Liu, and B. Mathur, in "Discarding Outliers Using a Nonlinear Resistive Network," Internadonal Conference on Neural Networks (IIEEE), Vol. I, pp. 501-06, July 8, 10 1991, the teachings of which a~e hereby incorporated by reference.
In operation, network 2û breaks one of the image plane resistive elements (i.e., opens switch 23) wherever a discontinuity occurs and breaks one of the data path resisdve elements (i.e., opens switch 25) wherever an outlier occurs. ~ach image plane resistive element may comprise a resisdve fuse or a sa~urating nonlinear resistor, for e~ample. As illus~ated in Figwe 15 2, the nonlinear resistive element in the data path con~prises ~ansconductance amplif~er 24 and switch 25. Connected in series, transconductance amplifier 24 and switch 25 have a nonlinear, sigmoid~ e I-V charactenstic that is bounded by dle opeIation of swi~ch 25.
Switch 25 of network 20 is controlled by ~e difference comparator 26. Ini~ally, all switches are closed and the network smoothes the input data values from all the sensor 20 elements. Comparator 26 then computes the difference between the input data value di and the smoothed data value at nocle i. If the difference is greater than a threshold value (i.e., greater than Vth), then the data value at node i is an outlier and switch 25 is opened. As a result, the image data at node i is smoothed without input ~om sensor element ~2. Highway lane marks are generally brighter than the road su~face and appear in ~he image as outliers (i.e., points 25 different from their immediate su~oundings). The position of the outliers, which is important in the detection and identification of lane marks, is indicated by the position of the open switches, such as switch 25, in netwo~k 20.
In the present invention~ the highway lane marks are detected as an outlier image by detector 12. Af~r a f~ame of the outlier image is transfe~red to mie~roprocessor RAM, the most 30 likely parameters are computed for the line ~hat goes through the detected lMe marks. Based on the known camera position and optical geometry, actual lane boundaries on the highway are computed from the lane mark parameters on the image plane. This measurement process, however, is inherently noisy. A Kalman filter may be used to smooth and track the distance . .

. , , ~, - :
, . , ~
.

~09~6~

and orientation of the vehicle with respece tO the ac~ lane boundaries. This data may be used to predict whetller or not the vehiele is deviating firom the desired lane position.
A well-known transform algorithm developed by Hough in 196~ can be used for finding the lane mark lines from the outlier images. 'The predicted intercept and angle of the 5 Kalman fil~er and the previous prediction errors can be used to limit the search region in both the image area and the line pararneter space in the cu~ent frame. The Hough transfo~m can also provide a count of the pixels on which the lane marks (i.e., the outliers) have ~llen. 13ased on the camera and highway geometry, an approximation of she number of pixels expected to be oudiers is known. This ~nfolmadon can be used to provide fieedback signals for adjusting the 10 final threshold voltage foq ou~lier detec~on.
Figure 3 illus~ates the basic functions of the p~esent inven~n in block diagram form.
Th5 sensor system, which may include opsics and a detector array 11 mounted on vehicle 12 as described above, generates an image of she highway ahead of the vehicle. Nonlinear resistive network 2û detects outliers tha~ ale analy~ed for the presence of highway lane ma~ks. The 15 microprocessor computes the position o~ the vehicle in ~he lane based on the detected lane marks and the known geome~y and position of the sensor system. The known sensor geometry and expected lane malic positions are used to provide feedback signals to adjust the threshold voltage net vork 20 for improved outlier detection and identificadon of lane mark.
Analysis of subsequent image frames produces a serles of data on lane position that is used f~r 20 tracking the position of the moving vehicle in the lane. Finally, the lane posi~on tracking data may be p~ovided to a warning and control system to alert the d~iver of the vehicle and/or p~ovide automatic steering co~ections to maintain the posi~on of dle vehicle within the lane.
Al~ough the plesent invention has been descnbed with respect to specific embodiments thereof, various changes and modifications can be car~ied out by those sldlled in the ar~ without 25 depar~ng fr~m the scope of ~he invention. Therefore, it is intended that the present invention encompass such changes and rnodificadons as fall wi~hin the sc~pe of the appended cl~s.

- ,. .. .
. ~ ,,, . ... ~ ,.... ~ .

Claims

1. A system for detecting the position of an automotive vehicle (14) with respect to lane marks on a highway, comprising:
an imaging sensor (13, 15) mounted on the vehicle (14) for generating an image of the lane ahead of the vehicle (14);
a nonlinear resistive network (20) connected to said imaging sensor (13, 15) for detecting lane marks in said image; and a microprocessor (16) connected to said resistive network (20) for determining position of the lane in said image and computing position of the vehicle (14) in the lane in real time.

2. The system of Claim 1, wherein said imaging sensor (13, 15) comprises a camera having a sensor array.

3. The system of Claim 1, wherein said nonlinear resistive network (20) comprises switches (23, 25) that are opened for discontinuities and outliers.

4. The system of Claim 1, wherein said microprocessor (16) performs a Hough transform to determine position of the lane in said image.

5. The system of Claim 1, wherein said microprocessor (16) tracks the position of the vehicle in the lane.

6. The system of Claim 5, further comprising a warning and control system connected to said microprocessor (16) for monitoring and controlling position of the vehicle (14) in the lane.

7. A method of determining position of an automotive vehicle (14) in a marked lane of a highway, comprising the steps of:
mounting a sensor system (13, 15, 16) on the vehicle;
generating a sensor image of the lane ahead of the vehicle;
detecting discontinuities and outliers in said sensor image of the lane;
performing a transform to determine lane marks from said discontinuities and outliers in said sensor image;
determining position of said lane marks in said sensor image; and computing position of the vehicle (14) in the lane in real time.

8. The method of Claim 7, wherein the step of performing a transform comprises the step of performing a Hough transform

9. The method of Claim 8, wherein the step of determining position of said lane marks in said image further comprises the step of providing feedback for the step of detecting discontinuities and outliers in said sensor image.

10. The method of Claim 9, further comprising the step of tracking position of the vehicle (14) in the lane.

11. The method of Claim 10, further comprising the steps of monitoring and controlling position of the vehicle (14) in the lane.