Advanced Lane Recognition - Fusing Vision and Radar

Proceedings of the

IEEE Intelligent Vehicles Symposium 2000

Dearborn (MI), USA October 3-5, 2000

Advanced Lane Recognition - Fusing Vision and Radar

Axel Gem, Uwe Franke DaimlerChrysler Research, T728 D-70546 Stuttgart, Germany

{ AxeLGern, Uwe.Franke} @DaimlerChrysler.com

Paul Levi Institute of Parallel and Distributed High-Performance Systems University of Stuttgart, Germany Paul.Levi @informatik.uni-stuttgart.de

Abstract One major problem of the common vision-based lane recognition systems is their susceptibility to weathel: These problems mainly stem from the fact, that they only look for road structures. From the position of other cars in front,

the run of the curve can be estimated. This paper presents our fusion approach, that takes leading vehicles into ac- count which have been detected by radal: The Kalman fil- ter applied here does not only deliver improved measure- ments of the run of the curve, but also a precise estimate of the lateral position of the observed cars. This informa-

tion can be used to improve the lane assignment of ACC

systems.

1 Introduction

In the past, many different vision-based lane recognition systems have been presented. Most of them try to find

road features such as lane markings or road surface textures which are matched against a specific geometrical model of the road (e.g. [l], [2]). Hereby, the parameters of the cho-

sen model and the position of the car in the lane are deter- mined, for example using a least-square fitting or a Kalman filter [2]. Problems occur when driving in adverse weather

conditions such as rain or snow. Often the contrast between the markings and the pavement is poor, sometimes the col- ors of the markings look negated. The range of sight is reduced enormously, causing a bad prediction of the lane parameters, particularly the curvature (see Fig. 1).

DaimlerChrysler introduced the DISTRONIC, a radar- based Adaptive Cruise Control (ACC) in the Mercedes S- Class in 1999. Advantages of a radar sensor is the relative independence of the weather and the range of sight of up to 150m. This paper presents a Kalman filter based approach fus- ing radar information and our vision-based lane recogni- tion system for improved lane recognition, which can be used e.g. for autonomous driving. In addition, the behavior of the ACC system can be improved by a better assignment

0-7803-6363-9/00/$10.00 0 2000 IEEE 45

Figure 1: The lane recognition system under rainy conditions, showing the tracked markings with found measurements, the.pre- dicted centerline of the lane and one tracked radar obstacle in

front.

of obstacles to specific lanes. The two following sections describe the optical lane recognition system and the Cruise Control separately. Section 4 sketches the fusion of both

systems in one joint Kalman filter. The approach for im- proved obstacle tracking in monocular images is described in Section 5. Section 6 shows results, obtained from simu-

lations and test drives.

2 Vision-Based Lane Recognition

According to the recommendations for highway construc- tion, highways are built under the constraint of slowly changing curvatures. Therefore, most lane recognition sys- tems are based on a clothoidal lane model, that is given by the following equation:

c(L) describes the curvature at the length L of the clothoid,

CO is the initial curvature and c1 the curvature-rate, which is called the clothoidal parameter. The curvature is defined

as c =

i, where R denotes the radius of the curve. Besides these curvature parameters, lateral position xuff and yaw angle AV relative to the lane axis are of interest for autonomous driving (see figure 2).

Assuming the pinhole-camera model and knowing the camera parameters focal length f, tilt angle a and height- over-ground H, the relation between a point on a marking and its image point Pi(xi,yi) can be described by the fol- lowing equations:

Xi CO c1 l(a. w -xUff - AV. L+ - . L2 + - .L3) (2)

L 2 6

H L= a + (Yi/f 1

(3)

w is the lane width and a = f0.5 is used for the left or the right marking. Hence, every measurement is projected onto a virtual measurement directly on the centerline of the lane. In all equations, the trigonometrical functions are ap- proximated by the argument (sinx = x, tanx = x), because we consider only small angles. These equations allow to determine the relevant run of the curve and vehicle posi- tion parameters. Driving at higher speeds, dynamic and kinematic restric- tions have to be taken into account. These constraints can be expressed by the following differential equations: f,ff = v.Av+v, (4) AV = qveh-CO'V (5 1 do = C1.V (6) i-1 = 0 (7) In these equations, v denotes the longitudinal speed of the vehicle, v, the lateral speed caused by a possible side slip angle and Vveh the yaw rate. v, and qv& are measured by inertial sensors. Based on the dynamic and kinematic model (Eqn. (4) through Eqn. (7)) the road markings are tracked from frame to frame by using Kalman filter techniques as first proposed by [2]. The geometrical equation (2) is used as the measurement equation updating the filter. The search areas are centered at the predicted position in the image. The size of the regions is determined by calculating the 30- area of the expected measurement, assuming a Gaussian noise process. Fig. 1 illustrates this common procedure. The above described system is independent of the image source, using a monocular or a stereo camera system. Our first approaches as e.g. described in [3] uses a monocular camera and relies on the assumption, that the road is flat. Sometimes problems occur because 'markings' are falsely found on cars cutting in or crash barriers. This causes a wrong state estimation. These problems can be solved using stereo information. Every point on the markings found in one image is cor- related against a small region in the second image. This delivers three-dimensional information allowing a vertical modeling. In our approach, we divide the vertical modeling into two components: 1. A linear part, described by the tilt angle a. 2. A non-linear part, described by the vertical curvature, approximated by a clothoid. German Highways are designed according to a parabolic vertical curvature cy. The vertical and horizontal curvature models are separated. The parabola curvature is approxi- mated using a clothoid as described in [4]: Tilt angle and vertical curvature are estimated in one joint Kalman filter using the following measurement equation for the height over ground y: