Pixel - Level Fusion of Image Sequences
using Wavelet Frames1
Oliver RockingerDaimler Benz AG, Systems Technology ResearchIntelligent Systems GroupAlt Moabit 96 A10559 BerlinGermany
rockinger@DBresearch-berlin.de
Abstract
In this paper we propose a novel approach to the pixel level fusion of spatially
registered image sequences. This fusion method incorporates a shift invariant
extension of the discrete Wavelet Transform, based on the concept of Wavelet
Frames which yields an overcomplete signal representation. The advantage of the
proposed fusion method is the improved temporal stability and consistency of the
fused image sequence. We show examples of the application of the Wavelet
Frame fusion scheme on both real world images and image sequences.
I Introduction
By the development of new imaging sensors arises the need of a meaningful combination of all
employed imaging sensors. This problem is addressed by image fusion. The actual fusion process
can take place at different levels of information representation, a generic categorization is to
consider the different levels as signal, pixel, feature and symbolic level [3].
In the following, we discuss the pixel level fusion process. To perform pixel level fusion
successfully, all input images must be exactly spatially registered, i.e. the pixel positions of all
input frames must correspond to the same location in real world. To date, the result of pixel level
image fusion is considered primarily to be presented to the human observer, especially in image
sequence fusion. A possible application is the fusion of infrared and visible images obtained by an
airborne sensor platform to aid a pilot navigate in poor weather conditions or darkness.
In case of pixel level fusion, some generic requirements can be imposed on the fusion result: The
fusion process should preserve all relevant information of the input imagery in the composite
image, while suppressing irrelevant image parts and noise in the fusion result. The fusion scheme
should not introduce any artifacts or inconsistencies which would distract the human observer or
following processing stages. In image sequence fusion arise the additional problems of temporal
stability and consistency of the fused image sequence.
The paper is organized as follows: In section II we briefly summarize the Discrete Wavelet
Transform and the extended concept of Discrete Wavelet Frames. In Section III the Wavelet Frame
1 In: Proceedings of the 16th Leeds Applied Shape Research workshop, Leeds University Press, 1996fusion scheme is introduced. Results of the proposed fusion scheme on real world imagery are
presented in section IV.
II Discrete Wavelet Transform and Discrete Wavelet Frames
A. The Discrete Wavelet Transform
The Wavelet Transform as initially described for square integrable functions of a continuos
variable, i. e. fxLR()()∈2, can be viewed as a multi-resolution approximation of the function
fx(). By the Wavelet Transform, fx() can be expressed in terms of limited support basis
functions ψabx,() of different resolution and extent. These basis functions are obtained by
translation and contraction/dilation from a "prototype" basis function ψ()x as
ψψabaxbax,()()=⋅−1.
Mallat [5] derived a recursive decomposition scheme to perform the Wavelet Transform of a
discrete sequence fn(). This recursive decomposition scheme for the discrete Wavelet Transform
(DWT) can be viewed as a subband coding scheme, known from speech signal coding [5], [7].
Due to the limited support of the basis functions ψabx,(), the Wavelet Transform signal
representation is shift variant, i.e. a shift of the input sequence leads to a nontrivial change of the
wavelet coefficients. This shift variance is a drawback in many applications.
There are several approaches to overcome this undesirable behaviour of the DWT. The
straightforward method is to compute the DWT for all possible circular shifts of the input sequence
with respect to the sequences length.
B. Discrete Wavelet Frames
Another approach to obtain a shift invariant wavelet representation is the concept of wavelet
frames, initially introduced for functions of a continuous variable [1]. In the case of discrete input
sequences, this is known as Discrete Wavelet Frame (DWF) decomposition [6]. For the DWF a
recursive decomposition scheme, similar to the DWT scheme, can be derived:
Each stage of the recursive DWF scheme splits the input sequence into the wavelet frame
sequence wni() which is stored, and the scale frame sequence sni() which serves as input for the
next decomposition level:
wngksnkiiik+=⋅⋅−∑12()()()(1)
snhksnkiiik+=⋅⋅−∑12()()()(2)
The zeroth level scale frame sequence is set equal to the input sequence snfn0()()=, thus
defining the complete DWF decomposition scheme. The analysis filters gki()2⋅ and hki()2⋅ at
level i are obtained by inserting the appropriate number of zeros between the filter taps of the
prototype filters gk() and hk().
The reconstruction of the input sequence is then performed by the inverse DWF (IDWF)
reconstruction as a convolution of both wavelet frame sequence and scale frame sequence with the
appropriate reconstruction filter ~()gki2⋅ and. ~()hki2⋅.
snhnksngnkwniiikiik()~()()~()()=⋅−⋅+⋅−⋅++∑∑2211(3)
