Learning Background and Shadow Appearance with 3-D Vehicle Models.pdf

Learning Background and Shadow Appearance with 3-D Vehicle Models ? Matthew J. Leotta Joseph L. Mundy Division of Engineering, Brown University Providence, RI, USA {mleotta,mundy}@lems.brown.edu Abstract This paper presents a novel algorithm for simultaneous background ap- pearance modeling and coarse-scale vehicle recognition in traffic surveil- lance applications. 3-d mesh models representing a small set of vehicle classes are used to the hypothesize image segmentations into background, shadow, and vehicle regions. The algorithm optimizes vehicle class and mo- tion parameters to best agree with a Hidden Markov Model for the image appearance. The best hypothesis, combined with image data, is used to adapt the parameters of the appearance model. Experiments on real video show that an appearance model trained in this way performs almost as well as one trained using manually segmented images. 1 Introduction Background modeling techniques are common in video surveillance to detect moving foreground objects. Often, this problem is confounded by the appearance of shadows cast by the moving foreground objects. While it is common to add an image-based model for shadow, this model does not exploit knowledge of shadow formation in the 3-d world. A correct 3-d model can accurately predict the image location of foreground and shadow pixels. However, in vehicle surveillance, the class of each observed vehicle is a priori unknown. Thus, the recognition and segmentation problems are intertwined. This paper addresses both detection and recognition simultaneously. The focus is on training and adapting an image appearance model using hypothesized image segmenta- tions. These hypotheses are generated from 3-d mesh models of vehicles, an illumination model, and an image projection model. The algorithm optimizes the vehicle class and motion parameters to best agree with the image data and appearance model. This appear- ance model is a Hidden Markov Model (HMM) at each pixel. The emissi