Adaptation to change of lighting conditions for interactive object recognition

This paper discusses the learning of color transformations for object recognition in various lighting conditions, a function which is needed in systems for object recognition by interaction with the user. The system first constructs a model of the target object under the reference lighting conditions. Since the lighting conditions in actual object recognition differ from those of object model construction, the system must transform the color of the object model according to the lighting conditions at the time of recognition. The color of a reference object observed in the recognition stage is used to estimate the color transformation. The system learns the relation between the color transformation and the color of the reference object by using the color of the object model and also the pair of colors of the target object and the reference object observed under various lighting conditions. In a new environment, such a set of colors is not stored, and the color transformation is applied by assuming a reasonable relation. Object recognition may fail when such a color transformation is applied. In such a case, the correct object is selected by interaction with the user. The relation is updated by using the obtained correctly matched pair. By improving the estimation accuracy of the color transformation at each instance of object recognition, the burden on the user in the interaction is reduced. Finally, the effectiveness of the proposed method is demonstrated through an experiment on a scene inside a refrigerator. © 2007 Wiley Periodicals, Inc. Syst Comp Jpn, 38(4): 52–62, 2007; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/scj.20228