PeopleVision: Human Aware Environments

Browser should automatically jump to new page in 5 seconds.

Developing easy-to-use interfaces between humans and computers is a major challenge in computer science and will continue to be one for a long time. New innovations in interfaces are slowly increasing the bandwidth and effectiveness of the human-computer-interface. While the currently emerging wave of human-computer interfaces is speech-based, the next innovation horizon in human-computer interfaces is likely to be visual. Cameras today cost less than $10 when purchased in bulk and the processing power on desktops allows significant tasks to be performed based on image and video processing. These combined advances in the hardware and sensing technology have caused a surge of research activity on various aspects of human-centric visual computing. The visual channel is interesting because it can provide a significantly larger bandwidth for human-to-computer communication. This mirrors the current high bandwidth of computer-to-human communication which relies primarily on CRT or LCD displays. The richness of visual input has the potential to greatly enhance the quality of human-to-computer interaction. In particular, there are three critical questions which are of interest regarding human activities: Who are the people in a given space? Where (physical location) are they? What is the person(s) doing? To achieve these aims, we note that the human activities of interest occur at various spatial scales. Facial expressions which occur at a fine scale, hand gestures at an intermediate scale, and body motions during walking are at a much coarser scale. Current academic research is covers this entire spectrum: lip-reading, facial expression analysis, and people tracking for surveillance. However, what is missing is a unified multi-scale approach to such human-centric visual computing. From a visual-computing perspective, the activities at a finer scale are always expressed within the frame of reference of the coarser scale. For example, a facial expression analysis module typically has to normalize for pose (rotation and translation) of the head in order to successfully deal with the analysis of facial expressions. Thus, for a system to be successful in real world applications, it is advantageous for the system to have a multi-scale model of the human and a mechanism for effectively utilizing coarse-scale information at finer scales. The explicit model of interaction between the coarse and fine scales of human activities is the driving philosophy behind our approach to multi-scale human perception. We intend to demonstrate applications which use a multi-scale tracking system to answer each of the above questions.   Video demos: See this page for videos of 3D tracking on a floorplan, outdoor person and vehicle tracking, articulated body motion estimation, head pose determination, and a video privacy method. All videos are in RealVideo (.rm) format.

 

Selected publications:

Appearance Models for Occlusion Handling
Andrew W. Senior, Arun Hampapur, Lisa M. Brown, Ying-Li Tian,
Sharathchandra Pankanti, Rudolf M. Bolle
Second International workshop on Performance Evaluation of Tracking
and Surveillance systems, Kauai, HI, USA Dec. 2001.

3D Head Tracking Using Motion Adaptive Texture-Mapping
L Brown
IEEE Conference in Computer Vision and Pattern Recognition,
Kauai, Hawaii, vol. no.1 pp.998-1005, December 8-14, 2001.
More

Related External Work:

Visual Analysis of Human Movement
D.M. Gavrilla
Computer Vision and Image Understanding, Vol 73, No 1, Jan 1999.

Ubiquitous sensing for smart and aware environments
I. Essa
in IEEE Personal Communications V7 N5, Oct 2000.

W4: Real-Time Surveillance of people and their activities
Haritaoglu, Harwood, and Davis
IEEE PAMI, Aug 2000.

Contact: Arun Hampapur			Last updated: 6/6/02