Multimedia Mining via Linguistic Associations

This AR project started as a collaboration between the Audio-Visual Speech Technologies Group and the Pervasive Media Management group.

Our goal is to build a trainable system for detecting semantics in Multimedia content. Specifically,

• Given
• Concepts vocabulary: Events, Objects, Static scenes
• Concept definition: Audio visual constituents and their spatial, temporal relationships
• Concept illustration: Examples or user specification 
• Build
• Statistical models from low-level and semantic features to detect concepts 
• Large, extensible inventory of low-level and semantic features
• Detect
• Temporal events, Static scenes, Objects in multimedia content

• Video TREC
• Video TREC 2001 homepage (IBM Research)
• IBM Research Video TREC 2001 System

Members:

Presentations and Publications:

• 2002 Worldwide Human Language Technology meeting (April 2002), IBM Yorktown
• Semantic Indexing of Multimedia Content-Vision Day Talk 
• G. Iyengar, H. Nock, C. Neti, M. Franz, Semantic Indexing of Multimedia using Audio, Text and Visual Cues, Proceedings of ICME2002, Lausanne, Switzerland, 2002