CS4241: Multimedia Information Systems

Course Home Page

Semester II, 2001-2002 (Wed 8am-10am, SR1)

Last update: Monday, 01-Apr-2002 13:04:27 +08


Table of Contents



General Information

Lecturer: Mohan S Kankanhalli

Lectures: 26 Hours (Wednesday, 8am-10am, SR1)

Tutorials: On a need basis

Open Laboratory: 13 Hours (3 programming assignments)

Midterm Examination: 22nd February, AM

Final Examination: 27th April, PM

Aims and Objectives:

This course introduces techniques for analysis, representation and retrieval of multimedia information. The media to be considered include image, audio and video. At the end of this course, the students should have the expertise and competence to design and implement retrieval software for multimedia data.

Brief Description:

Multimedia data is unstructured and rich in content. Conventional database systems, which are designed to handle structured data and support exact match retrieval, are inadequate for this type of data. This course covers a broad class of retrieval techniques based on similarity-based retrieval. Similarity-based retrieval relies on "best-match" rather than "exact match" and uses techniques to compute the "similarities" between the query and information items. As the users' information needs are also fuzzy, an important characteristic for this class of retrieval techniques is its support for the iterative process of retrieval.

In this course, we will discuss various attributes characterizing the multimedia data. The attributes to be discussed are text, color, texture and shapes. For each attribute, we will discuss its representation scheme, similarity-based retrieval model, iterative refinement technique, and other representation and retrieval models. We will discuss the use of these attributes to retrieve images, audio and video. Finally, we will discuss a framework for multimedia information retrieval and directions of the future work.

Assessment:

Assignments: 30%
Midterm Exam: 20%
Final Exam: 50%

Pre-requisites: CS3242 (Hypermedia Information Processing)

Office consultation hours: Wed 3-5pm (S17 #04-19)

<-- Table of Contents


Brief Course Outline

I. Introduction to multimedia information retrieval 6 hrs
characteristics of MM data; similarity-based retrieval model; attributes of MM data; similarity measures; multimedia retrieval framework; relevance feedback; benchmarking of multimedia information systems

II. Color-based Retrieval 4 hrs
color models; histogram model; indexing and retrieval; relevance feedback; histogram refinement; color cluster technique

III. Texture-based Retrieval 2 hrs
texture models; statistical models; combined color-texture representation

IV. Shape-based Retrieval 2 hrs
shape matching; contour-based method (Fourier descriptors); region-based method (moment invariants)

V. Audio Retrieval 4 hrs
characteristics of audio data; spectrum analysis; pitch tracking; techniques for audio feature extraction, similarity matching and retrieval

VI. Video Retrieval 4 hrs
video segmentation in raw and compressed domain; key-frame extraction; video summarization and retrieval techniques

VII. Multimedia Retrieval Framework 2 hrs
multi-attribute query processing; knowledge-based methods

VIII. Multimedia Retrieval Trends 2 hrs
applications; future

<-- Table of Contents


Course Material

A copy of the slides & all the relevant material will be put in the coop.

Some Books: (buying them is not at all necessary)

1. R. Jain, R. Kasturi, B.G. Schunck (1995), Machine Vision, McGraw-Hill.
[One of the many basic reference texts on image processing]

2. B. Furht, S.W. Smoliar, H.J. Zhang (1995), Video and Image Processing in Multimedia Systems, Kluwer, Boston.
[A reference text on multimedia in general]

3. J.K. Wu, M.S. Kankanhalli, J.H. Lim, D.Z. Hong (2000), Perspectives on Content-based Multimedia Systems, Kluwer Academic Publishers, Boston.

<-- Table of Contents


Tutorial Information

The tutorials will be held on Tuesdays 5-6pm and the classroom is SOC1 #04-TR7.

  Tutorial 1: Tutorial on Retrieval Models (05-02-2002)

  Tutorial 2: Tutorial on Color-based Retrieval (19-02-2002)

  Tutorial 3: Tutorial on Texture and Shape Retrieval (05-03-2002)

  Tutorial 4: Tutorial on Audio Retrieval (19-03-2002)

  Tutorial 5: Tutorial on Video Retrieval (02-04-2002)

<-- Table of Contents


Class Schedule

9th January (Week 1): Introduction to MMIR

16th January (Week 2): MM Retrieval Framework I

23rd January (Week 3): MM Retrieval Framework II

  Assignment 1: Image retrieval assignment (Due Mar 4)

30th January (Week 4): Color-based Retrieval I

6th February (Week 5): Color-based Retrieval II

16th February (Week 6): Texture-based Retrieval  8.00am - 10.00am @ SOC1 #04-TR7   Note: make-up class for February 13th

20th February (Week 7): Shape-based Retrieval

22nd February AM: Midterm Examination

TR9 (SOC1, 6th Floor) at 9.30am

This will be an open-book exam and it will be based on the material covered till February 6 (which is color-based retrieval).

27th February Semester Break

6th March (Week 8): Audio Retrieval I

  Assignment 2: Audio assignment (Due Mar 25)

13th March (Week 9): Audio Retrieval II

20th March (Week 10): Video Retrieval I

  Assignment 3: Video retrieval assignment (Due April 15)

27th March (Week 11): Video Retrieval II

3rd April (Week 12): Multi-attribute & Knowledge-based Retrieval

10th April (Week 13): Multimedia Retrieval Trends

27th April PM: Final Examination

 

<-- Table of Contents


Frequently Asked Questions

Lectures FAQ
Assignment 1 FAQ
Assignment 2 FAQ
Assignment 3 FAQ

<-- Table of Contents