Semester II, 2001-2002 (Wed 8am-10am, SR1)

Last update: Monday, 01-Apr-2002 13:04:27 +08

Table of Contents

• Announcements (last updated 31-01-2002)
• General Information
• Brief Course Outline
• Course Material
• Tutorial Information
• Class Schedule
• Frequently Asked Questions

General Information

Lecturer: Mohan S Kankanhalli

Aims and Objectives:

This course introduces techniques for analysis, representation and retrieval of multimedia information. The media to be considered include image, audio and video. At the end of this course, the students should have the expertise and competence to design and implement retrieval software for multimedia data.

Brief Description:

Multimedia data is unstructured and rich in content. Conventional database systems, which are designed to handle structured data and support exact match retrieval, are inadequate for this type of data. This course covers a broad class of retrieval techniques based on similarity-based retrieval. Similarity-based retrieval relies on "best-match" rather than "exact match" and uses techniques to compute the "similarities" between the query and information items. As the users' information needs are also fuzzy, an important characteristic for this class of retrieval techniques is its support for the iterative process of retrieval.

In this course, we will discuss various attributes characterizing the multimedia data. The attributes to be discussed are text, color, texture and shapes. For each attribute, we will discuss its representation scheme, similarity-based retrieval model, iterative refinement technique, and other representation and retrieval models. We will discuss the use of these attributes to retrieve images, audio and video. Finally, we will discuss a framework for multimedia information retrieval and directions of the future work.

Assignments: 30%
Midterm Exam: 20%
Final Exam: 50%

Pre-requisites: CS3242 (Hypermedia Information Processing)

Office consultation hours: Wed 3-5pm (S17 #04-19)

<-- Table of Contents

Brief Course Outline

I. Introduction to multimedia information retrieval 6 hrs
characteristics of MM data; similarity-based retrieval model; attributes of MM data; similarity measures; multimedia retrieval framework; relevance feedback; benchmarking of multimedia information systems

II. Color-based Retrieval 4 hrs
color models; histogram model; indexing and retrieval; relevance feedback; histogram refinement; color cluster technique

III. Texture-based Retrieval 2 hrs
texture models; statistical models; combined color-texture representation

IV. Shape-based Retrieval 2 hrs
shape matching; contour-based method (Fourier descriptors); region-based method (moment invariants)

V. Audio Retrieval 4 hrs
characteristics of audio data; spectrum analysis; pitch tracking; techniques for audio feature extraction, similarity matching and retrieval

VI. Video Retrieval 4 hrs
video segmentation in raw and compressed domain; key-frame extraction; video summarization and retrieval techniques

VII. Multimedia Retrieval Framework 2 hrs
multi-attribute query processing; knowledge-based methods

VIII. Multimedia Retrieval Trends 2 hrs
applications; future

<-- Table of Contents

Course Material

A copy of the slides & all the relevant material will be put in the coop.

• Lecture 1: Slides, Notes
• Lecture 2: Slides in the coop
• Lecture 3: Slides in the coop
• Lecture 4: Slides in the coop
  • Supplementary reading: G. Pass & R. Zabih, "Histogram Refinement for Content-based Image Retrieval", IEEE Workshop on Applications of Computer Vision, Sarasota, Florida, December 1996.
• Lecture 5: Slides in the coop
  • Supplementary reading: G.P. Babu, B.M. Mehtre & M.S. Kankanhalli, "Color Indexing for Efficient Image Retrieval", Multimedia Tools and Applications, Vol. 1, No. 4, pp. 327-348, 1995.
• Lecture 6: Slides in the coop
  • Supplementary reading: H. Tamura, S. Mori & T. Yamawaki, "Textural Features Corresponding to Visual Perception", IEEE Transactions on Systems, Man and Cybernetics, Vol. 8, No. 6, pp. 460-473, 1978.
• Lecture 7: Slides in the coop
  • Supplementary reading: B.M. Mehtre, M.S. Kankanhalli & W.F. Lee, "Shape Measures for Content Based Image Retrieval: A Comparison", Information Processing and Management, Vol. 33, No. 3, pp. 319-337, 1997.
• Lecture 8:
  • Supplementary reading: N. Kosugi, Y. Nishihara, S. Konya, M. Yamamuro & K. Kushima, Music Retrieval by Humming, IEEE Pacific Rim Conference on Communications, Computers and Signal Processing, 1999.
• Lecture 9:
  • Supplementary reading: Y.W. Zhu & M.S. Kankanhalli, Similarity Matching of Continuous Melody Contours for Humming Querying of Melody Databases, LIT Technical Report, February 2002.
• Lecture 10:
  • Supplementary reading: H.J. Zhang, A. Kankanhalli & S.W. Smoliar, "Automatic Partitioning of Full-motion Video", ACM Multimedia Systems, Vol. 1, No. 1, pp. 10-28, 1993.
• Lecture 11:
  • Supplementary reading: X. Sun & M.S. Kankanhalli, Video Summarization Using R-Sequences, Journal of Real-Time Imaging, Vol. 6, No. 6, pp. 449-459, December 2000.
• Lecture 12:
• Lecture 13:

Some Books: (buying them is not at all necessary)

1. R. Jain, R. Kasturi, B.G. Schunck (1995), Machine Vision, McGraw-Hill.
[One of the many basic reference texts on image processing]

2. B. Furht, S.W. Smoliar, H.J. Zhang (1995), Video and Image Processing in Multimedia Systems, Kluwer, Boston.
[A reference text on multimedia in general]

3. J.K. Wu, M.S. Kankanhalli, J.H. Lim, D.Z. Hong (2000), Perspectives on Content-based Multimedia Systems, Kluwer Academic Publishers, Boston.

<-- Table of Contents

Tutorial Information

  Tutorial 1: Tutorial on Retrieval Models (05-02-2002)

  Tutorial 2: Tutorial on Color-based Retrieval (19-02-2002)

  Tutorial 3: Tutorial on Texture and Shape Retrieval (05-03-2002)

  Tutorial 4: Tutorial on Audio Retrieval (19-03-2002)

  Tutorial 5: Tutorial on Video Retrieval (02-04-2002)

<-- Table of Contents

Class Schedule

9^th January (Week 1): Introduction to MMIR

16^th January (Week 2): MM Retrieval Framework I

23^rd January (Week 3): MM Retrieval Framework II

  Assignment 1: Image retrieval assignment (Due Mar 4)

30^th January (Week 4): Color-based Retrieval I

6^th February (Week 5): Color-based Retrieval II

16^th February (Week 6): Texture-based Retrieval  8.00am - 10.00am @ SOC1 #04-TR7   Note: make-up class for February 13^th

20^th February (Week 7): Shape-based Retrieval

22^nd February AM: Midterm Examination

TR9 (SOC1, 6^th Floor) at 9.30am

This will be an open-book exam and it will be based on the material covered till February 6 (which is color-based retrieval).

27^th February Semester Break

6^th March (Week 8): Audio Retrieval I

  Assignment 2: Audio assignment (Due Mar 25)

13^th March (Week 9): Audio Retrieval II

20^th March (Week 10): Video Retrieval I

  Assignment 3: Video retrieval assignment (Due April 15)

27^th March (Week 11): Video Retrieval II

3^rd April (Week 12): Multi-attribute & Knowledge-based Retrieval

10^th April (Week 13): Multimedia Retrieval Trends

27^th April PM: Final Examination

<-- Table of Contents

Frequently Asked Questions

Lectures FAQ
Assignment 1 FAQ
Assignment 2 FAQ
Assignment 3 FAQ

<-- Table of Contents