577 - Seminar in
AI: Advanced Topics in Pattern Recognition
Probability Models for Information Processing and Machine Perception
Lectures: Thursdays 2:00-4:40 in CSB 632
Course Description
This seminar will cover advanced probabilistic models and methods for pattern recognition and machine learning. The course will focus on models and algorithms but will go into application domain details required to understand state of the art techniques. Application areas will be selected from: computer vision and computational photography (~50%), text and natural language processing (~30%), bioinformatics and computational biology (~20%). There will be two assignments covering fundamental material. The bulk of the course will consist of reading and discussing recent research papers and presenting a project.
Syllabus
Date |
Topics: My Lectures, Readings and Student Presentations |
Jan. 24 |
Introduction & course overview |
Jan. 31 |
Introduction to probability models Mixture Models The EM Algorithm Introduction to Graphical Models Required Reading: Bilmes, J. A Gentle Tutorial on the EM Algorithm and its Application to
Parameter Estimation for Gaussian Mixture and Hidden Markov Models. Technical Report, UC Berkeley,
ICSI-TR-97-021, 1997.
Student Presentation: Ross Messing - Transformation Invariant Mixture Models |
Feb. 7 |
Inference and Estimation Factor Graphs Message Passing MAP and MPE inference Required Reading: Kschischang, Frey and Loeliger (2001) Factor Graphs and the Sum-Product Algorithm. IEEE Transactions on Information Theory. Additional
Reading: Jojic, Petrovic,
Frey and Huang. Transformed
Hidden Markov Models: Estimating Mixture Models of Images and Inferring
Spatial Transformations in Video Sequences. IEEE CVPR 200. Student
Presentation: Paul Ardis - Factor Graphs and the Sum Product Algorithm |
Feb. 14 |
Approximate Inference Techniques Graph Cuts Readings: Kolmogorov and Zabih, What Energy Functions can be Minimized via Graph Cuts? ECCV '02/PAMI '04. Additional Material: * Boykov, Veksler and Zabih Fast Approximate Energy Minimization via Graph Cuts PAMI '01. * Greig, Prteous and Seheuth. "Exact Maximum A Posteriori Estimation for Binary Images" J. Royal Statistical Soc., Series B, vol. 51, no. 2, pp. 271-279, 1989. * The Cornell Graph Cuts Page Student Presentation: Satyaki Mahalanabis - Graph Cuts |
Feb. 21 |
Bayesian Methods Exponential Families Variational Bayes Readings: Blei, Ng and
Jordan. Latent
Dirichlet Allocation. JMLR 2003. Student
Presentation: Nick Morsillo - LDA Student
Presentation: Bin Wei - Learning Bayesian Networks |
Feb. 28 |
More On Bayesian Methods Markov Chain Monte Carlo Hierarchical Bayesian Methods Readings: Yee Whye Teh,
Michael I Jordan, Matthew J. Beal and David M. Blei (2004) Sharing
Clusters among Related Groups: Hierarchical Dirichlet Processes. NIPS
(earlier version of the JASA paper). Beal, Ghahramani
and Rasmussen (2002) The Infinite Hidden
Markov Model. NIPS 14. Yee Whye Teh,
Michael I Jordan, Matthew J. Beal and David M. Blei (2006) Hierarchical
Dirichlet Processes, JASA. Student
Presentation: Ross Messing - Hierarchical Dirichlet Processes and Time Series
Models. |
March 6 |
Conditional Probability Models Parameterizations Readings
(Tentative): Zhu and Hastie
(2002) Kernel
Logistic Regression and the Import Vector Machine NIPS 14. Student
Presentation: Nick Morsillo - Kernel LR. |
March 13 |
No class / Spring Break |
March 20 |
Undirected Models I - MRFs and CRFs History and Fundamentals Their use in: Computer Vision - Image Segmentation Computational Photography - Focusing on CRFs for Stereo Natural Language Processing - Information Extraction Reading: Charles Sutton and Andrew McCallum. (2006) An Introduction to Conditional Random Fields for Relational Learning. In Introduction to Statistical Relational Learning. Edited by Lise Getoor and Ben Taskar. MIT Press. 2006. Additional Reading: Daniel Scharstein and Chris Pal (2007) Learning Conditional Random Fields for Stereo In the proceedings of IEEE Computer Vision and Pattern Recognition (CVPR). Student Presentation: Paul Ardis - CRFs |
March 27 |
Special Class Meeting with
Guest, David Stork Discussion
meeting: 2:15-3:15, meeting in office of CS Dept. Chair. ECE Talk 3:30 Goergen
101 When computers look at art: Image analysis in
humanistic studies of visual arts. Suggested Reading: Duda, Hart and
Stork. Pattern
Classification (2^{nd} Edition). |
April 3 |
Undirected Models II - Boltzmann Machines Reading: Bouchard, G., Bias-Variance tradeoff in Hybrid Generative-Discriminative models, In J. Antoch, editor, Proceedings in Computational Statistics, 16th Symposium of IASC, volume 16, Prague. Physica-Verlag. Student
Presentation: Bin Wei Additional Reading: Druck, G., Pal, C., Zhu, X., and Andrew McCallum. (2007) Semi-Supervised Classification with Hybrid
Generative/Discriminative Methods. In the proceedings of Knowledge
Discovery and Data Mining (KDD). |
April 10 |
Generalizing and
Understanding Belief Propagation Recent Views on
Unifying Logic and Probability - Markov Logic Reading: "Understanding Belief Propagation and its Generalizations" by Yeddidia, Freeman and Weiss (Mitsubishi TR-2001-22). Student Presentation (Tentative): Satyaki Mahalanabis - TBD Additional Readings: Matt Richardson and Pedro Domingos. Markov Logic Networks. Machine Learning, 62, 107-136, 2006. |
April 17 |
Student project
presentations: Satyaki Mahalanabis Nicholas Morsillo Relevant Reading: Morsillo et al., "Mining the Web for Visual Concepts." URCS Tech Report. Martin Zinkevich, Online Convex Programming and Generalized Infinitesimal Gradient Ascent, ICML 2003. With Some Possible
Additional Discussion On: More Markov Logic Markov Decision Processes |
April 24 |
Student project
presentations: Bin Wei Paul Ardis Tentative Additional
Discussion on: Bayesian Logic (BLOG) |
Textbooks, References and Additional Reading
Pattern Recognition and Machine Learning, Chris Bishop, Springer 2006.
Information Theory, Inference and Learning Algorithms, David MacKay, Cambridge University Press 2003.
Pattern
Classification (2^{nd} Edition), Duda, Hart and Stork.
The
Elements of Statistical Learning, Data Mining, Inference and Prediction. T.
Hastie, R. Tibshirani and J.H.Friedman
Grading Scheme
Component |
% |
Assignment #1 (Due March 20) |
10% |
Assignment #2 (Due April 24) |
20% |
Participation (Presenting papers) |
40% |
Project |
30% |