Machine Learning

CSCI 6505.03 Machine Learning

Instructor: Evangelos E. Milios

Course Description:

Machine Learning is the area of Artificial Intelligence concerned with the problem of building computer programs that automatically improve with experience. The intent of this course is to present a broad introduction to the principles and paradigms underlying machine learning, including discussions of each of the major approaches currently being investigated. Main topics covered in the course include a review of information theory, unsupervised learning or clustering (the K-means family, co-clustering, mixture models and the EM algorithm), supervised learning or classification (support vector machines, decision trees, rule learning, Bayesian learners, maximum entropy, ensemble methods), feature selection and feature transformations.
The focus of applications that will be discussed will be text classification and clustering.

Evaluation:

Course Component	Weight
Assignments (a1 , a2 equally weighted)	30%
Reading Presentation	25%
Class Participation - Reading short reports	10%
Project proposal	5%
Course project (Discussion paper guidelines)	30%

Notes:

Assignments, and presentation slides and project reports must be submitted electronically.
Reading presentations involve a formal presentation of a selected research paper by each student to the class.
Presentation files must be named as: paperNumber-studentLoginID-report.pdf, where the paperNumber is the one shown here.
Each presentation is alloted 40 min in total, 20 min for the presentation itself, and 20min for questions. Presentations will be cutoff at 25min.
Class participation is based on the contribution to the discussion of reading presentations by non-presenting students, and on the 250-word reports non-readers submit on the readings of the day.
No cover page or references (one sheet per day only).
First line of report: torchUserName, Last name, First Name, Date
Second line: title of first paper (in bold)
<First paper report>
Title of second paper (in bold)
<Second paper report>
Reports must be submitted on paper in the beginning of the class.
The best N-5 reports will be counted towards the course grade, where N is the total number of presentations given in class (N=24).
Presenters of a paper do not have to submit a report on the day of their presentation on any of the presented papers that day, but they should email the presentation to the instructor one hour before class starts.
Full title, authors, venue and year should be in the title page of the presentation slides.
Please ensure presentation slides have page numbers. In powerpoint, Insert=>Slide number => check "Slide number" checkbox => Apply to all
Please familiarize yourself with the Faculty's Plagiarism Policy and the University's Academic Integrity Policy. Read the statement summarizing the latter. Every suspected case of plagiarism will be referred to the Senate Discipline committee, and the standard penalty for a first offence is failure in the course.
All course components will be marked by letter grades, and averaged according to their GPA equivalent, shown in section 17.1 of the University Calendar. No rounding of partial grades takes place.

Students with disabilities are encouraged to register as quickly as possible at the Student Accessibility Services if they want to receive academic accommodations. To do so please phone 494-2836, e-mail access <at-symbol> dal.ca , drop in at the Killam, G28 or visit our website at www.studentaccessibility.dal.ca .

Prerequisites:
A course on Probability and Statistics is essential. General mathematical aptitude would really help. .

Required textbooks

[CB] is recommended, but it is not required.

Recommended Readings

[CB] Christopher Bishop: Pattern Recognition and Machine Learning,
Springer, 2006, ISBN: 0387310738

[ITILA] David J. C. MacKay: Information Theory, Inference and Learning Algorithms
Cambridge University Press, 2003, ISBN: 0521642981 (pdf file available on the Web for on-screen reading only)

[TK] S. Theodoridis, K. Koutroumbas
Pattern Recognition
Academic Press, 3rd edition, 2006, 0123695317

[IML] Ethem Alpaydin
Introduction to Machine Learning
The MIT Press, October 2004, ISBN 0-262-01211-1

[ESL] Trevor Hastie, Robert Tibshirani, Jerome Friedman
The Elements of Statistical Learning: Data Mining, Inference, and Prediction
Springer-Verlag, (August 2001), ISBN: 0387952845

[LV] John Lee, Michel Verleysen: Nonlinear Dimensionality Reduction
Springer Verlag, 2007, ISBN: 978-0-387-39350-6 (print), online (978-0-387-39351-3) (pdf file available on SpringerLink)

[ACD] A. Jain, R. Dubes: Algorithms for Clustering Data
Prentice Hall, 1988 (out of print, pdf file available on the Web)

[BL] Bing Liu: Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data
2007, 532 p., 177 illus., Hardcover, ISBN-10: 3-540-37881-2, ISBN-13: 978-3-540-37881-5

[SC] Soumen Chakrabarti:
Mining the Web: Discovering Knowledge from Hypertext Data
Hardcover - 344 pages - 9 October 2002
Morgan Kaufmann Publishers; ISBN: 1 55860 754 4

[SK] Scott Spangler and Jeffrey Kreulen
Mining the Talk: Unlocking the business value in unstructured information
ISBN-10: 0-13-233953-6; ISBN-13: 978-0-13-233953-7;
Published: Jul 19, 2007; Copyright 2008; Dimensions 7x9-1/4; Pages: 240; Edition: 1st.
(freely readable on Safari)

[TS] T. Segaran:
Programming Collective Intelligence: Building smart Web 2.0 applications
O'Reilly, Aug. 2007, 1st ed., ISBN 0-596-52932-5
(freely readable on Safari)

[FS] Ronen Feldman and James Sanger
The Text Mining Handbook: Advanced Approaches in Analyzing Unstructured Data
Cambridge University Press, 2007
(ISBN-13: 9780521836579 | ISBN-10: 0521836573)

[WF] Ian H. Witten, Eibe Frank:
Data Mining: Practical Machine Learning Tools and Techniques with Java Implementations (2nd edition)
Paperback - 525 pages (Jun. 2005)
Morgan Kaufmann Publishers; ISBN: 0-12-088407-0

Introduction to Machine Learning, Nils J. Nilsson (Draft of a Proposed New Textbook available on the Web)

Machine Learning, Tom Mitchell, McGraw Hill, 1997.

Manning & Schuetze: Foundations of Statistical NLP, MIT Press, 1998, ch. 14
pp. 495-527 (tot. pages: 680, ISBN: 0 262 13360 1)

Manning, Raghavan and Schuetze: Introduction to Information Retrieval, Cambridge University Press, 2008.

R. Sutton, A. Barto: "Reinforcement Learning, an Introduction", MIT Press, 1998.
Online version: http://www-anw.cs.umass.edu/~rich/book/the-book.html

Bibliography:

Background

Stuart Russell and Peter Norvig: Artificial Intelligence: A Modern Approach
Publisher: Prentice Hall; 2nd edition (December 20, 2002) ISBN: 0137903952
Hardcover: 1132 pages

Nils J. Nilsson: "Artificial Intelligence : A New Synthesis", Morgan Kaufmann Publishers (April 1998); ISBN: 1558604677

Journals, Conferences

AAAI and IJCAI conference proceedings
European Conference on Artificial Intelligence proceedings
Communications of the ACM
ACM SIGMOD Int. Conf. on Management of Data
Knowledge Engineering Review
Artificial Intelligence (Elsevier Publishers)
IEEE Trans. on Systems, Man and Cybernetics
IEEE Computer
IEEE Expert
IEEE Internet Computing
IEEE Transactions on Knowledge and Data Engineering
IEEE Transactions on Software Engineering
Journal of Artificial Intelligence Research
International Conference on Machine Learning
International Journal of Applied Artificial Intelligence
Int. Journal of Cooperative Information Systems

Note: The majority of the articles required for the course are available from the authors' web sites for downloading.

10 challenging problems and 10 most influential algorithms in data mining.
IEEE International Conference on Data Mining (ICDM)
http://www.cs.uvm.edu/~icdm/10Problems/index.shtml and
http://www.cs.uvm.edu/~icdm/algorithms/index.shtml

Related Courses at Dalhousie U.

CSCI 4144 & CSCI 6405: Data Mining and Data Warehousing. Focus on link to databases, pre/post processing of data.
MATH 5500. Statistical Data Mining: Statistical treatment of supervised and unsupervised learning.
CSCI 6501. Intelligent Systems.
CSCI 6504. Software Agents: Reinforcement learning, grammar induction (not offered)
CSCI 6506 Genetic Algorithms and Programming. In-depth treatment of this topic
CSCI 6507. Artificial Neural Networks (not offered)
CSCI 6508. Computational Neuroscience.
CSCI 6509. Advanced Topics in Natural Language Processing. Uses machine learning techniques (Hidden Markov Models, Naive Bayes, Clustering).

Machine Learning Resources on the Web:

UCI Machine Learning Repository
GMD Machine Learning Archive
Reinforcement Learning Repository at UMass

Scilab: a free scientific software package (a free equivalent of Matlab)
Octave (a free GNU project alternative to Matlab)
WEKA: Data Mining Software in Java
RapidMiner (formerly YALE: Yet Another Learning Environment (incorporates WEKA))
The R project for statistical computing (free comprehensive statistical programming environment)
---- TM: Text Mining in R
---- R/Weka: an R interface to WEKA
Dragon toolkit for Languge Modeling, Text Retrieval, and Text Data Mining
LingPipe (NLP applications of ML)
Topic Modelling Toolbox (Matlab) using Latent Dirichlet Allocation (LDA)