Supervised and Experiential Learning (SEL)


Master in Artificial Intelligence

(4.5 ECTS - 112.5 h)

Fall semester, Course 2019/2020

Responsible Professor:

KEMLG

Miquel Sànchez i Marrè
Knowledge Engineering & Machine Learning Group
Dept. of Computer Science
Universitat Politècnica de Catalunya
Campus Nord-Building OMEGA, 2nd floor, office 203
miquel@cs.upc.edu


Professors:

Professor  Office  E-mail  Phone
Tutoring 
Miquel Sànchez i Marrè OMEGA building,
2nd floor, office 203
Campus Nord
miquel@cs.upc.edu 93 413 78 41
 To convene by e-mail

Timetable:

Day Hour Room
Professor
Thursday
10:00-13:00 A6 102
M. Sànchez-Marrè



Schedule:

SEL Lecturing period: From February 13th, 2020 to May 21st, 2020.
        Holidays:  April 9th 2019 (Easter holidays)

        Change of scheduling: May 7th 2020 (It is Friday!)


Contents:
1. Machine Learning: Supervised and Unsupervised ML techniques

      Basic principles and classification of Machine Learning techniques

2. Important Challenges in Supervised Learning

     Quantity of data
     Quality of data: representability, imbalanced class distribution
     Overfitting & Underfitting of models
     Bias & Variance of models
     Feature relevance
         i. Reminder: Feature Selection vs Feature Weighting, Filters and wrappers
         ii. Feature weighting techniques
3. Supervised Learning techniques

      Rule-based Classifiers
         i. Decision Tree Classifiers (ID3, C4.5, CART). Pruning techniques
         ii. Classification Rule Classifiers (PRISM, RULES, CN2, RISE)
      Probabilistic/Bayesian Classifiers
         i. Bayes Optimal Classifier
         ii. Gibbs algorithm
         iii. Naïve Bayes Classifier
      Linear Predictors
         i. Linear Regression / Multiple Linear Regression
      Statistical Classifiers
         i. Linear Discriminant Analysis (LDA)
         ii. Logistic/Multinomial Regression
4. Diversification / Ensemble of classifiers
      a. Reminder: General scheme
      b. Random Forests
5. Evaluation Techniques
      a. Discriminant/Classification models
      b. Prediction/Regression models
6. Advanced Classification Challenges
      a. Multi-label classification
      b. Ordinal classification
      c. Imbalanced Dataset classification
      d. Using noise and diversification for improving classification
      e. Meta-Learning of classifiers
      f. Incremental Learning: Data stream/on-line learning
7. Experiential Learning

       Reminder: Fundamentals of Case-based Reasoning
           Cognitive Theories
           Basic Cycle of Reasoning

8. CBR Academic Demonstrators/Examples

9. CBR System Components
Case Structure
Case Library Structure
Retrieval
Adaptation (Reuse)
Evaluation (Repair)
Learning (Retain)

10. CBR Application
Supervision and Management of a WWTP: a complex real-world domain

11. CBR Development Problems
Performance criteria
    Competence
    Space Efficiency
    Time Efficiency

12. Reflective Reasoning in CBR
Case Base Maintenance

13. CBR Applications and Development Tools
Industrial Applications
Software Tools

14. CBR Systems' Evaluation

15. Advanced Research Issues
Temporal CBR
Spatial CBR
Hybrid CBR Systems
CBR as a recommendation tool: Feature Weighting Algorithm Recomendation based on CBR

Conclusions

Learning Resources
Literature References
Internet Resources


Course Material

2019/2020 Sylabus:
ML and Supervised Learning
Experiential Learning

Projects

All students in the course must do three practical works (PW1, PW2 and PW3). The practical works will be:

PW1 (Individual work) – Implementation and evaluation of a rule-based classifier.
PW1 will be assigned on March 5th, 2020
PW1 is due on April 2th, 2020

PW2 (Individual work) - Implementation and evaluation of an ensemble of classifiers.
PW2 will be assigned on April 16th, 2020
PW2 is due on May 7th, 2020

PW3 (Group work) - A practical work to build a CBR prototype for a synthetic task in a concrete domain.
PW3 will be assigned on May 7th, 2020
PW3 is due on June 16th, 2020
Public Presentation of PW3 on June 18th, 2020

Evaluation

Evaluation of the knowledge and skills obtained by the students will be assessed through three project works. The first two works (PW1 and PW2) will be on an individual basis and the third one (PW3) will be on a team group basis.

The individual works will consist on the implementation, application and evaluation of some supervised machine learning algorithms.
The team-group work will consist on the design, implementation, application and validation of a Case-Based Reasoning project to solve a synthetic problem.

The final grade will be computed as follows:

FinalGrade= 0.25 * PW1 + 0.25 * PW2 + 0.5 * PW3 * WFstud,        where 0 ≤ WFstud ≤ 1.2

WFstud is a Working Factor evaluating the work of a particular student within his/her teamwork in PW3. It will be obtained by observing and assessing the load of work and degree of participation of each student throughout the PW3. In normal conditions, the WFstud = 1.

The individual works will be evaluated according to the quality of the algorithm and software developed (0.6), the evaluation done (0.2) and the documentation delivered (0.2).

The Teamgroup work wil be evaluated according to:

- The methodology of the work (0.4)
- The quality of the report written (0.2)
- The quality of the oral exposition (both presentation and content assessed, as well as the ability to answer questions) (0.2)
- Planification, coordination and management of the team (0.05)
- The individual valoration of each student, including her/his integration level within the teamgroup (0.15)



Honour Code (adapted from Stanford University Honor Code)

A.    The Honor Code is an undertaking of the students, individually and collectively:
  1. that they will not give or receive aid in examinations; that they will not give or receive unpermitted aid in class work, in the preparation of reports, or in any other work that is to be used by the instructor as the basis of grading;
  2. that they will do their share and take an active part in seeing to it that others as well as themselves uphold the spirit and letter of the Honor Code.

B.    The faculty on its part manifests its confidence in the honor of its students by refraining from proctoring examinations and from taking unusual and unreasonable precautions to prevent the forms of dishonesty mentioned above. The faculty will also avoid, as far as practicable, academic procedures that create temptations to violate the Honor Code.

C.    While the faculty alone has the right and obligation to set academic requirements, the students and faculty will work together to establish optimal conditions for honorable academic work.

Examples of conduct which have been regarded as being in violation of the Honor Code include:



In recent years, most student disciplinary cases have involved Honor Code violations; of these, the most frequent arise when a student submits another’s work as his or her own, or gives or receives unpermitted aid. The standard penalty for a first offense or multiple violations will be proposed by the teachers in charge of the course, and approved by the Academic Commission of the Master MAI.

Learning Resources
Literature References
Basic References
Machine Learning and Supervised Learning


CBR foundations

CBR general introduction


CBR introductory papers, reviews and special issues

Complementary references

l
CBR application and tool reviews
l
CBR conferences and workshops

Both ECCBR and ICCBR joined in the new annual ICCBR conferences

Internet resources
Internet CBR resources
General Information
CBR Wiki
        http://cbrwiki.fdi.ucm.es/wiki/index.php/Main_Page
David Aha’s CBR Resources (No longer maintained)
http://home.earthlink.net/~dwaha/research/case-based-reasoning.html
ICCBR server
http://www.iccbr.org
Archive of CBR and other AI publications from Indiana University (David Leake)
http://www.cs.indiana.edu/~leake/INDEX.html

Software Tools
Source Code
Code from Inside Case-Based Explanation and Inside Case-Based Reasoning
JColibri2 (Universidad Complutense de Madrid, GAIA group)
JColibri web page project
AIA CBR shell (University of Edinburgh)
http://www.aiai.ed.ac.uk/project/cbr/CBRDistrib/
        CASPIAN (University of Wales)
        http://www.aber.ac.uk/~dcswww/Research/mbsg/cbrprojects/getting_caspian.shtml
        Selection Engine (by Baylor Wetzel, 2000)
http://selectionengine.sourceforge.net/
        IUCBRF (Indiana University CBR Framework)
        http://www.cs.indiana.edu/~sbogaert/CBR/
        MyCBR (German Research Center for Artificial Intelligence, DFKI)
        http://www.mycbr-project.net
        CASUEL (Universität Trier)
        http://www.wi2.uni-trier.de/de/cms/projects/CASUEL/CASUEL2_toc2.04.fm.html
        GESCONDA-CBR shell (Universitat Politècnica de Catalunya-BarcelonaTech, KEMLG group)
        http://upcommons.upc.edu/pfc/bitstream/2099.1/7726/1/MasterThesis-BeatrizSevillaVillanueva.pdf

Data Repository
UCI Repository of ML Databases
http://archive.ics.uci.edu/ml/