Syllabus

CS 457/557: Computational Intelligence

Spring 2014

Meeting Times

Lect: 12:00 - 12:50 M, Wed, Th, HB 106

Labs: 12:00 - 12:50 Tu, HB 204

Instructor

Dr. Razvan Andonie, HB 219-B, Office hours

Textbook

Stephen Marsland, Machine Learning: An Algorithmic Perspective, CRC Press, 2009

 

Objectives

Introducing concepts, models, algorithms, and tools for development of intelligent systems. Example topics include artificial neural networks, genetic algorithms, fuzzy systems, swarm intelligence, ant colony optimization, artificial life, and hybridizations of the above techniques. We will at all these techniques from a machine learning perspective. This domain is called Computational Intelligence, and is a numerical interpretation of biological intelligence.

Topics & Student Learning Outcomes

Neural networks, associative memories, vector quantization, self-organizing feature maps, support vector machines, genetic algorithms, fuzzy neural networks, swarm intelligence, ant colony optimization, decision trees, ensemble learning, nearest neighbor method, Gaussian mixture methods, principal component analysis, independent components analysis, hill climbing, reinforcement learning, Markov,  decision processes, simulated annealing, hidden Markov models, Bayesian networks.

On the completion of this course, the student will have:

An understanding of fundamental computational intelligence and machine learning models.

Implemented neural networks, genetic algorithms, and other computational intelligence and machine learning algorithms.

Applied computational intelligence and machine learning techniques to classification, prediction, pattern recognition, and optimization problems.

Grading

There are no exams. Grading will be based on 50% assignments and 50% Final Project.

Grade Distribution:

95 - 100   A

90 - 94     A-

87 - 89     B+

83 - 86     B

80 - 82     B-

77 - 79     C+

73 - 76     C

70 - 72     C-

67 - 69     D+

63 - 66     D

60 - 62     D-

0  - 59      F

Final Project

The Final Project should be submitted as a technical report and presented in class. The CS 557 projects are more complex and they will be graded with higher expectations.  The project will be graded as follows: 1/3 code, 1/3document, and 1/3 presentation. The report document should contain: problem description, benchmarking, and discussion. Guidelines can be found at here. The presentation should be made with 10-15 slides, in about 15 minutes, including questions and answers. Presentation guidelines can be found here. Submit the report by email and place your source code and your executable version in a Final Project folder in your account.

Here is the list of projects from previous years.

Current Final Projects

Lectures & Projects

The slides for lectures and additional materials can be found in the shared directory. The Python code and the datasets from the textbook can be found here.

Software

You are encouraged to use Python and Weka. The examples in the textbook are written in Python and can be downloaded from here. To install Python, use the WinPython installation package. Use the included Spyder scientific development environment. You may install the 32 or 64 bit version, but make sure you install Python 2 (not 3), because the examples in the book are in Python 2. It is a good idea to install WinPython on your computer or on a flash-drive, because in the lab you do not have administrator privileges to initialize Spyder.

Good links for scientific programming in Python are:

Numeric & Scientific Computation with Python

NumFocus

scikit-learn: machine learning algorithms in Python

Weka is excellent for benchmarking and testing. You may install it easily (read the manual). You may use the following books:

Witten, I. H., Frank, E., Hall, M. A. Data Mining; Practical Machine Learning Tools and Techniques, 2011.

Kaluza, B. INSTANT Weka How-to, Packt Publishing, 2013.

WinPython (with Spyder) and Weka are installed in our labs. Other free computational intelligence software are:

Stuttgart Neural Simulator

Computational Intelligence Library (CIlib)

Java Object Oriented Neural Engine (JOONE)

ECJ 20: A Java Based Evolutionary Computation and Genetic Programming Research System

LIBSVM – A Library for Support Vector Machines

Datasets for benchmarking:

UCI Machine Learning Repository

UCI Weka datasets in arff format

Free Additional Reading

Rojas, P. Neural Networks – A Systematic Introduction, Springer, 1996.

Hastie, T., Tibshirani, R., and Friedman, J. The Elements of Statistical Learning, Springer, 2009.

Course Schedule

Date

Topics

Reading

Assignments

4/2

Introduction

Syllabus

4/3

Preliminaries in Computational Intelligence, WinPython, WEKA

Ch 1

4/7

Overview of Neural Networks

Ch 2

4/9

Learning in NN

Ch 2

4/10

Learning in NN

Ch 2

Start HW 1 (graded)

4/14

The Perceptron Convergence Theorem

4/16

Multi-Layer Perceptron

Ch 3

HW 1 due

Start HW 2 (graded)

4/17

Multi-Layer Perceptron

Ch 3

4/21

Training, Testing, and Validation

Ch. 3

4/23

Data Pre-Processing, Regression, Universal Approximation, Classification, Time-Series Prediction

Ch. 3

HW 2 due

Start HW 3 (graded)

4/24

Unsupervised Learning: k-Means

Ch. 9

4/28

Unsupervised Learning  LVQ

Ch. 9

4/30

Unsupervised Learning: SOM

Ch. 9

5/1

Evolutionary Learning, Genetic Algorithms

Ch. 12

5/5

Genetic Algorithms, The Fundamental Theorem of Genetic Algorithms

Ch. 12

HW 3 due

Start HW 4 (graded)

5/7

Dimensionality Reduction & Feature Selection

Ch. 10

5/8

Dimensionality Reduction: PCA

Ch. 10

5/12

Radial Basis Function Network

Ch. 4

5/14

Support Vector Machines (SVM)

Ch. 5

LIBSVM

HW 4 due

Start HW 5 (graded)

5/15

Swarm Intelligence & Ant Colony Optimization

Dorigo and Parpinelli’s papers

5/19

Optimization and Search, Levenberg-Marquardt, Simulated Annealing

Ch. 11

5/21

Class cancelled

 

5/22

Class cancelled

 

 5/26

Memorial Day

No classes

Memorial Day

5/28

Probability and Learning: Naďve Bayes,, EM Algorithm, k-Nearest Neighbor

Ch. 8

HW 5 due

5/29

Fuzzy Systems

 

 

6/2

Hybrid Intelligent Systems, Expert Systems, Neuro-Fuzzy Systems

Cai & Kwan’s papers

6/4

Associative Memories: Hopfield NN & Boltzmann Machine

Meyder & Kiderlen’s paper

6/5

Project Presentations

6/9

Project Presentations: noon – 2:00

Laboratory Schedule

Date

Topics

Reading

Assignments

4/8

WinPython

Ch. 16

Use WinPython for Practice Questions (not graded)

4/15

Weka

Weka manual

Run Weka examples

(not graded)

4/22

Character Recognition

 

HW 2

4/29

Regression

 

HW 3

5/6

Regression

 

HW 3

5/13

GA Optimization

 

HW 4

5/20

GA for NN and Games

 

HW 5

5/27

Final Projects

 

6/3

Final Project

 

Honor Code: All work turned in for credit, including exams and all components of the project, are to be the work of the student whose name is on the exam or project. For all project components, the student can receive assistance from individuals other than the instructor only to ascertain the cause of errors. Thus you can get help if you need it to figure out why something doesn't work. You just can't get help from anyone, other than the instructor or TA, to figure out how to make something work. All solutions turned in for credit are to be your individual work and should demonstrate your problem solving skills, not someone else's. Help each other understand and debug the programming assignments. However, you should write the code for your programs yourself. Writing it yourself is the only way you will learn. Since everyone is writing their own code, no two programs should be the same or so similar that I could convert one to the other by a simple mechanical transformation (e.g. changing variable names and comments). I consider this plagiarism and a violation of academic code. The following text should appear on all assignments: “I pledge that I have neither given nor received help from anyone other than the instructor for all program components included here.

First violation: Students must meet with the instructor. In most cases, the grade will be split between the authors of the copied programs. Second violation: Students will receive no credit for the assignment, an incident letter will be placed on file in the Computer Science Department, and the matter referred to the Computer Science Department Chair.

Class Attendance: Class attendance is expected and recorded.

ADA Statement: Students with disabilities who wish to set up academic adjustment in this class should give me a copy of their "Confirmation of Eligibility for Academic Adjustment" from the Disability Support Services Office as soon as possible so we can discuss how the approved adjustment will be implemented in this class. Students without this form should contact the Disability Support Services Office, Buillon 205 or dssrecept@cwu.edu or 963-2171.

Caveat: The schedule and procedures for this course are subject to change. It is the student's responsibility to learn of and adjust to changes.