Prerequisites: CS 302

Syllabus

CS 473: Parallel Computing

Fall 2007

Meeting Times

Lect: 12:00 - 12:50 MWF, HB 112

Labs: 12:00 - 12:50 Tu, HB 207

Instructor

Dr. Razvan Andonie, HB 219-B, Office hours

TA Jonathan Thorsvik

Text

Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw Hill, 2004

 

 

Objectives

Parallel computation is becoming pervasive in all levels of computing, from massively parallel supercomputers used in large scale computational science, to multiprocessor servers supporting transaction processing and the World Wide Web. The major issues raised in each of the core areas of computer science (e.g., algorithms, systems, languages, architecture, etc.) become even more interesting when considered in the context of parallel computing. Hence, this course challenges students to apply in a new context the concepts and tools they have studied in earlier computer science courses. This hands-on course will also introduce students to a topic of fundamental importance to a wide variety of application areas.

Topics:

bullet Motivations for parallel processing
bullet Parallel computer architectures
bullet Parallel algorithm design and Foster's methodology
bullet Message passing programming with MPI
bullet Performance analysis
bullet Fundamental algorithms: backtracking, branch-and-bound, divide and conquer, sorting, searching
bullet Applications: data mining, artificial intelligence, computational intelligence, scientific computing
bullet Shared-memory programming with OpenMP

Student Learning Outcomes

On completion of this course, the student will have:
bullet Designed and analyzed algorithms that execute efficiently on parallel computers.
bullet Applied this knowledge to problems in a range of application areas.
bullet Implemented parallel programs in C, using the Message Passing Interface (MPI) and its MPICH2 implementation.
bullet Understand the shared-memory model and using OpenMP.

Lectures & Projects

The slides for lectures, additional materials (including several MPI examples from Quinn's textbook), and the projects can be found in the shared directory.

Grading

Exams (2 - 25% each)

50%

Lab Projects 1-3 (10% each) 30%

Final Project (Paper & Presentation)

20%

 

HWs will be not graded. Lab Project 4 is extra credit, 5% of the final grade.

Grade Distribution

95 - 100   A

90 - 94     A-

87 - 89     B+

83 - 86     B

80 - 82     B-

77 - 79     C+

73 - 76     C

70 - 72     C-

67 - 69     D+

63 - 66     D

60 - 62     D-

0  - 59      F

 

If you must miss an exam, contact your instructor prior to the exam to schedule a time to make it up. Late submission of assignments is generally not accepted. No partial credit for late assignments will be offered.

Software

We are going to use MPICH2, an implementation of the Message-Passing Interface (MPI). MPICH2 is distributed as source (with an open-source, freely available license), on several platforms, including Linux and Windows.

bullet Have a look at the MPICH2 Installer's Guide. If you have trouble installing or using MPICH2, first see the FAQ page. If you have trouble with the Windows version of MPICH2, see the tips for installing and running MPICH2 on Windows XP, or see the MPICH2 Windows Development Guide.
bullet Introduction to MPI, is an online course created by the PACS Training Group. It can be downloaded and printed.
bullet http://www.mpi-forum.org/ is the MPI Forum.

The OpenMP C and C++ application program interface lets you write applications that effectively use multiple processors. Visual C++ supports the OpenMP 2.0 standard. If you want to program in OpenMP under Windows, read this tutorial from Microsoft: OpenMP and C++. More information about OpenMP can be found at http://www.openmp.org/blog/resources/#Shareware/Freeware and http://www.llnl.gov/computing/tutorials/openMP/.

Links

bullet Designing and Building Parallel Programs, an online book by Ian Foster, Addison-Wesley, 1995.
bullet Nan's Parallel Computing Page. This list contains links related to parallel computing.
bullet Top 500 Supercomputer sites.

Additional Reading

bullet The Landscape of Parallel Computing Research: A View from Berkeley. A very important technical report.
bullet Wilkinson B. and M. Allen, Parallel Programming: Techniques and Applications Using Networked Workstations and Parallel Computers, Prentice Hall, 2005.
bullet Gropp, W., E. Lusk, and A. Skjellum, Using MPI: Portable Parallel Programming with the Message-Passing Interface, MIT Press, 1999.
bullet Grama, A., A. Gupta, G. Karypis, and V. Kumar, Introduction to Parallel Computing, 2nd ed., Addison-Wesley, 2003.
bullet Ghosh, S. Distributed Systems - An Algorithmic Approach, Chapman & Hall/CRC, 2007. An excellent textbook on distributed systems, with an emphasis on algorithms.
bullet Chapman, B., G. Jost, and R. V. D. Pas, Using OpenMP Portable Shared Memory Parallel Programming, MIT Press, 2008.

Final Projects

Final Project Guidelines

The final project is to be completed in groups of two, or individually. The project should contain the problem description, the code, the benchmarking, and a discussion. For a good presentation, read Quinn's effective presentation techniques guidelines.

Grading criteria: 10% Difficulty of the project, 50% Submitted document, 40% Presentation.
 
Projects:
Ex.  5.11 (Harmonic Progression in OpenMP) - Sorin Bucse
Ex.  5.11 (Harmonic Progression in MPI) - Christopher Stearns
Ex. 4.12 (Numerical Integration in MPI) - Abraham Dybvig
Ex. 4.10 (Student ID Shop in MPI)  - Jason White
Ex. 6.12 (Sunlight on Map in OpenMP) - Michael Wilson

Course Schedule

Date

 Topics Reading Assignments
9/19 Introduction Syllabus  
9/21 Motivation and History Ch. 1  
9/24 Parallel Algorithm Design 3.1-3.3  
9/26 Parallel Algorithm Design: Boundary Value Problem 3.4  
9/28 Parallel Algorithm Design: Finding the Maximum 3.5 HW 1: Ex. 3.12
10/1 Message-Passing Programming; Discuss HW 1 Ch. 4 HW 1 due

HW 2: Ex. 3.13

10/3 Message-Passing Programming; Discuss HW 2 Ch. 4 HW 2 due

HW 3: Ex. 3.14

10/5 The Sieve of Eratosthenes; Discuss HW 3 Ch. 5 HW 3 due

HW 4: Ex. 3.15

10/8 The Sieve of Eratosthenes; Discuss HW 4 Ch. 5 HW 4 due
10/10 Floyd's Algorithm Ch. 6 HW 5: Ex. 6.1
10/12 Discuss HW 5

Floyd's Algorithm

 Ch. 6 HW 5 due
10/15 Performance Analysis Ch. 7  
10/17 Exam I    
10/19 Matrix-Vector Multiplication Ch. 8  
10/22 Matrix-Vector Multiplication Ch. 8  
10/24 Document Classification Ch. 9  
10/26 Document Classification, Discuss Final Projects Ch. 9  
10/29 Combinatorial Search: Divide and Conquer, Backtracking Ch. 16 Final Project Proposal due
10/31 Combinatorial Search: Branch and Bound Ch. 16  
11/2 class canceled    
11/5 Combinatorial Search: Game Trees, Alpha-Beta Search Ch. 16  
11/7 Sorting: Quicksort 14.1-14.3  
11/9 Sorting: Hyperquicksort, Parallel Sorting by Regular Sampling 14.4-14.6  
11/12 Veterans day    
11/14 Shared-Memory Programming with OpenMP Ch. 17  
11/16 Shared-Memory Programming with OpenMP Ch. 17  
11/19 Q & A for Exam II, All Questions Will Be Answered    
11/21 Thanksgiving    
11/23 Thanksgiving    
11/26 Exam II    
11/28 Presentations 1:
Numerical Integration (MPI) - Abraham Dybvig
Sunlight on Map (OpenMP) - Michael Wilson
    
11/30 Presentations 2:

Harmonic Progression (OpenMP) - Sorin Bucse

Harmonic Progression (MPI) - Christopher Stearns

Student ID Shop (MPI) - Jason White

    
12/7 Presentations 3: 12:00 - 02:00    

Laboratory Schedule

Date

Topic

Item Due

9/25 Setup  
10/2 Project 0 (on Neve-cs473)  
10/9 Project 1: Ex. 4.4. Use program SATI.c on Neve-cs473 Project 0 due
10/16 Project 2: Ex. 4.8 Project 1 due
10/23 Project 3: Ex 6.11 (a, c). Use program Sieve.c on Neve-cs473 Project 2 due
10/30 Project 4 (on Neve-cs473) as extra credit, 5% of the final grade Project 3 due
11/6 Project 4 Project 4 due
11/13 Final Project  
11/20 Final Project  
11/27 Final Project  

Honor Code

All work turned in for credit, including exams and all components of the project, are to be the work of the student whose name is on the exam or project. For all project components, the student can receive assistance from individuals other than the instructor only to ascertain the cause of errors. Thus you can get help if you need it to figure out why something doesn't work. You just can't get help from anyone, other than the instructor or TA, to figure out how to make something work. All solutions turned in for credit are to be your individual work and should demonstrate your problem solving skills, not someone else's. The following text should appear on all assignments: " I pledge that I have neither given nor received help from anyone other than the instructor for all program components included here."

Help each other with the lab assignments. You may work together, but do the work on separate computers. The point is to understand the material enough so that you to go on and complete that week's programming assignment on your own.

Help each other understand and debug the programming assignments. However, you should write the code for your programs yourself. Writing it yourself is the only way you will learn. Do not work together to solve the programming assignments to the extent that two programs are essentially the same solution.

All program solutions turned in for credit are to be your individual work and should demonstrate your problem solving skills, not someone else's. Since everyone is writing their own code, no two programs should be the same or so similar that I could convert one to the other by a simple mechanical transformation (e.g. changing variable names and comments). I consider this plagiarism and a violation of academic code. First violation: Students must meet with the instructor. In most cases, the grade will be split between the authors of the copied programs. Second violation: Students will receive no credit for the assignment. An incident letter will be placed on file in the Computer Science Department and the matter referred to the Computer Science Department Chair.

Class Attendance

Class attendance is expected and recorded.

ADA Statement

Students with disabilities who wish to set up academic adjustment in this class should give me a copy of their "Confirmation of Eligibility for Academic Adjustment" from the Disability Support Services Office as soon as possible so we can discuss how the approved adjustment will be implemented in this class. Students without this form should contact the Disability Support Services Office, Buillon 205 or dssrecept@cwu.edu or 963-2171.

Caveat

The schedule and procedures for this course are subject to change. It is the student's responsibility to learn of and adjust to changes.