Prerequisites: CS 302
Syllabus
CS 473: Parallel Computing
Fall 2008
|
Meeting
Times |
Lect: 12:00 - 12:50 MWF, HB 112 Labs: 12:00 - 12:50 Tu, HB 207 |
|
Instructor |
Dr. Razvan Andonie, HB 219-B, Office hours |
|
TA |
|
|
Text |
Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw Hill, 2004 |
Objectives
Parallel
computation is becoming pervasive in all levels of computing, from massively
parallel supercomputers used in large scale computational science, to
multiprocessor servers supporting transaction processing and the World Wide
Web. The major issues raised in each of the core areas of computer science
(e.g., algorithms, systems, languages, architecture, etc.) become even more
interesting when considered in the context of parallel computing. Hence, this
course challenges students to apply in a new context the concepts and tools
they have studied in earlier computer science courses. This hands-on course
will also introduce students to a topic of fundamental importance to a wide
variety of application areas.
Topics:
|
|
Motivations
for parallel processing |
|
|
|
Parallel
computer architectures |
|
|
|
Multicore programming with OpenMP |
Multicore
programming with OpenMP |
|
|
Message
passing programming with MPI |
|
|
|
Performance
analysis |
|
|
|
Fundamental
algorithms: backtracking, branch-and-bound, divide and conquer, sorting,
searching |
|
|
|
Applications:
data mining, artificial intelligence, computational intelligence, scientific
computing |
|
|
Student Learning Outcomes
On
completion of this course, the student will have:
|
|
Designed
and analyzed algorithms that execute efficiently on parallel computers |
|
|
Implemented
distributed programs using the Message Passing Interface (MPI) |
|
|
Implemented
multicore programs using OpenMP |
Lectures & Projects
The
slides for lectures, additional materials (including several MPI examples from
Quinn's textbook), and the projects can be found in the shared directory.
Grading
Exams (2 - 25% each)
50%
Lab Projects
30%
Final Project (Paper & Presentation)
20%
HWs will be not graded. Lab Project 6 is extra credit, 5% of the final grade.
Grade Distribution
95 - 100 A
90 - 94 A-
87 - 89 B+
83 - 86 B
80 - 82 B-
77 - 79 C+
73 - 76 C
70 - 72 C-
67 - 69 D+
63 - 66 D
60 - 62 D-
0 - 59 F
If
you must miss an exam, contact your instructor prior to the
exam to schedule a time to make it up. Late submission of assignments is
generally not accepted. No partial credit for late assignments will be offered.
Software
We
are going to use MPICH2 and OpenMP. MPICH2 is an implementation of the
Message-Passing Interface (MPI). MPICH2 is distributed as source (with an
open-source, freely available license), on several platforms, including Linux
and Windows.
|
·
Download MPICH.
Have a look at the MPICH
documents. |
|
|
·
Introduction to MPI,
is an online course created by the PACS Training Group; it can be downloaded
and printed. |
|
|
·
|
The
OpenMP C and C++ application program interface lets you write applications that
effectively use multiple processors. Visual C++ supports the OpenMP 2.0
standard. If you want to program in OpenMP under Windows, read this tutorial
from Microsoft: OpenMP and C++. More
information about OpenMP can be found at http://www.openmp.org/blog/resources/#Shareware/Freeware
and
http://www.llnl.gov/computing/tutorials/openMP/.
Other software for
multicore processors:
·
Microsoft Task Parallel
Library and Microsoft
Parallel Extensions to .NET Framework
·
C++CSP2 (University of Kent)
·
Intel Threading Building Blocks
·
OpenCL (Open Computing Language) the
first open, royalty-free standard for general-purpose parallel programming of
heterogeneous systems.
Links
|
Designing and Building Parallel
Programs, an online book by Ian Foster, Addison-Wesley, 1995. |
|
|
Parallel
Computing Developer Center. Microsoft’s Parallel Computing
Platform. |
|
|
Nan's Parallel
Computing Page. This list contains links related to parallel computing. |
|
Additional Reading
|
The
Landscape of Parallel Computing Research: A View from Berkeley. A very
important technical report. |
|
|
Wilkinson
B. and M. Allen, Parallel
Programming: Techniques and Applications Using Networked Workstations and
Parallel Computers, Prentice Hall, 2005. |
|
|
Gropp,
W., E. Lusk, and A. Skjellum, Using MPI: Portable Parallel Programming
with the Message-Passing Interface, MIT Press, 1999. |
|
|
Grama,
A., A. Gupta, G. Karypis, and V. Kumar, Introduction to Parallel Computing,
2nd ed., Addison-Wesley, 2003. |
|
|
Ghosh,
S. Distributed
Systems - An Algorithmic Approach, Chapman & Hall/CRC, 2007. An
excellent textbook on distributed systems, with an emphasis on algorithms. |
|
|
Chapman,
B., G. Jost, and R. V. D. Pas, Using OpenMP Portable Shared Memory
Parallel Programming, MIT Press, 2008. |
Final Projects
Final Project Guidelines
The
final project is to be completed in groups of two, or individually. The project
should contain the problem description, the code, the benchmarking, and a
discussion. For a good presentation, read Quinn's effective
presentation techniques guidelines.
Course Schedule
|
Date |
Topics |
Reading |
Assignments |
|
9/24 |
Introduction |
|
|
|
9/26 |
Motivation and History |
Ch. 1 |
|
|
9/29 |
Parallel Algorithm Design |
3.1-3.3 |
|
|
10/1 |
Parallel Algorithm Design: Boundary Value Problem,
Finding the Maximum |
3.4-3.5 |
HW 1: Ex. 3.12 |
|
10/3 |
Shared-Memory Programming with OpenMP |
Ch. 17 |
HW 1 due; HW 2: Ex. 3.13 |
|
10/6 |
Shared-Memory Programming with OpenMP |
Ch. 17 |
HW 2 due; HW 3: Ex. 3.14 |
|
10/8 |
Shared-Memory Programming with OpenMP |
Ch. 17 |
HW 3 due; HW 4: Ex. 3.15 |
|
10/10 |
The Task Parallel Library (TPL) and Microsoft's Parallel
Computing Platform (PCP) |
|
HW 4 due |
|
10/13 |
Message-Passing Programming |
Ch. 4 |
|
|
10/15 |
Message-Passing Programming |
Ch. 4 |
|
|
10/17 |
The Sieve of Eratosthenes |
Ch. 5 |
|
|
10/20 |
The Sieve of Eratosthenes |
Ch. 5 |
|
|
10/22 |
Floyd's Algorithm |
Ch. 6 |
HW 5: Ex. 6.1 |
|
10/24 |
Floyd's Algorithm; Discuss HW 5 |
Ch. 6 |
HW 5 due |
|
10/27 |
Exam I |
|
|
|
10/29 |
Performance Analysis |
Ch. 7 |
|
|
10/31 |
Matrix-Vector Multiplication |
Ch. 8 |
|
|
11/3 |
Matrix-Vector Multiplication |
Ch. 8 |
|
|
11/5 |
Document Classification |
Ch. 9 |
|
|
11/7 |
Document Classification, Discuss Final Projects |
Ch. 9 |
|
|
11/10 |
Combinatorial Search: Divide and Conquer, Backtracking |
Ch. 16 |
Final Project Proposal due |
|
11/12 |
Combinatorial Search: Branch and Bound |
Ch. 16 |
|
|
11/14 |
Combinatorial Search: Game Trees, Alpha-Beta Search |
Ch. 16 |
|
|
11/17 |
Sorting: Quicksort |
14.1-14.3 |
|
|
11/19 |
Sorting: Hyperquicksort, Parallel Sorting by Regular Sampling |
14.4-14.6 |
|
|
11/21 |
Monte Carlo Methods |
Ch. 10 |
|
|
11/24 |
Monte Carlo Methods |
Ch. 10 |
|
|
11/26 |
Thanksgiving |
No classes |
|
|
11/28 |
Thanksgiving |
No classes |
|
|
12/1 |
Monte Carlo Methods |
Ch. 10 |
|
|
12/3 |
Exam II |
|
|
|
12/5 |
Presentations 1: |
|
|
|
12/9 |
Presentations 2: 12:00 - 02:00 |
|
|
Laboratory Schedule
|
Date |
Topic |
Item
Due |
|
9/30 |
Setup |
|
|
10/7 |
Project 1: Ex. 17.12 |
|
|
10/14 |
Project 2 (on Neve-cs473) |
Project 1 due |
|
10/21 |
Project 3: Ex. 4.4. Use program SATI.c on Neve-cs473 |
Project 2 due |
|
10/28 |
Project 4: Ex. 4.8 |
Project 3 due |
|
11/4 |
Project 5: Ex 6.11 (a, c). Use program Sieve.c on Neve-cs473 |
Project 4 due |
|
11/11 |
Veterans Day |
No classes |
|
11/18 |
Project 6 (on Neve-cs473)
as extra credit, 5% of the final grade |
Project 5 due |
|
11/25 |
Final Project |
|
|
12/2 |
Final Project |
Project 6 due |
Honor Code
All work turned in for credit, including exams and all components of the project, are to be the work of the student whose name is on the exam or project. For all project components, the student can receive assistance from individuals other than the instructor only to ascertain the cause of errors. Thus you can get help if you need it to figure out why something doesn't work. You just can't get help from anyone, other than the instructor or TA, to figure out how to make something work. All solutions turned in for credit are to be your individual work and should demonstrate your problem solving skills, not someone else's. The following text should appear on all assignments: " I pledge that I have neither given nor received help from anyone other than the instructor for all program components included here."
Help each other with the lab assignments. You may work together, but do the work on separate computers. The point is to understand the material enough so that you to go on and complete that week's programming assignment on your own.
Help each other understand and debug the programming assignments. However, you should write the code for your programs yourself. Writing it yourself is the only way you will learn. Do not work together to solve the programming assignments to the extent that two programs are essentially the same solution.
All program solutions turned in for credit are to be your individual
work and should demonstrate your problem solving skills, not someone else's.
Since everyone is writing their own code, no two programs should be the same or
so similar that I could convert one to the other by a simple mechanical
transformation (e.g. changing variable names and comments). I consider this
plagiarism and a violation of academic code. First violation: Students must
meet with the instructor. In most cases, the grade will be split between the
authors of the copied programs. Second violation: Students will receive no
credit for the assignment. An incident letter will be placed on file in the
Computer Science Department and the matter referred to the Computer Science
Department Chair.
Class Attendance
Class
attendance is expected and recorded.
ADA
Students
with disabilities who wish to set up academic adjustment in this class should
give me a copy of their "Confirmation of Eligibility for Academic
Adjustment" from the Disability Support Services Office as soon as
possible so we can discuss how the approved adjustment will be implemented in
this class. Students without this form should contact the Disability Support
Services Office, Buillon 205 or dssrecept@cwu.edu
or 963-2171.
Caveat
The
schedule and procedures for this course are subject to change. It is the
student's responsibility to learn of and adjust to changes.