Skip Navigation

AMS 530, Principles of Parallel Computing 
This course is designed for both academic and industrial scientists interested in parallel computing and its applications to large-scale scientific and engineering problems. It focuses on the three main issues in parallel computing: analysis of parallel hardware and software systems, design and implementation of parallel algorithms, and applications of parallel computing to selected problems in physical science and engineering. The course emphasizes hands-on practice and understanding of algorithmic concepts of parallel computing.

Prerequisite:  A course in basic computer science such as operating systems or architecture or some programming experience.
3 credits, ABCF grading 

Offered in Fall semester only.

Text: Professor Deng's Lecture Notes and Handouts

Recommended/Optional Text: "Introduction to Parallel Computing" by Ananth Grama, Anshul Gupta, George Karypis, and Vipin Kumar, 2nd edition, Pearson Publishing, 2003; ISBN: 978-0-201-64865-2


Learning Outcomes:

1) Demonstrate knowledge of supercomputer architectures:
      * Node architecture, central processing units, and accelerators;
      * Distributed- and shared-memory;
      * Various interconnection networks;
      * Network architectures and performance;
      * IO subsystem.

2) Demonstrate mastery of software architecture and MPI:
      * Communication patterns and protocols;
      * Message passing interface (MPI);
      * Process creation and management;
      * Single-sided communications;
      * Collective operations.

3) Demonstrate understanding of performance measurements:
      * Speedup, parallel efficiency, overhead, load imbalance ratio;
      * Amdahl’s law.

4) Demonstrate mastery of parallel algorithms:
      * Algorithm classification according to complexities;
      * Problem decomposition;
      * Communication handling;
      * Amdahl’s law.

5) Demonstrate mastery of applications of parallel computers:
      * Linear algebra: matrix multiplication, solution of AX=b, eigenvalues and eigenvectors;
      * Basic calculus including high-dimensional integration;
      * Solutions of ODEs and systems including molecular dynamics;
      * Solutions of a few PDES including wave, diffusion, and Poison equations;
      * Fast Fourier transforms;
      * Markov chain Monte Carlo methods.