Learning objectives:

In this lecture, you will learn:

• the basics of parallelism (in Python).
• how to calculate speedup, the definition of strong scaling and weak scaling.
• task-level parallelism in Dask.
• the basics of advanced Python data science tool Cylon.
• the performance benefits of Cylon on HPC (high-performance computing) infrastructure.

Lecture slides

• Lec5a: Parallelism in Python: slides pdf, slides+notes
• Lec5b: Cylon: slides

Readings

• Lec5a: Dask.distributed (optional), Python multiprocessing (optional)
• Lec5b: Cylon documentation (optional)

Quizzes

• Quiz 3: link

Lecture videos

• Lec5a: video
• Lec5b: video