Learning objectives:

In this lecture, you will learn:

• Lec7a: the design of HDFS and GFS
• Lec7a: the techniques of data partitioning, replication, and motivation behind them
• Lec7a: role clients, NameNode, DataNodes play for distributed file system reads and writes
• Lec7b: role mappers and reducers have in MapReduce jobs
• Lec7b: example MapReduce workflows
• Lec7b: techniques to tackle the challenges of stragglers

Lecture slides

• Lec7a: GFS and HDFS: slides pdf
• Lec7b: Hadoop MapReduce: slides pdf

Recordings

• Lec7a: video
• Lec7b: video
• Lec7c (performance result discussion): video