4.3 Parallel and Distributed Computing

1. Exam Points

Sequential computing vs. parallel computing.

• The efficiency of a parallel solution is still limited by the sequential portion.
• Optimize execution time.
  • Divide the workload of multiple processors as close to equal as possible.
• Compare the time taken by sequential computing and parallel computing for the same task.

2. Knowledge Points

(1) Sequential computing vs. Parallel computing

• Sequential computing is a computational model in which operations are performed in order one at a time. (one after another)
  • 任务严格按顺序执行，每个任务需等待前一个任务完成后才能开始，仅利用单个处理器资源。
• Parallel computing is a computational model where the program is broken into multiple smaller sequential computing operations, some of which are performed simultaneously.
  • 将大任务分解为子任务，通过多个处理器或核同时执行，实现任务重叠处理。
    
    
    
    
• Comparing efficiency of solutions can be done by comparing the time it takes them to perform the same task.
• A sequential computing solution takes as long as the sum of all of its steps.
  
  
  
  
• Parallel computing consists of a parallel portion and a sequential portion. So a parallel computing solution takes as long as its sequential tasks plus the longest of its parallel tasks.
  
  
• The ”speedup” of a parallel solution is measured in the time it took to complete the task sequentially divided by the time it took to complete the task when done in parallel. (加速比 是串行计算时间与并行计算时间的比值)
  
  • 𝐬𝐩𝐞𝐞𝐝𝐮𝐩=(𝐒𝐞𝐪𝐮𝐞𝐧𝐭𝐢𝐚𝐥 𝐭𝐢𝐦𝐞)/(𝐏𝐚𝐫𝐚𝐥𝐥𝐞𝐥 𝐭𝐢𝐦𝐞)
• The efficiency of a parallel solution is still limited by the sequential portion.

(2) Sequential computing vs. Parallel computing

• Distributed computing(分布式计算) is a computational model in which multiple devices are used to run a program.
  
  
• Distributed computing allows much larger problems to be solved quicker than they could be solved using a single computer.
• Divide the workload of multiple processors as close to equal as possible.
  • Example:
    
  • Solution:
    • (1). total time taken: 20+30+45+50 = 145
    • (2). divide the time by number of processors: 145/2 = 72.5
    • (3). divide tasks among two processors as equal as possible: X and Y (75), W and Z (70)

3. Exercises

• Practice
• Solution