1. 尝试一个计算量更大的任务，比如大矩阵的乘法，看看 CPU 和 GPU 之间的速度差异。再试一个计算量很小的任务呢？

import torch
from d2l import torch as d2l

# 设置随机种子以获得可复现的结果
torch.manual_seed(0)

# 大矩阵乘法
n = 10000
large_matrix1 = torch.randn(n, n, device='cpu')
large_matrix2 = torch.randn(n, n, device='cpu')

# 在 GPU 上进行矩阵乘法
timer = d2l.Timer()
result_large = torch.mm(large_matrix1, large_matrix2)
print(f"Large matrix multiplication on CPU took: {timer.stop():.5f} sec")

timer = d2l.Timer()
result_small = large_matrix1.sum()
print(f"Small matrix sum on CPU took: {timer.stop():.5f} sec")

# 移动到 GPU
large_matrix1 = large_matrix1.to('cuda')
large_matrix2 = large_matrix2.to('cuda')

# 在 GPU 上进行矩阵乘法
timer = d2l.Timer()
result_large = torch.mm(large_matrix1, large_matrix2)
print(f"Large matrix multiplication on GPU took: {timer.stop():.5f} sec")

timer = d2l.Timer()
result_small = large_matrix1.sum()
print(f"Small matrix sum on GPU took: {timer.stop():.5f} sec")

Large matrix multiplication on CPU took: 2.43366 sec
Small matrix sum on CPU took: 0.00916 sec
Large matrix multiplication on GPU took: 0.01598 sec
Small matrix sum on GPU took: 0.05159 sec

2. 我们应该如何在 GPU 上读写模型参数？

在深度学习中，模型参数通常存储在张量（tensor）中，这些张量可以位于 CPU 或 GPU 上。当你在 GPU 上训练模型时，通常希望模型参数也位于 GPU 上，以减少数据在 CPU 和 GPU 之间的传输。以下是如何在 GPU 上读写模型参数的一般步骤：

将模型参数移动到 GPU

1. 模型定义：首先，定义你的模型。如果你使用的是 PyTorch，模型应继承自 torch.nn.Module。

import torch.nn as nn

class MyModel(nn.Module):
    def __init__(self):
        super(MyModel, self).__init__()
        self.weight = nn.Parameter(torch.randn(10, 10))
        self.bias = nn.Parameter(torch.randn(10))

    def forward(self, x):
        return x.mm(self.weight) + self.bias

2. 模型实例化：创建模型的实例。

model = MyModel()

3. 移动到 GPU：使用 .to(device) 方法将模型参数移动到 GPU 上。

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
model.to(device)

在 GPU 上训练模型

1. 数据移动到 GPU：确保输入数据也在 GPU 上。

input_data = torch.randn(1, 10).to(device)
target = torch.randn(1, 10).to(device)

2. 执行前向传播：在 GPU 上执行模型的前向传播。

output = model(input_data)

3. 执行反向传播和优化：在 GPU 上执行损失函数的反向传播，并更新模型参数。

optimizer = torch.optim.SGD(model.parameters(), lr=0.01)

loss_fn = nn.MSELoss()
loss = loss_fn(output, target)

# 反向传播
loss.backward()

# 更新参数
optimizer.step()

保存和加载 GPU 上的模型参数

1. 保存模型参数：在训练完成后，你可以仅保存模型的参数，而不是整个模型。

torch.save(model.state_dict(), 'model_parameters.pth')

2. 加载模型参数：加载模型参数时，首先定义模型结构，然后将参数加载到模型的参数张量中。

model_loaded = MyModel()
model_loaded.load_state_dict(torch.load('model_parameters.pth'))

3. 将参数移动到 GPU：如果你在 GPU 上训练模型，加载参数后，将它们移动到 GPU 上。

model_loaded.to(device)

注意事项

• 模型参数的读写操作通常与设备无关，即参数张量被设计为可以在 CPU 和 GPU 之间无缝移动。
• 在训练过程中，模型参数的更新（如梯度清零）应该在参数所在的设备上执行。
• 一些框架（如 TensorFlow）可能需要你明确指定操作的设备上下文。

通过上述步骤，你可以有效地在 GPU 上进行模型参数的读写操作，从而优化模型训练和推理的性能。

3. 测量计算 1000 个 $100 \times 100$ 矩阵的矩阵乘法所需的时间，并记录输出矩阵的 Frobenius 范数，一次记录一个结果，而不是在 GPU 上保存日志并仅传输最终结果。

import torch
import time

# 确保CUDA可用
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
print(f"device: {device}")

# 创建1000个100x100的随机矩阵对，初始化在GPU上
matrix1 = torch.randn(1000, 100, 100, device=device)
matrix2 = torch.randn(1000, 100, 100, device=device)

# 用于存储Frobenius范数的列表
frobenius_norms = []

# 记录开始时间
start_time = time.time()

# 对每一对矩阵执行矩阵乘法并计算Frobenius范数
for i in range(1000):
    result = torch.matmul(matrix1[i], matrix2[i])
    # 计算Frobenius范数并将其添加到列表中
    frobenius_norms.append(torch.norm(result, p='fro'))

# 记录结束时间
end_time = time.time()

# 计算总时间
total_time = end_time - start_time

# 将Frobenius范数列表移动到CPU，以便可以轻松处理
frobenius_norms = [x.item() for x in frobenius_norms]

# 打印结果
print(f"Total time for matrix multiplications and Frobenius norm calculations: {total_time:.4f} seconds")
print(f"Frobenius norms of the resulting matrices: {frobenius_norms}")

device: cuda
Total time for matrix multiplications and Frobenius norm calculations: 0.0146 seconds
Frobenius norms of the resulting matrices: [1017.1538696289062, 994.1005249023438, 1019.8182373046875, 976.5487060546875, 996.4542236328125, 1010.9757080078125, 990.4861450195312, 1003.8755493164062, 1006.1589965820312, 994.14697265625, 992.165771484375, 994.4583740234375, 1008.2112426757812, 1001.319580078125, 1007.4982299804688, 1000.4361572265625, 996.3579711914062, 1011.021728515625, 991.4567260742188, 991.1979370117188, 1012.6778564453125, 1007.9273071289062, 996.900634765625, 1005.5061645507812, 1018.226318359375, 1004.3720703125, 1003.2996826171875, 998.1599731445312, 993.7078247070312, 987.1924438476562, 977.4632568359375, 1003.8605346679688, 1015.5083618164062, 1014.2286987304688, 981.67041015625, 1007.2933959960938, 997.6986083984375, 996.698486328125, 995.5963134765625, 990.97119140625, 1038.184814453125, 989.3598022460938, 1010.7578735351562, 1011.5593872070312, 989.8652954101562, 1035.0028076171875, 1016.3458251953125, 1010.9376220703125, 992.2198486328125, 1007.4716796875, 996.9808959960938, 987.9463500976562, 1004.7304077148438, 1000.767578125, 1017.2183227539062, 1017.29296875, 1006.6596069335938, 988.8013916015625, 999.884033203125, 991.9771728515625, 1001.7670288085938, 996.5944213867188, 994.8685913085938, 1001.1255493164062, 976.5785522460938, 1009.4751586914062, 994.654052734375, 1004.7172241210938, 997.4708251953125, 1012.830810546875, 1020.0433349609375, 988.2201538085938, 1025.104736328125, 1015.5918579101562, 1005.8145141601562, 1008.4390869140625, 1024.517822265625, 994.025390625, 976.8499755859375, 1009.3927001953125, 994.1780395507812, 977.0374145507812, 999.4506225585938, 990.5545654296875, 1010.7803955078125, 1006.278564453125, 1015.1917724609375, 998.611083984375, 1016.217529296875, 1005.0156860351562, 991.4480590820312, 968.5922241210938, 980.4558715820312, 992.5218505859375, 990.74462890625, 1011.4117431640625, 986.0603637695312, 1004.5045166015625, 993.8551635742188, 991.429443359375, 1005.6165161132812, 980.9917602539062, 1005.3969116210938, 977.3096313476562, 1008.6134033203125, 1000.0128173828125, 993.11376953125, 1022.4129028320312, 995.9287109375, 1004.7772216796875, 1007.7603759765625, 993.8242797851562, 1006.2771606445312, 1005.294189453125, 1001.5660400390625, 997.7781372070312, 1002.1535034179688, 1004.683837890625, 990.1777954101562, 997.8068237304688, 1004.189208984375, 998.3909301757812, 994.8164672851562, 998.0253295898438, 998.7249755859375, 999.7866821289062, 1013.2652587890625, 990.194091796875, 1014.750244140625, 1006.913330078125, 976.1336059570312, 1005.6521606445312, 995.1724853515625, 1006.66650390625, 995.488037109375, 987.5828247070312, 996.7553100585938, 987.3020629882812, 1016.7817993164062, 1001.7072143554688, 1013.8600463867188, 992.6307983398438, 998.2688598632812, 1017.1278076171875, 1014.210205078125, 1007.6275024414062, 1021.6859130859375, 1018.8344116210938, 994.6874389648438, 991.829345703125, 990.2803955078125, 1009.4072875976562, 1006.0741577148438, 1009.436279296875, 1004.186279296875, 998.511962890625, 1004.43115234375, 1005.3590087890625, 1008.79345703125, 998.5278930664062, 994.6074829101562, 996.15576171875, 1003.9343872070312, 999.5142211914062, 1020.6968383789062, 997.8656005859375, 997.2040405273438, 976.3314208984375, 1017.5911865234375, 1006.562255859375, 984.9722290039062, 994.8170776367188, 975.072021484375, 980.6249389648438, 1001.7012939453125, 975.3551025390625, 1011.7542114257812, 995.9155883789062, 1006.9625854492188, 1014.5380859375, 1015.8518676757812, 995.9840087890625, 987.3629760742188, 1007.1809692382812, 997.3641357421875, 1019.2618408203125, 983.763427734375, 979.8612060546875, 998.4417724609375, 1002.5838623046875, 992.8643798828125, 1009.117431640625, 999.3661499023438, 1001.5335083007812, 1019.9609985351562, 1008.0535888671875, 993.7745971679688, 997.3470458984375, 986.8492431640625, 1004.9653930664062, 997.8578491210938, 981.529296875, 1003.0723876953125, 986.3853759765625, 1011.6820068359375, 1000.0419311523438, 1012.1671142578125, 1014.9698486328125, 1001.5011596679688, 978.705078125, 1022.607666015625, 997.5687255859375, 1009.934814453125, 1021.7277221679688, 983.0974731445312, 1005.9005126953125, 1003.8726806640625, 988.6663818359375, 1024.531982421875, 994.1163330078125, 981.8489379882812, 1002.9684448242188, 994.7283325195312, 999.5060424804688, 982.1412963867188, 991.4252319335938, 1008.5546875, 990.4173583984375, 1008.5194091796875, 993.3617553710938, 1010.8427734375, 984.3529663085938, 974.7801513671875, 1025.4947509765625, 997.467529296875, 1003.7056274414062, 989.321533203125, 990.1168212890625, 998.4745483398438, 981.8483276367188, 1005.19189453125, 993.2596435546875, 1021.0465087890625, 1007.7028198242188, 1014.3692016601562, 999.032958984375, 1001.57421875, 991.9310913085938, 1003.470703125, 982.6017456054688, 1003.8162841796875, 1007.5313110351562, 993.982421875, 991.155517578125, 1004.3564453125, 999.1619262695312, 991.8817138671875, 1009.2344970703125, 974.2422485351562, 1014.5386962890625, 999.5076904296875, 980.1727294921875, 1000.6102905273438, 985.994873046875, 993.826904296875, 1000.2212524414062, 991.1804809570312, 1018.3226928710938, 1014.4978637695312, 994.2145385742188, 1004.8660278320312, 998.8656005859375, 1031.9560546875, 998.6813354492188, 980.3363037109375, 983.2780151367188, 992.4225463867188, 1002.630615234375, 1017.9630737304688, 1024.265625, 962.2919311523438, 1001.004638671875, 1004.7459716796875, 1008.5996704101562, 1029.8304443359375, 1008.9427490234375, 1009.7600708007812, 1007.1852416992188, 1019.0254516601562, 996.2719116210938, 999.2525634765625, 985.1588134765625, 998.2060546875, 995.5961303710938, 980.009765625, 1001.0667114257812, 1014.6389770507812, 1009.5944213867188, 1015.60546875, 1010.9227294921875, 994.2400512695312, 988.8575439453125, 989.6468505859375, 997.59423828125, 1004.416748046875, 992.8645629882812, 995.4790649414062, 986.2066040039062, 998.3768920898438, 989.628173828125, 998.6106567382812, 996.2763061523438, 989.816650390625, 992.7864379882812, 964.741943359375, 973.3789672851562, 997.0577392578125, 1008.130615234375, 1001.5797729492188, 1018.9520874023438, 1015.1868286132812, 989.1287231445312, 1016.8582763671875, 990.3129272460938, 989.240966796875, 1011.3787231445312, 993.0394287109375, 1013.4398803710938, 1002.4822998046875, 1017.6884155273438, 1004.870361328125, 983.4959716796875, 996.5506591796875, 1019.6244506835938, 1017.6296997070312, 984.5091552734375, 1013.66552734375, 991.1429443359375, 993.355224609375, 989.1423950195312, 998.6246337890625, 1003.5924682617188, 991.9061279296875, 991.92333984375, 996.449462890625, 985.0617065429688, 994.1025390625, 987.5969848632812, 1031.68896484375, 986.2462158203125, 1031.3001708984375, 1001.2178344726562, 999.8604125976562, 988.010009765625, 1016.8370361328125, 988.6778564453125, 1005.852294921875, 991.8128662109375, 1012.0091552734375, 1009.7847290039062, 993.5637817382812, 997.8951416015625, 1005.423095703125, 983.814697265625, 1008.6759033203125, 984.1990966796875, 997.8103637695312, 976.6843872070312, 993.799560546875, 1003.0878295898438, 1001.5765991210938, 1012.6994018554688, 983.37744140625, 995.6679077148438, 1002.9580688476562, 998.343994140625, 999.72509765625, 977.8327026367188, 1018.3513793945312, 990.0205078125, 996.9105224609375, 1029.3431396484375, 1016.8467407226562, 1016.0354614257812, 1015.9344482421875, 993.7523193359375, 988.3599243164062, 1003.4122314453125, 999.7429809570312, 992.173095703125, 995.5195922851562, 995.2606201171875, 1007.192138671875, 1002.9076538085938, 997.5140380859375, 1023.8084106445312, 987.1675415039062, 1002.8834228515625, 1000.4762573242188, 1003.8207397460938, 1009.1409301757812, 1010.0889282226562, 998.41015625, 1001.1316528320312, 986.6473999023438, 1001.3159790039062, 996.1746826171875, 1011.8079833984375, 1005.3672485351562, 988.2734375, 992.1544799804688, 993.488037109375, 996.0311279296875, 998.5387573242188, 1010.0753784179688, 980.9888916015625, 1006.151123046875, 1001.0821533203125, 989.3228759765625, 1003.57568359375, 1015.8931274414062, 995.1639404296875, 984.8231811523438, 989.4175415039062, 1006.7691650390625, 1021.3506469726562, 1015.4465942382812, 1005.9798583984375, 1002.4998168945312, 991.3881225585938, 997.1692504882812, 1007.5187377929688, 1012.3572387695312, 982.7108764648438, 977.5001220703125, 983.7129516601562, 975.0503540039062, 1011.7088623046875, 988.0769653320312, 993.4935302734375, 1016.8427734375, 1017.7904052734375, 997.53662109375, 1006.3212280273438, 987.6399536132812, 996.6499633789062, 1011.7398681640625, 998.1097412109375, 1004.9158935546875, 999.6876831054688, 998.29541015625, 1036.3048095703125, 988.8720703125, 994.7421264648438, 998.8275146484375, 1019.382568359375, 991.701416015625, 982.040283203125, 1014.2590942382812, 993.144775390625, 976.3253173828125, 1000.1658935546875, 981.36669921875, 1015.8656616210938, 1006.6417236328125, 992.7757568359375, 1004.2478637695312, 996.5673217773438, 997.4274291992188, 1001.4227294921875, 1018.118408203125, 990.2199096679688, 1013.7266845703125, 1001.2011108398438, 1014.1185913085938, 1003.2681884765625, 1016.0635986328125, 991.3792114257812, 999.2403564453125, 1006.2503051757812, 1001.717041015625, 991.1873168945312, 991.2544555664062, 1007.7902221679688, 973.7755737304688, 985.4977416992188, 997.87158203125, 1014.0445556640625, 978.621826171875, 1018.7953491210938, 1027.6097412109375, 1022.4765625, 1011.3067016601562, 990.0903930664062, 970.2365112304688, 994.1051635742188, 1003.696533203125, 986.0433349609375, 1007.4765014648438, 1014.7648315429688, 996.7695922851562, 992.3809814453125, 996.030029296875, 996.125, 980.6527099609375, 1010.3901977539062, 1004.5905151367188, 1002.3295288085938, 988.86376953125, 998.2445068359375, 1005.1199951171875, 992.934326171875, 995.6838989257812, 1021.2647094726562, 985.9216918945312, 1008.332763671875, 1013.6148071289062, 1002.9952392578125, 992.5791625976562, 1011.8684692382812, 1003.1002807617188, 994.660888671875, 1008.9127197265625, 1008.8080444335938, 1007.1912841796875, 989.1290893554688, 1013.9869995117188, 1014.501220703125, 998.8324584960938, 997.342529296875, 1001.359375, 983.9628295898438, 987.073486328125, 1021.5632934570312, 991.6016235351562, 1031.422119140625, 1002.3307495117188, 986.7777099609375, 986.7025756835938, 986.3648681640625, 1017.1181640625, 991.9849243164062, 987.3490600585938, 1007.8807373046875, 996.8131103515625, 1002.5682373046875, 990.834228515625, 991.8912353515625, 996.4434814453125, 1018.7001342773438, 1002.732666015625, 988.046630859375, 995.3675537109375, 1023.993896484375, 994.9285278320312, 1013.148681640625, 993.8716430664062, 1018.495849609375, 1018.9038696289062, 999.3428344726562, 997.2947387695312, 1029.970703125, 967.7144775390625, 1011.4215698242188, 1019.0669555664062, 993.355712890625, 996.072998046875, 1009.591796875, 995.5679931640625, 979.3378295898438, 997.0065307617188, 1005.0748291015625, 1001.5053100585938, 994.4246215820312, 1017.1673583984375, 1004.0501708984375, 979.8358764648438, 978.275390625, 993.575439453125, 994.7704467773438, 998.245849609375, 999.3946533203125, 993.5028686523438, 1014.3831176757812, 999.490478515625, 1002.2607421875, 1001.8602905273438, 981.6064453125, 994.5763549804688, 978.0537719726562, 998.3612670898438, 982.6378784179688, 1012.7417602539062, 1009.4373168945312, 1012.105712890625, 990.2240600585938, 1009.6126098632812, 981.4917602539062, 1005.4188842773438, 993.2816162109375, 975.3316650390625, 1002.7220458984375, 1004.0006103515625, 979.2216186523438, 995.943603515625, 1002.7631225585938, 1009.9290771484375, 959.9613037109375, 978.8505859375, 999.0321044921875, 993.12158203125, 972.775390625, 980.838134765625, 1018.1771850585938, 998.0579833984375, 991.602783203125, 1012.2890625, 998.3921508789062, 1014.4545288085938, 981.2846069335938, 1008.483154296875, 1004.5053100585938, 1010.8132934570312, 969.6649780273438, 992.6070556640625, 1015.7015991210938, 1001.8156127929688, 987.6817626953125, 1007.424560546875, 1010.6760864257812, 993.4402465820312, 1011.1005249023438, 1003.450439453125, 1002.56201171875, 1012.0348510742188, 995.1197509765625, 1013.912841796875, 988.6197509765625, 1016.8399658203125, 999.9243774414062, 999.4819946289062, 1003.8712158203125, 1011.9285278320312, 999.3246459960938, 988.5142211914062, 993.3571166992188, 982.6808471679688, 1004.2242431640625, 1008.1665649414062, 992.7423095703125, 1017.7481689453125, 1003.2393188476562, 985.1275024414062, 986.0277099609375, 996.2783203125, 983.9967041015625, 993.2989501953125, 992.5768432617188, 1002.9754028320312, 980.5664672851562, 997.4368896484375, 1003.553466796875, 974.1748657226562, 984.43896484375, 1000.6251220703125, 995.61669921875, 987.1814575195312, 990.5541381835938, 990.0877685546875, 1009.4140014648438, 1003.9510498046875, 994.9132690429688, 1008.360107421875, 1002.9154663085938, 1010.817138671875, 1017.1681518554688, 1009.2236328125, 998.5043334960938, 1005.48876953125, 1022.4688720703125, 999.6196899414062, 1007.1715698242188, 1004.1420288085938, 985.6704711914062, 976.096923828125, 999.2539672851562, 1011.3794555664062, 1004.900390625, 986.79052734375, 985.61376953125, 996.5308837890625, 979.875, 995.6734619140625, 997.5451049804688, 993.6224975585938, 992.6904296875, 981.3642578125, 986.7724609375, 1019.8555297851562, 1003.3471069335938, 1017.054443359375, 983.89501953125, 989.5504760742188, 1015.1813354492188, 1001.1950073242188, 1002.0502319335938, 992.4754638671875, 985.2805786132812, 985.551025390625, 1004.2861938476562, 998.4125366210938, 983.371826171875, 1001.8657836914062, 999.3462524414062, 1019.0427856445312, 986.0888061523438, 1005.7379150390625, 1012.4542236328125, 999.0997314453125, 982.38037109375, 998.5783081054688, 973.5562133789062, 996.6396484375, 1023.22705078125, 993.9512329101562, 1009.251220703125, 1009.1820678710938, 986.1122436523438, 1000.27197265625, 998.655029296875, 1006.1577758789062, 998.1149291992188, 968.0146484375, 990.8461303710938, 996.8008422851562, 1030.1121826171875, 1006.2313232421875, 998.695556640625, 981.8894653320312, 1022.8358154296875, 1001.8432006835938, 998.0527954101562, 998.851806640625, 1004.5804443359375, 1005.4210815429688, 1000.6353759765625, 990.9758911132812, 993.1123046875, 1001.9129638671875, 1004.3196411132812, 1013.0288696289062, 988.5238647460938, 1000.82421875, 981.290771484375, 992.3262329101562, 998.5184326171875, 1018.7284545898438, 996.5826416015625, 999.26220703125, 1011.4832153320312, 988.7879028320312, 979.0515747070312, 1012.689697265625, 1002.8492431640625, 1005.6196899414062, 991.0338745117188, 991.0057983398438, 1000.1575927734375, 982.70849609375, 1001.5527954101562, 993.1552124023438, 982.5993041992188, 994.2030029296875, 998.4617919921875, 985.8182373046875, 994.366455078125, 993.7509155273438, 1010.73876953125, 1011.2117309570312, 995.3343505859375, 1014.6305541992188, 1002.403564453125, 999.9559326171875, 993.6849365234375, 976.609375, 1000.1627807617188, 971.6099243164062, 986.2753295898438, 1002.7467041015625, 1009.4113159179688, 974.58447265625, 1004.7404174804688, 1000.870849609375, 1004.1853637695312, 1000.0333862304688, 1012.0671997070312, 989.5191650390625, 990.8786010742188, 995.4017333984375, 994.8360595703125, 1012.95654296875, 1005.7420043945312, 1002.2239379882812, 1000.0811767578125, 1011.0977172851562, 1002.5557861328125, 1013.2098388671875, 1006.27783203125, 1002.363525390625, 997.5533447265625, 1003.3416137695312, 988.433837890625, 978.705078125, 992.8304443359375, 996.2324829101562, 1007.45068359375, 1009.4588012695312, 988.8060302734375, 995.4365234375, 1010.6992797851562, 1001.894775390625, 1004.4688720703125, 981.9425659179688, 986.71875, 986.3672485351562, 999.4558715820312, 996.97509765625, 991.4132080078125, 999.7374877929688, 1002.2236328125, 994.1422729492188, 1001.3585205078125, 1003.4188232421875, 973.076171875, 1000.0256958007812, 991.4740600585938, 988.369140625, 1007.8214111328125, 1023.6136474609375, 989.1409301757812, 983.5599365234375, 1008.4443969726562, 1020.6265258789062, 996.8184204101562, 1013.5242309570312, 990.6552734375, 1011.41845703125, 995.4024658203125, 992.0567626953125, 1011.7145385742188, 1015.3118896484375, 1015.256103515625, 996.922119140625, 1004.71337890625, 1009.4837036132812, 1000.4320678710938, 985.3204345703125, 1007.28369140625, 1007.313720703125, 1007.0013427734375, 965.6690673828125, 990.693603515625, 990.42236328125, 1030.240234375, 993.2588500976562, 988.2843017578125, 1019.6034545898438, 998.6912231445312, 1020.764404296875, 1005.7849731445312, 994.7499389648438, 997.3994140625, 997.5021362304688, 971.6709594726562, 1003.6243286132812, 1008.7095947265625, 1002.5386962890625, 992.1788330078125, 994.9664306640625, 991.9518432617188, 992.4751586914062, 999.956298828125, 972.0277709960938, 1016.1978759765625, 1013.5011596679688, 991.6533813476562, 983.3887329101562, 999.777099609375, 991.2365112304688, 1000.5590209960938, 996.8184204101562, 1001.9032592773438, 998.8402099609375, 1003.4842529296875, 1011.2911987304688, 1013.5407104492188, 1000.5626831054688, 1008.2713012695312, 993.4784545898438, 1010.0947265625, 1018.62109375, 996.8099365234375, 993.9969482421875, 1018.0408935546875, 1006.7434692382812, 1005.6575927734375, 999.1968994140625, 1011.7129516601562, 1020.577880859375, 994.2716674804688, 1011.4623413085938, 997.6548461914062, 967.6307373046875, 997.3282470703125, 981.4429931640625, 1001.1870727539062, 1009.1238403320312, 1000.5778198242188, 1008.7935791015625, 991.4183349609375, 988.5365600585938, 1002.4605102539062, 994.1672973632812, 999.885986328125, 998.3533935546875, 1014.3081665039062, 997.0526123046875, 1007.5677490234375, 1001.6776123046875, 979.738525390625, 1003.6275024414062, 1007.6280517578125, 1009.5023803710938, 1001.9829711914062, 985.625732421875, 1009.9058837890625, 1003.0556030273438, 989.0628662109375, 1008.4418334960938, 1003.4070434570312, 1000.7747802734375, 1001.1533813476562, 983.9699096679688, 979.744873046875, 1017.6561889648438, 999.085693359375, 997.0099487304688, 990.58447265625, 983.3779907226562, 1004.9717407226562, 993.993896484375, 1010.6434936523438, 1019.89306640625, 991.4774780273438, 1006.1425170898438, 996.4913940429688, 1015.5972290039062, 1008.2631225585938, 991.3829956054688, 990.0894165039062, 967.6062622070312, 980.8825073242188, 1017.6199951171875, 1015.8839111328125, 968.4547119140625, 1002.2064208984375, 1002.8169555664062, 1005.94677734375, 1003.7781372070312, 998.491943359375, 999.8457641601562, 990.806640625, 995.9425659179688, 1005.0809936523438, 1007.1929931640625, 1007.1984252929688, 984.625, 1006.6495361328125, 1027.725341796875, 1004.06982421875, 1009.4842529296875, 1002.4030151367188, 983.93798828125, 1007.8916015625, 1014.0747680664062, 998.6422729492188, 1026.026123046875, 1006.53759765625, 1009.5279541015625, 998.3986206054688, 986.1914672851562, 1005.078857421875, 1001.0852661132812, 991.4698486328125, 1018.01025390625, 993.2970581054688, 997.6639404296875, 984.9723510742188, 1008.0287475585938, 995.1016845703125]

4. 测量同时在两个 GPU 上执行两个矩阵乘法与在一个 GPU 上按顺序执行两个矩阵乘法所需的时间。提示：应该看到近乎线性的缩放。