创建于 2020-12-02；最后更新于 2025-09-23；原文最后验证于 2024-11-05。
本篇介绍如何使用 PyTorch benchmark 模块，测量并比较代码性能。
简介
基准测试是编程的重要步骤，可以验证代码是否达到性能预期、比较同一问题的不同解法，并防止性能退化。
PyTorch 代码可以用多种工具测试，包括 Python 内置 timeit。但线程数量管理、CUDA 设备同步等细节容易被忽略，为基准测试生成张量输入也可能很繁琐。
本篇展示 benchmark 模块如何避免常见错误，并简化不同代码性能比较、输入生成等任务。
准备
如果尚未安装 torch，先安装：
pip install torch
步骤
1. 定义待测试函数。
2. 使用 timeit.Timer。
3. 使用 torch.utils.benchmark.Timer。
4. 使用 Blocked Autorange。
5. 比较结果。
6. 保存与加载结果。
7. 使用模糊参数生成输入。
8. 使用 Callgrind 收集指令计数。
1. 定义待测试函数
原文写作时，torch.dot 不支持批处理。本例比较两种由现有 torch 运算实现批量点积的方法：一种组合 mul 与 sum，另一种将问题转化为 bmm。
import torch
def batched_dot_mul_sum(a, b):
'''Computes batched dot by multiplying and summing'''
return a.mul(b).sum(-1)
def batched_dot_bmm(a, b):
'''Computes batched dot by reducing to ``bmm``'''
a = a.reshape(-1, 1, a.shape[-1])
b = b.reshape(-1, b.shape[-1], 1)
return torch.bmm (a, b).flatten(-3)
# Input for benchmarking
x = torch.randn (10000, 64)
# Ensure that both functions compute the same output
assert batched_dot_mul_sum(x, x).allclose(batched_dot_bmm(x, x))2. 使用 timeit.Timer
先用 Python 内置 timeit。这里保持代码简单，方便比较 timeit 与 torch.utils.benchmark 的默认行为。
import timeit
t0 = timeit.Timer(
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals={'x': x})
t1 = timeit.Timer(
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals={'x': x})
print(f'mul_sum(x, x): {t0.timeit(100) / 100 * 1e6:>5.1f} us')
print(f'bmm(x, x): {t1.timeit(100) / 100 * 1e6:>5.1f} us')**原文输出：**
mul_sum(x, x): 111.6 us
bmm(x, x): 70.0 us3. 使用 torch.utils.benchmark.Timer
PyTorch benchmark 为熟悉 timeit 的用户提供相近接口，但默认设置更适合安全方便地测试 PyTorch。先比较与前面相同的基本 API。
import torch.utils.benchmark as benchmark
t0 = benchmark.Timer (
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals={'x': x})
t1 = benchmark.Timer (
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals={'x': x})
print(t0.timeit(100))
print(t1.timeit(100))**原文输出：**
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb10400d0f0>
batched_dot_mul_sum(x, x)
setup: from __main__ import batched_dot_mul_sum
379.29 us
1 measurement, 100 runs , 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb103d67048>
batched_dot_bmm(x, x)
setup: from __main__ import batched_dot_bmm
716.42 us
1 measurement, 100 runs , 1 thread基本 API 虽然相同，却存在重要区别：benchmark.Timer.timeit() 返回每次运行的时间，timeit.Timer.timeit() 返回总时间。benchmark 还提供格式化结果字符串。
另一个导致结果不同的关键是 benchmark 默认使用单线程。可用 num_threads 修改线程数。
torch.utils.benchmark.Timer 还接受 label、sub_label、description、env，改变测量对象的 __repr__，并用于分组结果，后文会介绍。
num_threads = torch.get_num_threads ()
print(f'Benchmarking on {num_threads} threads')
t0 = benchmark.Timer (
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals={'x': x},
num_threads=num_threads,
label='Multithreaded batch dot',
sub_label='Implemented using mul and sum')
t1 = benchmark.Timer (
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals={'x': x},
num_threads=num_threads,
label='Multithreaded batch dot',
sub_label='Implemented using bmm')
print(t0.timeit(100))
print(t1.timeit(100))**原文输出：**
Benchmarking on 40 threads
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb103d54080>
Multithreaded batch dot: Implemented using mul and sum
setup: from __main__ import batched_dot_mul_sum
118.47 us
1 measurement, 100 runs , 40 threads
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb16935d2e8>
Multithreaded batch dot: Implemented using bmm
setup: from __main__ import batched_dot_bmm
68.21 us
1 measurement, 100 runs , 40 threads使用全部可用线程时，benchmark 与 timeit 结果接近。哪种实现更快取决于线程数，因此线程设置应代表真实用途。GPU 基准测试还必须同步 CPU 与 CUDA。下面对 CUDA 张量重跑测试：
x = torch.randn (10000, 1024, device='cuda')
t0 = timeit.Timer(
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals={'x': x})
t1 = timeit.Timer(
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals={'x': x})
# Ran each twice to show difference before/after warm-up
print(f'mul_sum(x, x): {t0.timeit(100) / 100 * 1e6:>5.1f} us')
print(f'mul_sum(x, x): {t0.timeit(100) / 100 * 1e6:>5.1f} us')
print(f'bmm(x, x): {t1.timeit(100) / 100 * 1e6:>5.1f} us')
print(f'bmm(x, x): {t1.timeit(100) / 100 * 1e6:>5.1f} us')**原文输出：**
mul_sum(x, x): 27.6 us
mul_sum(x, x): 25.3 us
bmm(x, x): 2775.5 us
bmm(x, x): 22.4 us
t0 = benchmark.Timer (
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals={'x': x})
t1 = benchmark.Timer (
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals={'x': x})
# Run only once since benchmark module does warm-up for us
print(t0.timeit(100))
print(t1.timeit(100))**原文输出：**
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb10400d080>
batched_dot_mul_sum(x, x)
setup: from __main__ import batched_dot_mul_sum
232.93 us
1 measurement, 100 runs , 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb10400d0f0>
batched_dot_bmm(x, x)
setup: from __main__ import batched_dot_bmm
181.04 us
1 measurement, 100 runs , 1 thread结果显示，timeit 中 bmm 的首次运行远慢于第二次，因为首次调用需要加载 cuBLAS。这说明测试前预热十分重要，benchmark 模块会自动处理。
timeit 与 benchmark 的结果差异来自 CUDA 同步：timeit 没有同步，因此只测到启动内核的时间；benchmark 会完成同步。
4. 使用 Blocked Autorange
timeit.Timer.autorange 进行一次至少 0.2 秒的连续测量；torch.utils.benchmark.Timer.blocked_autorange 进行多次测量，总时长至少 0.2 秒，可由 min_run_time 修改，同时确保计时开销仅占总测量的一小部分。
它先逐步增加每轮运行次数，直到运行时间远大于计时开销，这也起到预热作用；随后持续测量直至目标时长。这能减少数据浪费，并计算统计量评估测量可靠性。
m0 = t0.blocked_autorange()
m1 = t1.blocked_autorange()
print(m0)
print(m1)**原文输出：**
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb10400d0f0>
batched_dot_mul_sum(x, x)
setup: from __main__ import batched_dot_mul_sum
231.79 us
1 measurement, 1000 runs , 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb10400d080>
batched_dot_bmm(x, x)
setup: from __main__ import batched_dot_bmm
Median: 162.08 us
2 measurements, 1000 runs per measurement, 1 thread也可以从返回的测量对象中读取各项统计：
print(f"Mean: {m0.mean * 1e6:6.2f} us")
print(f"Median: {m0.median * 1e6:6.2f} us")
**原文输出：**
Mean: 231.79 us
Median: 231.79 us5. 比较测试结果
此前只针对一个输入比较两种批量点积。实际应尝试不同输入和线程数。Compare 把多次测量以表格呈现，依据 label、sub_label、num_threads 等注解以及 description 分组、组织。下面比较不同输入大小与线程数：
from itertools import product
# Compare takes a list of measurements which we'll save in results.
results = []
sizes = [1, 64, 1024, 10000]
for b, n in product(sizes, sizes):
# label and sub_label are the rows
# description is the column
label = 'Batched dot'
sub_label = f'[{b}, {n}]'
x = torch.ones ((b, n))
for num_threads in [1, 4, 16, 32]:
results.append(benchmark.Timer (
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals={'x': x},
num_threads=num_threads,
label=label,
sub_label=sub_label,
description='mul/sum',
).blocked_autorange(min_run_time=1))
results.append(benchmark.Timer (
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals={'x': x},
num_threads=num_threads,
label=label,
sub_label=sub_label,
description='bmm',
).blocked_autorange(min_run_time=1))
compare = benchmark.Compare (results)
compare.print()**原文输出：**
[--------------- Batched dot ----------------]
| mul/sum | bmm
1 threads: -----------------------------------
[1, 1] | 5.9 | 11.2
[1, 64] | 6.4 | 11.4
[1, 1024] | 6.7 | 14.2
[1, 10000] | 10.2 | 23.7
[64, 1] | 6.3 | 11.5
[64, 64] | 8.6 | 15.4
[64, 1024] | 39.4 | 204.4
[64, 10000] | 274.9 | 748.5
[1024, 1] | 7.7 | 17.8
[1024, 64] | 40.3 | 76.4
[1024, 1024] | 432.4 | 2795.9
[1024, 10000] | 22657.3 | 11899.5
[10000, 1] | 16.9 | 74.8
[10000, 64] | 300.3 | 609.4
[10000, 1024] | 23098.6 | 27246.1
[10000, 10000] | 267073.7 | 118823.7
4 threads: -----------------------------------
[1, 1] | 6.0 | 11.5
[1, 64] | 6.2 | 11.2
[1, 1024] | 6.8 | 14.3
[1, 10000] | 10.2 | 23.7
[64, 1] | 6.3 | 16.2
[64, 64] | 8.8 | 18.2
[64, 1024] | 41.5 | 189.1
[64, 10000] | 91.7 | 849.1
[1024, 1] | 7.6 | 17.4
[1024, 64] | 43.5 | 33.5
[1024, 1024] | 135.4 | 2782.3
[1024, 10000] | 7471.1 | 11874.0
[10000, 1] | 16.8 | 33.9
[10000, 64] | 118.7 | 173.2
[10000, 1024] | 7264.6 | 27824.7
[10000, 10000] | 100060.9 | 121499.0
16 threads: ----------------------------------
[1, 1] | 6.0 | 11.3
[1, 64] | 6.2 | 11.2
[1, 1024] | 6.9 | 14.2
[1, 10000] | 10.3 | 23.8
[64, 1] | 6.4 | 24.1
[64, 64] | 9.0 | 23.8
[64, 1024] | 54.1 | 188.5
[64, 10000] | 49.9 | 748.0
[1024, 1] | 7.6 | 23.4
[1024, 64] | 55.5 | 28.2
[1024, 1024] | 66.9 | 2773.9
[1024, 10000] | 6111.5 | 12833.7
[10000, 1] | 16.9 | 27.5
[10000, 64] | 59.5 | 73.7
[10000, 1024] | 6295.9 | 27062.0
[10000, 10000] | 71804.5 | 120365.8
32 threads: ----------------------------------
[1, 1] | 5.9 | 11.3
[1, 64] | 6.2 | 11.3
[1, 1024] | 6.7 | 14.2
[1, 10000] | 10.5 | 23.8
[64, 1] | 6.3 | 31.7
[64, 64] | 9.1 | 30.4
[64, 1024] | 72.0 | 190.4
[64, 10000] | 103.1 | 746.9
[1024, 1] | 7.6 | 28.4
[1024, 64] | 70.5 | 31.9
[1024, 1024] | 65.6 | 2804.6
[1024, 10000] | 6764.0 | 11871.4
[10000, 1] | 17.8 | 31.8
[10000, 64] | 110.3 | 56.0
[10000, 1024] | 6640.2 | 27592.2
[10000, 10000] | 73003.4 | 120083.2
Times are in microseconds (us).结果表明，多线程、大张量时，转为 bmm 的版本更优；较小张量或单线程时，另一种版本更优。
Compare 还提供调整表格格式的方法：
compare.trim_significant_figures()
compare.colorize()
compare.print()6. 保存与加载结果
测量结果及第 8 节的 CallgrindStats 可用 pickle 序列化，便于 A/B 测试：在不同环境采集、序列化，然后在同一环境加载。Timer 的 env 构造参数可以自然地区分这些环境。
假设 mul/sum 与 bmm 不只是两个 Python 函数，而是位于两个不同的 PyTorch 构建中，下例展示如何进行 A/B 测试。为简化，仅使用部分形状，只做 pickle 往返序列化，并未真的使用多个环境或写入磁盘。
import pickle
ab_test_results = []
for env in ('environment A: mul/sum', 'environment B: bmm'):
for b, n in ((1, 1), (1024, 10000), (10000, 1)):
x = torch.ones ((b, n))
dot_fn = (batched_dot_mul_sum if env == 'environment A: mul/sum' else batched_dot_bmm)
m = benchmark.Timer (
stmt='batched_dot(x, x)',
globals={'x': x, 'batched_dot': dot_fn},
num_threads=1,
label='Batched dot',
description=f'[{b}, {n}]',
env=env,
).blocked_autorange(min_run_time=1)
ab_test_results.append(pickle.dumps(m))
ab_results = [pickle.loads(i) for i in ab_test_results]
compare = benchmark.Compare (ab_results)
compare.trim_significant_figures()
compare.colorize()
compare.print()**原文输出：**
[------------------------------------- Batched dot -------------------------------------]
| [1, 1] | [1024, 10000] | [10000, 1]
1 threads: ------------------------------------------------------------------------------
(environment A: mul/sum) batched_dot(x, x) | 7 | 36000 | 21
(environment B: bmm) batched_dot(x, x) | 14 | 40000 | 85
Times are in microseconds (us).
# And just to show that we can round trip all of the results from earlier:
round_tripped_results = pickle.loads(pickle.dumps(results))
assert(str(benchmark.Compare (results)) == str(benchmark.Compare (round_tripped_results)))7. 使用模糊参数生成输入
输入张量不同，性能可能相差很大，所以应测试多种输入。手工创建很繁琐，torch.utils.benchmark.Fuzzer 及相关类可帮助生成测试用例：
from torch.utils.benchmark import Fuzzer, FuzzedParameter, FuzzedTensor, ParameterAlias
# Generates random tensors with 128 to 10000000 elements and sizes k0 and k1 chosen from a
# ``loguniform`` distribution in [1, 10000], 40% of which will be discontiguous on average.
example_fuzzer = Fuzzer(
parameters = [
FuzzedParameter('k0', minval=1, maxval=10000, distribution='loguniform'),
FuzzedParameter('k1', minval=1, maxval=10000, distribution='loguniform'),
],
tensors = [
FuzzedTensor('x', size=('k0', 'k1'), min_elements=128, max_elements=10000000, probability_contiguous=0.6)
],
seed=0,
)
results = []
for tensors, tensor_params, params in example_fuzzer.take(10):
# description is the column label
sub_label=f"{params['k0']:<6} x {params['k1']:<4} {'' if tensor_params['x']['is_contiguous'] else '(discontiguous)'}"
results.append(benchmark.Timer (
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals=tensors,
label='Batched dot',
sub_label=sub_label,
description='mul/sum',
).blocked_autorange(min_run_time=1))
results.append(benchmark.Timer (
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals=tensors,
label='Batched dot',
sub_label=sub_label,
description='bmm',
).blocked_autorange(min_run_time=1))
compare = benchmark.Compare (results)
compare.trim_significant_figures()
compare.print()**原文输出：**
[--------------------- Batched dot ---------------------]
| mul/sum | bmm
1 threads: ----------------------------------------------
725 x 257 | 87 | 180
49 x 383 | 15 | 30
34 x 1468 | 30 | 118
187 x 5039 | 400 | 1200
2140 x 1296 (discontiguous) | 2000 | 41000
78 x 1598 | 74 | 310
519 x 763 | 190 | 1500
141 x 1082 | 87 | 500
78 x 5 (discontiguous) | 9 | 20
187 x 1 | 12 | 10
Times are in microseconds (us).自定义 fuzzer 很灵活，能够构建强大的测试输入集。为进一步简化，模块还内置满足常见需求的 fuzzer，使用示例如下：
from torch.utils.benchmark.op_fuzzers import binary
results = []
for tensors, tensor_params, params in binary.BinaryOpFuzzer(seed=0).take(10):
sub_label=f"{params['k0']:<6} x {params['k1']:<4} {'' if tensor_params['x']['is_contiguous'] else '(discontiguous)'}"
results.append(benchmark.Timer (
stmt='batched_dot_mul_sum(x, x)',
setup='from __main__ import batched_dot_mul_sum',
globals=tensors,
label='Batched dot',
sub_label=sub_label,
description='mul/sum',
).blocked_autorange(min_run_time=1))
results.append(benchmark.Timer (
stmt='batched_dot_bmm(x, x)',
setup='from __main__ import batched_dot_bmm',
globals=tensors,
label='Batched dot',
sub_label=sub_label,
description='bmm',
).blocked_autorange(min_run_time=1))
compare = benchmark.Compare (results)
compare.trim_significant_figures()
compare.colorize(rowwise=True)
compare.print()**原文输出：**
[----------------------- Batched dot ------------------------]
| mul/sum | bmm
1 threads: ---------------------------------------------------
64 x 473 (discontiguous) | 10000 | 40000
16384 x 12642115 (discontiguous) | 31 | 78
8192 x 892 | 4800 | 20400
512 x 64 (discontiguous) | 110000 | 400000
493 x 27 (discontiguous) | 1100 | 2440
118 x 32 (discontiguous) | 870 | 2030
16 x 495 (discontiguous) | 23600 | 24000
488 x 62374 | 90000 | 100000
240372 x 69 | 40000 | 16000
40156 x 32 (discontiguous) | 2670 | 5000
Times are in microseconds (us).8. 使用 Callgrind 收集指令计数
代码优化的一项挑战是实际耗时的变化与不透明：动态时钟、其他进程资源争用等都会造成不确定性。端到端时间也不能说明时间具体花在哪里，而这正是优化关心的问题。
可以补充收集指令计数。它是替代指标，不能涵盖全部性能因素，例如内存或 I/O 瓶颈，但可重现、对环境变化不敏感，并细致揭示程序在哪些位置消耗周期。
为展示其价值，考虑减少 batched_dot_mul_sum 的开销。明显的方法是移到 C++，避免反复跨越 Python/C++ 边界。
源码几乎相同，但 C++ 需要考虑参数按值还是按引用传递。
batched_dot_src = """\
/* ---- Python ---- */
// def batched_dot_mul_sum(a, b):
// return a.mul(b).sum(-1)
torch::Tensor batched_dot_mul_sum_v0(
const torch::Tensor a,
const torch::Tensor b) {
return a.mul(b).sum(-1);
}
torch::Tensor batched_dot_mul_sum_v1(
const torch::Tensor& a,
const torch::Tensor& b) {
return a.mul(b).sum(-1);
}
"""
# PyTorch makes it easy to test our C++ implementations by providing a utility
# to JIT compile C++ source into Python extensions:
import os
from torch.utils import cpp_extension
cpp_lib = cpp_extension.load_inline (
name='cpp_lib',
cpp_sources=batched_dot_src,
extra_cflags=['-O3'],
extra_include_paths=[
# `load_inline` needs to know where to find ``pybind11`` headers.
os.path.join(os.getenv('CONDA_PREFIX'), 'include')
],
functions=['batched_dot_mul_sum_v0', 'batched_dot_mul_sum_v1']
)
# `load_inline` will create a shared object that is loaded into Python. When we collect
# instruction counts Timer will create a subprocess, so we need to re-import it. The
# import process is slightly more complicated for C extensions, but that's all we're
# doing here.
module_import_str = f"""\
# https://stackoverflow.com/questions/67631/how-to-import-a-module-given-the-full-path
import importlib.util
spec = importlib.util.spec_from_file_location("cpp_lib", {repr(cpp_lib.__file__)})
cpp_lib = importlib.util.module_from_spec(spec)
spec.loader.exec_module(cpp_lib)"""
import textwrap
def pretty_print(result):
"""Import machinery for ``cpp_lib.so`` can get repetitive to look at."""
print(repr(result).replace(textwrap.indent(module_import_str, " "), " import cpp_lib"))
t_baseline = benchmark.Timer (
stmt='batched_dot_mul_sum(x, x)',
setup='''\
from __main__ import batched_dot_mul_sum
x = torch.randn(2, 2)''')
t0 = benchmark.Timer (
stmt='cpp_lib.batched_dot_mul_sum_v0(x, x)',
setup=f'''\
{module_import_str}
x = torch.randn(2, 2)''')
t1 = benchmark.Timer (
stmt='cpp_lib.batched_dot_mul_sum_v1(x, x)',
setup=f'''\
{module_import_str}
x = torch.randn(2, 2)''')
# Moving to C++ did indeed reduce overhead, but it's hard to tell which
# calling convention is more efficient. v1 (call with references) seems to
# be a bit faster, but it's within measurement error.
pretty_print(t_baseline.blocked_autorange())
pretty_print(t0.blocked_autorange())
pretty_print(t1.blocked_autorange())**原文输出：**
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb16935d2e8>
batched_dot_mul_sum(x, x)
setup:
from __main__ import batched_dot_mul_sum
x = torch.randn(2, 2)
6.92 us
1 measurement, 100000 runs , 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb16935d2e8>
cpp_lib.batched_dot_mul_sum_v0(x, x)
setup:
import cpp_lib
x = torch.randn(2, 2)
5.29 us
1 measurement, 100000 runs , 1 thread
<torch.utils.benchmark.utils.common.Measurement object at 0x7fb16935d2e8>
cpp_lib.batched_dot_mul_sum_v1(x, x)
setup:
import cpp_lib
x = torch.randn(2, 2)
5.22 us
1 measurement, 100000 runs , 1 thread
# Let's use ``Callgrind`` to determine which is better.
stats_v0 = t0.collect_callgrind()
stats_v1 = t1.collect_callgrind()
pretty_print(stats_v0)
pretty_print(stats_v1)
# `.as_standardized` removes file names and some path prefixes, and makes
# it easier to read the function symbols.
stats_v0 = stats_v0.as_standardized()
stats_v1 = stats_v1.as_standardized()
# `.delta` diffs the instruction counts, and `.denoise` removes several
# functions in the Python interpreter that are known to have significant
# jitter.
delta = stats_v1.delta(stats_v0).denoise()
# `.transform` is a convenience API for transforming function names. It is
# useful for increasing cancelation when ``diff-ing`` instructions, as well as
# just generally improving readability.
replacements = (
("???:void pybind11", "pybind11"),
("batched_dot_mul_sum_v0", "batched_dot_mul_sum_v1"),
("at::Tensor, at::Tensor", "..."),
("at::Tensor const&, at::Tensor const&", "..."),
("auto torch::detail::wrap_pybind_function_impl_", "wrap_pybind_function_impl_"),
)
for before, after in replacements:
delta = delta.transform(lambda l: l.replace(before, after))
# We can use print options to control how much of the function to display.
torch.set_printoptions (linewidth=160)
# Once parsed, the instruction counts make clear that passing `a` and `b`
# by reference is more efficient as it skips some ``c10::TensorImpl`` bookkeeping
# for the intermediate Tensors, and is also works better with ``pybind11``. This
# is consistent with our noisy wall time observations.
print(delta)
<torch.utils.benchmark.utils.valgrind_wrapper.timer_interface.CallgrindStats object at 0x7fb0f06e7630>
cpp_lib.batched_dot_mul_sum_v0(x, x)
setup:
import cpp_lib
x = torch.randn(2, 2)
All Noisy symbols removed
Instructions: 2392671 2392671
Baseline: 4367 4367
100 runs per measurement, 1 thread
Warning: PyTorch was not built with debug symbols.
Source information may be limited. Rebuild with
REL_WITH_DEB_INFO=1 for more detailed results.
<torch.utils.benchmark.utils.valgrind_wrapper.timer_interface.CallgrindStats object at 0x7fb10400d208>
cpp_lib.batched_dot_mul_sum_v1(x, x)
setup:
import cpp_lib
x = torch.randn(2, 2)
All Noisy symbols removed
Instructions: 2378978 2378978
Baseline: 4367 4367
100 runs per measurement, 1 thread
Warning: PyTorch was not built with debug symbols.
Source information may be limited. Rebuild with
REL_WITH_DEB_INFO=1 for more detailed results.
<torch.utils.benchmark.utils.valgrind_wrapper.timer_interface.FunctionCounts object at 0x7fb1000ab358>
86 ???:0x000000000020d9e0
56 ???:0x000000000020db10
-1100 pybind11::cpp_function::initialize<wrap_pybind_function_impl_<at::Tensor ... r (&)(...), std::integer_sequence<unsigned long, 0ul, 1ul>)::{lambda(...)
-1600 ???:wrap_pybind_function_impl_<at::Tensor (&)(...), 0ul, 1ul>(at::Tensor (&)(...), std::integer_sequence<unsigned long, 0ul, 1ul>)::{lambda(...)
-5200 ???:c10::intrusive_ptr<c10::TensorImpl, c10::UndefinedTensorImpl>::reset_()
-5935 ???:0x000000000022c0e0
Total: -13693继续学习
可以继续阅读以下教程：
—
原文：PyTorch Benchmark。作者：PyTorch 教程贡献者。本文为中文翻译，保留原文代码、性能输出和验证日期；这些输出是原文数据。原始 notebook、Python 脚本与 ZIP 下载见原文末尾。
BSD 3-Clause License
Copyright (c) 2017-2022, Pytorch contributors
All rights reserved.
Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met:
* Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer.
* Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution.
* Neither the name of the copyright holder nor the names of its contributors may be used to endorse or promote products derived from this software without specific prior written permission.
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS “AS IS” AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
© 版权声明
文章版权归作者所有，未经允许请勿转载。
THE END
暂无评论内容