pytest：重新运行失败测试并在多次运行间保存状态
pytest支持在多次运行之间保存状态，并据此选择下次运行哪些测试。
用法
pytest提供两个命令行选项，用于重新运行上次失败的测试：
--lf、--last-failed：只重新运行上次失败的测试。--ff、--failed-first：先运行上次失败的测试，再运行其余测试。
对于其他用法，还可以使用--cache-clear在测试运行前删除所有缓存的跨会话数据。
插件可以访问config.cache对象，以在不同pytest调用之间读写JSON可编码的值。
只运行失败测试，或优先运行失败测试
先创建50个测试调用，其中只有两个失败：
# content of test_50.py
import pytest
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
pytest.fail("bad luck")第一次运行：
$ pytest -q
.................F.......F........................ [100%]
================================= FAILURES =================================
_______________________________ test_num[17] _______________________________
i = 17
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
> pytest.fail("bad luck")
E Failed: bad luck
test_50.py:7: Failed
_______________________________ test_num[25] _______________________________
i = 25
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
> pytest.fail("bad luck")
E Failed: bad luck
test_50.py:7: Failed
========================= short test summary info ==========================
FAILED test_50.py::test_num[17] - Failed: bad luck
FAILED test_50.py::test_num[25] - Failed: bad luck
2 failed, 48 passed in 0.12s现在使用--lf，只运行两个失败的测试：
$ pytest --lf
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 2 items
run-last-failure: rerun previous 2 failures
test_50.py FF [100%]
================================= FAILURES =================================
_______________________________ test_num[17] _______________________________
i = 17
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
> pytest.fail("bad luck")
E Failed: bad luck
test_50.py:7: Failed
_______________________________ test_num[25] _______________________________
i = 25
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
> pytest.fail("bad luck")
E Failed: bad luck
test_50.py:7: Failed
========================= short test summary info ==========================
FAILED test_50.py::test_num[17] - Failed: bad luck
FAILED test_50.py::test_num[25] - Failed: bad luck
============================ 2 failed in 0.12s =============================两个失败测试先前在中间位置运行，这次从头运行。再看--ff的行为：
$ pytest --ff
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
collected 50 items
run-last-failure: rerun previous 2 failures first
test_50.py FF................................................ [100%]
================================= FAILURES =================================
_______________________________ test_num[17] _______________________________
i = 17
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
> pytest.fail("bad luck")
E Failed: bad luck
test_50.py:7: Failed
_______________________________ test_num[25] _______________________________
i = 25
@pytest.mark.parametrize("i", range(50))
def test_num(i):
if i in (17, 25):
> pytest.fail("bad luck")
E Failed: bad luck
test_50.py:7: Failed
========================= short test summary info ==========================
FAILED test_50.py::test_num[17] - Failed: bad luck
FAILED test_50.py::test_num[25] - Failed: bad luck
======================= 2 failed, 48 passed in 0.12s =======================最后一个选项会先运行上次失败的测试，然后运行其余测试。因此，失败测试出现在前面。
优先运行新测试
--nf、--new-first会先运行新测试，然后运行其余测试。新测试和其余测试分别按文件修改时间排序，修改较新的文件优先。
未找到上次失败记录时的行为
没有发现上次失败的测试或没有缓存lastfailed数据时，可以用--lfnf、--last-failed-no-failures配置行为：
pytest --last-failed --last-failed-no-failures all # runs the full test suite (default behavior)
pytest --last-failed --last-failed-no-failures none # runs no tests and exits successfully新的config.cache对象
插件或conftest.py中的辅助代码，可以借助config.cache在不同pytest调用之间存储与获取值。下面的fixture会在不同测试运行之间复用计算结果：
# content of test_caching.py
import pytest
def expensive_computation():
print("running expensive computation...")
@pytest.fixture
def mydata(pytestconfig):
cache = getattr(pytestconfig, "cache", None)
if cache is None:
# pytestconfig not having the cache attribute means the
# cache plugin is disabled.
expensive_computation()
return 42
val = cache.get("example/value", None)
if val is None:
expensive_computation()
val = 42
cache.set("example/value", val)
return val
def test_function(mydata):
assert mydata == 23首次运行：
$ pytest -q
F [100%]
================================= FAILURES =================================
______________________________ test_function _______________________________
mydata = 42
def test_function(mydata):
> assert mydata == 23
E assert 42 == 23
test_caching.py:26: AssertionError
-------------------------- Captured stdout setup ---------------------------
running expensive computation...
========================= short test summary info ==========================
FAILED test_caching.py::test_function - assert 42 == 23
1 failed in 0.12s再次运行同一个测试时，计算值从缓存取回，不再重新执行计算：
$ pytest -q
F [100%]
================================= FAILURES =================================
______________________________ test_function _______________________________
mydata = 42
def test_function(mydata):
> assert mydata == 23
E assert 42 == 23
test_caching.py:26: AssertionError
========================= short test summary info ==========================
FAILED test_caching.py::test_function - assert 42 == 23
1 failed in 0.12s完整说明参见config.cache参考文档。
查看缓存内容
使用--cache-show可以查看缓存值：
$ pytest --cache-show
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
cachedir: /home/sweet/project/.pytest_cache
--------------------------- cache values for '*' ---------------------------
cache/lastfailed contains:
{'test_caching.py::test_function': True}
cache/nodeids contains:
['test_caching.py::test_function']
example/value contains:
42
========================== no tests ran in 0.12s ===========================也可以指定glob模式，筛选要显示的内容：
$ pytest --cache-show example/*
=========================== test session starts ============================
platform linux -- Python 3.x.y, pytest-9.x.y, pluggy-1.x.y
rootdir: /home/sweet/project
cachedir: /home/sweet/project/.pytest_cache
----------------------- cache values for 'example/*' -----------------------
example/value contains:
42
========================== no tests ran in 0.12s ===========================清除缓存内容
可在运行测试前清空缓存：
pytest --cache-clear在持续集成服务器上运行pytest时，建议使用此选项；在这类环境中，为了正确性，通常更希望隔离各次运行。
逐步运行
作为--lf -x的替代方案，尤其在预计大量测试会失败时，可以用--sw、--stepwise逐个修复：运行到首次失败时停止；下一次调用pytest时，从该失败测试继续。
--stepwise-skip会忽略一个失败测试，在第二个失败处停止。当某个失败已经明确但暂时不想处理时，这很有用。--stepwise-skip会自动启用--stepwise。
原始代码许可证全文
The MIT License (MIT) Copyright (c) 2004 Holger Krekel and others Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions: The above copyright notice and this permission notice shall be included in all copies or substantial portions of the Software. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
© 版权声明
文章版权归作者所有，未经允许请勿转载。
THE END
暂无评论内容