pytest / starting out

your first test · running it · what pytest looks for · reading a failure · fixtures: reusable setup · cleaning up afterwards · conftest.py: sharing fixtures

1 / 2sheet 1 front

01 your first test

# test_math.py
def test_two_plus_two():
    assert 2 + 2 == 4

$ pytest
1 passed in 0.01s

That is a complete test. No class, no self, no assertEqual, no imports. A test is a function whose name starts with test, and it passes if it does not raise.

The word assert is plain Python, not a pytest feature. It means "stop here if this is not true".

02 running it

pytest	run everything it can find
pytest -q	quieter: one dot per test
pytest test_math.py	just one file
pytest -k adds	only tests with "adds" in the name
pytest -x	stop at the first failure
pytest --lf	only what failed last time
pytest --collect-only	list what WOULD run, run nothing

Learn --lf early. Fix one thing, rerun only the failures, repeat. It changes how the day feels.

03 what pytest looks for

files	`test_.py` or `_test.py`
functions	name starts with `test`
classes	name starts with `Test`, and no `__init__`

Finding tests is called collection. If a test is not running, the name is nearly always why. pytest --collect-only shows exactly what pytest can see.

04 reading a failure

    def test_greeting():
>       assert greet("sam") == "hi sam"
E       AssertionError: assert 'Hi sam' == 'hi sam'
E         - hi sam
E         + Hi sam

> marks the failing line, E lines are the explanation, and pytest prints the actual values. Here: a capital H.

You get that diff from a plain assert. pytest rewrites your test file as it loads it so it can show both sides. This is why there is no assertion library to learn.

05 fixtures: reusable setup

A fixture is a named piece of setup. Ask for it by putting its name in your test's arguments, and pytest runs it and hands you the result.

import pytest

@pytest.fixture
def user():
    return {"name": "sam", "admin": False}

def test_not_admin(user):
    assert user["admin"] is False

The argument name is the wiring. Writing user in the signature is what finds the fixture called user. Nothing else connects them.

06 cleaning up afterwards

@pytest.fixture
def db():
    conn = connect()
    yield conn      # the test runs here
    conn.close()      # then this runs

Everything after yield is teardown. It runs when the test finishes.

Teardown runs even when the test FAILS. That is the reason to use a fixture instead of writing setup at the top of the test: a crash in the middle still closes the connection.

07 conftest.py: sharing fixtures

Move a fixture into a file called conftest.py and every test in that folder can use it, with no import.

tests/
  conftest.py      # fixtures live here
  test_users.py     # just ask for them by name
  test_billing.py

It is the one file where a name appears from nowhere, so it is the first place to look when you meet an argument you do not recognise.

turn over

pytest / starting out

factories: making several · fixtures you get for free · parametrize: same test, many inputs · glossary · glossary: fakes and patching · habits worth starting with · when something looks wrong

2 / 2sheet 1 back

08 factories: making several

A plain fixture gives you one thing. Sometimes a test needs three, or needs one with particular settings. So the fixture returns a function instead of a value. That is a factory fixture.

@pytest.fixture
def make_user():
    def _make(name, admin=False):
        return {"name": name, "admin": admin}
    return _make          # the FUNCTION, not a user

def test_two_users(make_user):
    boss  = make_user("ada", admin=True)
    other = make_user("sam")
    assert boss["admin"] and not other["admin"]

Reach for a factory the moment you would otherwise write admin_user, expired_user, user_with_no_email as separate fixtures. Those pile up fast, and a reader has to open another file to find out what each one is.

A factory can also keep a list of what it made and clean up after the test, using yield the same way the fixture above does.

09 fixtures you get for free

tmp_path	an empty folder, new for every test, cleaned up for you
capsys	captures anything printed, so you can assert on it
monkeypatch	change something temporarily and have it put back
caplog	captures log messages

def test_writes_a_file(tmp_path):
    out = tmp_path / "report.txt"
    save_report(out)
    assert out.read_text().startswith("REPORT")

Never write to a fixed path like /tmp/out.txt in a test. Two tests running at once will fight over it.

10 parametrize: same test, many inputs

@pytest.mark.parametrize("number,word", [
    (0, "zero"),
    (1, "one"),
    (2, "many"),
])
def test_naming(number, word):
    assert name(number) == word

$ pytest -v
test_naming[0-zero]  PASSED
test_naming[1-one]   PASSED
test_naming[2-many]  FAILED

Three separate tests, so one failing does not hide the others. A for loop inside a single test stops at the first problem and reports one result.

11 glossary

assertion	a line that says what must be true. `assert x == y`
test	a function whose name starts with `test`
fixture	named setup a test asks for by argument name
factory	a fixture returning a function, so a test can make several
teardown	cleanup after a test, written after `yield`
conftest.py	shared fixtures for a folder, no import needed
parametrize	run one test many times with different inputs
pytest.raises	assert that something DOES fail: `with pytest.raises(ValueError):`
skip	this test does not apply here, do not run it
flaky	passes and fails without the code changing. always a bug
coverage	which lines ran. NOT whether they were checked

12 glossary: fakes and patching

Words for "a stand-in used instead of the real thing". Collectively they are test doubles.

stub	always gives the same canned answer
fake	a real but simplified version, e.g. a dict instead of a database
mock	a spy that also checks it was called correctly
monkeypatch	temporarily replace something, put back automatically
injection	passing the collaborator in, instead of the code fetching it

Prefer a fake. A stub that always returns success cannot tell you what happens on failure, because it was never pretending to be anything.

If a test is painful to write, that is usually the code talking. Code that fetches its own database or network connection is hard to stand in for; code that is handed one is easy.

13 habits worth starting with

Name the behaviour, not the function. test_rejects_empty_name beats test_create_user_2.
One idea per test. Four unrelated asserts report the first failure and hide the rest.
Write the failing test first when fixing a bug. If it does not fail before your fix, it is not testing the fix.
Check the unhappy path. Empty input, missing file, wrong type. That is where the bugs are.

14 when something looks wrong

test not running	check the name, then `--collect-only`
fixture not found	spelling, or it is not in `conftest.py`
passes alone, fails together	something is shared between tests
passes here, fails on CI	a path, a clock, or an installed package
no output shown	pytest hides prints on pass. use `-s`

Stuck on a failure? pytest --lf -x --tb=short gets you the shortest useful loop: last failure only, stop at it, short traceback.