Python unittest, regression evidence and temporary fixtures

What you will learn

• Express contracts as automated assertions and expected failures.
• Use subtests for input boundaries and temporary directories for real file tests.
• Describe coverage limits without claiming an untested production integration.

Theory: test behavior, not a green screenshot

A test arranges inputs, performs an action and asserts an observable result. Expected exceptions are results too. A passing test proves only the behavior exercised under that environment; it is not a certificate that every input, OS or deployment works.

The standard-library unittest runner discovers test methods. Subtests label multiple cases so failures point at the responsible input. A temporary fixture gives each file test owned data and cleanup, avoiding edits to a real learner/company file.

The first tests are unit-style function checks. The file test crosses encoding/filesystem/JSON boundaries, so it also provides local integration evidence. It is not a network or database test. Keep assertions on returned values, rejected data and unchanged input rather than repeating the implementation line by line.

Worked example: repeatable contract tests

Required: Python3.13+ with the previous two modules. Save this file as test_py_study.py beside py_study_functions.py and py_study_json.py. No pip package, server or paid service. The tests create and remove their own temporary files, never application data.

"""Run only in a personal practice folder; no database or network is used."""
import json
from pathlib import Path
from tempfile import TemporaryDirectory
import unittest

from py_study_functions import count_work_states, read_minutes
from py_study_json import total_from_file


class StudyFunctionsTest(unittest.TestCase):
    def test_minutes_boundaries_and_invalid_types(self):
        for value in [0, 30, 480]:
            with self.subTest(value=value):
                self.assertEqual(value, read_minutes(value))
        for value in [-1, 481, True, False, "30", 30.0, None, []]:
            with self.subTest(value=value):
                with self.assertRaises(ValueError):
                    read_minutes(value)

    def test_report_is_repeatable_without_mutation(self):
        rows = [{"status": "done"}, {"status": []}, {}]
        original = json.loads(json.dumps(rows))
        expected = {"todo": 0, "in_progress": 0, "done": 1, "unknown": 2}
        self.assertEqual(expected, count_work_states(rows))
        self.assertEqual(expected, count_work_states(rows))
        self.assertEqual(original, rows)
        with self.assertRaises(ValueError):
            count_work_states([{"status": "done"}, None])

    def test_real_file_success_and_shape_errors(self):
        with TemporaryDirectory(prefix="study-fixture-") as directory:
            path = Path(directory) / "rows.json"
            for sessions, expected in [([], 0), ([{"minutes": 15}, {"minutes": 30}], 45)]:
                path.write_text(json.dumps({"sessions": sessions}), encoding="utf-8")
                self.assertEqual(expected, total_from_file(path))
            for document in [[], {}, {"sessions": None}, {"sessions": [True]}, {"sessions": [{"minutes": True}]}]:
                path.write_text(json.dumps(document), encoding="utf-8")
                with self.assertRaises(ValueError):
                    total_from_file(path)
            path.write_text("{", encoding="utf-8")
            with self.assertRaises(json.JSONDecodeError):
                total_from_file(path)
            path.write_bytes(b"\xff")
            with self.assertRaises(UnicodeDecodeError):
                total_from_file(path)
            path.write_text('{"sessions": [], "extra": NaN}', encoding="utf-8")
            with self.assertRaises(ValueError):
                total_from_file(path)
            path.write_bytes(b'\xef\xbb\xbf{"sessions": []}')
            self.assertEqual(0, total_from_file(path))
            path.write_bytes(b" " * 65537)
            with self.assertRaises(ValueError):
                total_from_file(path)


if __name__ == "__main__":
    unittest.main()

Run and read the result

Windows PowerShell, with all practice files in the current folder:

python -m unittest -v test_py_study

Ubuntu Bash, with all practice files in the current folder:

python3 -m unittest -v test_py_study

The three test methods should pass; exact runner timing/text can vary. A nonzero exit is a failure to investigate, not a result to delete until the suite looks green. Actual Python3.13 Windows execution was verified; Ubuntu was not.

The tests cover exact integer/bool/range rules, fresh repeatable report state, unchanged rows and real temporary JSON files with malformed syntax, invalid encoding, invalid shape, NaN, optional BOM and byte limits. The existing modules still own the implementation; tests import them rather than copying their algorithm.

Guided lab

1. Run the unchanged tests and record the real result.
2. In a disposable copy, replace the exact integer check with a loose isinstance check. Observe the boolean regression, then restore the rule.
3. Introduce shared report counters in that copy. Confirm repeated-call assertions detect it.
4. Remove shape validation and observe which malformed-data tests fail.
5. Add a missing-path case and assert an I/O error. Explain why permission failures differ by OS and user account rather than claiming every environment was checked.

Independent exercise

Write tests for your earlier max/accepted-minutes exercise before changing its implementation. Cover empty input, valid zero, repeated calls, a later invalid value and unchanged original data. Add a subprocess test for CLI exit1/2 and absence of a success line on failure.

Hint: exercise the real module/file, assert the chosen contract and use temporary owned fixtures. If a dependency is replaced by a fake later, keep separate integration tests for that boundary.

Common mistakes and interview

Avoid using real files, catching all exceptions so tests never fail, asserting only that code ran, copying the calculation into the expected result, ignoring process exit codes and claiming mocked tests prove a live service.

Interview: “What regression did your test prevent?” Give the failing input, expected behavior, observed failure, minimal fix and verified limits. Explain unit versus file integration scope in concrete terms.

English practice and glossary

Assertion: an expected condition. Fixture: controlled test data/resources. Subtest: a labelled case within a test. Regression: previously working behavior breaking. Integration test: a check across a real boundary. Coverage limit: what was not exercised.

Write: “The test failed when __. The rule requires __. I fixed __ and reran __. I have not verified __.” Store the real command/result in the journal.

Recap and next step

Turn contracts and failures into repeatable tests using owned fixtures. Keep implementation and test responsibilities separate. Next apply these skills to bounded HTTP/automation/provider adapters, using fakes before any optional paid API.