Testing — Call.Report

This page is for contributors modifying ffiec-data-connect. If you’re just using the library, you don’t need to run these tests.

Layout

tests/
├── conftest.py          # shared fixtures; autouse Config reset
├── unit/                # 606 fast, offline tests
├── integration/         # 26 live REST API tests (skipped without creds)
├── mocks/               # in-process mock server implementations
└── fixtures/            # static test data (XBRL, SOAP XML, WSDL)

Unit tests (tests/unit/) run fully offline against mocks and fixtures. No network access or credentials required.
Integration tests (tests/integration/) hit the real FFIEC REST API and auto-skip when credentials aren’t in the environment.

The unit suite hits 100% statement coverage and ~99.7% branch coverage. CI runs it on Python 3.11, 3.12, 3.13, and 3.14. A complete per-test catalog lives in TESTS.md in the repository root.

Setup

See Development setup for the full environment setup. The short version:

git clone https://github.com/call-report/ffiec-data-connect.git
cd ffiec-data-connect
python -m venv .venv && source .venv/bin/activate
pip install -e ".[dev,polars]"

Or use the Makefile target that installs every extra used by tests, docs, and notebooks:

make install-dev     # pip install -e ".[dev,docs,notebook,polars]"

Running unit tests

No credentials or network required.

make test            # python -m pytest tests/unit/ -v
make test-fast       # credentials + methods + calling convention only

Or invoke pytest directly for finer-grained runs:

pytest tests/unit/test_credentials.py                                     # one file
pytest tests/unit/test_credentials.py::TestOAuth2Credentials              # one class
pytest tests/unit/test_credentials.py::TestOAuth2Credentials::test_valid  # one method
pytest tests/unit/ -k "credentials and not legacy"                        # keyword expr
pytest tests/unit/ -m "not slow"                                          # by marker

Registered markers (defined in pyproject.toml):

unit — unit-test marker
integration — integration-test marker
slow — long-running tests (memory leak, thread safety, etc.)

Running integration tests

Live tests in tests/integration/test_rest_api_live.py require a JWT. They auto-skip without credentials, so make test-all on a fresh checkout won’t fail for that reason.

Required environment variables:

FFIEC_USERNAME — FFIEC portal username
FFIEC_BEARER_TOKEN — 90-day JWT from the CDR portal (see Auth)

FFIEC_USERNAME='your_username' \
FFIEC_BEARER_TOKEN='eyJ...' \
pytest tests/integration/test_rest_api_live.py -v

Full suite (unit + integration) with credentials:

FFIEC_USERNAME='your_username' \
FFIEC_BEARER_TOKEN='eyJ...' \
make test-all

The live tests exercise every public REST endpoint against a reference bank (JPMorgan Chase, RSSD 480228) and a fixed reporting period (12/31/2024):

collect_reporting_periods — Call and UBPR series, list and pandas outputs
collect_data — RetrieveFacsimile with multiple date input formats and force_null_types variants
collect_filers_on_reporting_period — PanelOfReporters, including ZIP leading-zero preservation and dual RSSD field validation
collect_filers_since_date — FilersSinceDate
collect_filers_submission_date_time
collect_ubpr_reporting_periods — REST-only
collect_ubpr_facsimile_data — REST-only
SOAP deprecation behavior when a session is supplied
JWT exp claim extraction and expiry detection

An expired token causes every test to skip at fixture setup with a clear message, not a cryptic auth failure.

Coverage

make coverage        # terminal report with missing lines
make coverage-html   # htmlcov/index.html
make coverage-full   # HTML + XML + JSON (for upload services)

Configuration lives in .coveragerc and the [tool.coverage.*] sections of pyproject.toml.

Quality checks

Before opening a PR, mirror CI locally:

make format          # black + isort
make lint            # flake8
make type-check      # mypy on src/ffiec_data_connect
make check-all       # format + lint + type-check + test, in sequence

Style conventions:

black — line length 88 ([tool.black] in pyproject.toml)
isort — black profile
flake8 — max-line-length = 120; ignores E203, W503, E501; F401 ignored in __init__.py
mypy — relaxed, ignore_missing_imports = true

Conventions for new tests

Mock credentials with a spec. Mock(spec=OAuth2Credentials) catches attribute typos and keeps the mock aligned with the real interface:

from unittest.mock import Mock
from ffiec_data_connect import OAuth2Credentials

def test_something():
    creds = Mock(spec=OAuth2Credentials)
    creds.username = "test_user"
    creds.bearer_token = "eyJtest.token."
    ...

Don’t mutate os.environ to change Config. The global Config is reset between tests by the reset_config_after_test autouse fixture in tests/conftest.py. Use Config.set_legacy_errors(...) (or the legacy_mode_enabled / legacy_mode_disabled fixtures) instead:

from ffiec_data_connect import config

def test_typed_exceptions_are_raised():
    config.Config.set_legacy_errors(False)
    # ... assert a typed FFIECError subclass is raised

Mark tests appropriately:

import pytest

@pytest.mark.integration
def test_live_endpoint(live_creds):
    ...

@pytest.mark.slow
def test_large_batch():
    ...

Integration tests must auto-skip without creds. Follow tests/integration/test_rest_api_live.py: read env vars in a module-scoped fixture and pytest.skip(...) if missing.

Unit tests stay offline. Anything that reaches the network belongs under tests/integration/. The unit suite must be runnable without credentials or connectivity.

CI

GitHub Actions runs the suite on every push and PR across Python 3.11–3.14 on Linux.

Pull request — unit tests, coverage, lint, type-check on the full matrix. Integration tests don’t run in PR CI (they need secret creds).
Main branch — also runs extended tests (memory, thread safety).

When CI fails, reproduce locally with make check-all before pushing.