Development setup

This page is for running ffiec-data-connect from source — contributors, people debugging a version that isn’t on PyPI yet, or anyone iterating on local changes without pip install each time. If you just want to use the library, start at Install instead.

Prerequisites

• Python 3.11+ (required for pandas 3.0 and modern type hints).
• git.
• A virtual environment tool — the examples below use venv.

Clone and install

git clone https://github.com/call-report/ffiec-data-connect.git
cd ffiec-data-connect

python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

pip install -e ".[dev]"

Editable (-e) installs mean source changes are picked up immediately; no reinstall is needed after each edit.

If .[dev] fails (older setuptools, missing extras), fall back to:

pip install -e .
pip install -r requirements-dev.txt

Verify:

python -c "import ffiec_data_connect as fdc; print(fdc.__version__)"

Running tests

The test suite is pytest:

# Everything
pytest

# One file
pytest tests/unit/test_methods.py

# By pattern
pytest -k collect_data

# Coverage report
pytest --cov=src/ffiec_data_connect --cov-report=html
open htmlcov/index.html

Unit tests live in tests/unit/; integration tests (hit live APIs, slower, credential-gated) live in tests/integration/. See Testing for credentials and marker conventions.

Linting and formatting

Ruff handles both:

ruff check src/          # lint
ruff check src/ --fix    # auto-fix what it can
ruff format src/         # format

If a mypy configuration is present:

mypy src/ffiec_data_connect

Building the Sphinx docs

pip install sphinx sphinx-rtd-theme
cd docs
make clean
make html
open build/html/index.html

The upstream Sphinx docs are the source of truth this site cross-checks against — see docs/source/*.rst in the library repo.

Project structure

ffiec-data-connect/
├── src/
│   └── ffiec_data_connect/
│       ├── __init__.py          # public re-exports
│       ├── credentials.py       # OAuth2Credentials
│       ├── methods.py           # the collect_* functions
│       ├── methods_enhanced.py  # REST-specific helpers
│       ├── protocol_adapter.py  # REST adapter
│       ├── xbrl_processor.py    # XBRL → DataFrame pipeline
│       ├── async_compatible.py  # AsyncCompatibleClient, RateLimiter
│       └── exceptions.py
├── tests/
│   ├── unit/
│   └── integration/
├── docs/
│   └── source/                  # RST sources for Sphinx
├── pyproject.toml
└── requirements-dev.txt

Anything underscore-prefixed or in methods_enhanced / xbrl_processor / utils is internal — Library reference is the stable surface.

Working with credentials during development

Tests and ad-hoc scripts look for these in the environment:

export FFIEC_USERNAME="your_username"
export FFIEC_BEARER_TOKEN="eyJ..."   # 90-day JWT, see /library/ffiec-data-connect/auth

Or use a .env file with python-dotenv:

from dotenv import load_dotenv
import os

load_dotenv()
username = os.environ["FFIEC_USERNAME"]
bearer_token = os.environ["FFIEC_BEARER_TOKEN"]

Never commit tokens. They’re 90-day credentials to a real CDR account.

Debugging

pdb works the usual way — breakpoint() in any source file, then run your script.

Library logs are under the ffiec_data_connect logger:

import logging
logging.basicConfig(level=logging.DEBUG)
logging.getLogger("ffiec_data_connect").setLevel(logging.DEBUG)

That will surface outgoing HTTP URLs, retry attempts, and token-expiration checks — the usual things you want visibility on when something fails.

Common setup problems

• ModuleNotFoundError: ffiec_data_connect — venv isn’t activated, or the editable install didn’t take. Verify with pip list | grep ffiec.
• Dependency resolver conflicts — easiest fix is a clean venv:

  deactivate && rm -rf venv
  python -m venv venv && source venv/bin/activate
  pip install -e ".[dev]"

• pandas version conflict — v3.0+ is now required. Uninstall any globally-installed older pandas before creating the venv.

Contributing changes upstream

1. Fork call-report/ffiec-data-connect on GitHub.
2. Branch from main.
3. Add tests for new behavior.
4. Run pytest and ruff check src/ before pushing.
5. Open a PR with a short description of the change and why.

See also Testing for the test layout and how to skip integration tests when you don’t have credentials.