Prowler MCP Testing

Critical Rules

• ALWAYS drive tools through an in-memory client: async with Client(mcp_root_server). Tool parameters use pydantic Field(default=...), and only FastMCP's wrapper resolves those defaults. Calling a tool method directly with an argument omitted leaves it as a raw FieldInfo — which is truthy, so if email: silently builds a filter out of the FieldInfo repr. Direct calls MUST pass every argument.
• NEVER open a fastmcp.Client inside a fixture. FastMCP warns this causes hard-to-diagnose event-loop issues; open it inline in the test.
• ALWAYS use the mock_api_client fixture; NEVER construct a ProwlerAPIClient. Tool instances captured the singleton by reference at import time, so only an in-place patch of .client reaches them.
• NEVER clear SingletonMeta._instances. It orphans every registered tool on an instance holding a real httpx.AsyncClient. Use isolated_api_client if you genuinely need a fresh instance.
• NEVER strip PROWLER_API_KEY. Tools are built at import time and a construction failure is swallowed, so the whole prowler_* namespace silently drops to zero tools. It is pinned in [tool.pytest_env].
• For ProwlerAppAuth, pass mode= / base_url= explicitly. Those are resolved in default arguments, evaluated once at module import, so monkeypatch.setenv has no effect on them.
• NEVER assert an exact tool count — every future branch would have to bump it.
• Assert on result.data (structured output), not result.content[0].text.
• Tests are test_*.py (prefix), like the API — not the SDK's *_test.py suffix.
• __init__.py IS required in every tests/ subdirectory here (unlike the SDK's repo-root tests/), or same-named modules collide under pytest's import mode.
• Async tests need no marker (asyncio_mode = "auto"). Do not use @pytest.mark.anyio.
• Use only obviously-fake credentials from tests.helpers.tokens (TruffleHog).
• One behaviour per test; keep tests self-contained and order-independent.

---

1. Layout

Mirror the source tree below the package root — drop the prowler_mcp_server/ level, exactly as the SDK maps prowler/providers/... to tests/providers/.... So prowler_mcp_server/prowler_app/tools/ is tested in tests/prowler_app/tools/.

mcp_server/tests/
├── conftest.py                  # all shared fixtures
├── helpers/                     # jsonapi.py, http.py, assertions.py, tokens.py
├── test_server.py               # mounted-server contract
├── test_health.py
├── prowler_app/{models,tools,utils}/
├── prowler_hub/
└── prowler_documentation/

---

2. Fixtures

Fixture	Autouse	What it gives you
_pinned_environment	yes	Deterministic env; blocks a developer's .env from leaking
_no_real_network	yes	Any real socket connect raises RuntimeError
_singleton_registry_guard	yes	Snapshots/restores SingletonMeta._instances
mock_router	no	Route registry + request recorder
api_client	no	The live ProwlerAPIClient singleton
mock_api_client	no	The workhorse — singleton with a mocked transport
isolated_api_client	no	Evicts the singleton, for construction/identity tests
mcp_root_server	no	The mounted root server (session-scoped)
health_client	no	Starlette TestClient for /health
http_request_headers	no	Injects headers for HTTP-mode auth
hub_router	no	Mocks the Hub sub-server's two sync clients
docs_router	no	Mocks the docs search engine's two sync clients

MockRouter

mock_router.add("GET", "/api/v1/users", json=jsonapi_collection([...]))
mock_router.add("GET", "/api/v1/tasks/t1", json=task_document("t1", "completed"))

mock_router.request_for("GET", "/api/v1/users")     # last request, for header asserts
mock_router.query_params("GET", "/api/v1/users")    # decoded query string
mock_router.paths()                                 # everything requested so far

Register a route more than once to return a sequence — the last response repeats. That is how you drive poll_task_until_complete (executing, executing, completed). An unregistered request raises, listing what was registered.

---

3. Patterns

Tool test — see assets/mcp_tool_test.py. Register routes, call through the in-memory client, assert on result.data and on the recorded request. When a tool chooses between endpoints, assert mock_router.paths() — a wrong choice is invisible in the response body.

Model test — see assets/mcp_model_test.py. Build the document with the jsonapi helpers, run from_api_response(), assert on both the model and model_dump(). MinimalSerializerMixin makes those differ, and an absent relationship (None) must never be conflated with an empty one ([]).

Contract test — see assets/mcp_contract_test.py. Namespacing and description coverage across every registered tool.

The worked example in the repo is findings, covered across both layers in tests/prowler_app/{models,tools}/test_findings.py. Read those first — they exercise every foundation capability in one feature.

Reading coverage

Coverage has a meaningless high floor. Model modules are almost entirely class-body Field(...) declarations that execute at import, and prowler_app/server.py imports every model module at import time. Importing the package with zero tests already reports 36% overall, and individual model modules 54–84%.

So a model module at ~68% with no tests has none of its logic covered — the missing ranges are the from_api_response() bodies, which is the only part worth testing. Compare against the import-only floor, never against zero, and do not set a Codecov target from the raw total.

Where fixture data lives

tests/helpers/ is feature-agnostic and must stay that way: it holds the JSON:API shape, not any feature's data. Per-feature attribute dictionaries (FINDING_ATTRIBUTES, CHECK_METADATA, …) belong as module-level constants in the test module that uses them. Do not add feature fixtures to helpers/.

---

4. Commands

From mcp_server/:

cd mcp_server

uv run pytest                              # whole suite
uv run pytest tests/prowler_app/models     # one area
uv run pytest --cov=./prowler_mcp_server   # with coverage

From the repository root:

make test-mcp   # runs the MCP suite exactly as CI does

---

5. Reference

• Fixtures and the reasoning behind them: mcp_server/tests/conftest.py
• Testing section of docs/developer-guide/mcp-server.mdx
• Official FastMCP testing guide: https://gofastmcp.com/development/tests