airbyte_ops_mcp.cli.cloud

CLI commands for Airbyte Cloud operations.

Commands:

airbyte-ops cloud connector get-version-info - Get connector version info airbyte-ops cloud connector set-version-override - Set connector version override airbyte-ops cloud connector clear-version-override - Clear connector version override airbyte-ops cloud connector regression-test - Run regression tests (single-version or comparison) airbyte-ops cloud connector fetch-connection-config - Fetch connection config to local file airbyte-ops cloud organization search - Search organizations by name/email airbyte-ops cloud organization info - Get organization info airbyte-ops cloud workspace search - Search workspaces by name/slug or email domain airbyte-ops cloud workspace info - Get workspace info

CLI reference

The commands below are regenerated by poe docs-generate via cyclopts's programmatic docs API; see docs/generate_cli.py.

airbyte-ops cloud COMMAND

Airbyte Cloud operations.

Commands:

  • connection: Connection operations in Airbyte Cloud.
  • connector: Deployed connector operations in Airbyte Cloud.
  • db: Database operations for Airbyte Cloud Prod DB Replica.
  • logs: GCP Cloud Logging operations for Airbyte Cloud.
  • organization: Organization operations in Airbyte Cloud.
  • workspace: Workspace operations in Airbyte Cloud.

airbyte-ops cloud connector

Deployed connector operations in Airbyte Cloud.

airbyte-ops cloud connector eu-runner

EU Cloud Batch regression runner.

Commands:

airbyte-ops cloud connector eu-runner build-job-spec
airbyte-ops cloud connector eu-runner build-job-spec JOB-ID SOURCE-ARCHIVE-URI RUN-PREFIX-URI CLI-ARGS-JSON GITHUB-RUN-ID CONNECTOR MAX-RUN-DURATION-SECONDS OUT [ARGS]

Write a Batch v1 Job JSON file.

Parameters:

  • JOB-ID, --job-id: Cloud Batch job ID. [required]
  • SOURCE-ARCHIVE-URI, --source-archive-uri: GCS source archive URI. [required]
  • RUN-PREFIX-URI, --run-prefix-uri: GCS run-prefix URI. [required]
  • CLI-ARGS-JSON, --cli-args-json: JSON array of CLI arguments. [required]
  • GITHUB-RUN-ID, --github-run-id: GitHub Actions run ID. [required]
  • CONNECTOR, --connector: Connector name for the Batch label. [required]
  • MAX-RUN-DURATION-SECONDS, --max-run-duration-seconds: Batch deadline in seconds. [required]
  • OUT, --out: Write the Batch Job JSON here. [required]
  • SEED-STATE-STAGED, --seed-state-staged, --no-seed-state-staged: A seed state was staged as state.json under the run prefix; the VM then stops when it cannot download it. [default: False]
  • TARGET-IMAGE, --target-image: The image the workflow built and staged as target-image.tar under the run prefix; the VM loads it and stops unless the archive holds that image alone. Empty when nothing was staged. [default: ""]
  • MACHINE-TYPE, --machine-type: [default: e2-standard-2]
  • CPU-MILLI, --cpu-milli: [default: 2000]
  • MEMORY-MIB, --memory-mib: [default: 6144]
  • DISK-GB, --disk-gb: [default: 30]
airbyte-ops cloud connector eu-runner render-summary
airbyte-ops cloud connector eu-runner render-summary BATCH-URL LOGS-URL REPORT-URL VERDICT

Print the GitHub step-summary Markdown.

Parameters:

  • BATCH-URL, --batch-url: Cloud Batch job URL. [required]
  • LOGS-URL, --logs-url: Cloud Logging URL. [required]
  • REPORT-URL, --report-url: Report object URL. [required]
  • VERDICT, --verdict: Metadata verdict JSON. [required]
airbyte-ops cloud connector eu-runner write-verdict
airbyte-ops cloud connector eu-runner write-verdict OUTPUT-DIR OUT

Write a masked metadata-only verdict JSON.

Parameters:

  • OUTPUT-DIR, --output-dir: Run output directory. [required]
  • OUT, --out: Write the metadata-only verdict here. [required]

airbyte-ops cloud connector get-version-info

airbyte-ops cloud connector get-version-info WORKSPACE-ID CONNECTOR-ID CONNECTOR-TYPE

Get the current version information for a deployed connector.

Parameters:

  • WORKSPACE-ID, --workspace-id: The Airbyte Cloud workspace ID. [required]
  • CONNECTOR-ID, --connector-id: The ID of the deployed connector (source or destination). [required]
  • CONNECTOR-TYPE, --connector-type: The type of connector. [required] [choices: source, destination]

airbyte-ops cloud connector set-version-override

airbyte-ops cloud connector set-version-override VERSION REASON ISSUE-URL APPROVAL-COMMENT-URL [ARGS]

Set a version override for a deployed connector.

Requires admin authentication via AIRBYTE_INTERNAL_ADMIN_FLAG and AIRBYTE_INTERNAL_ADMIN_USER environment variables.

The --override-level flag selects the scope at which the pin is applied:

  • actor (default): pins a single deployed connector instance. Requires --workspace-id, --connector-id, and --connector-type.
  • workspace: pins ALL instances of a connector type within a workspace. Requires --workspace-id and --actor-definition-id.
  • organization: pins ALL instances of a connector type across an organization. Requires --organization-id and --actor-definition-id.

The customer_tier_filter gates the operation: the call fails if the actual tier of the target organization does not match. Use ALL to proceed regardless of tier (a warning is shown for sensitive tiers).

Parameters:

  • VERSION, --version: The semver version string to pin to (e.g., '2.1.5-preview.abc1234'). [required]
  • REASON, --reason: Explanation for the override (min 10 characters). [required]
  • ISSUE-URL, --issue-url: GitHub issue URL providing context for this operation. [required]
  • APPROVAL-COMMENT-URL, --approval-comment-url: Slack approval record URL where admin authorized this deployment. [required]
  • OVERRIDE-LEVEL, --override-level: Scope at which to apply the version override: 'actor' (single deployed connector instance), 'workspace' (all instances of a connector type within a workspace), or 'organization' (all instances across an organization). Defaults to 'actor'. [choices: actor, workspace, organization] [default: actor]
  • WORKSPACE-ID, --workspace-id: The Airbyte Cloud workspace ID. Required for 'actor' and 'workspace' override levels.
  • ORGANIZATION-ID, --organization-id: The Airbyte Cloud organization ID. Required for 'organization' override level.
  • CONNECTOR-ID, --connector-id: The ID of the deployed connector (source or destination). Required for 'actor' override level.
  • CONNECTOR-TYPE, --connector-type: The type of connector. Required for 'actor' override level. [choices: source, destination]
  • ACTOR-DEFINITION-ID, --actor-definition-id: The connector definition UUID. Required for 'workspace' and 'organization' override levels.
  • AI-AGENT-SESSION-URL, --ai-agent-session-url: URL to AI agent session driving this operation (for auditability).
  • REASON-URL, --reason-url: Optional URL with more context (e.g., issue link).
  • CUSTOMER-TIER-FILTER, --customer-tier-filter: Tier filter: 'TIER_0', 'TIER_1', 'TIER_2', 'UNKNOWN', or 'ALL'. The operation will be rejected if the actual customer tier does not match. Defaults to 'TIER_2' (non-sensitive customers). [choices: TIER_0, TIER_1, TIER_2, UNKNOWN, ALL] [default: TIER_2]

airbyte-ops cloud connector clear-version-override

airbyte-ops cloud connector clear-version-override ISSUE-URL APPROVAL-COMMENT-URL [ARGS]

Clear a version override from a deployed connector.

Requires admin authentication via AIRBYTE_INTERNAL_ADMIN_FLAG and AIRBYTE_INTERNAL_ADMIN_USER environment variables.

The --override-level flag selects the scope at which the pin is removed:

  • actor (default): clears a single deployed connector instance pin. Requires --workspace-id, --connector-id, and --connector-type.
  • workspace: clears the workspace-level pin for a connector type. Requires --workspace-id and --actor-definition-id.
  • organization: clears the organization-level pin for a connector type. Requires --organization-id and --actor-definition-id.

Parameters:

  • ISSUE-URL, --issue-url: GitHub issue URL providing context for this operation. [required]
  • APPROVAL-COMMENT-URL, --approval-comment-url: Slack approval record URL where admin authorized this deployment. [required]
  • OVERRIDE-LEVEL, --override-level: Scope at which to clear the version override: 'actor', 'workspace', or 'organization'. Defaults to 'actor'. [choices: actor, workspace, organization] [default: actor]
  • WORKSPACE-ID, --workspace-id: The Airbyte Cloud workspace ID. Required for 'actor' and 'workspace' override levels.
  • ORGANIZATION-ID, --organization-id: The Airbyte Cloud organization ID. Required for 'organization' override level.
  • CONNECTOR-ID, --connector-id: The ID of the deployed connector (source or destination). Required for 'actor' override level.
  • CONNECTOR-TYPE, --connector-type: The type of connector. Required for 'actor' override level. [choices: source, destination]
  • ACTOR-DEFINITION-ID, --actor-definition-id: The connector definition UUID. Required for 'workspace' and 'organization' override levels.
  • AI-AGENT-SESSION-URL, --ai-agent-session-url: URL to AI agent session driving this operation (for auditability).
  • CUSTOMER-TIER-FILTER, --customer-tier-filter: Tier filter: 'TIER_0', 'TIER_1', 'TIER_2', 'UNKNOWN', or 'ALL'. The operation will be rejected if the actual customer tier does not match. Defaults to 'TIER_2' (non-sensitive customers). [choices: TIER_0, TIER_1, TIER_2, UNKNOWN, ALL] [default: TIER_2]

airbyte-ops cloud connector build-target-image

airbyte-ops cloud connector build-target-image CONNECTOR-NAME [ARGS]

Build the image regression-test --connector-name would build, and nothing else.

Writes the image to the image GitHub output. CI builds here, in a job that holds no credentials, because a Java connector's build runs the pull request's own Gradle on the host. The job that holds the credentials loads the image and passes it to regression-test as --test-image.

Parameters:

  • CONNECTOR-NAME, --connector-name: Connector to build from source (e.g., 'source-pokeapi'). [required]
  • REPO-ROOT, --repo-root: Path to the airbyte repo root, when it cannot be auto-detected.

airbyte-ops cloud connector regression-test

airbyte-ops cloud connector regression-test [ARGS]

Run regression tests on connectors.

This command supports two modes:

Comparison mode (skip_compare=False, default): Runs the specified Airbyte protocol command against both the target (new) and control (baseline) connector versions, then compares the results. This helps identify regressions between versions.

Single-version mode (skip_compare=True): Runs the specified Airbyte protocol command against a single connector and validates the output. No comparison is performed.

Results are written to the output directory and to GitHub Actions outputs if running in CI.

You can provide the test image in three ways:

  1. --test-image: Use a pre-built image from Docker registry
  2. --connector-name: Build the image locally from source code
  3. --connection-id: Auto-detect from an Airbyte Cloud connection

You can provide config/catalog either via file paths OR via a connection_id that fetches them from Airbyte Cloud.

Parameters:

  • SKIP-COMPARE, --skip-compare, --no-skip-compare: If True, skip comparison and run single-version tests only. If False (default), run comparison tests (target vs control). [default: False]
  • SKIP-RECORD-COMPARISON, --skip-record-comparison, --no-skip-record-comparison: If True, skip the record-level comparison for read commands (counts, PK presence, PK integrity, field values) and gate the verdict on command outcomes only. Escape hatch for sources whose data legitimately changes between the two live runs (rolling windows, feeds that age out records) until HTTP request caching makes strict comparison reliable. [default: False]
  • TEST-IMAGE, --test-image: Test connector image with tag (e.g., airbyte/source-github:1.0.0). When provided with connector_name, this image takes precedence over building from source. In comparison mode, it's compared against control_image.
  • CONTROL-IMAGE, --control-image: Control connector image (baseline version) with tag (e.g., airbyte/source-github:1.0.0). Ignored if skip_compare=True.
  • CONNECTOR-NAME, --connector-name: Connector name to build image from source (e.g., 'source-pokeapi'). If provided without test_image, builds the image locally with tag 'dev'. When both are provided, test_image takes precedence and no build occurs. For comparison tests (default), this builds the target image. For single-version tests (skip_compare=True), this builds the test image.
  • REPO-ROOT, --repo-root: Path to the airbyte repo root. Required if connector_name is provided and the repo cannot be auto-detected.
  • COMMAND, --command: The Airbyte command to run. [choices: spec, check, discover, read] [default: check]
  • CONNECTION-ID, --connection-id: Airbyte Cloud connection ID to fetch config/catalog from. Mutually exclusive with config-path/catalog-path. If provided, test_image/control_image can be auto-detected.
  • CONFIG-PATH, --config-path: Path to the connector config JSON file.
  • CATALOG-PATH, --catalog-path: Path to the configured catalog JSON file (required for read).
  • STATE-PATH, --state-path: Path to the state JSON file (optional for read).
  • OUTPUT-DIR, --output-dir: Directory to store test artifacts. [default: /tmp/regression_test_artifacts]
  • ENABLE-HTTP-METRICS, --enable-http-metrics, --no-enable-http-metrics: Capture HTTP traffic via mitmproxy. Comparison mode always captures, with or without replay -- the per-version request counts come from the capture -- so this flag adds nothing there, and --skip-compare runs a single version without a proxy. Requires mitmdump to be installed; without it, and for an image with no certifi bundle to trust the proxy with (a Java or Kotlin connector), the run proceeds without metrics. [default: False]
  • ENABLE-HTTP-REPLAY, --enable-http-replay, --no-http-replay: Replay the control run's recorded HTTP responses for the target run, so both versions see identical upstream data. Defaults to True in comparison mode; has no effect with --skip-compare, which has no control run to replay from. Pass --no-http-replay to run both versions against the live API, which is the fallback whenever replay cannot be delivered anyway; the traffic is still captured, so the report keeps its per-version request counts.
  • HTTP-REPLAY-MAX-BODY, --http-replay-max-body: Response bodies larger than this are streamed rather than recorded, and therefore re-fetched live instead of replayed. Raise it for a connector with large pages, at the cost of replay memory. Accepts a mitmproxy size such as '1m' or '10m' -- never a bare number, which means bytes. Unlike the other --http-replay-* options this one governs recording, so it applies to every comparison, replay or not. [default: 1m]
  • HTTP-REPLAY-MAX-DUMP-SIZE-MB, --http-replay-max-dump-size-mb: Skip replay and run the target live when the filtered replay corpus is larger than this, and discard a recorded dump this large once the run is done with it. mitmproxy holds the whole corpus in memory for the entire run, and a dump grows with everything the connector downloads, so this is what keeps a big connector from exhausting the CI runner's memory and disk. Dumps are never uploaded as artifacts, so on a local run -- where the dump is the only thing that can diagnose a coverage shortfall offline -- raise it to keep the dump. [default: 256]
  • HTTP-REPLAY-REUSE, --http-replay-reuse, --no-http-replay-reuse: Serve a recorded response more than once. By default each recorded flow is consumed on first match, which preserves recorded ordering and lets memory fall as the run proceeds; reuse keeps the whole corpus resident for the entire run. [default: False]
  • HTTP-REPLAY-IGNORE-PARAMS, --http-replay-ignore-params: Comma-separated query parameters to exclude from the replay match key, for connectors that put a timestamp or nonce in the URL.
  • STRICT-REPLAY, --strict-replay, --no-strict-replay: Kill requests that match nothing in the replay corpus instead of forwarding them upstream. Diagnostic only: it turns unmatched requests into connector errors, which is how you count them. mitmproxy stops intervening once its flowmap is empty, so combine this with --http-replay-reuse to cover the whole run rather than only the part before the corpus is consumed. [default: False]
  • SELECTED-STREAMS, --selected-streams: Comma-separated list of stream names to include in the read. Only these streams will be included in the configured catalog. This is useful to limit data volume by testing only specific streams.
  • DISCOVER-MISSING-STREAMS, --discover-missing-streams, --no-discover-missing-streams: Read selected streams the configured catalog does not have, such as a stream the PR adds: READ discovers them on the version under test and adds them to the catalog. A stream the control version also discovers is read by both versions and compared; one it does not discover has no control side and is read by the target only. Needs --selected-streams. A selected stream the target does not discover fails the run before READ, as a setup error. [default: False]
  • INJECTED-SYNC-MODE, --injected-sync-mode: Sync mode for streams added by --discover-missing-streams. Default: full refresh where the stream supports it. incremental uses the stream's default cursor and falls back to full refresh, with a note, for a stream that cannot be read incrementally. [choices: full_refresh, incremental]
  • ENABLE-DEBUG-LOGS, --enable-debug-logs, --no-enable-debug-logs: Enable debug-level logging for regression test output. Also passed as LOG_LEVEL=DEBUG to the connector Docker container. Switched off on a masked run for now: --output-mode masked, or a --connection-id run that leaves the mode unset. [default: False]
  • EXTRA-ENV, --extra-env, --empty-extra-env: Additional KEY=VALUE environment variables for the connector containers. In comparison mode, applies to both target and control. Values are visible in the debug log of the docker run command; do not use for secrets. LOG_LEVEL=DEBUG or TRACE here counts as debug logs and is dropped on a masked run.
  • WITH-STATE, --with-state, --no-state: Fetch and pass the connection's current state to the read command, producing a warm read instead of a cold read. Defaults to True when --connection-id is provided, False otherwise. Has no effect unless the command is read. Ignored when --state-path is explicitly provided.
  • READ-TIMEOUT, --read-timeout: The read deadline in seconds. A read still running at its deadline is stopped, keeps what it had written, and is reported as stopped early. In comparison mode it is the budget for the whole read command, from its start to its report: the two versions read one after the other and share it, ten minutes (or a tenth of the budget, if less) is kept for the comparison and the report, the control may use up to 60% of the read time left when it starts, and the target gets the rest; a version with no time left does not start. In single-version mode it covers the command the same way, with the same reserve, and the one read gets the rest. Only applies to read. In CI set it below the step's own timeout, so the comparison and the report still run on the partial output instead of the step being killed with nothing to show. [default: 14400]
  • MAX-RECORDS, --max-records: Stop each side of the read when a RECORD message arrives past this many; the record past the cap is not kept, and a read of no more than this many is complete. Bounds a read of a large connection; a read the cap stops is a smoke test of the records read before the cap, not a full regression, and the report says so on each side that stopped early. Has no effect unless the command is read.
  • OUTPUT-MODE, --output-mode: How the connector output is written to disk. masked masks every record value as a type-and-length token in the write pass -- primary keys are masked and a cursor stays readable only when its declared type is an integer or a date; DEBUG, TRACE, ERROR and FATAL log messages, stack traces, error internals and every stderr line keep only their length, while INFO and WARN log messages, a TRACE error's message and the CONNECTION_STATUS message stay readable. raw keeps the output as the connector emitted it. Unset is masked on a --connection-id run, which reads a customer's data, and raw otherwise; pass raw to keep a connection run's output as emitted, or masked to mask a GSM or local run. Debug logs are off on a masked run for now: --enable-debug-logs, and LOG_LEVEL=DEBUG or TRACE in --extra-env, are dropped with a note and the run goes on. [choices: masked, raw]
  • MASK-KEY-PATH, --mask-key-path: A file holding the run's masking key as 64 hex characters. Every command of one run (spec, check, discover, read) is given the same path, so their tokens pair within the run and with no other run's; without it the CLI makes a fresh key for this process, so a run by hand pairs its own two sides and nothing else. The key is never printed or written to any output. Ignored on a raw run.
  • REPRO-CASE-DIR, --repro-case-dir: Reproduction case directory (see regression-case-setup). Its catalog.json, state.json and expectations.json replace --catalog-path and --state-path, HTTP replay is turned off, and a read passes only when the control violates the case's expectations and the target satisfies them (see --repro-goal). Writes the verdict to result.json in the output directory. Requires --command read, --config-path, --sandbox-scenario and comparison mode; run the command under sandbox exec. Runs raw: --output-mode masked is refused.
  • REPRO-GOAL, --repro-goal: What a case read must show. fixed (default): the control violates the case and the target satisfies it. reproduced: both violate it -- the case gate's check, with the unmodified source of the affected release as the target. Requires --repro-case-dir. [choices: fixed, reproduced]
  • SANDBOX-SCENARIO, --sandbox-scenario: SaaS Sandbox scenario UUID the case was set up in. For a read, its baseline snapshot is restored immediately before each side runs, so control and target start from identical data and failure rules.

airbyte-ops cloud connector regression-case-setup

airbyte-ops cloud connector regression-case-setup CASE-DIR [ARGS]

Set up a reproduction case in the SaaS Sandbox.

Applies the case's sandbox/dataset.json together with its optional sandbox/behavior.json (failure rules), snapshots the result as the baseline that regression-test --repro-case-dir restores before each side of a read, and renders config.template.json with the scenario's synthetic credential. The sandbox connection comes from SAAS_API_URL, SAAS_API_KEY, SAAS_WORKSPACE_ID and XDG_STATE_HOME.

Writes scenario_id, config_path and the resolved case_dir to the GitHub step outputs.

Parameters:

  • CASE-DIR, --case-dir: Reproduction case directory to set up. [required]
  • CONFIG-OUTPUT, --config-output: Where to write the rendered connector config. Defaults to a private temporary directory outside the artifacts tree.

airbyte-ops cloud connector sandbox-support

airbyte-ops cloud connector sandbox-support CONNECTOR-NAME [ARGS]

Check statically whether a SaaS Sandbox twin covers a connector's needs.

The first gate before a reproduction case: provider host match, the HTTP operations, query parameters and page sizes the streams use, and optional dataset needs, against the sandbox's offline discovery data (sandbox ... --offline; needs the sandbox CLI, but no server). Prints a JSON report whose result is supported, unsupported or unknown. unknown means the requests could not be read statically; the case lane's twin-gap gate still decides during the run. With --host-only, only the provider match is decided (the oncall shadow gate's "has a twin" question).

Parameters:

  • CONNECTOR-NAME, --connector-name: Connector to check (e.g. 'source-slack'). [required]
  • STREAMS, --streams: Comma-separated streams the fix touches; their parent streams are included. Defaults to every stream in the manifest.
  • DATASET-NEEDS, --dataset-needs: Comma-separated entity or entity.field the reproduction needs in the twin's dataset (e.g. 'messages.thread_ts').
  • CONNECTOR-DIR, --connector-dir: Local connector directory. Without it, the files are read from airbytehq/airbyte at --ref.
  • REF, --ref: airbytehq/airbyte ref to read the connector from. [default: master]
  • HOST-ONLY, --host-only, --no-host-only: Only decide whether a twin serves the connector's provider: one serving an allowed host and, when the manifest's request hosts can be read, one of those (token endpoints don't count). No operation is checked, so operations is empty. Cannot be combined with --streams or --dataset-needs. [default: False]

airbyte-ops cloud connector regression-case-gate

airbyte-ops cloud connector regression-case-gate [ARGS]

Check whether a reproduction-case PR may merge without human review.

Prints a JSON report whose status is n/a (the PR touches no case), pass or fail, and exits nonzero on fail. Checks that the PR only adds one new case under regression_cases/<connector>/<case>/, as regular files with known names; that the case loads with an origin.json naming a published airbytehq/airbyte release whose source is at affected_ref, a commit on master; that the sandbox validates sandbox/ offline; and that the static support check says supported. The reproduction itself is a case run with goal reproduced. Writes connector, case, affected_image, affected_ref, problems (a JSON list) and status to the GitHub step outputs.

Parameters:

  • BASE-REF, --base-ref: The branch the PR merges into, e.g. origin/main. Required unless --case-dir is given.
  • HEAD-REF, --head-ref: The PR's head commit; the case is read at this commit. [default: HEAD]
  • REPO-DIR, --repo-dir: airbyte-ops-mcp clone holding both --base-ref and --head-ref. Its working tree is not read. [default: .]
  • CASE-DIR, --case-dir: Check this case directory on disk instead of a PR: no scope check, and the connector is the name of its parent directory.
  • REQUIRE-CASE, --require-case, --no-require-case: Fail, instead of answering n/a, when the PR adds no case. [default: False]
  • CONNECTOR-DIR, --connector-dir: Local connector directory at the case's affected_ref. Without it, the files are read from airbytehq/airbyte at affected_ref.
  • JSON-OUT, --json-out: Also write the JSON report to this file.
  • SANDBOX-REF, --sandbox-ref: The saas-sandbox commit the case runs use. Defaults to the pin in .github/saas-sandbox-ref of the current directory (the code checkout); the case's origin.json sandbox_ref must equal it.

airbyte-ops cloud connector regression-case-validate-decide

airbyte-ops cloud connector regression-case-validate-decide PR NEEDS GATE-REPORT [ARGS]

Decide what sandbox-case-validate.yml does after the gate and the runs.

Re-reads the PR through gh (set GH_REPO and GH_TOKEN) and prints a JSON decision: action (fail, report or merge), the commit status state and description, and on fail the PR comment, also written to --comment-out. Writes action, state, description and validated_sha to the GitHub step outputs.

Parameters:

  • PR, --pr: The case PR's number. [required]
  • NEEDS, --needs: File holding the workflow's toJSON(needs). [required]
  • GATE-REPORT, --gate-report: The gate's --json-out report; may be missing. [required]
  • MERGE-ENABLED, --merge-enabled, --no-merge-enabled: The merge switch (SANDBOX_CASE_AUTOMERGE_ENABLED). [default: False]
  • RUN-URL, --run-url: This run's URL, for the comment. [default: ""]
  • REPO, --repo: owner/name of the repository; defaults to GH_REPO.
  • COMMENT-OUT, --comment-out: Where to write the failure comment, when there is one. [default: comment.md]

airbyte-ops cloud connector regression-case-merge

airbyte-ops cloud connector regression-case-merge PR SHA CONNECTOR CASE [ARGS]

Merge a validated case PR at its validated head, waiting a bounded time.

Polls the PR through gh (set GH_TOKEN to a token that may merge) until GitHub reports it mergeable, then squash-merges it through the REST merge endpoint pinned to sha, with a fixed commit title. Never enables auto-merge. Prints the outcome and writes outcome, merge_state, state and description (the final commit status) to the GitHub step outputs. Exits nonzero unless the PR merged at sha or simply did not become mergeable in time.

Parameters:

  • PR, --pr: The case PR's number. [required]
  • SHA, --sha: The validated head commit. [required]
  • CONNECTOR, --connector: The case's connector, for the title. [required]
  • CASE, --case: The case's name, for the title. [required]
  • REPO, --repo: owner/name of the repository; defaults to GH_REPO.
  • RUN-URL, --run-url: The validating run, for the commit message. [default: ""]
  • TIMEOUT-MINUTES, --timeout-minutes: How long to wait for GitHub to report the PR mergeable. [default: 10]

airbyte-ops cloud connector consolidate-regression-reports

airbyte-ops cloud connector consolidate-regression-reports ARTIFACTS-DIR [ARGS]

Fold one run's per-command regression reports into a single page.

The regression test runs the CLI once per Airbyte command, so each command writes its own report into its own artifact. This produces the run-level view -- a verdict table across commands plus every report inlined -- so a reviewer downloads one file instead of four.

Never fails the run: a command whose report is missing or unreadable is skipped, because this is a reporting convenience and the per-command artifacts remain the source of truth.

Parameters:

  • ARTIFACTS-DIR, --artifacts-dir: Directory holding one subdirectory per command, each with the report.html that regression-test --output-dir wrote. [required]
  • OUTPUT, --output: Where to write the consolidated report. Defaults to report.html in --artifacts-dir.

airbyte-ops cloud connector fetch-connection-config

airbyte-ops cloud connector fetch-connection-config CONNECTION-ID [ARGS]

Fetch connection configuration from Airbyte Cloud to a local file.

This command retrieves the source configuration for a given connection ID and writes it to a JSON file. When --output-path is omitted the file is written to the platform temp directory to avoid accidentally committing secrets to a git repository.

Requires authentication via AIRBYTE_CLOUD_CLIENT_ID and AIRBYTE_CLOUD_CLIENT_SECRET environment variables.

When --with-secrets is specified, the command fetches unmasked secrets from the internal database using the connection-retriever. This additionally requires:

  • An OC issue URL for audit logging (--oc-issue-url)
  • GCP credentials via GCP_PROD_DB_ACCESS_CREDENTIALS env var or gcloud auth application-default login
  • Cloud SQL Python Connector access to the Prod DB replica.

Parameters:

  • CONNECTION-ID, --connection-id: The UUID of the Airbyte Cloud connection. [required]
  • OUTPUT-PATH, --output-path: Path to output file or directory. If directory, writes connection--config.json inside it. Default: platform temp directory (e.g. /tmp/connection--config.json)
  • WITH-SECRETS, --with-secrets, --no-secrets: If set, fetches unmasked secrets from the internal database. Requires GCP_PROD_DB_ACCESS_CREDENTIALS env var or gcloud auth application-default login. Must be used with --oc-issue-url. [default: False]
  • OC-ISSUE-URL, --oc-issue-url: OC issue URL for audit logging. Required when using --with-secrets.

airbyte-ops cloud connector rollout

Connector rollout operations.

Commands:

  • autopilot: AutoPilot rollout orchestration commands.
  • list: List connector rollouts from the production database.
airbyte-ops cloud connector rollout autopilot

AutoPilot rollout orchestration commands.

airbyte-ops cloud connector rollout autopilot auto-start
airbyte-ops cloud connector rollout autopilot auto-start [ARGS]

Start INITIALIZED rollouts that have autopilot auto-start enabled.

Queries the prod DB for rollouts in initialized state, checks the connector's autopilotConfig.autoStart gate, and starts eligible rollouts via the Cloud Config API.

Parameters:

  • CONNECTOR, --connector: Filter to a specific connector by canonical name (e.g., 'source-github').
  • DRY-RUN, --dry-run, --no-dry-run: Preview actions without executing them. [default: False]
airbyte-ops cloud connector rollout autopilot auto-advance
airbyte-ops cloud connector rollout autopilot auto-advance [ARGS]

Advance IN_PROGRESS rollouts within their current tier.

Advances in_progress rollouts that haven't reached their target percentage based on the configured strategy pacing (fast/slow/default).

Also recovers workflow_started rollouts: a rollout confirmed to have zero eligible actors is finalized as succeeded (0-of-0 eligible is complete, not stuck — promoting the RC to GA) instead of being left wedged; one with a positive estimate is re-driven; an unavailable estimate is skipped for a later retry.

Parameters:

  • CONNECTOR, --connector: Filter to a specific connector by canonical name (e.g., 'source-github').
  • DRY-RUN, --dry-run, --no-dry-run: Preview actions without executing them. [default: False]
airbyte-ops cloud connector rollout autopilot auto-promote
airbyte-ops cloud connector rollout autopilot auto-promote [ARGS]

Promote rollouts when at 100% of current tier.

Checks the autopilotConfig.autoPromoteStages gate, then scans the tier order forward for the next tier that has eligible actors and starts a rollout there, skipping empty intermediate tiers. If no later customer tier has actors (or the current tier is ALL), the current tier is finalized as succeeded (GA promotion).

Parameters:

  • CONNECTOR, --connector: Filter to a specific connector by canonical name (e.g., 'source-github').
  • DRY-RUN, --dry-run, --no-dry-run: Preview actions without executing them. [default: False]
airbyte-ops cloud connector rollout autopilot auto-triage-failed
airbyte-ops cloud connector rollout autopilot auto-triage-failed [ARGS]

Triage failed rollouts: log all failures, check unpin eligibility.

Finds rollouts in errored or paused state. Logs every failure unconditionally. Checks safe-to-downgrade eligibility per unsafeDowngrades. Actor-level unpinning is not yet implemented.

Parameters:

  • CONNECTOR, --connector: Filter to a specific connector by canonical name (e.g., 'source-github').
  • DRY-RUN, --dry-run, --no-dry-run: Preview actions without executing them. [default: False]
airbyte-ops cloud connector rollout autopilot auto-close
airbyte-ops cloud connector rollout autopilot auto-close [ARGS]

Close obsolete rollouts so only a connector's active candidate remains.

Closes a rollout when a newer RC supersedes it, when its RC is already the registry GA default, or when its RC is not the highest advertised release candidate for the connector. Uses retain_pins_on_cancellation=True so pinned actors are left undisturbed.

Parameters:

  • CONNECTOR, --connector: Filter to a specific connector by canonical name (e.g., 'source-github').
  • DRY-RUN, --dry-run, --no-dry-run: Preview actions without executing them. [default: False]
airbyte-ops cloud connector rollout autopilot auto-rollback-failed
airbyte-ops cloud connector rollout autopilot auto-rollback-failed [ARGS]

Full rollback/cancel of failed rollouts when safe to downgrade.

Finds rollouts in errored state. Checks the unsafeDowngrades gate. If the version is safe to downgrade, calls finalize_connector_rollout with state failed_rolled_back to cancel the entire rollout.

Parameters:

  • CONNECTOR, --connector: Filter to a specific connector by canonical name (e.g., 'source-github').
  • DRY-RUN, --dry-run, --no-dry-run: Preview actions without executing them. [default: False]
airbyte-ops cloud connector rollout list
airbyte-ops cloud connector rollout list [OPTIONS]

List connector rollouts from the production database.

Parameters:

  • --include-inactive, --no-include-inactive: Include terminal (canceled/succeeded/errored) rollouts. Requires --limit. [default: False]
  • --limit: Maximum number of rollouts to return. Required when --include-inactive is set. [default: 0]

airbyte-ops cloud db

Database operations for Airbyte Cloud Prod DB Replica.

airbyte-ops cloud db start-proxy

airbyte-ops cloud db start-proxy [ARGS]

Start the Cloud SQL Proxy for database access.

This command starts the Cloud SQL Auth Proxy to enable connections to the Airbyte Cloud Prod DB Replica. The proxy is required for database query tools.

By default, runs as a daemon (background process). Use --no-daemon to run in foreground mode where you can see logs and stop with Ctrl+C.

Credentials are read from the GCP_PROD_DB_ACCESS_CREDENTIALS environment variable, which should contain the service account JSON credentials.

After starting the proxy, set these environment variables to use database tools: export USE_CLOUD_SQL_PROXY=1 export DB_PORT={port}

Parameters:

  • PORT, --port: Port for the Cloud SQL Proxy to listen on. [default: 15432]
  • DAEMON, --daemon, --no-daemon: Run as daemon in background (default). Use --no-daemon for foreground. [default: True]

airbyte-ops cloud db stop-proxy

airbyte-ops cloud db stop-proxy

Stop the Cloud SQL Proxy daemon.

This command stops a Cloud SQL Proxy that was started with 'start-proxy'. It reads the PID from the PID file and sends a SIGTERM signal to stop the process.

airbyte-ops cloud connection

Connection operations in Airbyte Cloud.

airbyte-ops cloud connection state

Connection state operations in Airbyte Cloud.

Commands:

  • get: Get the current state for an Airbyte Cloud connection.
  • reset: Reset a configured stream's state so the next sync full-refreshes it.
  • set: Set the state for an Airbyte Cloud connection.
airbyte-ops cloud connection state get
airbyte-ops cloud connection state get CONNECTION-ID [ARGS]

Get the current state for an Airbyte Cloud connection.

Parameters:

  • CONNECTION-ID, --connection-id: The connection ID (UUID) to fetch state for. [required]
  • STREAM, --stream: Optional stream name to filter state for a single stream.
  • NAMESPACE, --namespace: Optional stream namespace to narrow the stream filter.
  • OUTPUT, --output: Path to write the state JSON output to a file.
airbyte-ops cloud connection state set
airbyte-ops cloud connection state set CONNECTION-ID [ARGS]

Set the state for an Airbyte Cloud connection.

State JSON can be provided as a positional argument, via --input , or piped through STDIN.

Uses the safe variant that prevents updates while a sync is running. When --stream is provided, only that stream's state is updated within the existing connection state.

Parameters:

  • CONNECTION-ID, --connection-id: The connection ID (UUID) to update state for. [required]
  • STATE-JSON, --state-json: The connection state as a JSON string. When --stream is used, this is just the stream's state blob (e.g., '{"cursor": "2024-01-01"}'). Otherwise, must include 'stateType', 'connectionId', and the appropriate state field. Can also be provided via --input or STDIN.
  • STREAM, --stream: Optional stream name to update state for a single stream only.
  • NAMESPACE, --namespace: Optional stream namespace to identify the stream.
  • INPUT, --input: Path to a JSON file containing the state to set.
airbyte-ops cloud connection state reset
airbyte-ops cloud connection state reset CONNECTION-ID STREAM [ARGS]

Reset a configured stream's state so the next sync full-refreshes it.

Uses the safe variant that prevents updates while a sync is running. Returns a restorable previous_state_backup in raw Config API format.

Parameters:

  • CONNECTION-ID, --connection-id: The connection ID (UUID) to update state for. [required]
  • STREAM, --stream: The configured stream name whose state should be reset. [required]
  • NAMESPACE, --namespace: Optional stream namespace to identify the stream.

airbyte-ops cloud connection catalog

Connection catalog operations in Airbyte Cloud.

Commands:

  • get: Get the configured catalog for an Airbyte Cloud connection.
  • set: Set the configured catalog for an Airbyte Cloud connection.
airbyte-ops cloud connection catalog get
airbyte-ops cloud connection catalog get CONNECTION-ID [ARGS]

Get the configured catalog for an Airbyte Cloud connection.

Parameters:

  • CONNECTION-ID, --connection-id: The connection ID (UUID) to fetch catalog for. [required]
  • OUTPUT, --output: Path to write the catalog JSON output to a file.
airbyte-ops cloud connection catalog set
airbyte-ops cloud connection catalog set CONNECTION-ID [ARGS]

Set the configured catalog for an Airbyte Cloud connection.

Catalog JSON can be provided as a positional argument, via --input , or piped through STDIN.

WARNING: This replaces the entire configured catalog.

Parameters:

  • CONNECTION-ID, --connection-id: The connection ID (UUID) to update catalog for. [required]
  • CATALOG-JSON, --catalog-json: The configured catalog as a JSON string. Can also be provided via --input or STDIN.
  • INPUT, --input: Path to a JSON file containing the catalog to set.

airbyte-ops cloud connection get-region

airbyte-ops cloud connection get-region CONNECTION-ID

Get a connection's dataplane group and derived residency.

Reads only the dataplane group, not connection or actor secrets. Requires GCP_PROD_DB_ACCESS_CREDENTIALS or Application Default Credentials. On any lookup failure it exits non-zero and writes no GitHub output.

Parameters:

  • CONNECTION-ID, --connection-id: The connection ID (UUID) to check. [required]

airbyte-ops cloud logs

GCP Cloud Logging operations for Airbyte Cloud.

airbyte-ops cloud logs lookup-cloud-backend-error

airbyte-ops cloud logs lookup-cloud-backend-error ERROR-ID [ARGS]

Look up error details from GCP Cloud Logging by error ID.

When an Airbyte Cloud API returns an error response with only an error ID (e.g., {"errorId": "3173452e-8f22-4286-a1ec-b0f16c1e078a"}), this command fetches the full stack trace and error details from GCP Cloud Logging.

Requires GCP credentials with Logs Viewer role on the target project. Set up credentials with: gcloud auth application-default login

Parameters:

  • ERROR-ID, --error-id: The error ID (UUID) to search for. This is typically returned in API error responses as {'errorId': '...'} [required]
  • LOOKBACK-DAYS, --lookback-days: Number of days to look back in logs. [default: 7]
  • MIN-SEVERITY-FILTER, --min-severity-filter: Optional minimum severity level to filter logs. [choices: debug, info, notice, warning, error, critical, alert, emergency]
  • RAW, --raw, --no-raw: Output raw JSON instead of formatted text. [default: False]

airbyte-ops cloud organization

Organization operations in Airbyte Cloud.

airbyte-ops cloud organization search NAME-CONTAINS [ARGS]

Search organizations by name or email substring.

Parameters:

  • NAME-CONTAINS, --name-contains: Case-insensitive substring to match against organization name or email. [required]
  • LIMIT, --limit: Maximum number of results to return. [default: 20]

airbyte-ops cloud organization info

airbyte-ops cloud organization info ORGANIZATION-ID

Get information about an organization (stub — not yet implemented).

Parameters:

  • ORGANIZATION-ID, --organization-id: The organization UUID. [required]

airbyte-ops cloud workspace

Workspace operations in Airbyte Cloud.

airbyte-ops cloud workspace search [ARGS]

Search workspaces by name substring or email domain.

Parameters:

  • NAME-CONTAINS, --name-contains: Case-insensitive substring to match against workspace name or slug.
  • EMAIL-DOMAIN, --email-domain: Email domain to search for (e.g. 'motherduck.com').
  • LIMIT, --limit: Maximum number of results to return. [default: 100]

airbyte-ops cloud workspace info

airbyte-ops cloud workspace info WORKSPACE-ID

Get information about a workspace.

Parameters:

  • WORKSPACE-ID, --workspace-id: The workspace UUID. [required]
   1# Copyright (c) 2025 Airbyte, Inc., all rights reserved.
   2"""CLI commands for Airbyte Cloud operations.
   3
   4Commands:
   5    airbyte-ops cloud connector get-version-info - Get connector version info
   6    airbyte-ops cloud connector set-version-override - Set connector version override
   7    airbyte-ops cloud connector clear-version-override - Clear connector version override
   8    airbyte-ops cloud connector regression-test - Run regression tests (single-version or comparison)
   9    airbyte-ops cloud connector fetch-connection-config - Fetch connection config to local file
  10    airbyte-ops cloud organization search - Search organizations by name/email
  11    airbyte-ops cloud organization info - Get organization info
  12    airbyte-ops cloud workspace search - Search workspaces by name/slug or email domain
  13    airbyte-ops cloud workspace info - Get workspace info
  14
  15## CLI reference
  16
  17The commands below are regenerated by `poe docs-generate` via cyclopts's
  18programmatic docs API; see `docs/generate_cli.py`.
  19
  20.. include:: ../../../docs/generated/cli/cloud.md
  21   :start-line: 2
  22"""
  23
  24from __future__ import annotations
  25
  26# Hide Python-level members from the pdoc page for this module; the rendered
  27# docs for this CLI group come entirely from the grafted `.. include::` in
  28# the module docstring above.
  29__all__: list[str] = []
  30
  31import atexit
  32import json
  33import logging
  34import os
  35import secrets
  36import shutil
  37import signal
  38import socket
  39import subprocess
  40import sys
  41import tempfile
  42import time
  43import uuid
  44from collections.abc import Callable
  45from contextlib import suppress
  46from pathlib import Path
  47from typing import Annotated, Any, Literal, NoReturn
  48
  49import google.auth.exceptions
  50import requests
  51import sqlalchemy.exc
  52import yaml
  53from airbyte.cloud.workspaces import CloudWorkspace
  54from airbyte.exceptions import PyAirbyteInputError
  55from airbyte_cdk.models.connector_metadata import MetadataFile
  56from airbyte_cdk.utils.connector_paths import find_connector_root_from_name
  57from airbyte_cdk.utils.docker import build_connector_image, verify_docker_installation
  58from airbyte_protocol.models import AirbyteCatalog, ConfiguredAirbyteCatalog
  59from cyclopts import Parameter
  60from fastmcp_extensions.cli import (
  61    exit_with_error,
  62    print_error,
  63    print_json,
  64    print_success,
  65    print_warning,
  66)
  67from google.api_core import exceptions as google_api_exceptions
  68
  69from airbyte_ops_mcp.cli._base import App, app
  70from airbyte_ops_mcp.cloud_admin.auth import CloudAuthError
  71from airbyte_ops_mcp.cloud_admin.connection_config import fetch_connection_config
  72from airbyte_ops_mcp.cloud_admin.connection_state import (
  73    reset_stream_state as reset_cloud_stream_state,
  74)
  75from airbyte_ops_mcp.cloud_admin.registry_lookup import (
  76    resolve_definition_id_to_canonical_info,
  77)
  78from airbyte_ops_mcp.cloud_admin.version_overrides import (
  79    ResolvedCloudAuth,
  80    VersionOverrideTarget,
  81    get_connector_version_info,
  82)
  83from airbyte_ops_mcp.cloud_admin.version_overrides import (
  84    set_version_override as set_cloud_version_override,
  85)
  86from airbyte_ops_mcp.connection_config_retriever import (
  87    ConnectionNotFoundError,
  88    get_connection_dataplane_group,
  89    get_connection_residency,
  90    residency_for_dataplane_group,
  91)
  92from airbyte_ops_mcp.connection_config_retriever.residency import (
  93    EU_REGION_EU_ONLY_MESSAGE,
  94    EU_REGION_REQUIRED_MESSAGE,
  95    is_running_in_eu_region,
  96)
  97from airbyte_ops_mcp.connector_secrets import (
  98    DEFAULT_GCP_PROJECT_ID,
  99    ConnectorSecretsUnavailableError,
 100)
 101from airbyte_ops_mcp.constants import (
 102    CLOUD_SQL_INSTANCE,
 103    CLOUD_SQL_PROXY_PID_FILE,
 104    DEFAULT_CLOUD_SQL_PROXY_PORT,
 105    ENV_GCP_PROD_DB_ACCESS_CREDENTIALS,
 106)
 107from airbyte_ops_mcp.gcp_logs import GCPSeverity, fetch_error_logs
 108from airbyte_ops_mcp.github_api import resolve_default_github_token
 109from airbyte_ops_mcp.prod_db_access.db_engine import (
 110    CloudRunProdDbAccessNotConfiguredError,
 111    ProdDbCloudSqlIamError,
 112    ProdDbSecretAccessError,
 113)
 114from airbyte_ops_mcp.prod_db_access.queries import (
 115    query_workspace_info,
 116    query_workspaces_by_email_domain,
 117)
 118from airbyte_ops_mcp.prod_db_access.queries import (
 119    search_organizations as _search_organizations,
 120)
 121from airbyte_ops_mcp.prod_db_access.queries import (
 122    search_workspaces as _search_workspaces,
 123)
 124from airbyte_ops_mcp.regression_tests.case_gate import (
 125    NAME_PATTERN,
 126    SANDBOX_PIN_FILE,
 127    gate_step_outputs,
 128    read_sandbox_pin,
 129    run_case_dir_gate,
 130    run_case_gate,
 131)
 132from airbyte_ops_mcp.regression_tests.case_validate import (
 133    decide_validation,
 134    gh_merge,
 135    gh_read_pr,
 136    runs_from_needs,
 137    wait_and_merge,
 138)
 139from airbyte_ops_mcp.regression_tests.cdk_secrets import (
 140    get_integration_test_config_from_secrets,
 141)
 142from airbyte_ops_mcp.regression_tests.ci_output import (
 143    ComparisonOutcome,
 144    command_result_succeeded,
 145    config_error_streams,
 146    could_not_test,
 147    describe_could_not_test,
 148    describe_setup_error,
 149    evaluate_comparison_outcome,
 150    generate_regression_report,
 151    generate_single_version_report,
 152    is_github_actions,
 153    run_readable_fields,
 154    stopped_early_sides,
 155    target_errors_only_from,
 156    undefined_streams,
 157    write_github_output,
 158    write_github_outputs,
 159    write_github_summary,
 160    write_json_output,
 161)
 162from airbyte_ops_mcp.regression_tests.config_overrides import (
 163    InjectedCatalogs,
 164    InjectedSyncMode,
 165    StreamsNotDiscoveredError,
 166    build_injected_catalogs,
 167    filter_configured_catalog_file,
 168)
 169from airbyte_ops_mcp.regression_tests.connection_fetcher import (
 170    enrich_catalog_schemas_from_cloud,
 171    fetch_connection_data,
 172    fetch_connection_state,
 173    save_connection_data_to_files,
 174    streams_missing_schemas,
 175)
 176from airbyte_ops_mcp.regression_tests.connection_secret_retriever import (
 177    SecretRetrievalError,
 178    enrich_catalog_schemas_from_platform_db,
 179    enrich_config_with_secrets,
 180    should_use_secret_retriever,
 181)
 182from airbyte_ops_mcp.regression_tests.connector_runner import (
 183    DEFAULT_TIMEOUT_SECONDS,
 184    ConnectorRunner,
 185    ensure_image_available,
 186    proxy_trust_gap,
 187)
 188from airbyte_ops_mcp.regression_tests.eu_runner.cli import eu_runner_app
 189from airbyte_ops_mcp.regression_tests.http_metrics import (
 190    DEFAULT_MAX_REPLAY_DUMP_SIZE_MB,
 191    DEFAULT_REPLAY_EXTRA,
 192    DEFAULT_STREAM_LARGE_BODIES,
 193    MitmproxyManager,
 194    ReplayOptions,
 195    discard_oversized_dump,
 196    parse_connector_cache_metrics,
 197    parse_http_dump,
 198)
 199from airbyte_ops_mcp.regression_tests.masking import (
 200    MaskPolicy,
 201    mask_input_state,
 202)
 203from airbyte_ops_mcp.regression_tests.models import (
 204    Command,
 205    ComparableOutputs,
 206    ConnectorUnderTest,
 207    ExecutionInputs,
 208    OutputMode,
 209    StoppedEarly,
 210    TargetOrControl,
 211    duplicate_stream_names,
 212    get_primary_keys_per_stream,
 213    split_records_per_stream,
 214)
 215from airbyte_ops_mcp.regression_tests.normalization import stream_label, to_stream_id
 216from airbyte_ops_mcp.regression_tests.read_budget import ReadBudget
 217from airbyte_ops_mcp.regression_tests.regression import (
 218    ComparisonResult,
 219    RecordComparisonSummary,
 220    compare_catalog_schemas,
 221    compare_final_states,
 222    compare_specs,
 223    run_record_comparisons_from_files,
 224)
 225from airbyte_ops_mcp.regression_tests.regression.comparators import (
 226    _read_records_file,
 227    _scan_pk_integrity,
 228)
 229from airbyte_ops_mcp.regression_tests.report import (
 230    HTML_REPORT_FILENAME,
 231    CheckResult,
 232    DiffBlock,
 233    DiffLine,
 234    build_comparison_report_model,
 235    build_diff_block,
 236    build_json_block,
 237    build_single_version_report_model,
 238    consolidate_reports,
 239    enrich_report_model,
 240    failed_checks,
 241    render_inline_summary,
 242    report_section,
 243    write_html_report,
 244)
 245from airbyte_ops_mcp.regression_tests.repro_case import (
 246    CASE_RESULT_FILE,
 247    SHA_PATTERN,
 248    ReproCase,
 249    ReproductionVerdict,
 250    ReproGoal,
 251    SandboxCli,
 252    SandboxError,
 253    SandboxSide,
 254    case_result,
 255    check_twin_gaps,
 256    evaluate_reproduction,
 257    evaluate_side,
 258    restore_side,
 259    set_up_scenario,
 260)
 261from airbyte_ops_mcp.regression_tests.sandbox_support import (
 262    run_sandbox_support_check,
 263)
 264from airbyte_ops_mcp.regression_tests.seed_state import (
 265    seed_state_error,
 266    seeded_streams,
 267)
 268from airbyte_ops_mcp.regression_tests.validation.catalog_validators import (
 269    validate_cursors_exist_in_schema,
 270    validate_primary_keys_exist_in_schema,
 271)
 272from airbyte_ops_mcp.regression_tests.validation.record_validators import (
 273    check_incremental_cursor,
 274    check_record_schema,
 275    declared_types,
 276)
 277from airbyte_ops_mcp.telemetry import track_regression_test
 278from airbyte_ops_mcp.tier_cache import resolve_workspace
 279
 280# Path to connectors directory within the airbyte repo
 281CONNECTORS_SUBDIR = Path("airbyte-integrations") / "connectors"
 282
 283# Create the cloud sub-app
 284cloud_app = App(name="cloud", help="Airbyte Cloud operations.")
 285app.command(cloud_app)
 286
 287# Create the connector sub-app under cloud
 288connector_app = App(
 289    name="connector", help="Deployed connector operations in Airbyte Cloud."
 290)
 291cloud_app.command(connector_app)
 292connector_app.command(eu_runner_app)
 293
 294# Create the db sub-app under cloud
 295db_app = App(name="db", help="Database operations for Airbyte Cloud Prod DB Replica.")
 296cloud_app.command(db_app)
 297
 298# Create the connection sub-app under cloud
 299connection_app = App(name="connection", help="Connection operations in Airbyte Cloud.")
 300cloud_app.command(connection_app)
 301
 302# Create the state sub-app under connection
 303state_app = App(name="state", help="Connection state operations in Airbyte Cloud.")
 304connection_app.command(state_app)
 305
 306# Create the catalog sub-app under connection
 307catalog_app = App(
 308    name="catalog", help="Connection catalog operations in Airbyte Cloud."
 309)
 310connection_app.command(catalog_app)
 311
 312# Create the logs sub-app under cloud
 313logs_app = App(name="logs", help="GCP Cloud Logging operations for Airbyte Cloud.")
 314cloud_app.command(logs_app)
 315
 316# Create the organization sub-app under cloud
 317organization_app = App(
 318    name="organization", help="Organization operations in Airbyte Cloud."
 319)
 320cloud_app.command(organization_app)
 321
 322# Create the workspace sub-app under cloud
 323workspace_app = App(name="workspace", help="Workspace operations in Airbyte Cloud.")
 324cloud_app.command(workspace_app)
 325
 326
 327@db_app.command(name="start-proxy")
 328def start_proxy(
 329    port: Annotated[
 330        int,
 331        Parameter(help="Port for the Cloud SQL Proxy to listen on."),
 332    ] = DEFAULT_CLOUD_SQL_PROXY_PORT,
 333    daemon: Annotated[
 334        bool,
 335        Parameter(
 336            help="Run as daemon in background (default). Use --no-daemon for foreground."
 337        ),
 338    ] = True,
 339) -> None:
 340    """Start the Cloud SQL Proxy for database access.
 341
 342    This command starts the Cloud SQL Auth Proxy to enable connections to the
 343    Airbyte Cloud Prod DB Replica. The proxy is required for database query tools.
 344
 345    By default, runs as a daemon (background process). Use --no-daemon to run in
 346    foreground mode where you can see logs and stop with Ctrl+C.
 347
 348    Credentials are read from the GCP_PROD_DB_ACCESS_CREDENTIALS environment variable,
 349    which should contain the service account JSON credentials.
 350
 351    After starting the proxy, set these environment variables to use database tools:
 352        export USE_CLOUD_SQL_PROXY=1
 353        export DB_PORT={port}
 354
 355    Example:
 356        airbyte-ops cloud db start-proxy
 357        airbyte-ops cloud db start-proxy --port 15432
 358        airbyte-ops cloud db start-proxy --no-daemon
 359    """
 360    # Check if proxy is already running on the requested port (idempotency)
 361    try:
 362        with socket.create_connection(("127.0.0.1", port), timeout=0.5):
 363            # Something is already listening on this port
 364            pid_file = Path(CLOUD_SQL_PROXY_PID_FILE)
 365            pid_info = ""
 366            if pid_file.exists():
 367                pid_info = f" (PID: {pid_file.read_text().strip()})"
 368            print_success(
 369                f"Cloud SQL Proxy is already running on port {port}{pid_info}"
 370            )
 371            print_success("")
 372            print_success("To use database tools, set these environment variables:")
 373            print_success("  export USE_CLOUD_SQL_PROXY=1")
 374            print_success(f"  export DB_PORT={port}")
 375            return
 376    except (OSError, TimeoutError, ConnectionRefusedError):
 377        pass  # Port not in use, proceed with starting proxy
 378
 379    # Check if cloud-sql-proxy is installed
 380    proxy_path = shutil.which("cloud-sql-proxy")
 381    if not proxy_path:
 382        exit_with_error(
 383            "cloud-sql-proxy not found in PATH. "
 384            "Install it from: https://cloud.google.com/sql/docs/mysql/sql-proxy"
 385        )
 386
 387    # Get credentials from environment
 388    creds_json = os.getenv(ENV_GCP_PROD_DB_ACCESS_CREDENTIALS)
 389    if not creds_json:
 390        exit_with_error(
 391            f"{ENV_GCP_PROD_DB_ACCESS_CREDENTIALS} environment variable is not set. "
 392            "This should contain the GCP service account JSON credentials."
 393        )
 394
 395    # Build the command using --json-credentials to avoid writing to disk
 396    cmd = [
 397        proxy_path,
 398        CLOUD_SQL_INSTANCE,
 399        f"--port={port}",
 400        f"--json-credentials={creds_json}",
 401    ]
 402
 403    print_success(f"Starting Cloud SQL Proxy on port {port}...")
 404    print_success(f"Instance: {CLOUD_SQL_INSTANCE}")
 405    print_success("")
 406    print_success("To use database tools, set these environment variables:")
 407    print_success("  export USE_CLOUD_SQL_PROXY=1")
 408    print_success(f"  export DB_PORT={port}")
 409    print_success("")
 410
 411    if daemon:
 412        # Run in background (daemon mode) with log file for diagnostics
 413        log_file_path = Path("/tmp/airbyte-cloud-sql-proxy.log")
 414        log_file = log_file_path.open("ab")
 415        process = subprocess.Popen(
 416            cmd,
 417            stdout=subprocess.DEVNULL,
 418            stderr=log_file,
 419            start_new_session=True,
 420        )
 421
 422        # Brief wait to verify the process started successfully
 423        time.sleep(0.5)
 424        if process.poll() is not None:
 425            # Process exited immediately - read any error output
 426            log_file.close()
 427            error_output = ""
 428            if log_file_path.exists():
 429                error_output = log_file_path.read_text()[-1000:]  # Last 1000 chars
 430            exit_with_error(
 431                f"Cloud SQL Proxy failed to start (exit code: {process.returncode}).\n"
 432                f"Check logs at {log_file_path}\n"
 433                f"Recent output: {error_output}"
 434            )
 435
 436        # Write PID to file for stop-proxy command
 437        pid_file = Path(CLOUD_SQL_PROXY_PID_FILE)
 438        pid_file.write_text(str(process.pid))
 439        print_success(f"Cloud SQL Proxy started as daemon (PID: {process.pid})")
 440        print_success(f"Logs: {log_file_path}")
 441        print_success("To stop: airbyte-ops cloud db stop-proxy")
 442    else:
 443        # Run in foreground - replace current process
 444        # Signals (Ctrl+C) will be handled directly by the cloud-sql-proxy process
 445        print_success("Running in foreground. Press Ctrl+C to stop the proxy.")
 446        print_success("")
 447        os.execv(proxy_path, cmd)
 448
 449
 450@db_app.command(name="stop-proxy")
 451def stop_proxy() -> None:
 452    """Stop the Cloud SQL Proxy daemon.
 453
 454    This command stops a Cloud SQL Proxy that was started with 'start-proxy'.
 455    It reads the PID from the PID file and sends a SIGTERM signal to stop the process.
 456
 457    Example:
 458        airbyte-ops cloud db stop-proxy
 459    """
 460    pid_file = Path(CLOUD_SQL_PROXY_PID_FILE)
 461
 462    if not pid_file.exists():
 463        exit_with_error(
 464            f"PID file not found at {CLOUD_SQL_PROXY_PID_FILE}. "
 465            "No Cloud SQL Proxy daemon appears to be running."
 466        )
 467
 468    pid_str = pid_file.read_text().strip()
 469    if not pid_str.isdigit():
 470        pid_file.unlink()
 471        exit_with_error(f"Invalid PID in {CLOUD_SQL_PROXY_PID_FILE}: {pid_str}")
 472
 473    pid = int(pid_str)
 474
 475    # Check if process is still running
 476    try:
 477        os.kill(pid, 0)  # Signal 0 just checks if process exists
 478    except ProcessLookupError:
 479        pid_file.unlink()
 480        print_success(
 481            f"Cloud SQL Proxy (PID: {pid}) is not running. Cleaned up PID file."
 482        )
 483        return
 484    except PermissionError:
 485        exit_with_error(f"Permission denied to check process {pid}.")
 486
 487    # Send SIGTERM to stop the process
 488    try:
 489        os.kill(pid, signal.SIGTERM)
 490        print_success(f"Sent SIGTERM to Cloud SQL Proxy (PID: {pid}).")
 491    except ProcessLookupError:
 492        print_success(f"Cloud SQL Proxy (PID: {pid}) already stopped.")
 493    except PermissionError:
 494        exit_with_error(f"Permission denied to stop process {pid}.")
 495
 496    # Clean up PID file
 497    pid_file.unlink(missing_ok=True)
 498    print_success("Cloud SQL Proxy stopped.")
 499
 500
 501@connector_app.command(name="get-version-info")
 502def get_version_info(
 503    workspace_id: Annotated[
 504        str,
 505        Parameter(help="The Airbyte Cloud workspace ID."),
 506    ],
 507    connector_id: Annotated[
 508        str,
 509        Parameter(help="The ID of the deployed connector (source or destination)."),
 510    ],
 511    connector_type: Annotated[
 512        Literal["source", "destination"],
 513        Parameter(help="The type of connector."),
 514    ],
 515) -> None:
 516    """Get the current version information for a deployed connector."""
 517    result = get_connector_version_info(
 518        auth=_resolve_cli_cloud_auth(),
 519        workspace_id=workspace_id,
 520        actor_id=connector_id,
 521        actor_type=connector_type,
 522    )
 523    print_json(result.model_dump())
 524
 525
 526_OverrideLevel = Literal["actor", "workspace", "organization"]
 527_TierFilter = Literal["TIER_0", "TIER_1", "TIER_2", "UNKNOWN", "ALL"]
 528
 529
 530def _resolve_cli_cloud_auth() -> ResolvedCloudAuth:
 531    """Resolve Airbyte Cloud auth from environment variables for CLI use.
 532
 533    Mirrors the priority order the MCP server uses:
 534
 535    1. `AIRBYTE_CLOUD_BEARER_TOKEN`
 536    2. `AIRBYTE_CLOUD_CLIENT_ID` + `AIRBYTE_CLOUD_CLIENT_SECRET`
 537
 538    Raises a CLI error via `exit_with_error` if no usable credentials are present.
 539    """
 540    bearer_token = os.environ.get("AIRBYTE_CLOUD_BEARER_TOKEN")
 541    if bearer_token:
 542        return ResolvedCloudAuth(bearer_token=bearer_token)
 543
 544    client_id = os.environ.get("AIRBYTE_CLOUD_CLIENT_ID")
 545    client_secret = os.environ.get("AIRBYTE_CLOUD_CLIENT_SECRET")
 546    if client_id and client_secret:
 547        return ResolvedCloudAuth(client_id=client_id, client_secret=client_secret)
 548
 549    exit_with_error(
 550        "Missing Airbyte Cloud credentials. Set AIRBYTE_CLOUD_BEARER_TOKEN, or "
 551        "both AIRBYTE_CLOUD_CLIENT_ID and AIRBYTE_CLOUD_CLIENT_SECRET."
 552    )
 553
 554
 555def _validate_override_scope_args(
 556    *,
 557    override_level: _OverrideLevel,
 558    workspace_id: str | None,
 559    organization_id: str | None,
 560    connector_id: str | None,
 561    connector_type: Literal["source", "destination"] | None,
 562    actor_definition_id: str | None,
 563) -> None:
 564    """Validate scope-specific arguments for `set/clear-version-override`.
 565
 566    Calls `exit_with_error` (which terminates the process) if the supplied
 567    arguments are inconsistent with the requested `override_level`.
 568    """
 569    if override_level == "actor":
 570        missing = [
 571            name
 572            for name, value in (
 573                ("--workspace-id", workspace_id),
 574                ("--connector-id", connector_id),
 575                ("--connector-type", connector_type),
 576            )
 577            if not value
 578        ]
 579        if missing:
 580            exit_with_error(
 581                "Override level 'actor' requires " + ", ".join(missing) + "."
 582            )
 583        if actor_definition_id is not None:
 584            exit_with_error(
 585                "Override level 'actor' must not be combined with "
 586                "--actor-definition-id."
 587            )
 588        if organization_id is not None:
 589            exit_with_error(
 590                "Override level 'actor' must not be combined with --organization-id."
 591            )
 592        return
 593
 594    if override_level == "workspace":
 595        missing = [
 596            name
 597            for name, value in (
 598                ("--workspace-id", workspace_id),
 599                ("--actor-definition-id", actor_definition_id),
 600            )
 601            if not value
 602        ]
 603        if missing:
 604            exit_with_error(
 605                "Override level 'workspace' requires " + ", ".join(missing) + "."
 606            )
 607        if connector_id is not None or connector_type is not None:
 608            exit_with_error(
 609                "Override level 'workspace' must not be combined with "
 610                "--connector-id or --connector-type."
 611            )
 612        if organization_id is not None:
 613            exit_with_error(
 614                "Override level 'workspace' must not be combined with "
 615                "--organization-id."
 616            )
 617        return
 618
 619    # override_level == "organization"
 620    missing = [
 621        name
 622        for name, value in (
 623            ("--organization-id", organization_id),
 624            ("--actor-definition-id", actor_definition_id),
 625        )
 626        if not value
 627    ]
 628    if missing:
 629        exit_with_error(
 630            "Override level 'organization' requires " + ", ".join(missing) + "."
 631        )
 632    if connector_id is not None or connector_type is not None:
 633        exit_with_error(
 634            "Override level 'organization' must not be combined with "
 635            "--connector-id or --connector-type."
 636        )
 637    if workspace_id is not None:
 638        exit_with_error(
 639            "Override level 'organization' must not be combined with --workspace-id."
 640        )
 641
 642
 643def _dispatch_version_override(
 644    *,
 645    override_level: _OverrideLevel,
 646    workspace_id: str | None,
 647    organization_id: str | None,
 648    connector_id: str | None,
 649    connector_type: Literal["source", "destination"] | None,
 650    actor_definition_id: str | None,
 651    version: str | None,
 652    unset: bool,
 653    reason: str | None,
 654    reason_url: str | None,
 655    issue_url: str,
 656    approval_comment_url: str,
 657    ai_agent_session_url: str | None,
 658    customer_tier_filter: _TierFilter,
 659) -> None:
 660    """Route a version-override request through the normalized core helper.
 661
 662    Resolves `--actor-definition-id` to its canonical name and connector type
 663    via the cloud registry for `workspace`/`organization` scopes. Prints the
 664    operation result as JSON.
 665    """
 666    _validate_override_scope_args(
 667        override_level=override_level,
 668        workspace_id=workspace_id,
 669        organization_id=organization_id,
 670        connector_id=connector_id,
 671        connector_type=connector_type,
 672        actor_definition_id=actor_definition_id,
 673    )
 674
 675    auth = _resolve_cli_cloud_auth()
 676
 677    if override_level == "actor":
 678        assert workspace_id is not None
 679        assert connector_id is not None
 680        assert connector_type is not None
 681        assert organization_id is None
 682        try:
 683            ws_resolution = resolve_workspace(workspace_id)
 684            if not ws_resolution.organization_id:
 685                exit_with_error("Could not resolve organization for workspace.")
 686            result = set_cloud_version_override(
 687                auth=auth,
 688                target=VersionOverrideTarget(
 689                    scope="actor",
 690                    organization_id=ws_resolution.organization_id,
 691                    workspace_id=workspace_id,
 692                    actor_id=connector_id,
 693                    connector_type=connector_type,
 694                ),
 695                approval_comment_url=approval_comment_url,
 696                version=version,
 697                unset=unset,
 698                override_reason=reason,
 699                override_reason_reference_url=reason_url,
 700                issue_url=issue_url,
 701                ai_agent_session_url=ai_agent_session_url,
 702                customer_tier_filter=customer_tier_filter,
 703            )
 704        except (PyAirbyteInputError, CloudAuthError) as e:
 705            exit_with_error(str(e))
 706        _print_override_result(result.success, result.message)
 707        print_json(result.model_dump())
 708        return
 709
 710    try:
 711        assert actor_definition_id is not None
 712        canonical_name, resolved_connector_type = (
 713            resolve_definition_id_to_canonical_info(actor_definition_id)
 714        )
 715    except (PyAirbyteInputError, requests.RequestException) as e:
 716        exit_with_error(f"Failed to resolve --actor-definition-id: {e}")
 717    typed_connector_type: Literal["source", "destination"] = (
 718        "source" if resolved_connector_type == "source" else "destination"
 719    )
 720
 721    if override_level == "workspace":
 722        assert workspace_id is not None
 723        try:
 724            ws_resolution = resolve_workspace(workspace_id)
 725            if not ws_resolution.organization_id:
 726                exit_with_error("Could not resolve organization for workspace.")
 727            target = VersionOverrideTarget(
 728                scope="workspace",
 729                organization_id=ws_resolution.organization_id,
 730                workspace_id=workspace_id,
 731                connector_name=canonical_name,
 732                connector_type=typed_connector_type,
 733            )
 734        except PyAirbyteInputError as e:
 735            exit_with_error(str(e))
 736    else:
 737        assert organization_id is not None
 738        target = VersionOverrideTarget(
 739            scope="organization",
 740            organization_id=organization_id,
 741            connector_name=canonical_name,
 742            connector_type=typed_connector_type,
 743        )
 744
 745    try:
 746        result = set_cloud_version_override(
 747            auth=auth,
 748            target=target,
 749            approval_comment_url=approval_comment_url,
 750            version=version,
 751            unset=unset,
 752            override_reason=reason,
 753            override_reason_reference_url=reason_url,
 754            issue_url=issue_url,
 755            ai_agent_session_url=ai_agent_session_url,
 756            customer_tier_filter=customer_tier_filter,
 757        )
 758    except (PyAirbyteInputError, CloudAuthError) as e:
 759        exit_with_error(str(e))
 760    _print_override_result(result.success, result.message)
 761    print_json(result.model_dump())
 762
 763
 764def _print_override_result(success: bool, message: str) -> None:
 765    """Emit an override-operation status message via the shared CLI helpers."""
 766    if success:
 767        print_success(message)
 768    else:
 769        print_error(message)
 770
 771
 772@connector_app.command(name="set-version-override")
 773def set_version_override(
 774    version: Annotated[
 775        str,
 776        Parameter(
 777            help="The semver version string to pin to (e.g., '2.1.5-preview.abc1234')."
 778        ),
 779    ],
 780    reason: Annotated[
 781        str,
 782        Parameter(help="Explanation for the override (min 10 characters)."),
 783    ],
 784    issue_url: Annotated[
 785        str,
 786        Parameter(help="GitHub issue URL providing context for this operation."),
 787    ],
 788    approval_comment_url: Annotated[
 789        str,
 790        Parameter(
 791            help="Slack approval record URL where admin authorized this deployment."
 792        ),
 793    ],
 794    override_level: Annotated[
 795        _OverrideLevel,
 796        Parameter(
 797            help=(
 798                "Scope at which to apply the version override: 'actor' (single "
 799                "deployed connector instance), 'workspace' (all instances of a "
 800                "connector type within a workspace), or 'organization' (all "
 801                "instances across an organization). Defaults to 'actor'."
 802            ),
 803        ),
 804    ] = "actor",
 805    workspace_id: Annotated[
 806        str | None,
 807        Parameter(
 808            help=(
 809                "The Airbyte Cloud workspace ID. Required for 'actor' and "
 810                "'workspace' override levels."
 811            )
 812        ),
 813    ] = None,
 814    organization_id: Annotated[
 815        str | None,
 816        Parameter(
 817            help=(
 818                "The Airbyte Cloud organization ID. Required for "
 819                "'organization' override level."
 820            )
 821        ),
 822    ] = None,
 823    connector_id: Annotated[
 824        str | None,
 825        Parameter(
 826            help=(
 827                "The ID of the deployed connector (source or destination). "
 828                "Required for 'actor' override level."
 829            )
 830        ),
 831    ] = None,
 832    connector_type: Annotated[
 833        Literal["source", "destination"] | None,
 834        Parameter(help="The type of connector. Required for 'actor' override level."),
 835    ] = None,
 836    actor_definition_id: Annotated[
 837        str | None,
 838        Parameter(
 839            help=(
 840                "The connector definition UUID. Required for 'workspace' and "
 841                "'organization' override levels."
 842            )
 843        ),
 844    ] = None,
 845    ai_agent_session_url: Annotated[
 846        str | None,
 847        Parameter(
 848            help="URL to AI agent session driving this operation (for auditability)."
 849        ),
 850    ] = None,
 851    reason_url: Annotated[
 852        str | None,
 853        Parameter(help="Optional URL with more context (e.g., issue link)."),
 854    ] = None,
 855    customer_tier_filter: Annotated[
 856        _TierFilter,
 857        Parameter(
 858            help=(
 859                "Tier filter: 'TIER_0', 'TIER_1', 'TIER_2', 'UNKNOWN', or 'ALL'. "
 860                "The operation will be rejected if the actual customer tier does not match. "
 861                "Defaults to 'TIER_2' (non-sensitive customers)."
 862            ),
 863        ),
 864    ] = "TIER_2",
 865) -> None:
 866    """Set a version override for a deployed connector.
 867
 868    Requires admin authentication via `AIRBYTE_INTERNAL_ADMIN_FLAG` and
 869    `AIRBYTE_INTERNAL_ADMIN_USER` environment variables.
 870
 871    The `--override-level` flag selects the scope at which the pin is applied:
 872
 873    - `actor` (default): pins a single deployed connector instance. Requires
 874      `--workspace-id`, `--connector-id`, and `--connector-type`.
 875    - `workspace`: pins ALL instances of a connector type within a workspace.
 876      Requires `--workspace-id` and `--actor-definition-id`.
 877    - `organization`: pins ALL instances of a connector type across an
 878      organization. Requires `--organization-id` and `--actor-definition-id`.
 879
 880    The `customer_tier_filter` gates the operation: the call fails if the
 881    actual tier of the target organization does not match. Use `ALL` to
 882    proceed regardless of tier (a warning is shown for sensitive tiers).
 883    """
 884    _dispatch_version_override(
 885        override_level=override_level,
 886        workspace_id=workspace_id,
 887        organization_id=organization_id,
 888        connector_id=connector_id,
 889        connector_type=connector_type,
 890        actor_definition_id=actor_definition_id,
 891        version=version,
 892        unset=False,
 893        reason=reason,
 894        reason_url=reason_url,
 895        issue_url=issue_url,
 896        approval_comment_url=approval_comment_url,
 897        ai_agent_session_url=ai_agent_session_url,
 898        customer_tier_filter=customer_tier_filter,
 899    )
 900
 901
 902@connector_app.command(name="clear-version-override")
 903def clear_version_override(
 904    issue_url: Annotated[
 905        str,
 906        Parameter(help="GitHub issue URL providing context for this operation."),
 907    ],
 908    approval_comment_url: Annotated[
 909        str,
 910        Parameter(
 911            help="Slack approval record URL where admin authorized this deployment."
 912        ),
 913    ],
 914    override_level: Annotated[
 915        _OverrideLevel,
 916        Parameter(
 917            help=(
 918                "Scope at which to clear the version override: 'actor', "
 919                "'workspace', or 'organization'. Defaults to 'actor'."
 920            ),
 921        ),
 922    ] = "actor",
 923    workspace_id: Annotated[
 924        str | None,
 925        Parameter(
 926            help=(
 927                "The Airbyte Cloud workspace ID. Required for 'actor' and "
 928                "'workspace' override levels."
 929            )
 930        ),
 931    ] = None,
 932    organization_id: Annotated[
 933        str | None,
 934        Parameter(
 935            help=(
 936                "The Airbyte Cloud organization ID. Required for "
 937                "'organization' override level."
 938            )
 939        ),
 940    ] = None,
 941    connector_id: Annotated[
 942        str | None,
 943        Parameter(
 944            help=(
 945                "The ID of the deployed connector (source or destination). "
 946                "Required for 'actor' override level."
 947            )
 948        ),
 949    ] = None,
 950    connector_type: Annotated[
 951        Literal["source", "destination"] | None,
 952        Parameter(help="The type of connector. Required for 'actor' override level."),
 953    ] = None,
 954    actor_definition_id: Annotated[
 955        str | None,
 956        Parameter(
 957            help=(
 958                "The connector definition UUID. Required for 'workspace' and "
 959                "'organization' override levels."
 960            )
 961        ),
 962    ] = None,
 963    ai_agent_session_url: Annotated[
 964        str | None,
 965        Parameter(
 966            help="URL to AI agent session driving this operation (for auditability)."
 967        ),
 968    ] = None,
 969    customer_tier_filter: Annotated[
 970        _TierFilter,
 971        Parameter(
 972            help=(
 973                "Tier filter: 'TIER_0', 'TIER_1', 'TIER_2', 'UNKNOWN', or 'ALL'. "
 974                "The operation will be rejected if the actual customer tier does not match. "
 975                "Defaults to 'TIER_2' (non-sensitive customers)."
 976            ),
 977        ),
 978    ] = "TIER_2",
 979) -> None:
 980    """Clear a version override from a deployed connector.
 981
 982    Requires admin authentication via `AIRBYTE_INTERNAL_ADMIN_FLAG` and
 983    `AIRBYTE_INTERNAL_ADMIN_USER` environment variables.
 984
 985    The `--override-level` flag selects the scope at which the pin is removed:
 986
 987    - `actor` (default): clears a single deployed connector instance pin.
 988      Requires `--workspace-id`, `--connector-id`, and `--connector-type`.
 989    - `workspace`: clears the workspace-level pin for a connector type.
 990      Requires `--workspace-id` and `--actor-definition-id`.
 991    - `organization`: clears the organization-level pin for a connector type.
 992      Requires `--organization-id` and `--actor-definition-id`.
 993    """
 994    _dispatch_version_override(
 995        override_level=override_level,
 996        workspace_id=workspace_id,
 997        organization_id=organization_id,
 998        connector_id=connector_id,
 999        connector_type=connector_type,
1000        actor_definition_id=actor_definition_id,
1001        version=None,
1002        unset=True,
1003        reason=None,
1004        reason_url=None,
1005        issue_url=issue_url,
1006        approval_comment_url=approval_comment_url,
1007        ai_agent_session_url=ai_agent_session_url,
1008        customer_tier_filter=customer_tier_filter,
1009    )
1010
1011
1012def _load_json_file(file_path: Path) -> dict | None:
1013    """Load a JSON file and return its contents.
1014
1015    Returns None if the file doesn't exist or contains invalid JSON.
1016    """
1017    if not file_path.exists():
1018        return None
1019    try:
1020        return json.loads(file_path.read_text())
1021    except json.JSONDecodeError as e:
1022        print_error(f"Failed to parse JSON in file: {file_path}\nError: {e}")
1023        return None
1024
1025
1026def _parse_extra_env(pairs: list[str] | None) -> dict[str, str]:
1027    """Parse repeatable KEY=VALUE arguments for connector container env."""
1028    parsed: dict[str, str] = {}
1029    for item in pairs or []:
1030        if "=" not in item:
1031            exit_with_error(f"Invalid --extra-env value (expected KEY=VALUE): {item}")
1032        key, value = item.split("=", 1)
1033        key = key.strip()
1034        if not key:
1035            exit_with_error(f"Invalid --extra-env value (expected KEY=VALUE): {item}")
1036        parsed[key] = value
1037    return parsed
1038
1039
1040_DEBUG_LOG_LEVELS = frozenset({"DEBUG", "TRACE"})
1041
1042
1043def _private_dir(prefix: str) -> Path:
1044    """A temp directory for the connector's secrets, removed when the CLI exits.
1045
1046    `mkdtemp` makes it 0700. On a runner the VM dies with the job; on a laptop
1047    nothing else would remove it, and it holds the config and the raw state.
1048    `atexit` runs on a normal exit and on Ctrl-C, not on SIGTERM or `kill -9`,
1049    so a run killed that way still leaves the directory behind.
1050    """
1051    path = Path(tempfile.mkdtemp(prefix=prefix))
1052    atexit.register(shutil.rmtree, path, ignore_errors=True)
1053    return path
1054
1055
1056def _without_debug_logs(
1057    *, enable_debug_logs: bool, extra_env: dict[str, str]
1058) -> tuple[dict[str, str], list[str]]:
1059    """The container env without a debug log level, and the settings dropped.
1060
1061    Debug logs are off on masked runs for now, to keep masking simple. A
1062    `LOG_LEVEL` passed through `--extra-env` reaches the connector exactly as
1063    `--enable-debug-logs` does, so a DEBUG or TRACE one is dropped too.
1064    """
1065    dropped = ["--enable-debug-logs"] if enable_debug_logs else []
1066    level = extra_env.get("LOG_LEVEL")
1067    if level is not None and level.strip().upper() in _DEBUG_LOG_LEVELS:
1068        dropped.append(f"LOG_LEVEL={level} from --extra-env")
1069        extra_env = {
1070            key: value for key, value in extra_env.items() if key != "LOG_LEVEL"
1071        }
1072    return extra_env, dropped
1073
1074
1075# The masking key's length: HMAC-SHA256's block is 64 bytes, and 32 random
1076# bytes is the full strength of the hash it feeds.
1077MASK_KEY_BYTES = 32
1078
1079
1080def _resolve_mask_key(mask_key_path: str | None) -> bytes:
1081    """The key every token of this run is hashed under.
1082
1083    From the file when given: the workflow makes one key per run and hands
1084    every command the path, so the four commands' tokens pair and the key
1085    itself goes through no argument the runner would print. Without it a
1086    fresh key for this process, so a run by hand pairs its own two sides and
1087    nothing else. The key is returned, never printed; the errors name the
1088    file, not what it held.
1089    """
1090    if mask_key_path is None:
1091        return secrets.token_bytes(MASK_KEY_BYTES)
1092    try:
1093        text = Path(mask_key_path).read_text().strip()
1094    except OSError as exc:
1095        error = f"--mask-key-path cannot be read: {exc}"
1096    else:
1097        try:
1098            key = bytes.fromhex(text)
1099        except ValueError:
1100            key = b""
1101        if len(key) == MASK_KEY_BYTES:
1102            return key
1103        error = (
1104            f"--mask-key-path must hold the run's masking key as "
1105            f"{2 * MASK_KEY_BYTES} hex characters: {mask_key_path}"
1106        )
1107    write_github_output("success", False)
1108    write_github_output("error", error)
1109    exit_with_error(error)
1110
1111
1112def _mask_policy(
1113    catalog: Any, output_mode: OutputMode, mask_key: bytes | None
1114) -> MaskPolicy | None:
1115    """The policy a masked run masks `catalog`'s streams under; `None` on a raw run.
1116
1117    A masked run without a key is a programming error, not a raw run:
1118    refusing it keeps a missing key from writing output under the wrong
1119    mode.
1120    """
1121    if output_mode != "masked":
1122        return None
1123    if mask_key is None:
1124        raise ValueError("A masked run needs its masking key")
1125    return MaskPolicy.from_configured_catalog(catalog, key=mask_key)
1126
1127
1128def _run_connector_command(
1129    connector_image: str,
1130    command: Command,
1131    output_dir: Path,
1132    target_or_control: TargetOrControl,
1133    config_path: Path | None = None,
1134    catalog_path: Path | None = None,
1135    state_path: Path | None = None,
1136    proxy_url: str | None = None,
1137    ca_cert_path: Path | None = None,
1138    enable_debug_logs: bool = False,
1139    timeout_seconds: int | None = None,
1140    max_records: int | None = None,
1141    output_mode: OutputMode = "raw",
1142    mask_key: bytes | None = None,
1143    mask_catalog_path: Path | None = None,
1144    extra_env: dict[str, str] | None = None,
1145) -> tuple[dict, ComparableOutputs]:
1146    """Run a connector command and return its results and its declared objects.
1147
1148    The dict is a flattened view of `ExecutionResult`: counts, exit status and
1149    artifact paths, which is all the reports need. What a comparison needs
1150    instead -- the spec, the discovered catalog -- travels beside it as
1151    `ComparableOutputs` rather than growing a dict that several consumers read.
1152    The `ExecutionResult` itself is still dropped here: it holds every parsed
1153    message, and a comparison keeps both sides alive at once.
1154
1155    Args:
1156        connector_image: Full connector image name with tag.
1157        command: The Airbyte command to run.
1158        output_dir: Directory to store output files.
1159        target_or_control: Whether this is target or control version.
1160        config_path: Path to connector config JSON file.
1161        catalog_path: Path to configured catalog JSON file.
1162        state_path: Path to state JSON file.
1163        proxy_url: Optional HTTP proxy URL for traffic capture.
1164        ca_cert_path: Path to the proxy's CA certificate, so the container trusts
1165            it. Without it every HTTPS request through the proxy fails TLS
1166            verification and the capture comes back near-empty.
1167        enable_debug_logs: If `True`, pass `LOG_LEVEL=DEBUG` to the connector container.
1168        timeout_seconds: How long the container may run before the harness stops
1169            it and reports the run as stopped early; the runner's default when
1170            `None`.
1171        max_records: Stop the container after this many RECORD messages and
1172            report the run as stopped early; no cap when `None`.
1173        output_mode: `masked` masks every stdout line as it is written,
1174            under a policy built from the configured catalog that masks primary
1175            keys and keeps a cursor readable only when its declared type is an
1176            integer or a date; `raw` writes the output as emitted.
1177        mask_key: The run's masking key, which every token is hashed under;
1178            both sides of a comparison are given the same one. Required on a
1179            masked run, unused on a raw one.
1180        mask_catalog_path: The catalog the mask policy is built from, when it
1181            is not `catalog_path`'s: both sides of a comparison must mask under
1182            one policy, so their tokens pair.
1183        extra_env: Additional environment variables for the connector container.
1184
1185    Returns:
1186        The flattened results, and the objects the connector declared. A run
1187        the harness stopped carries `stopped_early` -- its reason, the records
1188        kept and the seconds it ran -- and `None` there otherwise.
1189    """
1190    connector = ConnectorUnderTest.from_image_name(connector_image, target_or_control)
1191
1192    config = _load_json_file(config_path) if config_path else None
1193    state = _load_json_file(state_path) if state_path else None
1194
1195    configured_catalog = None
1196    if catalog_path and catalog_path.exists():
1197        catalog_json = catalog_path.read_text()
1198        configured_catalog = ConfiguredAirbyteCatalog.parse_raw(catalog_json)  # ty: ignore[deprecated]
1199
1200    mask_catalog = configured_catalog
1201    if mask_catalog_path is not None and output_mode == "masked":
1202        mask_catalog = ConfiguredAirbyteCatalog.parse_raw(mask_catalog_path.read_text())  # ty: ignore[deprecated]
1203
1204    # Pass log level as an environment variable to the connector container
1205    container_env: dict[str, str] = {}
1206    if enable_debug_logs:
1207        container_env["LOG_LEVEL"] = "DEBUG"
1208    container_env.update(extra_env or {})
1209
1210    execution_inputs = ExecutionInputs(
1211        connector_under_test=connector,
1212        command=command,
1213        output_dir=output_dir,
1214        config=config,
1215        configured_catalog=configured_catalog,
1216        state=state,
1217        environment_variables=container_env or None,
1218    )
1219
1220    mask_policy = _mask_policy(mask_catalog, output_mode, mask_key)
1221    runner = ConnectorRunner(
1222        execution_inputs,
1223        timeout_seconds=(
1224            DEFAULT_TIMEOUT_SECONDS if timeout_seconds is None else timeout_seconds
1225        ),
1226        proxy_url=proxy_url,
1227        ca_cert_path=ca_cert_path,
1228        max_records=max_records,
1229        mask_policy=mask_policy,
1230    )
1231    result = runner.run()
1232
1233    result.save_artifacts(output_dir)
1234
1235    result_dict: dict[str, object] = {
1236        "connector": connector_image,
1237        "command": command.value,
1238        "success": result.success,
1239        "exit_code": result.exit_code,
1240        "stdout_file": str(result.stdout_file_path),
1241        "stderr_file": str(result.stderr_file_path),
1242        "message_counts": {
1243            k.value: v for k, v in result.get_message_count_per_type().items()
1244        },
1245        "record_counts_per_stream": result.get_record_count_per_stream(),
1246        "stopped_early": (
1247            result.stopped_early.to_dict() if result.stopped_early else None
1248        ),
1249        "output_mode": output_mode,
1250    }
1251    if mask_policy is not None and command.needs_catalog():
1252        # Per stream, what the mask left readable and why, for the report. Only
1253        # a command that reads records has a schema-enriched catalog and output
1254        # to keep readable; anywhere else every stream reads `schema missing`.
1255        result_dict["readable_fields"] = {
1256            name: info.to_dict() for name, info in mask_policy.streams.items()
1257        }
1258
1259    # Under `http_metrics` because that is what they are, but set here rather
1260    # than in `_run_with_optional_http_metrics`: the connector's own cache is
1261    # measurable with or without the proxy, so gating it on the proxy would
1262    # hide the one metric that needs none. The wrapper merges its proxy
1263    # counters into whatever is already here. Read off the summary that
1264    # `save_artifacts` collected above, not off a second walk of stdout: a
1265    # `read`'s stdout is the dataset.
1266    connector_cache = parse_connector_cache_metrics(result.summary.analytics)
1267    if connector_cache is not None:
1268        result_dict["http_metrics"] = {
1269            "connector_request_count": connector_cache.request_count,
1270            "connector_cache_hits_count": connector_cache.cache_hits_count,
1271            "connector_cache_hit_ratio": connector_cache.cache_hit_ratio,
1272        }
1273
1274    # For CHECK, include the parsed connectionStatus from the Airbyte protocol.
1275    # The CDK exits 0 regardless of check outcome; the real pass/fail signal
1276    # lives in the CONNECTION_STATUS message (status=SUCCEEDED or FAILED).
1277    if command == Command.CHECK:
1278        conn_status = result.get_connection_status()
1279        if conn_status is not None:
1280            result_dict["connection_status"] = conn_status.status.value
1281            result_dict["connection_status_message"] = conn_status.message or ""
1282        else:
1283            result_dict["connection_status"] = None
1284            result_dict["connection_status_message"] = ""
1285
1286    return result_dict, ComparableOutputs.from_execution_result(result)
1287
1288
1289def _read_not_started(
1290    connector_image: str,
1291    command: Command,
1292    output_dir: Path,
1293    output_mode: OutputMode,
1294) -> tuple[dict, ComparableOutputs]:
1295    """The result of a side whose deadline passed before its turn came.
1296
1297    No container runs: a read given no time would only push the comparison and
1298    the report past the step's own timeout. The side reads like any other run
1299    stopped at its deadline -- a clean exit, `stopped_early` with reason
1300    `timeout` and no records -- so the verdict is incomplete, never a pass,
1301    and the report is still written. Its output files exist, empty but for a
1302    note in stderr, so every reader of a run's files finds them, and it
1303    carries the run's `output_mode`, so the report still says how the run's
1304    output was written.
1305    """
1306    output_dir.mkdir(parents=True, exist_ok=True)
1307    stdout_path = output_dir / "stdout.txt"
1308    stderr_path = output_dir / "stderr.txt"
1309    stdout_path.write_text("")
1310    note = (
1311        f"{connector_image} was not started: nothing was left of --read-timeout "
1312        "for its read"
1313    )
1314    stderr_path.write_text(f"{note}\n")
1315    print_warning(note)
1316    return (
1317        {
1318            "connector": connector_image,
1319            "command": command.value,
1320            "success": True,
1321            "exit_code": 0,
1322            "stdout_file": str(stdout_path),
1323            "stderr_file": str(stderr_path),
1324            "message_counts": {},
1325            "record_counts_per_stream": {},
1326            "stopped_early": StoppedEarly(
1327                reason="timeout", records_seen=0, elapsed_s=0.0
1328            ).to_dict(),
1329            "output_mode": output_mode,
1330        },
1331        ComparableOutputs(),
1332    )
1333
1334
1335def _record_read_share(result: dict, share_s: int | None, budget_s: int) -> None:
1336    """Name the share of `--read-timeout` a side stopped at, for its report line.
1337
1338    Only on a deadline stop under the budget: a capped side stopped for
1339    another reason, and a side that finished needs no excuse. `share_s` 0 is a
1340    side that never started. A copy replaces the entry rather than adding to
1341    it, since the dict may be shared with whoever produced it.
1342    """
1343    stopped = result.get("stopped_early")
1344    if share_s is None or not stopped or stopped.get("reason") != "timeout":
1345        return
1346    result["stopped_early"] = {**stopped, "share_s": share_s, "budget_s": budget_s}
1347
1348
1349def _annotate_run_verdict(command: str, result: dict[str, Any]) -> dict[str, Any]:
1350    """Copy a run result with `success` as the verdict, exit status kept separately.
1351
1352    `ExecutionResult.success` is the container exit status, which for `check` says
1353    nothing about whether the connector could connect -- the CDK exits 0 even when a
1354    check fails. Any payload read as the verdict must therefore not expose the exit
1355    status under `success`, at any nesting level. The raw value stays available as
1356    `exited_cleanly`; for every command but `check` the two are equal -- except on
1357    a side the harness stopped early, including one it never started for lack of
1358    time: that side read part of what it would have, so its `success` is false,
1359    as the run's is, and `stopped_early` says why.
1360
1361    Returns a copy: callers pass the unannotated originals to the report generator,
1362    which reads `success` as the exit status.
1363    """
1364    return {
1365        **result,
1366        "success": (
1367            command_result_succeeded(command, result)
1368            and not result.get("stopped_early")
1369        ),
1370        "exited_cleanly": result["success"],
1371    }
1372
1373
1374# The lists in the run's JSON payload that hold record values: missing and
1375# duplicated primary keys, and the field diffs of paired records. Named alike,
1376# so a new example list is covered without a change here.
1377_EXAMPLE_LIST_SUFFIX = "_examples"
1378# What a raw run's printed payload says in place of a stream's state diff.
1379_STATE_DIFF_OMITTED = (
1380    "Omitted from the job log on a raw run; both versions' state messages are "
1381    "in the saved artifacts"
1382)
1383
1384
1385def _without_examples(payload: dict[str, Any]) -> dict[str, Any]:
1386    """A copy of the payload with every example list emptied, keys kept, and
1387    each stream's final-state diff replaced by a `diff_omitted` note.
1388
1389    The job log outlives the artifacts and is not masked, so on a raw run the
1390    printed payload carries the shape and the counts of what was found and
1391    none of the record values; `examples_omitted` says why the lists are
1392    empty. The state diff is the one diff that holds customer values rather
1393    than types: a moved cursor puts both sides' values, and any partition
1394    value, in its `values_changed` entry, where catalog and spec diffs hold
1395    schema. The `regression_report` output and the artifacts keep the full
1396    payload: the artifact is where the detail belongs.
1397    """
1398
1399    def strip(value: Any, key: str = "") -> Any:
1400        if isinstance(value, dict):
1401            return {name: strip(item, name) for name, item in value.items()}
1402        if isinstance(value, list):
1403            if key.endswith(_EXAMPLE_LIST_SUFFIX):
1404                return []
1405            return [strip(item) for item in value]
1406        return value
1407
1408    stripped = strip(payload)
1409    final_state = stripped.get("comparison", {}).get("final_state")
1410    if isinstance(final_state, dict):
1411        for entry in final_state.get("streams", {}).values():
1412            if isinstance(entry, dict) and "diff" in entry:
1413                del entry["diff"]
1414                entry["diff_omitted"] = _STATE_DIFF_OMITTED
1415    return {**stripped, "examples_omitted": True}
1416
1417
1418def _printable_payload(
1419    payload: dict[str, Any], output_mode: OutputMode
1420) -> dict[str, Any]:
1421    """The payload as `print_json` may show it: whole on a masked run, where
1422    every example is a token, and without its examples on a raw run."""
1423    return payload if output_mode == "masked" else _without_examples(payload)
1424
1425
1426# The checks the output comparisons contribute, named for the row a reviewer
1427# reads in the report and in the step summary.
1428_SPEC_CHECK = "Spec compatibility"
1429_CATALOG_SCHEMA_CHECK = "Catalog schema"
1430_FINAL_STATE_CHECK = "Final state per stream"
1431
1432# How many individual changes a check's one-line summary names before it says
1433# how many more there are. The full list is in the report and in the run's JSON
1434# payload, and the comparators order their findings worst-first, so the ones
1435# named here are the ones worth naming.
1436_MAX_LISTED_CHANGES = 3
1437
1438# How many changes the JSON payload lists per comparison. Same reason as
1439# `_MAX_DIFF_PAYLOAD_CHARS`: the payload is written to `GITHUB_OUTPUT`, and a
1440# reordered `oneOf` on a wide spec produces hundreds of sentences.
1441_MAX_PAYLOAD_CHANGES = 100
1442
1443# How many per-stream entries it lists. A wide source has thousands of streams,
1444# and one line each is enough to exceed the same output budget without a single
1445# diff being attached.
1446_MAX_PAYLOAD_STREAMS = 100
1447
1448# How much of a comparison's detail the run's JSON payload carries. That payload
1449# is written to `GITHUB_OUTPUT`, which is size-limited, and a wide connector's
1450# catalog diff has no natural bound -- so the payload keeps as many stream diffs
1451# as fit and says how many it dropped. The catalogs themselves are saved
1452# artifacts, which is where an unbounded diff belongs.
1453_MAX_DIFF_PAYLOAD_CHARS = 50_000
1454
1455
1456def _compare_outputs(
1457    *,
1458    command: str,
1459    outcome: ComparisonOutcome,
1460    control_outputs: ComparableOutputs,
1461    target_outputs: ComparableOutputs,
1462    partial: bool = False,
1463    no_control_streams: tuple[str, ...] = (),
1464    could_not_test: bool = False,
1465    deferred_diff_blocks: list[Callable[[], tuple[DiffBlock, ...]]] | None = None,
1466) -> tuple[tuple[CheckResult, ...], tuple[DiffBlock, ...], dict[str, Any]]:
1467    """Compare what the two versions produced, for the commands that produce it.
1468
1469    This is the content check for `spec`, `discover` and `read`: without it their
1470    verdict is the exit code, and a connector that drops a config field, changes
1471    a stream's schema or corrupts the state the next sync resumes from exits 0
1472    all the way to a release. `check` reports its own status, so it is the one
1473    command with nothing to diff here.
1474
1475    Only runs when both commands succeeded. A failed run has nothing to compare,
1476    and reporting "no spec to compare" beside the command failure that caused it
1477    would state the same problem twice, in weaker terms the second time.
1478
1479    A comparator that raises is left to propagate. The comparators normalise what
1480    they are given, so the only ways here are a programming error in a future
1481    caller or a schema nested past the recursion limit -- and the workflow
1482    already has the right vocabulary for both: a step that dies is an
1483    `internal_failure`, reported as infrastructure rather than as a regression,
1484    with the connector's artifacts uploaded either way. Catching it here would
1485    only let a tooling bug read as "the new version broke its spec".
1486
1487    Args:
1488        command: The Airbyte command that was run, as the CLI names it. This is
1489            always `"read"` for a read, including one given a state file: the
1490            `READ` -> `READ_WITH_STATE` upgrade only changes the `Command` the
1491            container is run with.
1492        outcome: The already-evaluated command outcome.
1493        control_outputs: What the control version produced.
1494        target_outputs: What the target version produced.
1495        partial: Whether a side was stopped early. Only the final-state check
1496            is affected: the states of a stopped read are checkpoints from
1497            wherever each side got to, so the check is reported rather than
1498            folded.
1499
1500    Returns:
1501        The checks to fold into the verdict, the diff blocks that show what they
1502        found, and the comparison detail for the run's JSON payload -- all empty
1503        when there was nothing to compare.
1504    """
1505    if not outcome.success and not could_not_test:
1506        return (), (), {}
1507
1508    if command == "spec":
1509        return _spec_comparison(
1510            control_outputs.spec, target_outputs.spec, deferred_diff_blocks
1511        )
1512
1513    if command == "discover":
1514        return _catalog_comparison(
1515            control_outputs.catalog, target_outputs.catalog, deferred_diff_blocks
1516        )
1517
1518    if command == "read":
1519        return _final_state_comparison(
1520            control_outputs.final_states,
1521            target_outputs.final_states,
1522            partial,
1523            no_control_streams,
1524            deferred_diff_blocks,
1525        )
1526
1527    return (), (), {}
1528
1529
1530def _spec_comparison(
1531    control_spec: Any,
1532    target_spec: Any,
1533    deferred_diff_blocks: list[Callable[[], tuple[DiffBlock, ...]]] | None = None,
1534) -> tuple[tuple[CheckResult, ...], tuple[DiffBlock, ...], dict[str, Any]]:
1535    """Run the spec comparison and shape it for every surface."""
1536    result = compare_specs(control_spec, target_spec)
1537
1538    check = CheckResult(
1539        name=_SPEC_CHECK,
1540        passed=result.passed,
1541        summary=_change_summary(
1542            result.message,
1543            # A passing spec check still says what was added: an additive change
1544            # is the thing a reviewer most wants to see confirmed.
1545            result.errors if result.errors else result.warnings,
1546        ),
1547        details=tuple(result.errors),
1548    )
1549    # The incompatible changes are the check's `details`, which the report renders
1550    # under the check row and the step summary lists -- a block would repeat them.
1551    # The compatible ones have nowhere else to go, and a reviewer asking "what
1552    # else moved?" is asking about exactly those.
1553    blocks = _report_diff_blocks(
1554        lambda: tuple(
1555            block
1556            for block in (
1557                _change_block("Spec — compatible changes", result.warnings, "add"),
1558            )
1559            if block is not None
1560        ),
1561        deferred_diff_blocks,
1562    )
1563
1564    return (check,), blocks, {"spec": _spec_detail(result)}
1565
1566
1567def _catalog_comparison(
1568    control_catalog: Any,
1569    target_catalog: Any,
1570    deferred_diff_blocks: list[Callable[[], tuple[DiffBlock, ...]]] | None = None,
1571) -> tuple[tuple[CheckResult, ...], tuple[DiffBlock, ...], dict[str, Any]]:
1572    """Run the discovered-catalog comparison and shape it for every surface."""
1573    result = compare_catalog_schemas(control_catalog, target_catalog)
1574
1575    check = CheckResult(
1576        name=_CATALOG_SCHEMA_CHECK,
1577        passed=result.passed,
1578        # A catalog that only grew -- new streams, new fields, nothing removed
1579        # or retyped -- is reported and does not gate the release. Connectors
1580        # add streams constantly, and a check that goes red on every one of them
1581        # is a check people learn to click past.
1582        severity="diagnostic" if result.additive_only else "strict",
1583        summary=_change_summary(result.message, result.errors),
1584        details=tuple(result.errors),
1585    )
1586
1587    # The findings themselves are the check's `details`, which the report renders
1588    # under the check row: a stream that is missing or new has no schema diff, so
1589    # that list is the only place it appears.
1590    blocks = _report_diff_blocks(
1591        lambda: _stream_diff_blocks(
1592            result,
1593            block_title="Schema diff",
1594            omission_title="Schema diffs omitted for size",
1595            artifact_hint=(
1596                "Both discovered catalogs are in the artifact, under "
1597                "airbyte_messages/catalog.jsonl."
1598            ),
1599        ),
1600        deferred_diff_blocks,
1601    )
1602
1603    return (check,), blocks, {"catalog_schema": _catalog_schema_detail(result)}
1604
1605
1606def _final_state_comparison(
1607    control_states: dict[tuple[str | None, str], Any] | None,
1608    target_states: dict[tuple[str | None, str], Any] | None,
1609    partial: bool = False,
1610    no_control_streams: tuple[str, ...] = (),
1611    deferred_diff_blocks: list[Callable[[], tuple[DiffBlock, ...]]] | None = None,
1612) -> tuple[tuple[CheckResult, ...], tuple[DiffBlock, ...], dict[str, Any]]:
1613    """Run the final-state comparison and shape it for every surface."""
1614    excluded = set(no_control_streams)
1615    compared_control = (
1616        {key: value for key, value in control_states.items() if key[1] not in excluded}
1617        if control_states is not None
1618        else None
1619    )
1620    compared_target = (
1621        {key: value for key, value in target_states.items() if key[1] not in excluded}
1622        if target_states is not None
1623        else None
1624    )
1625    result = compare_final_states(compared_control, compared_target)
1626
1627    # Structural findings first, then the value-only ones: the check lists
1628    # everything it found, and the summary quotes the front of that list.
1629    findings = [*result.errors, *result.warnings]
1630
1631    summary = _change_summary(result.message, findings)
1632    not_compared = sorted(no_control_streams)
1633    if not_compared:
1634        summary += (
1635            f"; {len(not_compared)} stream(s) read by the target only were not "
1636            f"compared: {', '.join(not_compared)}"
1637        )
1638    passed = result.passed
1639    if partial and not result.inconclusive:
1640        # Under a stop neither side emitted a *final* state: what each left
1641        # behind is the checkpoint it had reached, and the two were cut at
1642        # different points -- so a state present on one side only, a cursor
1643        # that differs, or two that agree, says where each stopped, not what
1644        # changed. Never a pass, for the same reason the count row is not.
1645        passed = False
1646        summary += (
1647            " (inconclusive on a stopped-early read: these are the checkpoints "
1648            "each side had reached, not final states)"
1649        )
1650
1651    check = CheckResult(
1652        name=_FINAL_STATE_CHECK,
1653        passed=passed,
1654        # Three ways a non-passing state check does not gate. A cursor that only
1655        # advanced: until HTTP replay lands, the two versions call the live API
1656        # in separate runs and a timestamp cursor moves between them on its own.
1657        # A read with no state on either side, where there was nothing to
1658        # compare rather than something wrong. And a read that was stopped
1659        # early, whose states are not final.
1660        severity=(
1661            "diagnostic"
1662            if partial or result.value_only or result.inconclusive
1663            else "strict"
1664        ),
1665        summary=summary,
1666        details=tuple(findings),
1667    )
1668
1669    blocks = _report_diff_blocks(
1670        lambda: _stream_diff_blocks(
1671            result,
1672            block_title="Final state diff",
1673            omission_title="Final state diffs omitted for size",
1674            artifact_hint=(
1675                "Both versions' state messages are in the artifact, under "
1676                "airbyte_messages/state.jsonl."
1677            ),
1678        ),
1679        deferred_diff_blocks,
1680    )
1681
1682    detail = _final_state_detail(result)
1683    detail["not_compared"] = not_compared
1684    return (check,), blocks, {"final_state": detail}
1685
1686
1687def _report_diff_blocks(
1688    build: Callable[[], tuple[DiffBlock, ...]],
1689    deferred: list[Callable[[], tuple[DiffBlock, ...]]] | None,
1690) -> tuple[DiffBlock, ...]:
1691    """Defer display-only diffs while keeping comparison checks and JSON eager."""
1692    if deferred is not None:
1693        deferred.append(build)
1694        return ()
1695    return build()
1696
1697
1698def _stream_diff_blocks(
1699    result: ComparisonResult,
1700    *,
1701    block_title: str,
1702    omission_title: str,
1703    artifact_hint: str,
1704) -> tuple[DiffBlock, ...]:
1705    """One block per changed stream, within the report's diff budget.
1706
1707    In comparison order -- only a *changed* stream has a diff, so there is no
1708    severity left to sort by here -- and within the same budget the payload uses:
1709    a report that inlines every schema of a 200-stream source is not a report
1710    anyone reads. What the budget drops is named rather than skipped, since a
1711    block that is simply absent reads as "there was nothing to see".
1712    """
1713    blocks: list[DiffBlock] = []
1714    budget = _MAX_DIFF_PAYLOAD_CHARS
1715    dropped: list[str] = []
1716
1717    for name in result.failed_streams:
1718        stream = result.stream_results[name]
1719        if not stream.schema_diff:
1720            continue
1721
1722        payload = json.dumps(stream.schema_diff, indent=2, sort_keys=True)
1723        if len(payload) > budget:
1724            dropped.append(name)
1725            continue
1726
1727        budget -= len(payload)
1728        blocks.append(build_json_block(f"{block_title} — {name}", payload))
1729
1730    if dropped:
1731        blocks.append(
1732            build_diff_block(
1733                omission_title,
1734                [
1735                    DiffLine(
1736                        kind="meta",
1737                        text=(
1738                            "These streams changed, and their diffs were too "
1739                            f"large for this report. {artifact_hint}"
1740                        ),
1741                    ),
1742                    *(DiffLine(kind="del", text=name) for name in dropped),
1743                ],
1744                language="text",
1745            )
1746        )
1747
1748    return tuple(blocks)
1749
1750
1751def _change_block(title: str, changes: list[str], kind: str) -> DiffBlock | None:
1752    """One diff block listing a comparison's findings, or `None` if there are none."""
1753    if not changes:
1754        return None
1755
1756    return build_diff_block(
1757        title,
1758        [DiffLine(kind=kind, text=change) for change in changes],
1759        language="text",
1760    )
1761
1762
1763def _change_summary(message: str, changes: list[str]) -> str:
1764    """One line: what the comparison concluded, and the first few reasons."""
1765    if not changes:
1766        return message
1767
1768    listed = "; ".join(changes[:_MAX_LISTED_CHANGES])
1769    remaining = len(changes) - _MAX_LISTED_CHANGES
1770    if remaining > 0:
1771        listed = f"{listed}; and {remaining} more"
1772
1773    return f"{message}: {listed}"
1774
1775
1776def _bounded_changes(changes: list[str]) -> list[str]:
1777    """A change list the JSON payload can carry, saying what it left out."""
1778    if len(changes) <= _MAX_PAYLOAD_CHANGES:
1779        return changes
1780
1781    dropped = len(changes) - _MAX_PAYLOAD_CHANGES
1782
1783    return [
1784        *changes[:_MAX_PAYLOAD_CHANGES],
1785        f"... and {dropped} more, in the report and the console output",
1786    ]
1787
1788
1789def _spec_detail(result: ComparisonResult) -> dict[str, Any]:
1790    """The spec comparison, for the run's JSON payload."""
1791    return {
1792        "passed": result.passed,
1793        "message": result.message,
1794        "breaking_changes": _bounded_changes(result.errors),
1795        "compatible_changes": _bounded_changes(result.warnings),
1796    }
1797
1798
1799def _catalog_schema_detail(result: ComparisonResult) -> dict[str, Any]:
1800    """The catalog comparison, for the run's JSON payload."""
1801    streams, omitted, dropped = _bounded_stream_diffs(
1802        result,
1803        omission_note="Too large for this payload; diff the catalogs",
1804        artifact_hint="Both discovered catalogs are in the saved artifacts.",
1805    )
1806
1807    detail: dict[str, Any] = {
1808        "passed": result.passed,
1809        # Why a `passed: false` did not fail the run: everything found was an
1810        # addition, which is reported as a warning rather than a regression.
1811        "additive_only": result.additive_only,
1812        "message": result.message,
1813        "changed_streams": _bounded_changes(result.failed_streams),
1814        "streams": streams,
1815    }
1816    if omitted:
1817        detail["streams_with_omitted_diffs"] = omitted
1818    if dropped:
1819        detail["streams_omitted"] = dropped
1820
1821    return detail
1822
1823
1824def _final_state_detail(result: ComparisonResult) -> dict[str, Any]:
1825    """The final-state comparison, for the run's JSON payload."""
1826    streams, omitted, dropped = _bounded_stream_diffs(
1827        result,
1828        omission_note="Too large for this payload; diff the state messages",
1829        artifact_hint="Both versions' state messages are in the saved artifacts.",
1830    )
1831
1832    detail: dict[str, Any] = {
1833        "passed": result.passed,
1834        # The two reasons a `passed: false` did not fail the run, kept apart so
1835        # the payload never says "every state that moved moved in value only"
1836        # about a read where nothing moved because nothing was compared.
1837        "value_only": result.value_only,
1838        "inconclusive": result.inconclusive,
1839        "message": result.message,
1840        "changed_streams": _bounded_changes(result.failed_streams),
1841        "streams": streams,
1842    }
1843    if omitted:
1844        detail["streams_with_omitted_diffs"] = omitted
1845    if dropped:
1846        detail["streams_omitted"] = dropped
1847
1848    return detail
1849
1850
1851def _bounded_stream_diffs(
1852    result: ComparisonResult,
1853    *,
1854    omission_note: str,
1855    artifact_hint: str,
1856) -> tuple[dict[str, Any], list[str], int]:
1857    """A comparison's per-stream verdicts and diffs, sized for the JSON payload.
1858
1859    Every stream that changed is named with its verdict -- so the payload says
1860    what moved, not just that something did -- while the diffs themselves are
1861    kept only while they fit in `_MAX_DIFF_PAYLOAD_CHARS`, and the number of
1862    entries is capped in turn: a thousand-stream source would otherwise put a
1863    thousand of them into `GITHUB_OUTPUT`. Both bounds say what they dropped,
1864    on the console and in the payload: a bound nobody mentions reads as "there
1865    was nothing more to see".
1866    """
1867    streams: dict[str, Any] = {}
1868    omitted: list[str] = []
1869    budget = _MAX_DIFF_PAYLOAD_CHARS
1870
1871    # Non-passing first, so the cap falls on the streams with nothing to say.
1872    ordered = sorted(result.stream_results.items(), key=lambda item: item[1].passed)
1873    kept, dropped_entries = (
1874        ordered[:_MAX_PAYLOAD_STREAMS],
1875        ordered[_MAX_PAYLOAD_STREAMS:],
1876    )
1877
1878    for name, stream in kept:
1879        entry: dict[str, Any] = {"passed": stream.passed, "message": stream.message}
1880        streams[name] = entry
1881
1882        if not stream.schema_diff:
1883            continue
1884
1885        rendered = len(json.dumps(stream.schema_diff, separators=(",", ":")))
1886        if rendered > budget:
1887            omitted.append(name)
1888            entry["diff_omitted"] = omission_note
1889            continue
1890
1891        budget -= rendered
1892        entry["diff"] = stream.schema_diff
1893
1894    if omitted:
1895        print_warning(
1896            f"Comparison detail for {len(omitted)} streams was too large for the "
1897            f"run's JSON output ({', '.join(omitted)}); their diffs are omitted "
1898            f"there. {artifact_hint}"
1899        )
1900
1901    if dropped_entries:
1902        print_warning(
1903            f"The run's JSON output lists {_MAX_PAYLOAD_STREAMS} of "
1904            f"{len(ordered)} streams, changed ones first. The report carries "
1905            f"them all. {artifact_hint}"
1906        )
1907
1908    return streams, omitted, len(dropped_entries)
1909
1910
1911def _build_connector_image_from_source(
1912    connector_name: str,
1913    repo_root: Path | None = None,
1914    tag: str = "dev",
1915    *,
1916    prefer_repo_root: bool = False,
1917) -> str | None:
1918    """Build a connector image from source code.
1919
1920    Args:
1921        connector_name: Name of the connector (e.g., 'source-pokeapi').
1922        repo_root: Optional path to the airbyte repo root. If not provided,
1923            will attempt to auto-detect from current directory.
1924        tag: Tag to apply to the built image (default: 'dev').
1925        prefer_repo_root: Use `repo_root` even when auto-discovery would find
1926            a connector directory. Reproduction cases set it: auto-discovery
1927            searches from the working directory and can pick up a different
1928            airbyte checkout on the machine, silently building the wrong source.
1929
1930    Returns:
1931        The full image name with tag if successful, None if build fails.
1932    """
1933    if not verify_docker_installation():
1934        print_error("Docker is not installed or not running")
1935        return None
1936
1937    connector_directory: Path | None = None
1938    if not (prefer_repo_root and repo_root):
1939        with suppress(FileNotFoundError):
1940            connector_directory = find_connector_root_from_name(connector_name)
1941    if connector_directory is None:
1942        if repo_root:
1943            connector_directory = repo_root / CONNECTORS_SUBDIR / connector_name
1944            if not connector_directory.exists():
1945                print_error(f"Connector directory not found: {connector_directory}")
1946                return None
1947        else:
1948            print_error(
1949                f"Could not find connector '{connector_name}'. "
1950                "Try providing --repo-root to specify the airbyte repo location."
1951            )
1952            return None
1953    metadata_file_path = connector_directory / "metadata.yaml"
1954    if not metadata_file_path.exists():
1955        print_error(f"metadata.yaml not found at {metadata_file_path}")
1956        return None
1957
1958    metadata = MetadataFile.from_file(metadata_file_path)
1959    print_success(f"Building image for connector: {connector_name}")
1960
1961    built_image = build_connector_image(
1962        connector_name=connector_name,
1963        connector_directory=connector_directory,
1964        metadata=metadata,
1965        tag=tag,
1966        no_verify=False,
1967    )
1968    print_success(f"Successfully built image: {built_image}")
1969    return built_image
1970
1971
1972def _fetch_control_image_from_metadata(connector_name: str) -> str | None:
1973    """Fetch the current released connector image from metadata.yaml on main branch.
1974
1975    This fetches the connector's metadata.yaml from the airbyte monorepo's master branch
1976    and extracts the dockerRepository and dockerImageTag to construct the control image.
1977
1978    Args:
1979        connector_name: The connector name (e.g., 'source-github').
1980
1981    Returns:
1982        The full connector image with tag (e.g., 'airbyte/source-github:1.0.0'),
1983        or None if the metadata could not be fetched or parsed.
1984    """
1985    metadata_url = (
1986        f"https://raw.githubusercontent.com/airbytehq/airbyte/master/"
1987        f"airbyte-integrations/connectors/{connector_name}/metadata.yaml"
1988    )
1989    response = requests.get(metadata_url, timeout=30)
1990    if not response.ok:
1991        print_error(
1992            f"Failed to fetch metadata for {connector_name}: "
1993            f"HTTP {response.status_code} from {metadata_url}"
1994        )
1995        return None
1996
1997    metadata = yaml.safe_load(response.text)
1998    if not isinstance(metadata, dict):
1999        print_error(f"Invalid metadata format for {connector_name}: expected dict")
2000        return None
2001
2002    data = metadata.get("data", {})
2003    docker_repository = data.get("dockerRepository")
2004    docker_image_tag = data.get("dockerImageTag")
2005
2006    if not docker_repository or not docker_image_tag:
2007        print_error(
2008            f"Could not find dockerRepository/dockerImageTag in metadata for {connector_name}"
2009        )
2010        return None
2011
2012    return f"{docker_repository}:{docker_image_tag}"
2013
2014
2015def _existing_http_metrics(result: dict[str, Any]) -> dict[str, object]:
2016    """The HTTP metrics a run already reported, for a payload that adds to them.
2017
2018    `_run_connector_command` sets the connector's own cache counters, which need
2019    no proxy; the proxy counters are added here afterwards. Overwriting rather
2020    than merging would silently drop the connector's numbers on every
2021    comparison run -- the runs most likely to be read.
2022    """
2023    return dict(result.get("http_metrics") or {})
2024
2025
2026def _run_with_optional_http_metrics(
2027    connector_image: str,
2028    command: Command,
2029    output_dir: Path,
2030    target_or_control: TargetOrControl,
2031    enable_http_metrics: bool,
2032    config_path: Path | None,
2033    catalog_path: Path | None,
2034    state_path: Path | None,
2035    enable_debug_logs: bool = False,
2036    extra_env: dict[str, str] | None = None,
2037    replay_expected: bool = False,
2038    replay_from: Path | None = None,
2039    replay_options: ReplayOptions | None = None,
2040    stream_large_bodies: str = DEFAULT_STREAM_LARGE_BODIES,
2041    timeout_seconds: int | None = None,
2042    max_records: int | None = None,
2043    output_mode: OutputMode = "raw",
2044    mask_key: bytes | None = None,
2045    mask_catalog_path: Path | None = None,
2046) -> tuple[dict, ComparableOutputs]:
2047    """Run a connector command with optional HTTP metrics capture.
2048
2049    When enable_http_metrics is True, starts mitmproxy to capture HTTP traffic.
2050    If mitmproxy fails to start, or the image cannot trust its CA (see
2051    `proxy_trust_gap`), falls back to running without metrics.
2052
2053    Args:
2054        connector_image: Full connector image name with tag.
2055        command: The Airbyte command to run.
2056        output_dir: Directory to store output files.
2057        target_or_control: Whether this is target or control version.
2058        enable_http_metrics: Whether to capture HTTP metrics via mitmproxy.
2059        config_path: Path to connector config JSON file.
2060        catalog_path: Path to configured catalog JSON file.
2061        state_path: Path to state JSON file.
2062        enable_debug_logs: If `True`, pass `LOG_LEVEL=DEBUG` to the connector container.
2063        extra_env: Additional environment variables for the connector container.
2064        replay_expected: Whether this run was *meant* to replay. Separate from
2065            `replay_from`, which is only non-None once a recording actually
2066            exists -- when the proxy is unavailable there is no recording to
2067            point at, and that is exactly the case that has to be reported.
2068        replay_from: A previous run's HTTP dump to serve responses from, so this
2069            run sees the same upstream data the recorded one did. Requires
2070            `enable_http_metrics`; ignored without it.
2071        replay_options: Replay tuning; defaults apply when omitted.
2072        stream_large_bodies: Body-size threshold above which mitmproxy streams
2073            rather than records a body.
2074        timeout_seconds: Passed through to `_run_connector_command`. At 0 the
2075            deadline has already passed, and neither the proxy nor the
2076            container starts (see `_read_not_started`).
2077        max_records: Passed through to `_run_connector_command`.
2078        output_mode: Passed through to `_run_connector_command`.
2079        mask_key: Passed through to `_run_connector_command`.
2080        mask_catalog_path: Passed through to `_run_connector_command`.
2081
2082    Returns:
2083        What `_run_connector_command` returns, with `http_metrics` added to the
2084        results dict when they were captured -- plus `http_dump_file`, the dump
2085        this run recorded, which is what a subsequent run replays from.
2086    """
2087    if timeout_seconds is not None and timeout_seconds <= 0:
2088        return _read_not_started(connector_image, command, output_dir, output_mode)
2089
2090    # An image the proxy's CA cannot be handed to runs direct, as with capture
2091    # off: through the proxy each of its HTTPS calls would fail verification,
2092    # which reads as a connector bug. Its requests then read as not captured.
2093    capture_gap = proxy_trust_gap(connector_image) if enable_http_metrics else None
2094    if capture_gap:
2095        print_warning(f"Running without HTTP capture: {capture_gap}")
2096    if not enable_http_metrics or capture_gap:
2097        result, comparable = _run_connector_command(
2098            connector_image=connector_image,
2099            command=command,
2100            output_dir=output_dir,
2101            target_or_control=target_or_control,
2102            config_path=config_path,
2103            catalog_path=catalog_path,
2104            state_path=state_path,
2105            enable_debug_logs=enable_debug_logs,
2106            timeout_seconds=timeout_seconds,
2107            max_records=max_records,
2108            output_mode=output_mode,
2109            mask_key=mask_key,
2110            mask_catalog_path=mask_catalog_path,
2111            extra_env=extra_env,
2112        )
2113        if capture_gap:
2114            skipped = {"capture_skipped_reason": capture_gap}
2115            if replay_expected:
2116                skipped["replay_skipped_reason"] = (
2117                    f"{capture_gap}, so nothing could be replayed"
2118                )
2119            result["http_metrics"] = {**_existing_http_metrics(result), **skipped}
2120        return result, comparable
2121
2122    # The manager rather than `MitmproxyManager.start`: the reason a startup
2123    # failed and the fact that a shutdown had to be killed are only reachable
2124    # through it -- there is no session to carry the first, and the second is
2125    # not known until the session is gone.
2126    manager = MitmproxyManager(
2127        output_dir,
2128        replay_from=replay_from,
2129        replay_options=replay_options,
2130        stream_large_bodies=stream_large_bodies,
2131    )
2132    with manager.running() as session:
2133        if session is None:
2134            startup_failure = (
2135                manager.startup_failure_reason or "mitmproxy is unavailable"
2136            )
2137            print_error(
2138                f"Mitmproxy unavailable ({startup_failure}), "
2139                "running without HTTP metrics"
2140            )
2141            result, comparable = _run_connector_command(
2142                connector_image=connector_image,
2143                command=command,
2144                output_dir=output_dir,
2145                target_or_control=target_or_control,
2146                config_path=config_path,
2147                catalog_path=catalog_path,
2148                state_path=state_path,
2149                enable_debug_logs=enable_debug_logs,
2150                timeout_seconds=timeout_seconds,
2151                max_records=max_records,
2152                output_mode=output_mode,
2153                mask_key=mask_key,
2154                mask_catalog_path=mask_catalog_path,
2155                extra_env=extra_env,
2156            )
2157            if replay_expected:
2158                # The one skip path that produces no session, and so reported
2159                # nothing at all: no metrics, no reason, an ordinary green
2160                # comparison against live data. Keyed on `replay_expected` and
2161                # not on `replay_from`: when mitmdump is missing the *control*
2162                # run records no dump either, so there is nothing to point at
2163                # and the condition that looks natural here is never true in
2164                # the failure it defends against.
2165                #
2166                # The precise reason, not a fixed "mitmproxy is unavailable":
2167                # `wait_for_proxy_listener` distinguishes a proxy that died on
2168                # startup and a corpus that took longer than its ceiling to
2169                # load, and those point at a code fix where the hardcoded
2170                # string tells a reader to install mitmproxy.
2171                result["http_metrics"] = {
2172                    **_existing_http_metrics(result),
2173                    "replay_skipped_reason": (
2174                        f"{startup_failure}, so nothing could be replayed"
2175                    ),
2176                }
2177            return result, comparable
2178
2179        if session.replay_source:
2180            print_success(
2181                f"Started mitmproxy on {session.proxy_url} "
2182                f"(replaying {session.replay_source}, "
2183                f"{session.unreplayable_flow_count} flow(s) not replayable)"
2184            )
2185        else:
2186            print_success(f"Started mitmproxy on {session.proxy_url}")
2187            if session.replay_skipped_reason:
2188                print_warning(
2189                    f"HTTP replay unavailable ({session.replay_skipped_reason}); "
2190                    "this run hits the live API, so it cannot be compared against "
2191                    "identical upstream data"
2192                )
2193
2194        result, comparable = _run_connector_command(
2195            connector_image=connector_image,
2196            command=command,
2197            output_dir=output_dir,
2198            target_or_control=target_or_control,
2199            config_path=config_path,
2200            catalog_path=catalog_path,
2201            state_path=state_path,
2202            proxy_url=session.proxy_url,
2203            ca_cert_path=session.ca_cert_path,
2204            enable_debug_logs=enable_debug_logs,
2205            timeout_seconds=timeout_seconds,
2206            max_records=max_records,
2207            output_mode=output_mode,
2208            mask_key=mask_key,
2209            mask_catalog_path=mask_catalog_path,
2210            extra_env=extra_env,
2211        )
2212
2213        # The path, not the metrics: this is what the next run replays from.
2214        result["http_dump_file"] = str(session.dump_file_path)
2215
2216        # Everything the metrics are built from, carried out of the block. The
2217        # dump itself cannot be read here: mitmproxy's `save` addon writes each
2218        # flow through a block-buffered file and only flushes and closes it in
2219        # `done()`, holding in-flight flows until shutdown -- so parsing inside
2220        # the `with` misses the buffered tail, and `live_flow_count` is the
2221        # feature's acceptance criterion, not a rough count.
2222        dump_file_path = session.dump_file_path
2223        replay_source = session.replay_source
2224        replay_skipped_reason = session.replay_skipped_reason
2225        unreplayable_flow_count = session.unreplayable_flow_count
2226
2227    http_metrics = parse_http_dump(dump_file_path, replay_source)
2228    metrics_payload: dict[str, object] = {
2229        # Whatever the connector reported about its own cache, which
2230        # `_run_connector_command` set and this must not drop: the two are
2231        # different metrics of different things, and the report shows both.
2232        **_existing_http_metrics(result),
2233        "flow_count": http_metrics.flow_count,
2234        "duplicate_flow_count": http_metrics.duplicate_flow_count,
2235        # On a replaying target run, `live_flow_count` is the acceptance
2236        # criterion for replay -- so it belongs in the report rather than
2237        # only in the raw dump. `replay_hit_ratio` is what the legacy report
2238        # called its cache-hit ratio: the share mitmproxy served from the
2239        # recording, a property of the harness rather than of the connector.
2240        "replayed_flow_count": http_metrics.replayed_flow_count,
2241        "live_flow_count": http_metrics.live_flow_count,
2242        "replay_hit_ratio": http_metrics.replay_hit_ratio,
2243    }
2244    if replay_from is not None:
2245        # Keyed on replay having been *asked* for, not on a corpus having
2246        # survived: when every recorded flow is unreplayable there is no
2247        # corpus, and that is exactly the run whose dropped count matters.
2248        # Poor replay coverage is usually this, and the fix is a bigger
2249        # `--http-replay-max-body`.
2250        metrics_payload["unreplayable_flow_count"] = unreplayable_flow_count
2251    if replay_source:
2252        metrics_payload["replay_source"] = str(replay_source)
2253        # Only on a replaying run, where a live request is a request the corpus
2254        # could not answer and so the one thing that explains a coverage
2255        # shortfall. On a recording run every request is live, so the same list
2256        # would be a request log, which is not a surface to widen for no
2257        # diagnostic gain. Query-parameter values are masked either way -- these
2258        # URLs reach the step summary, the workflow log and `report.html`, and
2259        # plenty of APIs put credentials in the query string; see `redact_url`.
2260        live_urls = http_metrics.top_live_urls()
2261        if live_urls:
2262            metrics_payload["live_urls"] = live_urls
2263            metrics_payload["live_unique_url_count"] = len(http_metrics.live_url_counts)
2264    if manager.metrics_incomplete_reason:
2265        # Read after the block, not inside it: a killed shutdown is only known
2266        # once `_stop` has run. Recorded because every count below is then a
2267        # lower bound -- and short in the direction that makes a run look closer
2268        # to the replay acceptance criterion than it was.
2269        metrics_payload["metrics_incomplete"] = manager.metrics_incomplete_reason
2270    skipped_reason = replay_skipped_reason
2271    if replay_expected and replay_source is None and not skipped_reason:
2272        # The proxy came up but there was no recording to replay -- the
2273        # control run captured nothing. Same outcome as any other skip, so
2274        # it has to read the same way rather than as a clean replayed run.
2275        skipped_reason = "the control run recorded no HTTP dump to replay from"
2276    if skipped_reason:
2277        metrics_payload["replay_skipped_reason"] = skipped_reason
2278    result["http_metrics"] = metrics_payload
2279
2280    print_success(
2281        f"Captured {http_metrics.flow_count} HTTP flows "
2282        f"({http_metrics.duplicate_flow_count} duplicates)"
2283    )
2284    if replay_source:
2285        print_success(
2286            f"Replayed {http_metrics.replayed_flow_count} of "
2287            f"{http_metrics.flow_count} request(s) "
2288            f"({http_metrics.replay_hit_ratio}); "
2289            f"{http_metrics.live_flow_count} went live"
2290        )
2291    return result, comparable
2292
2293
2294def _discard_oversized_http_dumps(*results: dict, max_bytes: int) -> None:
2295    """Drop recorded dumps too large to keep, and say so in the report.
2296
2297    "Too large to keep" is about the runner's disk, not the artifact upload:
2298    dumps are excluded from every upload step regardless of size.
2299
2300    Deleting is not silent: the size that was discarded lands in the run's
2301    `http_metrics`, so a reader looking for the dump learns why it is not there
2302    instead of finding a dangling path.
2303    """
2304    for result in results:
2305        dump_file = result.get("http_dump_file")
2306        if not dump_file:
2307            continue
2308
2309        discarded = discard_oversized_dump(Path(dump_file), max_bytes)
2310        if discarded is None:
2311            continue
2312
2313        result.pop("http_dump_file", None)
2314        metrics = result.get("http_metrics")
2315        if isinstance(metrics, dict):
2316            metrics["discarded_dump_mb"] = round(discarded / 1024 / 1024)
2317        print_warning(
2318            f"Discarded {dump_file}: {discarded / 1024 / 1024:.0f} MB exceeds the "
2319            "retention cap, and a dump this large is what fills the runner's disk. "
2320            "Dumps are never uploaded as artifacts; raise "
2321            "--http-replay-max-dump-size-mb to keep it for a local run"
2322        )
2323
2324
2325def _compare_read_records(
2326    target_result: dict,
2327    control_result: dict,
2328    catalog_file: Path | None,
2329    target_record_files: dict[str, Path] | None = None,
2330) -> tuple[RecordComparisonSummary | None, str | None]:
2331    """Run record-level comparisons between control and target read outputs.
2332
2333    Uses primary keys from the configured catalog (source-defined primary
2334    keys come from the discovered catalog) to compare record counts, PK
2335    presence, PK integrity, and field values. Streams without a primary key
2336    fall back to a directional record count comparison.
2337
2338    Each run's stdout is first split into per-stream record files (a single
2339    streaming pass), and the comparison then takes one stream at a time.
2340    The per-stream files land next to each run's stdout and double as
2341    debugging artifacts. Within one stream the records stay on disk: each
2342    check reads them back from the file, and the field comparison keeps
2343    only a position and a fingerprint per record, loading a PK's records
2344    back only where the two versions' fingerprints differ. Peak memory is
2345    the largest stream's PK indexes -- a few hundred bytes per record --
2346    not its records: a synthetic PayPal-shaped stream of 40k records
2347    (~127 MB a side) added ~35 MB to the process, where holding both sides
2348    had added ~2.8 GB. Start here when debugging an OOM.
2349
2350    Returns:
2351        `(summary, skip_reason)`. The summary is None when the comparison
2352        cannot run at all -- then `skip_reason` says why, so the caller can
2353        surface a diagnostic check instead of letting the run read as a pass
2354        that compared records. An unexpected error during the comparison
2355        itself returns a FAILED summary instead of raising: by this point
2356        both (expensive, live-API) runs have completed, and a comparison bug
2357        must fail the verdict, not destroy the run's outputs and reports.
2358    """
2359    if catalog_file is None or not catalog_file.exists():
2360        reason = "no configured catalog available"
2361        print_warning(f"Skipping record-level comparison: {reason}")
2362        return None, reason
2363
2364    try:
2365        configured_catalog = ConfiguredAirbyteCatalog.model_validate_json(
2366            catalog_file.read_text()
2367        )
2368    except Exception as exc:
2369        reason = f"configured catalog could not be parsed ({exc})"
2370        print_warning(f"Skipping record-level comparison: {reason}")
2371        return None, reason
2372
2373    try:
2374        primary_keys = get_primary_keys_per_stream(configured_catalog)
2375        # A stream name that repeats across namespaces merges two streams in
2376        # this name-keyed pipeline; those names degraded to count-only above,
2377        # and the reader must hear why from the summary, not just stdout.
2378        collision_warnings = [
2379            f"Stream name {name!r} appears in more than one namespace; the "
2380            "record comparison keys streams by name, so its records are "
2381            "compared by count only"
2382            for name in duplicate_stream_names(configured_catalog)
2383        ]
2384        control_record_files = split_records_per_stream(
2385            Path(control_result["stdout_file"])
2386        )
2387        target_record_files = target_record_files or split_records_per_stream(
2388            Path(target_result["stdout_file"])
2389        )
2390
2391        summary = run_record_comparisons_from_files(
2392            control_record_files=control_record_files,
2393            target_record_files=target_record_files,
2394            primary_keys_per_stream=primary_keys,
2395            # The two outputs end at different points when a side was
2396            # stopped, so counts and PK presence are reported, not folded.
2397            partial=bool(
2398                stopped_early_sides(
2399                    ("control", control_result), ("target", target_result)
2400                )
2401            ),
2402        )
2403        summary.warnings[:0] = collision_warnings
2404    except Exception as exc:
2405        summary = RecordComparisonSummary(
2406            passed=False,
2407            errored=True,
2408            count_comparison=ComparisonResult(
2409                passed=False, message="Record comparison errored"
2410            ),
2411            errors=[f"Record comparison errored: {exc!r}"],
2412        )
2413
2414    if summary.passed and summary.partial:
2415        print_success(
2416            "Record comparison passed on a stopped-early read (PK integrity and "
2417            "field values on shared PKs; counts and PK presence are diagnostic)"
2418        )
2419    elif summary.passed:
2420        print_success("Record comparison passed (counts, PKs, field values)")
2421    else:
2422        print_error("Record comparison failed:")
2423        for error in summary.errors[:20]:
2424            print_error(f"  - {error}")
2425    for warning in summary.warnings[:20]:
2426        print_warning(warning)
2427
2428    return summary, None
2429
2430
2431@connector_app.command(name="build-target-image")
2432def build_target_image(
2433    connector_name: Annotated[
2434        str,
2435        Parameter(help="Connector to build from source (e.g., 'source-pokeapi')."),
2436    ],
2437    repo_root: Annotated[
2438        str | None,
2439        Parameter(
2440            help="Path to the airbyte repo root, when it cannot be auto-detected."
2441        ),
2442    ] = None,
2443) -> None:
2444    """Build the image `regression-test --connector-name` would build, and nothing else.
2445
2446    Writes the image to the `image` GitHub output. CI builds here, in a job
2447    that holds no credentials, because a Java connector's build runs the pull
2448    request's own Gradle on the host. The job that holds the credentials loads
2449    the image and passes it to `regression-test` as `--test-image`.
2450    """
2451    built_image = _build_connector_image_from_source(
2452        connector_name=connector_name,
2453        repo_root=Path(repo_root) if repo_root else None,
2454        tag="dev",
2455    )
2456    if not built_image:
2457        exit_with_error(f"Failed to build image for {connector_name}")
2458    write_github_output("image", built_image)
2459
2460
2461def _iter_record_data(path: Path):
2462    with path.open() as records_file:
2463        for line in records_file:
2464            message = json.loads(line)
2465            record = message.get("record")
2466            data = record.get("data") if isinstance(record, dict) else None
2467            if isinstance(data, dict):
2468                yield data
2469
2470
2471def _record_count(path: Path) -> int:
2472    return sum(1 for _ in _iter_record_data(path))
2473
2474
2475def _new_stream_record_checks(
2476    *,
2477    stream_names: tuple[str, ...],
2478    record_files: dict[str, Path],
2479    catalog_file: Path | None,
2480    target_outputs: ComparableOutputs,
2481    record_comparison: RecordComparisonSummary | None,
2482    masked: bool,
2483    mask_policy: MaskPolicy | None,
2484    stopped: bool,
2485) -> tuple[tuple[CheckResult, ...], dict[str, dict[str, Any]], bool]:
2486    configured_catalog = (
2487        ConfiguredAirbyteCatalog.model_validate_json(catalog_file.read_text())
2488        if catalog_file is not None and catalog_file.exists()
2489        else None
2490    )
2491    configured = {
2492        item.stream.name: item
2493        for item in (configured_catalog.streams if configured_catalog else [])
2494    }
2495    primary_keys = (
2496        get_primary_keys_per_stream(configured_catalog) if configured_catalog else {}
2497    )
2498    stream_results: dict[str, dict[str, Any]] = {}
2499    schema_summaries: list[str] = []
2500    pk_summaries: list[str] = []
2501    cursor_summaries: list[str] = []
2502    schema_failed = False
2503    pk_failed = False
2504    cursor_failed = False
2505    cursor_diagnostic = False
2506    schema_diagnostic = False
2507    regression_found = False
2508    has_pk = False
2509    has_incremental = False
2510
2511    for name in stream_names:
2512        path = record_files.get(name)
2513        if path is None:
2514            continue
2515        count = _record_count(path)
2516        if not count:
2517            continue
2518        item = configured.get(name)
2519        stream = item.stream if item else None
2520        result: dict[str, Any] = {
2521            "records": count,
2522            "partial": stopped,
2523            "schema_mismatched_records": None,
2524            "undeclared_field_records": None,
2525            "pk_null": None,
2526            "pk_duplicates": None,
2527            "null_cursor_records": None,
2528            "state_has_cursor_value": None,
2529            "state_below_max": None,
2530            "check_failed": False,
2531        }
2532        stream_results[name] = result
2533        try:
2534            if stream is None:
2535                raise ValueError("configured stream missing")
2536            schema_result = check_record_schema(
2537                _iter_record_data(path), stream.json_schema or {}, masked=masked
2538            )
2539            result["schema_mismatched_records"] = schema_result.mismatched_records
2540            result["undeclared_field_records"] = schema_result.undeclared_field_records
2541            if schema_result.mismatched_records:
2542                result["check_failed"] = True
2543                schema_failed = True
2544                regression_found = True
2545            elif schema_result.undeclared_field_records:
2546                schema_diagnostic = True
2547            if schema_result.undeclared_field_records:
2548                schema_summaries.append(
2549                    f"{name}: {schema_result.undeclared_field_records} record(s) "
2550                    "have undeclared field(s)"
2551                )
2552            properties = (stream.json_schema or {}).get("properties", {})
2553            for field_name, mismatch_count in sorted(
2554                schema_result.mismatches_by_field.items()
2555            ):
2556                observed_types = schema_result.observed_types_by_field.get(field_name)
2557                observed = ", ".join(observed_types) if observed_types else "unknown"
2558                declared = declared_types(properties.get(field_name))
2559                expected = (
2560                    "unconstrained" if declared is None else "|".join(sorted(declared))
2561                )
2562                schema_summaries.append(
2563                    f"{name}.{field_name}: {mismatch_count} record(s), observed "
2564                    f"{observed}, declared {expected}"
2565                )
2566            if (
2567                stopped
2568                and not schema_result.mismatched_records
2569                and not schema_result.undeclared_field_records
2570            ):
2571                schema_summaries.append(
2572                    f"{name}: declared top-level types match on observed records"
2573                )
2574        except Exception as exc:
2575            schema_failed = True
2576            result["check_failed"] = True
2577            schema_summaries.append(f"{name}: errored ({type(exc).__name__})")
2578
2579        pk_paths = primary_keys.get(name)
2580        if pk_paths:
2581            has_pk = True
2582            try:
2583                for pk_path in pk_paths:
2584                    catalog_stream = stream.model_copy(  # ty: ignore[unresolved-attribute]
2585                        update={"source_defined_primary_key": [pk_path]}
2586                    )
2587                    validation = validate_primary_keys_exist_in_schema(
2588                        AirbyteCatalog(streams=[catalog_stream])
2589                    )
2590                    if not validation.passed:
2591                        pk_failed = True
2592                        result["check_failed"] = True
2593                        regression_found = True
2594                        pk_summaries.append(
2595                            f"{name}: primary key {'.'.join(pk_path)} missing from schema"
2596                        )
2597                if record_comparison is not None:
2598                    integrity = (
2599                        record_comparison.target_pk_uniqueness.stream_results.get(name)  # ty: ignore[unresolved-attribute]
2600                    )
2601                    presence = record_comparison.pk_presence.stream_results.get(name)  # ty: ignore[unresolved-attribute]
2602                    null_count = integrity.records_missing_pk if integrity else 0
2603                    duplicate_count = integrity.duplicate_pk_count if integrity else 0
2604                    presence_detail = (
2605                        "presence pass"
2606                        if presence is None or presence.passed
2607                        else "presence failed"
2608                    )
2609                else:
2610                    integrity = _scan_pk_integrity(_read_records_file(path), pk_paths)
2611                    null_count = integrity.records_missing_pk
2612                    duplicate_count = len(integrity.duplicates)
2613                    presence_detail = "presence not compared"
2614                result["pk_null"] = null_count
2615                result["pk_duplicates"] = duplicate_count
2616                pk_failed |= bool(null_count or duplicate_count)
2617                result["check_failed"] |= bool(null_count or duplicate_count)
2618                regression_found |= bool(null_count or duplicate_count)
2619                control_note = (
2620                    " (stream not read by control)"
2621                    if record_comparison is not None and duplicate_count
2622                    else ""
2623                )
2624                pk_summaries.append(
2625                    f"{name}: {null_count} null/missing PK record(s), "
2626                    f"{duplicate_count} duplicate PK group(s){control_note}, "
2627                    f"{presence_detail}"
2628                )
2629            except Exception as exc:
2630                pk_failed = True
2631                result["check_failed"] = True
2632                pk_summaries.append(f"{name}: errored ({type(exc).__name__})")
2633
2634        cursor_path = (
2635            list(item.cursor_field or [])
2636            if item and item.cursor_field
2637            else list(stream.default_cursor_field or [])
2638            if stream and stream.source_defined_cursor
2639            else []
2640        )
2641        sync_mode = (
2642            item.sync_mode.value
2643            if item is not None and hasattr(item.sync_mode, "value")
2644            else item.sync_mode
2645            if item is not None
2646            else None
2647        )
2648        incremental = bool(sync_mode == "incremental" and cursor_path)
2649        if incremental:
2650            has_incremental = True
2651            try:
2652                validation = validate_cursors_exist_in_schema(
2653                    AirbyteCatalog(streams=[stream]),  # ty: ignore[invalid-argument-type]
2654                    cursor_paths_by_stream={name: cursor_path},
2655                )
2656                if not validation.passed:
2657                    cursor_failed = True
2658                    result["check_failed"] = True
2659                    regression_found = True
2660                    cursor_summaries.append(
2661                        f"{name}: cursor {'.'.join(cursor_path)} missing from schema"
2662                    )
2663                state = (target_outputs.final_states or {}).get(
2664                    (stream.namespace, stream.name)  # ty: ignore[unresolved-attribute]
2665                )
2666                cursor_readable = (
2667                    mask_policy.cursor_readable(name)
2668                    if masked and mask_policy is not None
2669                    else not masked
2670                )
2671                cursor_result = check_incremental_cursor(
2672                    _iter_record_data(path),
2673                    cursor_path,
2674                    state,
2675                    cursor_readable=cursor_readable,
2676                    stopped=stopped,
2677                )
2678                result["null_cursor_records"] = cursor_result.null_cursor_records
2679                result["state_has_cursor_value"] = cursor_result.state_has_cursor_value
2680                result["state_below_max"] = cursor_result.state_below_max
2681                cursor_failed |= cursor_result.strict_failed
2682                result["check_failed"] |= cursor_result.strict_failed
2683                regression_found |= cursor_result.strict_failed
2684                diagnostics = []
2685                if cursor_result.null_cursor_records:
2686                    diagnostics.append(
2687                        f"{cursor_result.null_cursor_records} null/missing cursor record(s)"
2688                    )
2689                if cursor_result.state_below_max:
2690                    diagnostics.append("state below maximum record cursor")
2691                if cursor_result.cursor_key_not_found:
2692                    diagnostics.append("cursor key not found in state")
2693                if diagnostics:
2694                    cursor_diagnostic = True
2695                cursor_summaries.append(
2696                    f"{name}: {'; '.join(diagnostics) if diagnostics else 'cursor checks passed'}"
2697                )
2698            except Exception as exc:
2699                cursor_failed = True
2700                result["check_failed"] = True
2701                cursor_summaries.append(f"{name}: errored ({type(exc).__name__})")
2702
2703    checks: list[CheckResult] = []
2704    if stream_results:
2705        checks.append(
2706            CheckResult(
2707                name="Record schema (new streams)",
2708                passed=not (schema_failed or schema_diagnostic),
2709                severity="diagnostic"
2710                if schema_diagnostic and not schema_failed
2711                else "strict",
2712                summary="; ".join(schema_summaries)
2713                or (
2714                    f"{len(stream_results)} new stream(s): declared top-level types match"
2715                ),
2716            )
2717        )
2718    if has_pk:
2719        checks.append(
2720            CheckResult(
2721                name="Primary key (new streams)",
2722                passed=not pk_failed,
2723                summary="; ".join(pk_summaries),
2724            )
2725        )
2726    if has_incremental:
2727        checks.append(
2728            CheckResult(
2729                name="Incremental cursor (new streams)",
2730                passed=not (cursor_failed or cursor_diagnostic),
2731                severity="diagnostic"
2732                if cursor_diagnostic and not cursor_failed
2733                else "strict",
2734                summary="; ".join(cursor_summaries),
2735            )
2736        )
2737    return tuple(checks), stream_results, regression_found
2738
2739
2740@connector_app.command(name="regression-test")
2741def regression_test(
2742    skip_compare: Annotated[
2743        bool,
2744        Parameter(
2745            help="If True, skip comparison and run single-version tests only. "
2746            "If False (default), run comparison tests (target vs control)."
2747        ),
2748    ] = False,
2749    skip_record_comparison: Annotated[
2750        bool,
2751        Parameter(
2752            help="If True, skip the record-level comparison for read commands "
2753            "(counts, PK presence, PK integrity, field values) and gate the "
2754            "verdict on command outcomes only. Escape hatch for sources whose "
2755            "data legitimately changes between the two live runs (rolling "
2756            "windows, feeds that age out records) until HTTP request caching "
2757            "makes strict comparison reliable."
2758        ),
2759    ] = False,
2760    test_image: Annotated[
2761        str | None,
2762        Parameter(
2763            help="Test connector image with tag (e.g., airbyte/source-github:1.0.0). "
2764            "When provided with connector_name, this image takes precedence over "
2765            "building from source. In comparison mode, it's compared against control_image."
2766        ),
2767    ] = None,
2768    control_image: Annotated[
2769        str | None,
2770        Parameter(
2771            help="Control connector image (baseline version) with tag (e.g., airbyte/source-github:1.0.0). "
2772            "Ignored if `skip_compare=True`."
2773        ),
2774    ] = None,
2775    connector_name: Annotated[
2776        str | None,
2777        Parameter(
2778            help="Connector name to build image from source (e.g., 'source-pokeapi'). "
2779            "If provided without test_image, builds the image locally with tag 'dev'. "
2780            "When both are provided, test_image takes precedence and no build occurs. "
2781            "For comparison tests (default), this builds the target image. "
2782            "For single-version tests (skip_compare=True), this builds the test image."
2783        ),
2784    ] = None,
2785    repo_root: Annotated[
2786        str | None,
2787        Parameter(
2788            help="Path to the airbyte repo root. Required if connector_name is provided "
2789            "and the repo cannot be auto-detected."
2790        ),
2791    ] = None,
2792    command: Annotated[
2793        Literal["spec", "check", "discover", "read"],
2794        Parameter(help="The Airbyte command to run."),
2795    ] = "check",
2796    connection_id: Annotated[
2797        str | None,
2798        Parameter(
2799            help="Airbyte Cloud connection ID to fetch config/catalog from. "
2800            "Mutually exclusive with config-path/catalog-path. "
2801            "If provided, test_image/control_image can be auto-detected."
2802        ),
2803    ] = None,
2804    config_path: Annotated[
2805        str | None,
2806        Parameter(help="Path to the connector config JSON file."),
2807    ] = None,
2808    catalog_path: Annotated[
2809        str | None,
2810        Parameter(help="Path to the configured catalog JSON file (required for read)."),
2811    ] = None,
2812    state_path: Annotated[
2813        str | None,
2814        Parameter(help="Path to the state JSON file (optional for read)."),
2815    ] = None,
2816    output_dir: Annotated[
2817        str,
2818        Parameter(help="Directory to store test artifacts."),
2819    ] = "/tmp/regression_test_artifacts",
2820    enable_http_metrics: Annotated[
2821        bool,
2822        Parameter(
2823            help="Capture HTTP traffic via mitmproxy. Comparison mode always "
2824            "captures, with or without replay -- the per-version request counts "
2825            "come from the capture -- so this flag adds nothing there, and "
2826            "`--skip-compare` runs a single version without a proxy. Requires "
2827            "mitmdump to be installed; without it, and for an image with no "
2828            "certifi bundle to trust the proxy with (a Java or Kotlin "
2829            "connector), the run proceeds without metrics."
2830        ),
2831    ] = False,
2832    enable_http_replay: Annotated[
2833        bool | None,
2834        Parameter(
2835            negative="--no-http-replay",
2836            help="Replay the control run's recorded HTTP responses for the target "
2837            "run, so both versions see identical upstream data. Defaults to "
2838            "`True` in comparison mode; has no effect with `--skip-compare`, "
2839            "which has no control run to replay from. Pass --no-http-replay to "
2840            "run both versions against the live API, which is the fallback "
2841            "whenever replay cannot be delivered anyway; the traffic is still "
2842            "captured, so the report keeps its per-version request counts.",
2843        ),
2844    ] = None,
2845    http_replay_max_body: Annotated[
2846        str,
2847        Parameter(
2848            help="Response bodies larger than this are streamed rather than "
2849            "recorded, and therefore re-fetched live instead of replayed. Raise it "
2850            "for a connector with large pages, at the cost of replay memory. "
2851            "Accepts a mitmproxy size such as '1m' or '10m' -- never a bare number, "
2852            "which means bytes. Unlike the other --http-replay-* options this one "
2853            "governs recording, so it applies to every comparison, replay or not."
2854        ),
2855    ] = DEFAULT_STREAM_LARGE_BODIES,
2856    http_replay_max_dump_size_mb: Annotated[
2857        int,
2858        Parameter(
2859            help="Skip replay and run the target live when the filtered replay "
2860            "corpus is larger than this, and discard a recorded dump this large "
2861            "once the run is done with it. mitmproxy holds the whole corpus in "
2862            "memory for the entire run, and a dump grows with everything the "
2863            "connector downloads, so this is what keeps a big connector from "
2864            "exhausting the CI runner's memory and disk. Dumps are never "
2865            "uploaded as artifacts, so on a local run -- where the dump is the "
2866            "only thing that can diagnose a coverage shortfall offline -- raise "
2867            "it to keep the dump."
2868        ),
2869    ] = DEFAULT_MAX_REPLAY_DUMP_SIZE_MB,
2870    http_replay_reuse: Annotated[
2871        bool,
2872        Parameter(
2873            help="Serve a recorded response more than once. By default each "
2874            "recorded flow is consumed on first match, which preserves recorded "
2875            "ordering and lets memory fall as the run proceeds; reuse keeps the "
2876            "whole corpus resident for the entire run."
2877        ),
2878    ] = False,
2879    http_replay_ignore_params: Annotated[
2880        str | None,
2881        Parameter(
2882            help="Comma-separated query parameters to exclude from the replay match "
2883            "key, for connectors that put a timestamp or nonce in the URL."
2884        ),
2885    ] = None,
2886    strict_replay: Annotated[
2887        bool,
2888        Parameter(
2889            help="Kill requests that match nothing in the replay corpus instead of "
2890            "forwarding them upstream. Diagnostic only: it turns unmatched requests "
2891            "into connector errors, which is how you count them. mitmproxy stops "
2892            "intervening once its flowmap is empty, so combine this with "
2893            "--http-replay-reuse to cover the whole run rather than only the part "
2894            "before the corpus is consumed."
2895        ),
2896    ] = False,
2897    selected_streams: Annotated[
2898        str | None,
2899        Parameter(
2900            help="Comma-separated list of stream names to include in the read. "
2901            "Only these streams will be included in the configured catalog. "
2902            "This is useful to limit data volume by testing only specific streams."
2903        ),
2904    ] = None,
2905    discover_missing_streams: Annotated[
2906        bool,
2907        Parameter(
2908            help="Read selected streams the configured catalog does not have, such "
2909            "as a stream the PR adds: READ discovers them on the version under test "
2910            "and adds them to the catalog. A stream the control version also "
2911            "discovers is read by both versions and compared; one it does not "
2912            "discover has no control side and is read by the target only. Needs "
2913            "--selected-streams. A selected stream the target does not discover "
2914            "fails the run before READ, as a setup error."
2915        ),
2916    ] = False,
2917    injected_sync_mode: Annotated[
2918        InjectedSyncMode | None,
2919        Parameter(
2920            help="Sync mode for streams added by --discover-missing-streams. "
2921            "Default: full refresh where the stream supports it. `incremental` "
2922            "uses the stream's default cursor and falls back to full refresh, "
2923            "with a note, for a stream that cannot be read incrementally."
2924        ),
2925    ] = None,
2926    enable_debug_logs: Annotated[
2927        bool,
2928        Parameter(
2929            help="Enable debug-level logging for regression test output. "
2930            "Also passed as `LOG_LEVEL=DEBUG` to the connector Docker container. "
2931            "Switched off on a masked run for now: `--output-mode masked`, or a "
2932            "`--connection-id` run that leaves the mode unset."
2933        ),
2934    ] = False,
2935    extra_env: Annotated[
2936        list[str] | None,
2937        Parameter(
2938            help=(
2939                "Additional KEY=VALUE environment variables for the connector "
2940                "containers. In comparison mode, applies to both target and "
2941                "control. Values are visible in the debug log of the `docker run` "
2942                "command; do not use for secrets. `LOG_LEVEL=DEBUG` or `TRACE` here "
2943                "counts as debug logs and is dropped on a masked run."
2944            )
2945        ),
2946    ] = None,
2947    with_state: Annotated[
2948        bool | None,
2949        Parameter(
2950            negative="--no-state",
2951            help="Fetch and pass the connection's current state to the read command, "
2952            "producing a warm read instead of a cold read. Defaults to `True` when "
2953            "`--connection-id` is provided, `False` otherwise. Has no effect unless "
2954            "the command is `read`. Ignored when `--state-path` is explicitly provided.",
2955        ),
2956    ] = None,
2957    read_timeout: Annotated[
2958        int,
2959        Parameter(
2960            help="The read deadline in seconds. A read still running at its "
2961            "deadline is stopped, keeps what it had written, and is reported as "
2962            "stopped early. In comparison mode it is the budget for the whole "
2963            "`read` command, from its start to its report: the two versions read "
2964            "one after the other and share it, ten minutes (or a tenth of the "
2965            "budget, if less) is kept for the comparison and the report, the "
2966            "control may use up to 60% of the read time left when it starts, and "
2967            "the target gets the rest; a version with no time left does not "
2968            "start. In single-version mode it covers the command the same way, "
2969            "with the same reserve, and the one read gets the rest. Only "
2970            "applies to `read`. In CI set it below the step's own "
2971            "timeout, so the comparison and the report still run on the partial "
2972            "output instead of the step being killed with nothing to show."
2973        ),
2974    ] = DEFAULT_TIMEOUT_SECONDS,
2975    max_records: Annotated[
2976        int | None,
2977        Parameter(
2978            help="Stop each side of the read when a RECORD message arrives past "
2979            "this many; the record past the cap is not kept, and a read of no "
2980            "more than this many is complete. Bounds a read of a large "
2981            "connection; a read the cap stops is a smoke test of the records "
2982            "read before the cap, not a full regression, and the report says so "
2983            "on each side that stopped early. Has no effect unless the command "
2984            "is `read`."
2985        ),
2986    ] = None,
2987    output_mode: Annotated[
2988        Literal["masked", "raw"] | None,
2989        Parameter(
2990            help="How the connector output is written to disk. `masked` "
2991            "masks every record value as a type-and-length token in the write "
2992            "pass -- primary keys are masked and a cursor stays readable only "
2993            "when its declared type is an integer or a date; DEBUG, TRACE, ERROR "
2994            "and FATAL log messages, stack traces, error internals and every "
2995            "stderr line keep only their length, while INFO and WARN log messages, "
2996            "a TRACE error's message and the CONNECTION_STATUS message stay "
2997            "readable. `raw` keeps the output as the connector emitted it. "
2998            "Unset is `masked` on a `--connection-id` run, which reads a "
2999            "customer's data, and `raw` otherwise; pass `raw` to keep a "
3000            "connection run's output as emitted, or `masked` to mask a GSM or "
3001            "local run. Debug logs are off on a masked run for now: "
3002            "`--enable-debug-logs`, and `LOG_LEVEL=DEBUG` or `TRACE` in "
3003            "`--extra-env`, are dropped with a note and the run goes on."
3004        ),
3005    ] = None,
3006    mask_key_path: Annotated[
3007        str | None,
3008        Parameter(
3009            help="A file holding the run's masking key as 64 hex characters. "
3010            "Every command of one run (spec, check, discover, read) is given "
3011            "the same path, so their tokens pair within the run and with no "
3012            "other run's; without it the CLI makes a fresh key for this "
3013            "process, so a run by hand pairs its own two sides and nothing "
3014            "else. The key is never printed or written to any output. Ignored "
3015            "on a raw run."
3016        ),
3017    ] = None,
3018    repro_case_dir: Annotated[
3019        str | None,
3020        Parameter(
3021            help="Reproduction case directory (see `regression-case-setup`). Its "
3022            "catalog.json, state.json and expectations.json replace --catalog-path "
3023            "and --state-path, HTTP replay is turned off, and a read passes only "
3024            "when the control violates the case's expectations and the target "
3025            "satisfies them (see --repro-goal). Writes the verdict to "
3026            "`result.json` in the output directory. Requires `--command read`, "
3027            "--config-path, --sandbox-scenario and comparison mode; run the "
3028            "command under `sandbox exec`. Runs raw: `--output-mode masked` is "
3029            "refused."
3030        ),
3031    ] = None,
3032    repro_goal: Annotated[
3033        Literal["fixed", "reproduced"] | None,
3034        Parameter(
3035            help="What a case read must show. `fixed` (default): the control "
3036            "violates the case and the target satisfies it. `reproduced`: both "
3037            "violate it -- the case gate's check, with the unmodified source of "
3038            "the affected release as the target. Requires --repro-case-dir."
3039        ),
3040    ] = None,
3041    sandbox_scenario: Annotated[
3042        str | None,
3043        Parameter(
3044            help="SaaS Sandbox scenario UUID the case was set up in. For a read, "
3045            "its baseline snapshot is restored immediately before each side runs, "
3046            "so control and target start from identical data and failure rules."
3047        ),
3048    ] = None,
3049) -> None:
3050    """Run regression tests on connectors.
3051
3052    This command supports two modes:
3053
3054    Comparison mode (skip_compare=False, default):
3055        Runs the specified Airbyte protocol command against both the target (new)
3056        and control (baseline) connector versions, then compares the results.
3057        This helps identify regressions between versions.
3058
3059    Single-version mode (skip_compare=True):
3060        Runs the specified Airbyte protocol command against a single connector
3061        and validates the output. No comparison is performed.
3062
3063    Results are written to the output directory and to GitHub Actions outputs
3064    if running in CI.
3065
3066    You can provide the test image in three ways:
3067    1. --test-image: Use a pre-built image from Docker registry
3068    2. --connector-name: Build the image locally from source code
3069    3. --connection-id: Auto-detect from an Airbyte Cloud connection
3070
3071    You can provide config/catalog either via file paths OR via a connection_id
3072    that fetches them from Airbyte Cloud.
3073    """
3074    # Before anything else: `--read-timeout` covers the whole command, so the
3075    # setup below spends it as surely as the reads do.
3076    read_budget = ReadBudget.start(read_timeout)
3077    parsed_extra_env = _parse_extra_env(extra_env)
3078
3079    output_path = Path(output_dir)
3080    output_path.mkdir(parents=True, exist_ok=True)
3081
3082    cmd = Command(command)
3083
3084    if read_timeout <= 0:
3085        write_github_output("success", False)
3086        write_github_output(
3087            "error", "--read-timeout must be a positive number of seconds"
3088        )
3089        exit_with_error("--read-timeout must be a positive number of seconds")
3090    if max_records is not None and max_records < 1:
3091        write_github_output("success", False)
3092        write_github_output("error", "--max-records must be at least 1")
3093        exit_with_error("--max-records must be at least 1")
3094    if max_records is not None and cmd is not Command.READ:
3095        print_warning("--max-records only applies to `read`; ignoring it")
3096        max_records = None
3097    if max_records is not None:
3098        print_success(
3099            f"Capping each read at {max_records} record(s): a read that goes "
3100            "past them is stopped there, and its comparison covers only the "
3101            "records read before the cap"
3102        )
3103    connection_residency: str | None = None
3104    if connection_id:
3105        try:
3106            connection_residency = get_connection_residency(connection_id)
3107        except _RESIDENCY_LOOKUP_ERRORS as exc:
3108            message = _describe_residency_lookup_error(connection_id, exc)
3109            write_github_output("success", False)
3110            write_github_output("error", message)
3111            exit_with_error(message)
3112        if connection_residency == "eu" and output_mode == "raw":
3113            message = (
3114                "EU-resident connections always run masked; output_mode=raw is refused."
3115            )
3116            write_github_output("success", False)
3117            write_github_output("error", message)
3118            exit_with_error(message)
3119        if connection_residency == "eu":
3120            _refuse_eu_connection_outside_eu_region(connection_id, connection_residency)
3121            output_mode = "masked"
3122        else:
3123            _refuse_non_eu_connection_in_eu_region(connection_id)
3124    if output_mode is None:
3125        # A connection reads a customer's data, so unset is masked there and
3126        # `raw` is the opt-in; a GSM or local config is the team's own. Kept
3127        # apart from an explicit value until here so the EU run path can
3128        # resolve an unset mode by residency.
3129        output_mode = "masked" if connection_id else "raw"
3130        if output_mode == "masked":
3131            print_success(
3132                "Output mode: masked (the default for a connection run); pass "
3133                "--output-mode raw to keep the output as emitted"
3134            )
3135    if output_mode == "masked":
3136        parsed_extra_env, dropped = _without_debug_logs(
3137            enable_debug_logs=enable_debug_logs, extra_env=parsed_extra_env
3138        )
3139        if dropped:
3140            enable_debug_logs = False
3141            print_warning(
3142                "Debug logs are off on masked runs; ignoring " + " and ".join(dropped)
3143            )
3144        print_success(
3145            "Masking connector output: record values become tokens; primary keys "
3146            "are masked; only integer and date cursors stay readable"
3147        )
3148    # The one key every policy of this invocation masks under, resolved before
3149    # anything is fetched so a bad key file fails the run in seconds. A raw
3150    # run has none, and does not read the file it may have been given.
3151    mask_key = _resolve_mask_key(mask_key_path) if output_mode == "masked" else None
3152
3153    # Configure debug logging for the regression test harness when requested
3154    if enable_debug_logs:
3155        logging.basicConfig(
3156            level=logging.DEBUG,
3157            format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
3158            force=True,
3159        )
3160
3161    repro_case: ReproCase | None = None
3162    sandbox: SandboxCli | None = None
3163    if repro_case_dir or sandbox_scenario or repro_goal:
3164        case_error = _repro_case_argument_error(
3165            command=cmd,
3166            repro_case_dir=repro_case_dir,
3167            sandbox_scenario=sandbox_scenario,
3168            skip_compare=skip_compare,
3169            connection_id=connection_id,
3170            output_mode=output_mode,
3171            config_path=config_path,
3172            catalog_path=catalog_path,
3173            state_path=state_path,
3174            selected_streams=selected_streams,
3175        )
3176        if case_error is None and repro_case_dir:
3177            try:
3178                repro_case = ReproCase.load(Path(repro_case_dir))
3179                sandbox = SandboxCli()
3180            except (OSError, ValueError, SandboxError) as exc:
3181                case_error = f"Invalid reproduction case: {exc}"
3182        if case_error is None and repro_case is not None:
3183            case_error = _repro_case_command_error(cmd, repro_case.expectations.command)
3184        if case_error is not None:
3185            write_github_output("success", False)
3186            write_github_output("error", case_error)
3187            exit_with_error(case_error)
3188        assert repro_case is not None
3189        if repro_case.expectations.command == "read":
3190            catalog_path = str(repro_case.catalog_path)
3191            state_path = str(repro_case.state_path) if repro_case.state_path else None
3192        # The sandbox is the controlled upstream; replaying one side's traffic
3193        # to the other would bypass the per-side baseline restore.
3194        if enable_http_replay or enable_http_metrics:
3195            print_warning("HTTP replay and metrics are off for reproduction cases")
3196        enable_http_replay = False
3197        enable_http_metrics = False
3198
3199    # On by default, but only where it can work: single-version mode has no
3200    # control run to replay from. Asking for it there explicitly is a mistake
3201    # worth naming; the default landing there is not, so it is not a warning.
3202    if skip_compare:
3203        if enable_http_replay:
3204            print_warning("--enable-http-replay needs a control run; ignoring it")
3205        replay_enabled = False
3206    else:
3207        replay_enabled = True if enable_http_replay is None else enable_http_replay
3208        # Every comparison captures, replay or not: the per-version request
3209        # counts come from the capture, and a `--no-http-replay` run is the one
3210        # where they matter most. Replay stays a separate decision; a
3211        # reproduction case keeps the capture off (see above), and an image
3212        # that cannot trust the proxy runs without it.
3213        if repro_case is None:
3214            enable_http_metrics = True
3215
3216    if not replay_enabled and (
3217        http_replay_reuse or strict_replay or http_replay_ignore_params
3218    ):
3219        # Otherwise someone debugging poor coverage with --strict-replay alone
3220        # concludes strict mode found nothing wrong. `--http-replay-max-body`
3221        # and `--http-replay-max-dump-size-mb` are deliberately not in this
3222        # list: the first governs recording and the second decides whether a
3223        # recorded dump is kept, and every comparison records.
3224        print_warning(
3225            "Replay tuning flags do nothing with --no-http-replay; ignoring them"
3226        )
3227
3228    replay_options = ReplayOptions(
3229        reuse=http_replay_reuse,
3230        extra="kill" if strict_replay else DEFAULT_REPLAY_EXTRA,
3231        ignore_params=tuple(
3232            param.strip()
3233            for param in (http_replay_ignore_params or "").split(",")
3234            if param.strip()
3235        ),
3236        max_dump_bytes=http_replay_max_dump_size_mb * 1024 * 1024,
3237    )
3238
3239    config_file: Path | None = None
3240    catalog_file: Path | None = None
3241    state_file = Path(state_path) if state_path else None
3242    connection_image: str | None = None
3243
3244    selected_stream_names: set[str] = (
3245        {s.strip() for s in selected_streams.split(",") if s.strip()}
3246        if selected_streams
3247        else set()
3248    )
3249
3250    # Resolve the test image (used in both single-version and comparison modes)
3251    resolved_test_image: str | None = test_image
3252    resolved_control_image: str | None = control_image
3253
3254    # Validate conflicting parameters
3255    # Single-version mode: reject comparison-specific parameters
3256    if skip_compare and control_image:
3257        write_github_output("success", False)
3258        write_github_output(
3259            "error", "Cannot specify control_image with skip_compare=True"
3260        )
3261        exit_with_error(
3262            "Cannot specify --control-image with --skip-compare. "
3263            "Control image is only used in comparison mode."
3264        )
3265
3266    built_test_image_locally = False
3267    if connector_name and not resolved_test_image:
3268        repo_root_path = Path(repo_root) if repo_root else None
3269        built_image = _build_connector_image_from_source(
3270            connector_name=connector_name,
3271            repo_root=repo_root_path,
3272            tag="dev",
3273            prefer_repo_root=repro_case is not None,
3274        )
3275        if not built_image:
3276            write_github_output("success", False)
3277            write_github_output("error", f"Failed to build image for {connector_name}")
3278            exit_with_error(f"Failed to build image for {connector_name}")
3279        resolved_test_image = built_image
3280        built_test_image_locally = True
3281
3282    if connection_id:
3283        if config_path or catalog_path:
3284            write_github_output("success", False)
3285            write_github_output(
3286                "error", "Cannot specify both connection_id and file paths"
3287            )
3288            exit_with_error(
3289                "Cannot specify both connection_id and config_path/catalog_path"
3290            )
3291
3292        print_success(f"Fetching config/catalog from connection: {connection_id}")
3293        connection_data = fetch_connection_data(connection_id)
3294
3295        # Check if we should retrieve unmasked secrets
3296        if should_use_secret_retriever():
3297            print_success(
3298                "USE_CONNECTION_SECRET_RETRIEVER enabled - enriching config with unmasked secrets..."
3299            )
3300            try:
3301                connection_data = enrich_config_with_secrets(
3302                    connection_data,
3303                    retrieval_reason="Regression test with USE_CONNECTION_SECRET_RETRIEVER=true",
3304                )
3305                print_success("Successfully retrieved unmasked secrets from database")
3306            except SecretRetrievalError as e:
3307                write_github_output("success", False)
3308                write_github_output("error", str(e))
3309                exit_with_error(
3310                    f"{e}\n\n"
3311                    f"This connection cannot be used for regression testing. "
3312                    f"Please use a connection from a non-EU workspace, or use GSM-based "
3313                    f"integration test credentials instead (by omitting --connection-id)."
3314                )
3315
3316        # Fill in stream schemas: the minimal catalog from the public API has
3317        # empty json_schema values, which silently yields zero records for
3318        # connectors that derive request fields from the catalog schema.
3319        # Only the read command consumes the catalog, so spec/check/discover
3320        # skip the extra Cloud API round trip.
3321        enrichment_state: list[dict[str, Any]] | None = None
3322        enrichment_state_fetched = False
3323        if cmd.needs_catalog():
3324            merged_count = 0
3325            try:
3326                enrichment = enrich_catalog_schemas_from_cloud(connection_data)
3327                connection_data = enrichment.connection_data
3328                merged_count = enrichment.merged_count
3329                enrichment_state = enrichment.state
3330                enrichment_state_fetched = True
3331            except Exception as exc:
3332                print_error(f"Failed to fetch stream schemas from Cloud API: {exc}")
3333            missing_schemas = streams_missing_schemas(connection_data.catalog)
3334            if selected_stream_names:
3335                missing_schemas = [
3336                    s for s in missing_schemas if s in selected_stream_names
3337                ]
3338            if missing_schemas and should_use_secret_retriever():
3339                try:
3340                    connection_data, db_merged_count = (
3341                        enrich_catalog_schemas_from_platform_db(
3342                            connection_data,
3343                            retrieval_reason="Regression test catalog schema enrichment",
3344                        )
3345                    )
3346                    merged_count += db_merged_count
3347                except Exception as exc:
3348                    print_error(f"Failed to merge stored stream schemas: {exc}")
3349                missing_schemas = streams_missing_schemas(connection_data.catalog)
3350                if selected_stream_names:
3351                    missing_schemas = [
3352                        s for s in missing_schemas if s in selected_stream_names
3353                    ]
3354            total_streams = len(connection_data.catalog.get("streams", []))
3355            checked_streams = (
3356                f"{len(missing_schemas)} of {len(selected_stream_names)} "
3357                "selected stream(s)"
3358                if selected_stream_names
3359                else f"{len(missing_schemas)} of {total_streams} stream(s)"
3360            )
3361            if not total_streams:
3362                error_msg = (
3363                    "No configured streams found in the connection catalog. "
3364                    "A read with zero streams would produce zero records on "
3365                    "both versions, so failing instead of producing a false pass."
3366                )
3367                write_github_output("success", False)
3368                write_github_output("error", error_msg)
3369                exit_with_error(error_msg)
3370            if missing_schemas:
3371                error_msg = (
3372                    f"Stream schemas missing for {checked_streams}: "
3373                    f"{', '.join(missing_schemas)}. "
3374                    "Connectors that derive request fields from the catalog "
3375                    "schema would silently return zero records, so failing "
3376                    "instead of producing a false pass."
3377                )
3378                write_github_output("success", False)
3379                write_github_output("error", error_msg)
3380                exit_with_error(error_msg)
3381            print_success(
3382                f"Stream schemas ready: merged {merged_count} of {total_streams} "
3383                "stream(s)"
3384                + (
3385                    f"; checked {len(selected_stream_names)} selected stream(s)"
3386                    if selected_stream_names
3387                    else ""
3388                )
3389            )
3390
3391        # Resolve with_state default: True when connection_id is provided
3392        resolved_with_state = with_state if with_state is not None else True
3393
3394        if not (resolved_with_state and not state_file and command == "read"):
3395            connection_data.state = None
3396        elif enrichment_state_fetched:
3397            connection_data.state = enrichment_state
3398            if connection_data.state:
3399                print_success(
3400                    f"Using state fetched during schema enrichment "
3401                    f"({len(connection_data.state)} stream(s))"
3402                )
3403            else:
3404                print_success("No state available for this connection (initial sync)")
3405        else:
3406            print_success("Fetching connection state for warm read...")
3407            try:
3408                connection_data.state = fetch_connection_state(connection_data)
3409                if connection_data.state:
3410                    print_success(
3411                        f"Fetched state with {len(connection_data.state)} stream(s)"
3412                    )
3413                else:
3414                    connection_data.state = None
3415                    print_success(
3416                        "No state available for this connection (initial sync)"
3417                    )
3418            except Exception as exc:
3419                print_error(f"Failed to fetch connection state: {exc}")
3420                print_success("Falling back to cold read (no state)")
3421
3422        # The output directory is uploaded whole as the run's artifact, so the
3423        # config -- the one object that carries the secrets -- is written to a
3424        # private directory outside it, as the GSM path below already does. A
3425        # masked run keeps the raw state there too: the artifact gets a copy
3426        # masked under the same rule as the records.
3427        config_file, catalog_file, state_file_from_conn = save_connection_data_to_files(
3428            connection_data,
3429            output_path / "connection_data",
3430            config_dir=_private_dir("connection-config-"),
3431            mask_policy=_mask_policy(connection_data.catalog, output_mode, mask_key),
3432        )
3433
3434        if state_file_from_conn and not state_file:
3435            state_file = state_file_from_conn
3436
3437        print_success(
3438            f"Fetched config for source: {connection_data.source_name} "
3439            f"with {len(connection_data.stream_names)} streams"
3440        )
3441
3442        connection_image = connection_data.connector_image
3443
3444        # Auto-detect test/control image from connection if not provided
3445        if not resolved_test_image and connection_data.connector_image:
3446            resolved_test_image = connection_data.connector_image
3447            print_success(f"Auto-detected test image: {resolved_test_image}")
3448
3449        if (
3450            not skip_compare
3451            and not resolved_control_image
3452            and connection_data.connector_image
3453        ):
3454            resolved_control_image = connection_data.connector_image
3455            print_success(f"Auto-detected control image: {resolved_control_image}")
3456    elif config_path:
3457        config_file = Path(config_path)
3458        catalog_file = Path(catalog_path) if catalog_path else None
3459    elif connector_name:
3460        # Prefer the default config secret, falling back to other GSM secrets.
3461        print_success(
3462            f"No connection_id or config_path provided. "
3463            f"Attempting to fetch integration test config from GSM for {connector_name}..."
3464        )
3465        gsm_unavailable_message = None
3466        try:
3467            gsm_config = get_integration_test_config_from_secrets(connector_name)
3468        except ConnectorSecretsUnavailableError as exc:
3469            gsm_unavailable_message = (
3470                f"{exc} Or pass a connection_id to test against a customer connection."
3471            )
3472            if cmd != Command.SPEC:
3473                write_github_output("success", False)
3474                write_github_output("error", gsm_unavailable_message)
3475                exit_with_error(gsm_unavailable_message)
3476            print_error(gsm_unavailable_message)
3477            gsm_config = None
3478
3479        if gsm_config:
3480            # Write config to a temp file (not in output_path to avoid artifact upload)
3481            gsm_config_dir = Path(
3482                tempfile.mkdtemp(prefix=f"gsm-config-{connector_name}-")
3483            )
3484            gsm_config_dir.chmod(0o700)
3485            gsm_config_file = gsm_config_dir / "config.json"
3486            gsm_config_file.write_text(json.dumps(gsm_config.config, indent=2))
3487            gsm_config_file.chmod(0o600)
3488            config_file = gsm_config_file
3489            # Use catalog_path if provided (e.g., generated from discover output)
3490            catalog_file = Path(catalog_path) if catalog_path else None
3491            print_success(
3492                f"Fetched integration test config from GSM secret "
3493                f"{gsm_config.secret_name} for {connector_name}"
3494            )
3495        else:
3496            if gsm_unavailable_message is None:
3497                msg = (
3498                    f"No usable GSM integration-test secret was found for "
3499                    f"{connector_name} in {DEFAULT_GCP_PROJECT_ID}. "
3500                    f"The secret may be missing, have no enabled version, or contain "
3501                    f"an empty placeholder (every value blank or null). Pass a "
3502                    f"connection_id to test against a customer connection, or have "
3503                    f"the connector's integration-test secret added or refreshed."
3504                )
3505                if cmd != Command.SPEC:
3506                    write_github_output("success", False)
3507                    write_github_output("error", msg)
3508                    exit_with_error(msg)
3509                print_error(msg)
3510            config_file = None
3511            # Use catalog_path if provided (e.g., generated from discover output)
3512            catalog_file = Path(catalog_path) if catalog_path else None
3513    else:
3514        config_file = None
3515        catalog_file = Path(catalog_path) if catalog_path else None
3516
3517    # Auto-detect control_image from metadata.yaml if connector_name is provided (comparison mode only)
3518    if not skip_compare and not resolved_control_image and connector_name:
3519        resolved_control_image = _fetch_control_image_from_metadata(connector_name)
3520        if resolved_control_image:
3521            print_success(
3522                f"Auto-detected control image from metadata.yaml: {resolved_control_image}"
3523            )
3524
3525    # Validate that we have the required images
3526    if not resolved_test_image:
3527        write_github_output("success", False)
3528        write_github_output("error", "No test image specified")
3529        exit_with_error(
3530            "You must provide one of the following: a test_image, a connector_name "
3531            "to build the image from source, or a connection_id to auto-detect the image."
3532        )
3533
3534    if not skip_compare and not resolved_control_image:
3535        write_github_output("success", False)
3536        write_github_output("error", "No control image specified")
3537        exit_with_error(
3538            "You must provide one of the following: a control_image, a connection_id "
3539            "for a connection that has an associated connector image, or a connector_name "
3540            "to auto-detect the control image from the airbyte repo's metadata.yaml."
3541        )
3542
3543    # Pull images if they weren't just built locally
3544    if not built_test_image_locally and not ensure_image_available(resolved_test_image):
3545        write_github_output("success", False)
3546        write_github_output("error", f"Failed to pull image: {resolved_test_image}")
3547        exit_with_error(f"Failed to pull test image: {resolved_test_image}")
3548
3549    if (
3550        not skip_compare
3551        and resolved_control_image
3552        and not ensure_image_available(resolved_control_image)
3553    ):
3554        write_github_output("success", False)
3555        write_github_output("error", f"Failed to pull image: {resolved_control_image}")
3556        exit_with_error(
3557            f"Failed to pull control connector image: {resolved_control_image}"
3558        )
3559
3560    if discover_missing_streams and not selected_stream_names:
3561        error_msg = (
3562            "--discover-missing-streams needs --selected-streams naming the "
3563            "streams to add to the catalog."
3564        )
3565        write_github_output("success", False)
3566        write_github_output("error", error_msg)
3567        exit_with_error(error_msg)
3568
3569    # Streams the connection does not sync yet, typically one the PR adds, are
3570    # configured from DISCOVER. The two versions can then get different
3571    # catalogs: a stream with no control side goes to the target only.
3572    control_catalog_file = catalog_file
3573    injection: InjectedCatalogs | None = None
3574    if discover_missing_streams and cmd.needs_catalog():
3575        injected = _inject_discovered_streams(
3576            catalog_file=catalog_file,
3577            selected_stream_names=selected_stream_names,
3578            test_image=resolved_test_image,
3579            reference_image=(
3580                resolved_control_image
3581                if not skip_compare
3582                else connection_image
3583                or (
3584                    _fetch_control_image_from_metadata(connector_name)
3585                    if connector_name
3586                    else None
3587                )
3588            ),
3589            skip_compare=skip_compare,
3590            config_file=config_file,
3591            output_path=output_path,
3592            enable_debug_logs=enable_debug_logs,
3593            extra_env=parsed_extra_env,
3594            sync_mode=injected_sync_mode,
3595            output_mode=output_mode,
3596            mask_key=mask_key,
3597            read_budget=read_budget,
3598        )
3599        if injected is not None:
3600            catalog_file, control_catalog_file, injection = injected
3601
3602    # Apply selected_streams filter to catalog if requested. Commands that read
3603    # no catalog skip it under --discover-missing-streams: the connection's
3604    # catalog does not have the streams being added, so the filter would fail
3605    # SPEC, CHECK and DISCOVER on a stream only READ will configure.
3606    if (
3607        selected_stream_names
3608        and catalog_file
3609        and injection is None
3610        and not (discover_missing_streams and not cmd.needs_catalog())
3611    ):
3612        print_success(
3613            f"Filtering catalog to {len(selected_stream_names)} selected streams: "
3614            f"{', '.join(sorted(selected_stream_names))}"
3615        )
3616        filter_configured_catalog_file(catalog_file, selected_stream_names)
3617
3618    # A seed the catalog cannot apply would run as a cold read that the log
3619    # and the report still called seeded. A reproduction case brings its own
3620    # catalog and state, judged when the case is loaded.
3621    if cmd == Command.READ and state_path and repro_case is None:
3622        seed_error, unseeded_streams = _judge_seed_file(Path(state_path), catalog_file)
3623        if seed_error:
3624            write_github_output("success", False)
3625            write_github_output("error", seed_error)
3626            exit_with_error(seed_error)
3627        for stream_name in unseeded_streams:
3628            print_warning(
3629                f"The configured catalog does not sync '{stream_name}' "
3630                "incrementally, so the connector will ignore its seeded state"
3631            )
3632
3633    # Upgrade READ → READ_WITH_STATE when a state file is available
3634    state_seeded = cmd == Command.READ and bool(state_path)
3635    if cmd == Command.READ and state_file:
3636        cmd = Command.READ_WITH_STATE
3637        if state_seeded:
3638            print_success(
3639                f"Using seeded state from {state_file} — read-with-state starts "
3640                "from its cursors, not the connection's"
3641            )
3642        else:
3643            print_success("State available — using read-with-state (warm read)")
3644
3645    # Only a read runs long enough for the deadline to matter, and only a read
3646    # has partial output worth keeping when it does.
3647    read_timeout_seconds = (
3648        read_timeout if cmd in (Command.READ, Command.READ_WITH_STATE) else None
3649    )
3650
3651    # The report embeds the connection's input state; in a masked run it is
3652    # masked under the catalog the run uses and under the run's key, so its
3653    # tokens are the ones the records carry. A seeded state shows its outline
3654    # only (`_seed_outline`).
3655    report_mask_policy = _mask_policy(
3656        _load_json_file(catalog_file) if catalog_file else None, output_mode, mask_key
3657    )
3658
3659    # Track telemetry for the regression test
3660    # Extract version from image tag (e.g., "airbyte/source-github:1.0.0" -> "1.0.0")
3661    target_version = (
3662        resolved_test_image.rsplit(":", 1)[-1]
3663        if ":" in resolved_test_image
3664        else "unknown"
3665    )
3666    control_version = None
3667    if resolved_control_image and ":" in resolved_control_image:
3668        control_version = resolved_control_image.rsplit(":", 1)[-1]
3669
3670    # Get tester identity from environment (GitHub Actions sets GITHUB_ACTOR)
3671    tester = os.getenv("GITHUB_ACTOR") or os.getenv("USER")
3672
3673    track_regression_test(
3674        user_id=tester,
3675        connector_image=resolved_test_image,
3676        command=command,
3677        target_version=target_version,
3678        control_version=control_version,
3679        additional_properties={
3680            "connection_id": connection_id,
3681            "skip_compare": skip_compare,
3682            "with_state": cmd == Command.READ_WITH_STATE,
3683            "state_seeded": state_seeded,
3684            "max_records": max_records,
3685            "output_mode": output_mode,
3686        },
3687    )
3688
3689    # Execute the appropriate mode
3690    if skip_compare:
3691        # Single-version mode: run only the connector image. There is no control
3692        # to compare its spec or catalog against, so the declared objects go
3693        # unread here. Its read keeps the same reserve as a comparison's, so
3694        # the report is written before the step's own timeout.
3695        single_timeout_s = (
3696            read_budget.read_time_left_s() if read_timeout_seconds is not None else None
3697        )
3698        if single_timeout_s is not None and single_timeout_s <= 0:
3699            result, single_connector_outputs = _read_not_started(
3700                resolved_test_image, cmd, output_path, output_mode
3701            )
3702        else:
3703            result, single_connector_outputs = _run_connector_command(
3704                connector_image=resolved_test_image,
3705                command=cmd,
3706                output_dir=output_path,
3707                target_or_control=TargetOrControl.TARGET,
3708                config_path=config_file,
3709                catalog_path=catalog_file,
3710                state_path=state_file,
3711                enable_debug_logs=enable_debug_logs,
3712                timeout_seconds=single_timeout_s,
3713                output_mode=output_mode,
3714                mask_key=mask_key,
3715                extra_env=parsed_extra_env,
3716                max_records=max_records,
3717            )
3718        _record_read_share(result, single_timeout_s, read_timeout)
3719
3720        configured_streams = _configured_stream_names(cmd, catalog_file)
3721        report_model = build_single_version_report_model(
3722            connector_image=resolved_test_image,
3723            command=command,
3724            result=result,
3725            configured_streams=configured_streams,
3726            target_only_streams=(
3727                tuple(sorted((*injection.new_streams, *injection.kept_target_only)))
3728                if injection is not None
3729                else ()
3730            ),
3731            stream_statuses=single_connector_outputs.stream_statuses,
3732            stream_errors=single_connector_outputs.stream_errors,
3733            include_report_sections=False,
3734        )
3735
3736        # Same per-version rule as comparison mode: for CHECK the exit code alone
3737        # is not the verdict, connectionStatus is.
3738        annotated_result = _annotate_run_verdict(command, result)
3739        incomplete = report_model.incomplete
3740        single_success = report_model.passed
3741        annotated_result.update(report_model.verdict_flags())
3742
3743        print_json(_printable_payload(annotated_result, output_mode))
3744
3745        single_outputs: dict[str, object] = {
3746            **report_model.verdict_flags(github=True),
3747            "connector": resolved_test_image,
3748            "command": command,
3749            "exit_code": result["exit_code"],
3750            "output_mode": output_mode,
3751        }
3752        if command == "check":
3753            # Named to match the comparison-mode output the workflow summary reads.
3754            single_outputs["target_connection_status"] = result.get("connection_status")
3755
3756        write_github_outputs(single_outputs)
3757
3758        write_json_output(
3759            "regression_report",
3760            {
3761                **report_model.verdict_flags(),
3762                "command": command,
3763                "output_mode": output_mode,
3764                "target": _annotate_run_verdict(command, result),
3765                "check_results": report_model.check_results(),
3766                "partial_streams": list(report_model.partial_streams),
3767            },
3768        )
3769
3770        connection_objects: tuple[DiffBlock, ...] = ()
3771        with report_section("connection objects"):
3772            connection_objects = _collect_connection_objects(
3773                catalog_file,
3774                state_file,
3775                output_path / "airbyte_messages",
3776                state_seeded=state_seeded,
3777                mask_policy=report_mask_policy,
3778            )
3779        report_model = enrich_report_model(
3780            report_model,
3781            target_result=result,
3782            configured_streams=configured_streams,
3783            connection_objects=connection_objects,
3784        )
3785
3786        # Generate report.md with detailed metrics
3787        with report_section("markdown report"):
3788            report_path = generate_single_version_report(
3789                connector_image=resolved_test_image,
3790                command=command,
3791                result=result,
3792                output_dir=output_path,
3793                report_model=report_model,
3794            )
3795            print_success(f"Generated report: {report_path}")
3796
3797        # The detailed surface is report.html in the uploaded artifact; the step
3798        # summary gets the concise block instead of the whole report. Both are
3799        # built from the unannotated result: the model derives the verdict itself.
3800        with report_section("HTML report"):
3801            html_report_path = write_html_report(report_model, output_path)
3802            print_success(f"Generated HTML report: {html_report_path}")
3803
3804        # Write the concise summary to GITHUB_STEP_SUMMARY (if env var exists)
3805        with report_section("GitHub step summary"):
3806            write_github_summary(render_inline_summary(report_model))
3807
3808        if incomplete:
3809            print_error(
3810                f"Single-version regression test incomplete for "
3811                f"{resolved_test_image}: the read was stopped early."
3812            )
3813        elif single_success:
3814            print_success(
3815                f"Single-version regression test passed for {resolved_test_image}"
3816            )
3817        elif command == "check" and result["success"]:
3818            print_error(
3819                f"Single-version regression test failed for {resolved_test_image}: "
3820                f"connectionStatus {result.get('connection_status') or 'missing'} "
3821                "(the connector itself exited 0)."
3822            )
3823        else:
3824            print_error(
3825                f"Single-version regression test failed for {resolved_test_image}"
3826            )
3827    else:
3828        # Comparison mode: run both control and target images. The control
3829        # version runs FIRST on purpose: the record comparison is
3830        # directional (extra records in the target are allowed, missing
3831        # ones fail), so data created upstream between the two live runs
3832        # must land in the target output — where it is tolerated — rather
3833        # than in the control output, where it would read as records
3834        # missing from the target and fail the comparison. Accepted side
3835        # effect (applies to every command, not just read): a hanging or
3836        # timing-out control run means the target never runs, so such a
3837        # run says nothing about the version under test.
3838        #
3839        # HTTP replay needs the same order for its own reason: the target
3840        # replays the dump the control run recorded, so that dump has to
3841        # exist before the target starts.
3842        target_output = output_path / "target"
3843        control_output = output_path / "control"
3844
3845        # The control reads `control_catalog_file`: the configured catalog, or
3846        # under --discover-missing-streams one without the target-only streams
3847        # (set above). The NOT EXERCISED rule below judges the difference.
3848
3849        # A read's two sides share `--read-timeout` with the comparison and
3850        # the report, each taking its share as it starts (see `ReadBudget`):
3851        # given the whole budget each, two long reads back to back outlast the
3852        # step, which then dies with no report. Other commands keep the
3853        # runner's default.
3854        control_sandbox = _restore_sandbox_baseline(sandbox, sandbox_scenario)
3855        control_timeout_s = (
3856            read_budget.control_share_s() if read_timeout_seconds is not None else None
3857        )
3858        # Use the target catalog so changed shared-stream cursor or primary-key
3859        # declarations cannot change record tokens; catalog comparison shows them.
3860        control_result, control_outputs = _run_with_optional_http_metrics(
3861            connector_image=resolved_control_image,  # ty: ignore[invalid-argument-type]
3862            command=cmd,
3863            output_dir=control_output,
3864            target_or_control=TargetOrControl.CONTROL,
3865            enable_http_metrics=enable_http_metrics,
3866            config_path=config_file,
3867            catalog_path=control_catalog_file,
3868            state_path=state_file,
3869            enable_debug_logs=enable_debug_logs,
3870            extra_env=parsed_extra_env,
3871            stream_large_bodies=http_replay_max_body,
3872            timeout_seconds=control_timeout_s,
3873            max_records=max_records,
3874            output_mode=output_mode,
3875            mask_key=mask_key,
3876            mask_catalog_path=catalog_file,
3877        )
3878        _record_read_share(control_result, control_timeout_s, read_timeout)
3879        control_sandbox = _check_sandbox_twin_gaps(
3880            sandbox, sandbox_scenario, control_sandbox, repro_case
3881        )
3882
3883        replay_from: Path | None = None
3884        if replay_enabled:
3885            control_dump = control_result.get("http_dump_file")
3886            if control_dump:
3887                replay_from = Path(control_dump)
3888            else:
3889                print_warning(
3890                    "Control run recorded no HTTP dump; the target will run live"
3891                )
3892
3893        target_sandbox = _restore_sandbox_baseline(sandbox, sandbox_scenario)
3894        target_timeout_s = (
3895            read_budget.target_share_s() if read_timeout_seconds is not None else None
3896        )
3897        target_result, target_outputs = _run_with_optional_http_metrics(
3898            connector_image=resolved_test_image,
3899            command=cmd,
3900            output_dir=target_output,
3901            target_or_control=TargetOrControl.TARGET,
3902            enable_http_metrics=enable_http_metrics,
3903            config_path=config_file,
3904            catalog_path=catalog_file,
3905            state_path=state_file,
3906            enable_debug_logs=enable_debug_logs,
3907            extra_env=parsed_extra_env,
3908            replay_expected=replay_enabled,
3909            replay_from=replay_from,
3910            replay_options=replay_options,
3911            stream_large_bodies=http_replay_max_body,
3912            timeout_seconds=target_timeout_s,
3913            max_records=max_records,
3914            output_mode=output_mode,
3915            mask_key=mask_key,
3916        )
3917        _record_read_share(target_result, target_timeout_s, read_timeout)
3918        target_sandbox = _check_sandbox_twin_gaps(
3919            sandbox, sandbox_scenario, target_sandbox, repro_case
3920        )
3921
3922        # Both dumps have served their purpose by now -- metrics parsed, replay
3923        # corpus built -- so anything too large to be a usable artifact goes,
3924        # rather than being uploaded from a runner with ~14 GB of disk.
3925        _discard_oversized_http_dumps(
3926            control_result,
3927            target_result,
3928            max_bytes=http_replay_max_dump_size_mb * 1024 * 1024,
3929        )
3930
3931        outcome = evaluate_comparison_outcome(command, target_result, control_result)
3932        # A side the harness stopped makes the run incomplete: the comparison
3933        # sees part of what each version would have emitted, so it can still
3934        # find a regression but can never clear one.
3935        incomplete = bool(
3936            stopped_early_sides(("control", control_result), ("target", target_result))
3937        )
3938        # A stream only the target read and read nothing from proves nothing
3939        # about the change: never a pass, whatever the compared streams say.
3940        target_only_streams = _target_only_stream_names(
3941            cmd, catalog_file, control_catalog_file
3942        )
3943        setup_error_streams = undefined_streams(
3944            target_only_streams,
3945            target_outputs.stream_statuses or {},
3946            target_result.get("record_counts_per_stream", {}),
3947            target_outputs.stream_errors or {},
3948        )
3949        config_errors = config_error_streams(
3950            target_only_streams, target_outputs.stream_errors or {}
3951        )
3952        setup_error = bool(setup_error_streams)
3953        could_not_test_result = could_not_test(
3954            target_only_streams, target_outputs.stream_errors or {}
3955        )
3956        setup_error_clears_regression = setup_error and target_errors_only_from(
3957            target_only_streams, target_outputs.stream_errors or {}
3958        )
3959
3960        # A reproduction case replaces the record and final-state comparisons
3961        # for a read: each side is held to the case's expectations instead of
3962        # to the other side.
3963        reproduction: ReproductionVerdict | None = None
3964        if repro_case is not None and sandbox is not None:
3965            reproduction = evaluate_reproduction(
3966                evaluate_side(
3967                    control_result,
3968                    repro_case,
3969                    sandbox_side=control_sandbox,
3970                    final_states=control_outputs.final_states,
3971                ),
3972                evaluate_side(
3973                    target_result,
3974                    repro_case,
3975                    sandbox_side=target_sandbox,
3976                    final_states=target_outputs.final_states,
3977                ),
3978                ReproGoal(repro_goal or ReproGoal.FIXED.value),
3979            )
3980
3981        # Record-level comparison for READ commands (counts, PK presence,
3982        # PK integrity, field values). Only meaningful when both runs
3983        # succeeded; a comparison failure is a regression and fails the
3984        # verdict even though both containers exited cleanly. `outcome`
3985        # itself stays the pure exit-status/connectionStatus layer — the
3986        # folded verdict lives in `success`/`regression_detected` below.
3987        record_comparison: RecordComparisonSummary | None = None
3988        target_record_files: dict[str, Path] | None = None
3989        new_record_checks: tuple[CheckResult, ...] = ()
3990        record_checks_payload: dict[str, dict[str, Any]] = {}
3991        record_checks_regression = False
3992        if (
3993            cmd in (Command.READ, Command.READ_WITH_STATE)
3994            and bool(target_result.get("success"))
3995            and reproduction is None
3996            and not skip_record_comparison
3997            and target_only_streams
3998        ):
3999            try:
4000                target_record_files = split_records_per_stream(
4001                    Path(target_result["stdout_file"])
4002                )
4003            except Exception as exc:
4004                record_checks_regression = False
4005                new_record_checks = (
4006                    CheckResult(
4007                        name="Record schema (new streams)",
4008                        passed=False,
4009                        summary=f"errored ({type(exc).__name__})",
4010                    ),
4011                )
4012        # A skipped comparison must be visible on the report surfaces, not
4013        # only on stdout: without this row the run reads as a pass that
4014        # compared records. Diagnostic, because the skip is not a regression.
4015        record_comparison_skipped: CheckResult | None = None
4016        if (
4017            cmd in (Command.READ, Command.READ_WITH_STATE)
4018            and outcome.both_succeeded
4019            and not skip_record_comparison
4020            and reproduction is None
4021        ):
4022            record_comparison, skip_reason = _compare_read_records(
4023                target_result=target_result,
4024                control_result=control_result,
4025                catalog_file=catalog_file,
4026                target_record_files=target_record_files,
4027            )
4028            if record_comparison is None and skip_reason is not None:
4029                record_comparison_skipped = CheckResult(
4030                    name="Record comparison",
4031                    passed=False,
4032                    severity="diagnostic",
4033                    summary=f"Skipped — {skip_reason}; records were not compared",
4034                )
4035        elif (
4036            skip_record_comparison
4037            and cmd in (Command.READ, Command.READ_WITH_STATE)
4038            and reproduction is None
4039        ):
4040            print_warning(
4041                "Record-level comparison skipped (--skip-record-comparison); "
4042                "the verdict gates on command outcomes only."
4043            )
4044            record_comparison_skipped = CheckResult(
4045                name="Record comparison",
4046                passed=False,
4047                severity="diagnostic",
4048                summary="Skipped (--skip-record-comparison); records were not compared",
4049            )
4050        if target_record_files is not None:
4051            try:
4052                (
4053                    new_record_checks,
4054                    record_checks_payload,
4055                    record_checks_regression,
4056                ) = _new_stream_record_checks(
4057                    stream_names=target_only_streams,
4058                    record_files=target_record_files,
4059                    catalog_file=catalog_file,
4060                    target_outputs=target_outputs,
4061                    record_comparison=record_comparison,
4062                    masked=output_mode == "masked",
4063                    mask_policy=report_mask_policy,
4064                    stopped=bool(stopped_early_sides(("target", target_result))),
4065                )
4066            except Exception as exc:
4067                record_checks_regression = False
4068                new_record_checks = (
4069                    CheckResult(
4070                        name="Record schema (new streams)",
4071                        passed=False,
4072                        summary=f"errored ({type(exc).__name__})",
4073                    ),
4074                )
4075
4076        # Spec/catalog-schema comparison of what the two versions declared
4077        # about themselves; a failed check here is a regression as much as
4078        # a dropped record.
4079        comparison_checks: tuple[CheckResult, ...] = ()
4080        comparison_diff_blocks: tuple[DiffBlock, ...] = ()
4081        deferred_diff_blocks: list[Callable[[], tuple[DiffBlock, ...]]] = []
4082        comparison_detail: dict[str, Any] = {}
4083        if reproduction is None:
4084            (
4085                comparison_checks,
4086                comparison_diff_blocks,
4087                comparison_detail,
4088            ) = _compare_outputs(
4089                command=command,
4090                outcome=outcome,
4091                control_outputs=control_outputs,
4092                target_outputs=target_outputs,
4093                partial=incomplete,
4094                no_control_streams=target_only_streams,
4095                could_not_test=could_not_test_result,
4096                deferred_diff_blocks=deferred_diff_blocks,
4097            )
4098        failed_comparisons = failed_checks(comparison_checks)
4099        extra_checks = (
4100            *new_record_checks,
4101            *comparison_checks,
4102            *([record_comparison_skipped] if record_comparison_skipped else []),
4103        )
4104        report_target_result = {
4105            **target_result,
4106            "setup_error": setup_error,
4107            "setup_error_clears_regression": setup_error_clears_regression,
4108            "record_checks_regression": record_checks_regression,
4109            "could_not_test": could_not_test_result,
4110            "setup_error_detail": describe_setup_error(setup_error_streams),
4111            "could_not_test_detail": describe_could_not_test(
4112                config_errors, target_outputs.stream_errors or {}
4113            ),
4114            "excluded_not_exercised_streams": (
4115                *setup_error_streams,
4116                *config_errors,
4117            ),
4118        }
4119        configured_streams = _configured_stream_names(cmd, catalog_file)
4120        report_model = build_comparison_report_model(
4121            target_image=resolved_test_image,
4122            control_image=resolved_control_image,  # ty: ignore[invalid-argument-type]
4123            command=command,
4124            target_result=report_target_result,
4125            control_result=control_result,
4126            outcome=outcome,
4127            record_comparison=record_comparison,
4128            extra_checks=extra_checks,
4129            configured_streams=configured_streams,
4130            include_report_sections=False,
4131            reproduction=reproduction,
4132            target_only_streams=target_only_streams,
4133            failed_new_streams=tuple(
4134                name
4135                for name, checks in record_checks_payload.items()
4136                if checks.get("check_failed")
4137            ),
4138        )
4139
4140        # Annotate copies of each side, never the originals: `generate_regression_report`
4141        # below reads `result["success"]` as the exit status, so handing it a dict whose
4142        # `success` is the verdict would corrupt the report's per-version wording.
4143        combined_result = {
4144            "target": _annotate_run_verdict(command, target_result),
4145            "control": _annotate_run_verdict(command, control_result),
4146            "both_succeeded": outcome.both_succeeded,
4147            **report_model.verdict_flags(),
4148            "output_mode": output_mode,
4149            "check_results": report_model.check_results(),
4150        }
4151        if record_checks_payload:
4152            combined_result["record_checks"] = record_checks_payload
4153        readable_fields = run_readable_fields(target_result, control_result)
4154        if readable_fields is not None:
4155            combined_result["readable_fields"] = readable_fields
4156        if record_comparison is not None:
4157            combined_result["record_comparison"] = record_comparison.to_dict()
4158        if comparison_detail:
4159            combined_result["comparison"] = comparison_detail
4160        if reproduction is not None:
4161            combined_result["reproduction"] = reproduction.to_dict()
4162
4163        print_json(_printable_payload(combined_result, output_mode))
4164
4165        gh_outputs: dict[str, object] = {
4166            **report_model.verdict_flags(github=True),
4167            "target_image": resolved_test_image,
4168            "control_image": resolved_control_image,
4169            "command": command,
4170            "target_exit_code": target_result["exit_code"],
4171            "control_exit_code": control_result["exit_code"],
4172            "output_mode": output_mode,
4173        }
4174
4175        if command == "check":
4176            gh_outputs["target_connection_status"] = target_result.get(
4177                "connection_status"
4178            )
4179            gh_outputs["control_connection_status"] = control_result.get(
4180                "connection_status"
4181            )
4182
4183        if record_comparison is not None:
4184            # The record comparison's own verdict: on an incomplete run it says
4185            # the strict checks passed on what was read, so it can be true
4186            # while `success` is false.
4187            gh_outputs["record_comparison_passed"] = record_comparison.passed
4188
4189        if reproduction is not None:
4190            gh_outputs["reproduction_verdict"] = reproduction.label
4191
4192        write_github_outputs(gh_outputs)
4193
4194        write_json_output(
4195            "regression_report",
4196            {
4197                **combined_result,
4198                "partial_streams": list(report_model.partial_streams),
4199            },
4200        )
4201
4202        if reproduction is not None and repro_case is not None:
4203            result_path = output_path / CASE_RESULT_FILE
4204            result_path.write_text(
4205                json.dumps(
4206                    case_result(
4207                        reproduction,
4208                        repro_case,
4209                        control_image=resolved_control_image or "",
4210                        target_image=resolved_test_image,
4211                    ),
4212                    indent=2,
4213                )
4214                + "\n"
4215            )
4216            print_success(f"Wrote reproduction result: {result_path}")
4217
4218        connection_objects = ()
4219        with report_section("connection objects"):
4220            connection_objects = _collect_connection_objects(
4221                catalog_file,
4222                state_file,
4223                target_output / "airbyte_messages",
4224                state_seeded=state_seeded,
4225                mask_policy=report_mask_policy,
4226            )
4227        for build_blocks in deferred_diff_blocks:
4228            with report_section("comparison diff blocks"):
4229                comparison_diff_blocks += build_blocks()
4230        report_model = enrich_report_model(
4231            report_model,
4232            target_result=report_target_result,
4233            control_result=control_result,
4234            configured_streams=configured_streams,
4235            record_comparison=record_comparison,
4236            extra_diff_blocks=comparison_diff_blocks,
4237            connection_objects=connection_objects,
4238        )
4239
4240        with report_section("markdown report"):
4241            report_path = generate_regression_report(
4242                report_model=report_model,
4243                target_image=resolved_test_image,
4244                control_image=resolved_control_image,  # ty: ignore[invalid-argument-type]
4245                command=command,
4246                target_result=report_target_result,
4247                control_result=control_result,
4248                output_dir=output_path,
4249                record_comparison=(
4250                    record_comparison.to_dict()
4251                    if record_comparison is not None
4252                    else None
4253                ),
4254                # Without these the markdown report would announce "no regression"
4255                # over the same run the HTML report and the CLI call a failure --
4256                # and, for a warning, would show no finding at all where the other
4257                # two surfaces show one.
4258                comparison_findings=[
4259                    (check.status, f"{check.name}: {check.summary}")
4260                    for check in (*comparison_checks, *new_record_checks)
4261                    if check.status in {"fail", "warn"}
4262                ],
4263                inconclusive_findings=[
4264                    f"{check.name}: {check.summary}"
4265                    for check in new_record_checks
4266                    if check.status == "fail" and not record_checks_regression
4267                ],
4268                reproduction=reproduction,
4269                target_only_streams=target_only_streams,
4270            )
4271            print_success(f"Generated regression report: {report_path}")
4272
4273        with report_section("HTML report"):
4274            html_report_path = write_html_report(report_model, output_path)
4275            print_success(f"Generated regression HTML report: {html_report_path}")
4276
4277        # Write the concise summary to GITHUB_STEP_SUMMARY (if env var exists)
4278        with report_section("GitHub step summary"):
4279            write_github_summary(render_inline_summary(report_model))
4280
4281        if (
4282            report_model.incomplete
4283            or report_model.setup_error
4284            or report_model.could_not_test
4285            or report_model.not_exercised
4286        ):
4287            print_error(f"{report_model.verdict_label} — {report_model.verdict_detail}")
4288        elif reproduction is not None:
4289            report_line = (
4290                f"Reproduction case {repro_case.name if repro_case else ''}: "
4291                f"{reproduction.label} -- {reproduction.detail}"
4292            )
4293            if report_model.passed:
4294                print_success(report_line)
4295            else:
4296                print_error(report_line)
4297                for finding in reproduction.details:
4298                    print_error(f"  - {finding}")
4299        elif failed_comparisons:
4300            # Only reachable when both commands ran cleanly -- a comparison is
4301            # skipped otherwise -- so this says what the exit codes could not.
4302            print_error(
4303                f"Regression detected between {resolved_test_image} and "
4304                f"{resolved_control_image}: "
4305                + "; ".join(
4306                    f"{check.name.lower()} — {check.summary}"
4307                    for check in failed_comparisons
4308                )
4309            )
4310        elif report_model.outcome and report_model.outcome.check_improvement:
4311            print_success(
4312                f"Regression test passed for {resolved_test_image} vs "
4313                f"{resolved_control_image}: target connectionStatus SUCCEEDED "
4314                "while control FAILED (improvement)."
4315            )
4316        elif report_model.comparison_errored:
4317            print_error(
4318                f"Record comparison errored for {resolved_test_image} vs "
4319                f"{resolved_control_image}. Test failed/inconclusive; this "
4320                "is a tooling failure, not evidence of a regression."
4321            )
4322        elif report_model.strict_failed:
4323            print_error(
4324                f"Regression detected between {resolved_test_image} and "
4325                f"{resolved_control_image}: both versions ran, but the "
4326                "record comparison failed."
4327            )
4328        elif report_model.outcome and report_model.outcome.both_succeeded:
4329            print_success(
4330                f"Regression test passed for {resolved_test_image} vs {resolved_control_image}"
4331            )
4332        elif report_model.regression_detected:
4333            print_error(
4334                f"Regression detected between {resolved_test_image} and {resolved_control_image}"
4335            )
4336        elif report_model.both_failed:
4337            print_error(
4338                f"Both versions failed for {resolved_test_image} vs {resolved_control_image}. "
4339                "Test failed/inconclusive; cannot rule out a regression."
4340            )
4341        elif report_model.outcome and report_model.outcome.control_failed_only:
4342            print_error(
4343                f"Control {resolved_control_image} failed while target "
4344                f"{resolved_test_image} succeeded. Test failed/inconclusive; "
4345                "nothing to compare the target against."
4346            )
4347        else:
4348            print_error(
4349                f"Regression test failed for {resolved_test_image} vs "
4350                f"{resolved_control_image}: {report_model.verdict_label} — "
4351                f"{report_model.verdict_detail}"
4352            )
4353
4354
4355def _repro_case_argument_error(
4356    *,
4357    command: Command,
4358    repro_case_dir: str | None,
4359    sandbox_scenario: str | None,
4360    skip_compare: bool,
4361    connection_id: str | None,
4362    output_mode: OutputMode,
4363    config_path: str | None,
4364    catalog_path: str | None,
4365    state_path: str | None,
4366    selected_streams: str | None,
4367) -> str | None:
4368    """Why the reproduction-case options cannot be used as given, if they cannot."""
4369    if not (repro_case_dir and sandbox_scenario):
4370        return (
4371            "--repro-case-dir and --sandbox-scenario must be given together, "
4372            "and --repro-goal needs them"
4373        )
4374    if command not in (Command.READ, Command.READ_WITH_STATE, Command.CHECK):
4375        # Only a read or a CHECK is judged against a case.
4376        return "Reproduction cases run only `--command read` or `--command check`"
4377    if skip_compare:
4378        return "Reproduction cases need comparison mode; drop --skip-compare"
4379    if connection_id:
4380        return "Reproduction cases cannot use --connection-id"
4381    if output_mode == "masked":
4382        # The case is judged on raw values: masked records would read as
4383        # missing, and a masked error as not the expected one.
4384        return (
4385            "Reproduction cases run raw: the sandbox holds no customer data and "
4386            "the case's expectations are raw values; drop --output-mode masked"
4387        )
4388    if not config_path:
4389        return "Reproduction cases need --config-path (from regression-case-setup)"
4390    if catalog_path or state_path:
4391        return "The reproduction case supplies the catalog and state"
4392    if selected_streams:
4393        # The filter rewrites the catalog file in place, i.e. the committed case.
4394        return "The reproduction case's catalog selects its own streams"
4395    return None
4396
4397
4398def _repro_case_command_error(command: Command, case_command: str) -> str | None:
4399    """Why `--command` is not the command the case is judged by, if it is not."""
4400    runs = "check" if command is Command.CHECK else "read"
4401    if runs != case_command:
4402        return f"This case runs only `--command {case_command}`"
4403    return None
4404
4405
4406def _restore_sandbox_baseline(
4407    sandbox: SandboxCli | None, scenario_id: str | None
4408) -> SandboxSide:
4409    """Restore the case's baseline before one side of a case read, else a no-op."""
4410    if sandbox is None or not scenario_id:
4411        return SandboxSide()
4412    side = restore_side(sandbox, scenario_id)
4413    if side.restore_error:
4414        print_error(f"Could not restore the sandbox baseline: {side.restore_error}")
4415    return side
4416
4417
4418def _check_sandbox_twin_gaps(
4419    sandbox: SandboxCli | None,
4420    scenario_id: str | None,
4421    side: SandboxSide,
4422    repro_case: ReproCase | None = None,
4423) -> SandboxSide:
4424    """Gate one side on the twin supporting every request it made."""
4425    if sandbox is None or not scenario_id:
4426        return side
4427    side = check_twin_gaps(
4428        sandbox,
4429        scenario_id,
4430        side,
4431        repro_case.injected_failures if repro_case else (),
4432    )
4433    for gap in side.twin_gaps:
4434        print_error(f"Twin gap: {gap}")
4435    if side.gap_check_error:
4436        print_error(f"Twin-gap check failed: {side.gap_check_error}")
4437    return side
4438
4439
4440@connector_app.command(name="regression-case-setup")
4441def regression_case_setup(
4442    case_dir: Annotated[
4443        str,
4444        Parameter(help="Reproduction case directory to set up."),
4445    ],
4446    config_output: Annotated[
4447        str | None,
4448        Parameter(
4449            help="Where to write the rendered connector config. Defaults to a "
4450            "private temporary directory outside the artifacts tree."
4451        ),
4452    ] = None,
4453) -> None:
4454    """Set up a reproduction case in the SaaS Sandbox.
4455
4456    Applies the case's sandbox/dataset.json together with its optional
4457    sandbox/behavior.json (failure rules), snapshots the result as the
4458    `baseline` that `regression-test --repro-case-dir` restores before each side
4459    of a read, and renders config.template.json with the scenario's
4460    synthetic credential. The sandbox connection comes from `SAAS_API_URL`,
4461    `SAAS_API_KEY`, `SAAS_WORKSPACE_ID` and `XDG_STATE_HOME`.
4462
4463    Writes `scenario_id`, `config_path` and the resolved `case_dir` to the
4464    GitHub step outputs.
4465    """
4466    try:
4467        case = ReproCase.load(Path(case_dir))
4468        sandbox = SandboxCli()
4469        name = f"{case.name}-{os.getenv('GITHUB_RUN_ID') or uuid.uuid4().hex[:8]}"
4470        destination = (
4471            Path(config_output)
4472            if config_output
4473            else Path(tempfile.mkdtemp(prefix=f"repro-case-{case.name}-"))
4474            / "config.json"
4475        )
4476        scenario_id, config_file, credential = set_up_scenario(
4477            case, sandbox, scenario_name=name, config_destination=destination
4478        )
4479    except (OSError, ValueError, SandboxError) as exc:
4480        exit_with_error(f"Could not set up reproduction case {case_dir}: {exc}")
4481
4482    if is_github_actions():
4483        # The workflow log must never show the credential, even a synthetic one.
4484        print(f"::add-mask::{credential}")
4485    outputs = {
4486        "scenario_id": scenario_id,
4487        "config_path": str(config_file),
4488        "case_dir": str(case.directory),
4489        # From a fixed set, so case content cannot inject an output line.
4490        "command": case.expectations.command,
4491    }
4492    write_github_outputs(outputs)
4493    print_json(outputs)
4494
4495
4496@connector_app.command(name="sandbox-support")
4497def sandbox_support(
4498    connector_name: Annotated[
4499        str,
4500        Parameter(help="Connector to check (e.g. 'source-slack')."),
4501    ],
4502    streams: Annotated[
4503        str | None,
4504        Parameter(
4505            help="Comma-separated streams the fix touches; their parent streams "
4506            "are included. Defaults to every stream in the manifest."
4507        ),
4508    ] = None,
4509    dataset_needs: Annotated[
4510        str | None,
4511        Parameter(
4512            help="Comma-separated `entity` or `entity.field` the reproduction "
4513            "needs in the twin's dataset (e.g. 'messages.thread_ts')."
4514        ),
4515    ] = None,
4516    connector_dir: Annotated[
4517        str | None,
4518        Parameter(
4519            help="Local connector directory. Without it, the files are read from "
4520            "airbytehq/airbyte at --ref."
4521        ),
4522    ] = None,
4523    ref: Annotated[
4524        str,
4525        Parameter(help="airbytehq/airbyte ref to read the connector from."),
4526    ] = "master",
4527    host_only: Annotated[
4528        bool,
4529        Parameter(
4530            help="Only decide whether a twin serves the connector's provider: "
4531            "one serving an allowed host and, when the manifest's request hosts "
4532            "can be read, one of those (token endpoints don't count). No "
4533            "operation is checked, so `operations` is empty. Cannot be combined "
4534            "with --streams or --dataset-needs."
4535        ),
4536    ] = False,
4537) -> None:
4538    """Check statically whether a SaaS Sandbox twin covers a connector's needs.
4539
4540    The first gate before a reproduction case: provider host match, the HTTP
4541    operations, query parameters and page sizes the streams use, and optional
4542    dataset needs, against the sandbox's offline discovery data (`sandbox ...
4543    --offline`; needs the `sandbox` CLI, but no server). Prints a JSON report
4544    whose `result` is `supported`, `unsupported` or `unknown`. `unknown` means
4545    the requests could not be read statically; the case lane's twin-gap gate
4546    still decides during the run. With --host-only, only the provider match is
4547    decided (the oncall shadow gate's "has a twin" question).
4548    """
4549    # Checked on presence, so `--streams ""` is refused too.
4550    if host_only and (streams is not None or dataset_needs is not None):
4551        exit_with_error(
4552            "--host-only checks no stream or dataset need; drop --streams and "
4553            "--dataset-needs"
4554        )
4555    print_json(
4556        run_sandbox_support_check(
4557            connector_name=connector_name,
4558            streams=_split_csv(streams),
4559            dataset_needs=_split_csv(dataset_needs),
4560            connector_dir=Path(connector_dir) if connector_dir else None,
4561            ref=ref,
4562            host_only=host_only,
4563        )
4564    )
4565
4566
4567@connector_app.command(name="regression-case-gate")
4568def regression_case_gate(
4569    base_ref: Annotated[
4570        str | None,
4571        Parameter(
4572            help="The branch the PR merges into, e.g. `origin/main`. Required "
4573            "unless --case-dir is given."
4574        ),
4575    ] = None,
4576    head_ref: Annotated[
4577        str,
4578        Parameter(help="The PR's head commit; the case is read at this commit."),
4579    ] = "HEAD",
4580    repo_dir: Annotated[
4581        str,
4582        Parameter(
4583            help="airbyte-ops-mcp clone holding both --base-ref and --head-ref. "
4584            "Its working tree is not read."
4585        ),
4586    ] = ".",
4587    case_dir: Annotated[
4588        str | None,
4589        Parameter(
4590            help="Check this case directory on disk instead of a PR: no scope "
4591            "check, and the connector is the name of its parent directory."
4592        ),
4593    ] = None,
4594    require_case: Annotated[
4595        bool,
4596        Parameter(help="Fail, instead of answering `n/a`, when the PR adds no case."),
4597    ] = False,
4598    connector_dir: Annotated[
4599        str | None,
4600        Parameter(
4601            help="Local connector directory at the case's `affected_ref`. Without "
4602            "it, the files are read from airbytehq/airbyte at `affected_ref`."
4603        ),
4604    ] = None,
4605    json_out: Annotated[
4606        str | None,
4607        Parameter(help="Also write the JSON report to this file."),
4608    ] = None,
4609    sandbox_ref: Annotated[
4610        str | None,
4611        Parameter(
4612            help="The saas-sandbox commit the case runs use. Defaults to the pin "
4613            "in `.github/saas-sandbox-ref` of the current directory (the code "
4614            "checkout); the case's `origin.json` `sandbox_ref` must equal it."
4615        ),
4616    ] = None,
4617) -> None:
4618    """Check whether a reproduction-case PR may merge without human review.
4619
4620    Prints a JSON report whose `status` is `n/a` (the PR touches no case),
4621    `pass` or `fail`, and exits nonzero on `fail`. Checks that the PR only adds
4622    one new case under `regression_cases/<connector>/<case>/`, as regular files
4623    with known names; that the case loads with an `origin.json` naming a
4624    published airbytehq/airbyte release whose source is at `affected_ref`, a
4625    commit on `master`; that the sandbox validates `sandbox/` offline; and that
4626    the static support check says `supported`. The reproduction itself is a
4627    case run with goal `reproduced`. Writes `connector`, `case`,
4628    `affected_image`, `affected_ref`, `problems` (a JSON list) and `status` to
4629    the GitHub step outputs.
4630    """
4631    if sandbox_ref is None:
4632        try:
4633            sandbox_ref = read_sandbox_pin(Path(SANDBOX_PIN_FILE))
4634        except ValueError as exc:
4635            exit_with_error(f"{exc}; run from the code checkout or pass --sandbox-ref")
4636    elif not SHA_PATTERN.fullmatch(sandbox_ref):
4637        exit_with_error("--sandbox-ref must be a full 40-character commit SHA")
4638    token = resolve_default_github_token(allow_none=True)
4639    connector_path = Path(connector_dir) if connector_dir else None
4640    if case_dir is not None:
4641        report = run_case_dir_gate(
4642            Path(case_dir),
4643            sandbox_ref=sandbox_ref,
4644            github_token=token,
4645            connector_dir=connector_path,
4646        )
4647    elif base_ref is None:
4648        exit_with_error("Pass --base-ref, or --case-dir to check a case directory.")
4649    else:
4650        try:
4651            report = run_case_gate(
4652                Path(repo_dir),
4653                base_ref,
4654                head_ref,
4655                sandbox_ref=sandbox_ref,
4656                require_case=require_case,
4657                github_token=token,
4658                connector_dir=connector_path,
4659            )
4660        except subprocess.CalledProcessError as exc:
4661            exit_with_error(
4662                f"Could not compare {base_ref}...{head_ref}: {exc.stderr.strip()}"
4663            )
4664    if json_out:
4665        Path(json_out).write_text(json.dumps(report.to_dict(), indent=2))
4666    write_github_outputs(gate_step_outputs(report))
4667    print_json(report.to_dict())
4668    if report.status == "fail":
4669        exit_with_error("Case gate failed: " + "; ".join(report.problems))
4670
4671
4672@connector_app.command(name="regression-case-validate-decide")
4673def regression_case_validate_decide(
4674    pr: Annotated[int, Parameter(help="The case PR's number.")],
4675    needs: Annotated[
4676        str,
4677        Parameter(help="File holding the workflow's `toJSON(needs)`."),
4678    ],
4679    gate_report: Annotated[
4680        str,
4681        Parameter(help="The gate's `--json-out` report; may be missing."),
4682    ],
4683    merge_enabled: Annotated[
4684        bool,
4685        Parameter(help="The merge switch (`SANDBOX_CASE_AUTOMERGE_ENABLED`)."),
4686    ] = False,
4687    run_url: Annotated[str, Parameter(help="This run's URL, for the comment.")] = "",
4688    repo: Annotated[
4689        str | None,
4690        Parameter(help="`owner/name` of the repository; defaults to `GH_REPO`."),
4691    ] = None,
4692    comment_out: Annotated[
4693        str,
4694        Parameter(help="Where to write the failure comment, when there is one."),
4695    ] = "comment.md",
4696) -> None:
4697    """Decide what `sandbox-case-validate.yml` does after the gate and the runs.
4698
4699    Re-reads the PR through `gh` (set `GH_REPO` and `GH_TOKEN`) and prints a
4700    JSON decision: `action` (`fail`, `report` or `merge`), the commit status
4701    `state` and `description`, and on `fail` the PR comment, also written to
4702    --comment-out. Writes `action`, `state`, `description` and `validated_sha`
4703    to the GitHub step outputs.
4704    """
4705    needs_data = json.loads(Path(needs).read_text())
4706    gate_outputs = (needs_data.get("gate") or {}).get("outputs") or {}
4707    validated_sha = str(gate_outputs.get("head_sha") or "")
4708    # It is written back to the step outputs, so it must be a plain SHA.
4709    if not SHA_PATTERN.fullmatch(validated_sha):
4710        validated_sha = ""
4711    report_path = Path(gate_report)
4712    gate = json.loads(report_path.read_text()) if report_path.is_file() else None
4713    try:
4714        pr_now = gh_read_pr(repo or os.environ.get("GH_REPO", ""), pr)
4715    except (RuntimeError, ValueError, subprocess.SubprocessError):
4716        pr_now = None
4717    decision = decide_validation(
4718        gate=gate,
4719        gate_refusal=gate_outputs.get("refusal") or None,
4720        runs=runs_from_needs(needs_data),
4721        pr=pr_now,
4722        validated_sha=validated_sha,
4723        merge_enabled=merge_enabled,
4724        run_url=run_url,
4725    )
4726    if decision.comment is not None:
4727        Path(comment_out).write_text(decision.comment)
4728    write_github_outputs(
4729        {
4730            "action": decision.action,
4731            "state": decision.state,
4732            "description": decision.description,
4733            "validated_sha": validated_sha,
4734        }
4735    )
4736    print_json(decision.to_dict())
4737
4738
4739@connector_app.command(name="regression-case-merge")
4740def regression_case_merge(
4741    pr: Annotated[int, Parameter(help="The case PR's number.")],
4742    sha: Annotated[str, Parameter(help="The validated head commit.")],
4743    connector: Annotated[str, Parameter(help="The case's connector, for the title.")],
4744    case: Annotated[str, Parameter(help="The case's name, for the title.")],
4745    repo: Annotated[
4746        str | None,
4747        Parameter(help="`owner/name` of the repository; defaults to `GH_REPO`."),
4748    ] = None,
4749    run_url: Annotated[
4750        str, Parameter(help="The validating run, for the commit message.")
4751    ] = "",
4752    timeout_minutes: Annotated[
4753        float,
4754        Parameter(help="How long to wait for GitHub to report the PR mergeable."),
4755    ] = 10,
4756) -> None:
4757    """Merge a validated case PR at its validated head, waiting a bounded time.
4758
4759    Polls the PR through `gh` (set `GH_TOKEN` to a token that may merge) until
4760    GitHub reports it mergeable, then squash-merges it through the REST merge
4761    endpoint pinned to `sha`, with a fixed commit title. Never enables
4762    auto-merge. Prints the outcome and writes `outcome`, `merge_state`, `state`
4763    and `description` (the final commit status) to the GitHub step outputs.
4764    Exits nonzero unless the PR merged at `sha` or simply did not become
4765    mergeable in time.
4766    """
4767    repo = repo or os.environ.get("GH_REPO")
4768    if not repo:
4769        exit_with_error("Pass --repo or set GH_REPO.")
4770    if not (
4771        SHA_PATTERN.fullmatch(sha)
4772        and NAME_PATTERN.fullmatch(connector)
4773        and NAME_PATTERN.fullmatch(case)
4774    ):
4775        exit_with_error("--sha must be a full SHA; --connector and --case plain names.")
4776    title = f"test(regression-cases): add {connector} {case} reproduction case (#{pr})"
4777    message = (
4778        f"Validated by {run_url}" if run_url else "Validated by sandbox-case-validate"
4779    )
4780    outcome = wait_and_merge(
4781        sha,
4782        timeout_seconds=timeout_minutes * 60,
4783        read_pr=lambda: gh_read_pr(repo, pr),
4784        merge=lambda: gh_merge(repo, pr, sha, title, message),
4785    )
4786    result = outcome.to_dict()
4787    write_github_outputs(
4788        {
4789            "outcome": outcome.result,
4790            "merge_state": outcome.merge_state or "",
4791            "state": result["state"],
4792            "description": result["description"],
4793        }
4794    )
4795    print_json(result)
4796    if not outcome.succeeded:
4797        exit_with_error(f"Case PR #{pr} was not merged: {outcome.result}")
4798
4799
4800def _split_csv(value: str | None) -> list[str]:
4801    return [item.strip() for item in (value or "").split(",") if item.strip()]
4802
4803
4804def _stop_with_setup_error(message: str) -> NoReturn:
4805    """Report a failed setup verdict but exit 0 to avoid infrastructure failures."""
4806    write_github_output("success", False)
4807    write_github_output("setup_error", True)
4808    write_github_output("error", message)
4809    write_github_summary(
4810        f"## `READ` Test Results\n\n**Result:** SETUP ERROR — {message}"
4811    )
4812    print_error(message)
4813    raise SystemExit(0)
4814
4815
4816def _injection_discover_timeout_message(
4817    image: str, timeout_s: int, read_budget: ReadBudget
4818) -> str:
4819    if timeout_s <= 0:
4820        return (
4821            f"Setup error: DISCOVER of {image} for --discover-missing-streams had "
4822            "no read time left to run in "
4823            f"(--read-timeout {read_budget.budget_s} s). Raise read_timeout_minutes, "
4824            "or check why DISCOVER hangs."
4825        )
4826    return (
4827        f"Setup error: DISCOVER of {image} for --discover-missing-streams did not "
4828        f"finish within the {timeout_s} s left of the read budget "
4829        f"(--read-timeout {read_budget.budget_s} s). Raise read_timeout_minutes, "
4830        "or check why DISCOVER hangs."
4831    )
4832
4833
4834def _discovered_catalog(
4835    connector_image: str,
4836    target_or_control: TargetOrControl,
4837    output_dir: Path,
4838    config_file: Path | None,
4839    enable_debug_logs: bool,
4840    extra_env: dict[str, str] | None,
4841    output_mode: OutputMode,
4842    mask_key: bytes | None,
4843    timeout_seconds: int,
4844) -> tuple[dict[str, Any] | None, dict[str, Any] | None]:
4845    """Run a budgeted DISCOVER and return its catalog and stopped-early result."""
4846    result, outputs = _run_connector_command(
4847        connector_image=connector_image,
4848        command=Command.DISCOVER,
4849        output_dir=output_dir,
4850        target_or_control=target_or_control,
4851        timeout_seconds=timeout_seconds,
4852        config_path=config_file,
4853        enable_debug_logs=enable_debug_logs,
4854        output_mode=output_mode,
4855        mask_key=mask_key,
4856        extra_env=extra_env,
4857    )
4858    if outputs.catalog is None:
4859        return None, result.get("stopped_early")
4860    return (
4861        outputs.catalog.model_dump(mode="json", exclude_none=True),
4862        result.get("stopped_early"),
4863    )
4864
4865
4866def _inject_discovered_streams(
4867    *,
4868    catalog_file: Path | None,
4869    selected_stream_names: set[str],
4870    test_image: str,
4871    reference_image: str | None,
4872    skip_compare: bool,
4873    config_file: Path | None,
4874    output_path: Path,
4875    enable_debug_logs: bool,
4876    extra_env: dict[str, str] | None,
4877    sync_mode: InjectedSyncMode | None,
4878    output_mode: OutputMode,
4879    mask_key: bytes | None,
4880    read_budget: ReadBudget,
4881) -> tuple[Path, Path, InjectedCatalogs] | None:
4882    """Add the selected streams the configured catalog lacks, from DISCOVER.
4883
4884    The target's DISCOVER configures the streams. The reference version's
4885    DISCOVER -- the control image, or in single-version mode the released
4886    image -- says whether each one is new: a stream it also discovers is only
4887    unselected on the connection, so both versions read it using their own
4888    discovered entries. Without the reference catalog the novelty is unknown
4889    and the stream is read by the target only, as if new, and reported as
4890    unknown. Each DISCOVER gets the read time left when it starts.
4891
4892    A selected stream already in the catalog that the reference version does
4893    not discover is kept from the control, as a new stream would be. That is
4894    what a run without a connection needs: its catalog comes from the target's
4895    DISCOVER, so the PR's new streams are in it already.
4896
4897    Returns:
4898        The target catalog path, the control catalog path and the
4899        classification, or `None` when neither catalog would change.
4900    """
4901    configured = (
4902        json.loads(catalog_file.read_text())
4903        if catalog_file is not None and catalog_file.exists()
4904        else None
4905    )
4906    configured_names = {
4907        stream.get("stream", stream).get("name", "")
4908        for stream in (configured or {}).get("streams", [])
4909    }
4910    missing = selected_stream_names - configured_names
4911
4912    discover_dir = output_path / "discover_for_injection"
4913    target_timeout_s = read_budget.setup_share_s()
4914    if target_timeout_s <= 0:
4915        _stop_with_setup_error(
4916            _injection_discover_timeout_message(
4917                test_image, target_timeout_s, read_budget
4918            )
4919        )
4920    target_catalog, target_stopped_early = _discovered_catalog(
4921        test_image,
4922        TargetOrControl.TARGET,
4923        discover_dir / "target",
4924        config_file,
4925        enable_debug_logs,
4926        extra_env,
4927        output_mode,
4928        mask_key,
4929        target_timeout_s,
4930    )
4931    if target_stopped_early:
4932        _stop_with_setup_error(
4933            _injection_discover_timeout_message(
4934                test_image, target_timeout_s, read_budget
4935            )
4936        )
4937    if target_catalog is None and not missing:
4938        print_warning(
4939            f"{test_image} returned no catalog from DISCOVER; the selected streams "
4940            "are read as configured"
4941        )
4942        return None
4943    if target_catalog is None:
4944        error_msg = (
4945            f"Setup error: {test_image} returned no catalog from DISCOVER, so the selected "
4946            "streams missing from the configured catalog cannot be added."
4947        )
4948        _stop_with_setup_error(error_msg)
4949
4950    reference_catalog: dict[str, Any] | None = None
4951    if reference_image and reference_image != test_image:
4952        if skip_compare and not ensure_image_available(reference_image):
4953            print_warning(
4954                f"Could not pull {reference_image}; whether the added streams are "
4955                "new is unknown, so they are read by the target only"
4956            )
4957        else:
4958            reference_timeout_s = read_budget.setup_share_s()
4959            if reference_timeout_s <= 0:
4960                _stop_with_setup_error(
4961                    _injection_discover_timeout_message(
4962                        reference_image, reference_timeout_s, read_budget
4963                    )
4964                )
4965            reference_catalog, reference_stopped_early = _discovered_catalog(
4966                reference_image,
4967                TargetOrControl.CONTROL,
4968                discover_dir / "control",
4969                config_file,
4970                enable_debug_logs,
4971                extra_env,
4972                output_mode,
4973                mask_key,
4974                reference_timeout_s,
4975            )
4976            if reference_stopped_early:
4977                _stop_with_setup_error(
4978                    _injection_discover_timeout_message(
4979                        reference_image, reference_timeout_s, read_budget
4980                    )
4981                )
4982            if reference_catalog is None:
4983                print_warning(
4984                    f"{reference_image} returned no catalog from DISCOVER; whether "
4985                    "the added streams are new is unknown, so they are read by the "
4986                    "target only"
4987                )
4988    else:
4989        print_warning(
4990            "No control version to compare the discovered streams with; whether "
4991            "the added streams are new is unknown, so they are read by the target "
4992            "only"
4993        )
4994
4995    try:
4996        injection = build_injected_catalogs(
4997            configured,
4998            selected_stream_names,
4999            target_catalog,
5000            reference_catalog,
5001            sync_mode=sync_mode,
5002        )
5003    except StreamsNotDiscoveredError as exc:
5004        _stop_with_setup_error(f"Setup error: {exc}")
5005    if not injection.changes_anything:
5006        return None
5007
5008    if not skip_compare and not injection.control_catalog["streams"]:
5009        error_msg = (
5010            "Every selected stream has no control side: "
5011            f"{', '.join(injection.target_only_streams)}. The control version would "
5012            "read nothing, so add one stream the connection already syncs to "
5013            "--selected-streams, or use --skip-compare for a single-version run."
5014        )
5015        _stop_with_setup_error(f"Setup error: {error_msg}")
5016
5017    catalog_dir = output_path / "connection_data"
5018    catalog_dir.mkdir(parents=True, exist_ok=True)
5019    target_catalog_file = (
5020        catalog_file if catalog_file is not None else catalog_dir / "catalog.json"
5021    )
5022    control_catalog_file = target_catalog_file.with_name("control_catalog.json")
5023    target_catalog_file.write_text(json.dumps(injection.target_catalog, indent=2))
5024    control_catalog_file.write_text(json.dumps(injection.control_catalog, indent=2))
5025    # The image novelty was decided against: the control, a pinned connection
5026    # version or metadata.yaml, so a reader never has to guess which.
5027    summary = {
5028        **injection.to_dict(),
5029        "reference_image": reference_image if reference_catalog is not None else None,
5030    }
5031    (output_path / "stream_injection.json").write_text(json.dumps(summary, indent=2))
5032    write_github_output("stream_injection", json.dumps(summary))
5033
5034    for label, names in (
5035        (f"new, not in {reference_image}, target only", injection.new_streams),
5036        (f"already in {reference_image}, read by both", injection.shared_streams),
5037        ("novelty unknown, target only", injection.unknown_streams),
5038    ):
5039        if names:
5040            print_success(f"Added stream(s) ({label}): {', '.join(names)}")
5041    if injection.kept_target_only:
5042        print_success(
5043            f"Kept from the control (not in {reference_image}, target only): "
5044            f"{', '.join(injection.kept_target_only)}"
5045        )
5046    for note in injection.notes:
5047        print_warning(note)
5048    return target_catalog_file, control_catalog_file, injection
5049
5050
5051def _configured_stream_names(
5052    command: Command, catalog_file: Path | None
5053) -> tuple[str, ...]:
5054    """Every stream the run was configured to read.
5055
5056    The report's per-stream tables are keyed on emitted records, so a stream
5057    that returned nothing has no entry and would be missing from the report
5058    entirely. Reading the configured catalog is what lets it be shown as zero.
5059
5060    Only for commands that read records. `--connection-id` fetches a catalog for
5061    every command, and handing it to `spec`, `check` or `discover` would give
5062    them an all-zero stream table and a coverage warning saying no stream
5063    returned anything -- true, meaningless, and enough noise to train a reviewer
5064    to ignore the warning on the one command where it means something.
5065    """
5066    if command not in (Command.READ, Command.READ_WITH_STATE):
5067        return ()
5068
5069    if catalog_file is None or not catalog_file.exists():
5070        return ()
5071
5072    try:
5073        catalog = ConfiguredAirbyteCatalog.parse_raw(catalog_file.read_text())  # ty: ignore[deprecated]
5074    except Exception as exc:
5075        print_warning(f"Could not read the configured catalog for the report: {exc}")
5076        return ()
5077
5078    return tuple(
5079        sorted(
5080            configured.stream.name
5081            for configured in catalog.streams
5082            if configured.stream
5083        )
5084    )
5085
5086
5087def _configured_streams_by_id(catalog: Any) -> dict[tuple[str | None, str], dict]:
5088    """A configured catalog's stream entries, keyed by `(namespace, name)`.
5089
5090    Keyed as `to_stream_id` keys a stream, so `users` in two namespaces is two
5091    entries. An entry without a stream name is skipped.
5092    """
5093    streams = catalog.get("streams", []) if isinstance(catalog, dict) else []
5094    return {
5095        to_stream_id(
5096            stream["stream"].get("namespace"), stream["stream"]["name"]
5097        ): stream
5098        for stream in streams
5099        if isinstance(stream, dict)
5100        and isinstance(stream.get("stream"), dict)
5101        and isinstance(stream["stream"].get("name"), str)
5102    }
5103
5104
5105def _target_only_stream_names(
5106    command: Command, target_catalog: Path | None, control_catalog: Path | None
5107) -> tuple[str, ...]:
5108    """The streams configured for the target and not for the control.
5109
5110    One of these that returns no records makes a comparison read NOT
5111    EXERCISED (see `not_exercised_streams`). Both sides read the same catalog
5112    today, so this is empty and the rule is dormant; it fires once a run
5113    configures a catalog per side, with the streams a connector PR adds on
5114    the target's only.
5115    """
5116    if control_catalog == target_catalog:
5117        return ()
5118
5119    control = set(_configured_stream_names(command, control_catalog))
5120
5121    return tuple(
5122        stream
5123        for stream in _configured_stream_names(command, target_catalog)
5124        if stream not in control
5125    )
5126
5127
5128def _judge_seed_file(
5129    state_file: Path, catalog_file: Path | None
5130) -> tuple[str | None, list[str]]:
5131    """Why `--state-path` cannot seed this read, and the seeded streams it will not reach.
5132
5133    A connector ignores the state of a stream it syncs as full_refresh, so a
5134    seed is refused when none of the streams it names is one the configured
5135    catalog syncs incrementally -- the catalog generated from DISCOVER when
5136    there is no connection puts every stream on full_refresh, and
5137    `--selected-streams` can drop a seeded stream -- since the read would be
5138    cold while the log and the report called it seeded. A seeded stream the
5139    catalog reads in full is returned for a warning; the rest still seed.
5140    Streams match on namespace and name, as the connector matches a state to
5141    its stream: a seed for `public.users` is not taken by an incremental
5142    `reporting.users`. A seed in another shape (a GLOBAL state, say) names no
5143    stream the harness can match, and is held only to the coarse rule that
5144    some stream syncs incrementally. Without a catalog there is nothing to
5145    judge against, and the read fails on its own.
5146
5147    Returns:
5148        The error, or `None`; and the seeded streams the catalog reads in full,
5149        as `namespace.name` labels.
5150    """
5151    try:
5152        text = state_file.read_text()
5153        state = json.loads(text)
5154    except (OSError, ValueError) as exc:
5155        return f"--state-path cannot be read as JSON: {exc}", []
5156    if state == []:
5157        return (
5158            "--state-path holds an empty list, which seeds nothing; drop it for "
5159            "a cold read",
5160            [],
5161        )
5162    if catalog_file is None:
5163        return None, []
5164    try:
5165        catalog = json.loads(catalog_file.read_text())
5166    except (OSError, ValueError) as exc:
5167        return f"--catalog-path cannot be read as JSON: {exc}", []
5168    incremental = {
5169        stream_id
5170        for stream_id, stream in _configured_streams_by_id(catalog).items()
5171        if stream.get("sync_mode") == "incremental"
5172    }
5173    fix = (
5174        "A catalog generated from DISCOVER puts every stream on full_refresh; "
5175        "give --connection-id so the catalog is the connection's own, or a "
5176        "--catalog-path that syncs the seeded streams incrementally"
5177    )
5178    if seed_state_error(text) is not None:
5179        if incremental:
5180            return None, []
5181        return (
5182            "--state-path cannot seed this read: no stream of the configured "
5183            "catalog syncs incrementally, so the connector would ignore the "
5184            f"state and the run would still report it as seeded. {fix}",
5185            [],
5186        )
5187    seeded = [to_stream_id(*stream) for stream in seeded_streams(text)]
5188    if not set(seeded) & incremental:
5189        named = ", ".join(stream_label(*stream) for stream in seeded)
5190        synced = (
5191            ", ".join(sorted(stream_label(*stream) for stream in incremental))
5192            or "no stream"
5193        )
5194        return (
5195            f"--state-path cannot seed this read: it names {named}, and the "
5196            f"configured catalog syncs {synced} incrementally, so the connector "
5197            "would ignore the state and the run would still report it as "
5198            f"seeded. {fix}",
5199            [],
5200        )
5201    return None, [
5202        stream_label(*stream) for stream in seeded if stream not in incremental
5203    ]
5204
5205
5206def _seed_outline(seed_text: str, catalog_file: Path | None) -> str:
5207    """What the report shows of a seeded state: the streams it names, no value.
5208
5209    A seed is customer state, and the report is an uploaded artifact, so the
5210    `State (seeded)` block lists each seeded stream's namespace and name with
5211    the sync mode and cursor field the configured catalog gives it; a stream
5212    the catalog does not configure has neither. A seed that is not a list of
5213    STREAM messages names no stream to list, and gives an empty outline, so
5214    the block is left off.
5215    """
5216    if seed_state_error(seed_text) is not None:
5217        return ""
5218    try:
5219        catalog = json.loads(catalog_file.read_text()) if catalog_file else None
5220    except (OSError, ValueError):
5221        catalog = None
5222    configured = _configured_streams_by_id(catalog)
5223    outline = []
5224    for namespace, name in seeded_streams(seed_text):
5225        stream = configured.get(to_stream_id(namespace, name), {})
5226        cursor_field = stream.get("cursor_field") or stream.get("stream", {}).get(
5227            "default_cursor_field"
5228        )
5229        outline.append(
5230            {
5231                "namespace": namespace,
5232                "name": name,
5233                "sync_mode": stream.get("sync_mode"),
5234                "cursor_field": cursor_field or [],
5235            }
5236        )
5237    return json.dumps(outline)
5238
5239
5240def _collect_connection_objects(
5241    catalog_file: Path | None,
5242    state_file: Path | None,
5243    messages_dir: Path | None,
5244    *,
5245    state_seeded: bool = False,
5246    mask_policy: MaskPolicy | None = None,
5247) -> tuple[DiffBlock, ...]:
5248    """Gather the objects the run was given or discovered, for the report.
5249
5250    The connector config is deliberately absent: it is the one object that
5251    carries secrets, and the report is an artifact anyone with repo access can
5252    download. A seeded state is customer state the caller brought, so its
5253    values are left out too: only the streams it names are shown (see
5254    `_seed_outline`).
5255
5256    Args:
5257        catalog_file: The configured catalog handed to the run.
5258        state_file: The state handed to the run, for a warm read.
5259        messages_dir: The target side's `airbyte_messages` directory, which
5260            holds the catalog the connector discovered.
5261        state_seeded: Whether the caller chose the state (`--state-path`)
5262            rather than the run fetching the connection's own. The label says
5263            so, since a reader otherwise assumes the connection's cursors.
5264        mask_policy: The masked run's policy, under which the connection's
5265            own state is masked before it goes on the page (a seeded state
5266            shows its outline only, in either mode); the file itself is the
5267            raw one the connector read. A state that does not parse is left
5268            off the page rather than embedded raw. `None` in a raw run.
5269
5270    Returns:
5271        One JSON block per object that exists, in the order a reader wants them.
5272    """
5273    sources: list[tuple[str, Path | None, bool]] = [
5274        ("Configured catalog (input)", catalog_file, False),
5275        ("State (seeded)" if state_seeded else "State (input)", state_file, True),
5276        (
5277            "Discovered catalog (from the target)",
5278            (messages_dir / "catalog.jsonl") if messages_dir else None,
5279            False,
5280        ),
5281    ]
5282
5283    blocks: list[DiffBlock] = []
5284    for title, path, is_state in sources:
5285        if path is None or not path.is_file():
5286            continue
5287
5288        try:
5289            payload = path.read_text()
5290        except OSError:
5291            continue
5292
5293        if is_state and state_seeded:
5294            payload = _seed_outline(payload, catalog_file)
5295        elif is_state and mask_policy is not None:
5296            try:
5297                state = json.loads(payload)
5298            except ValueError:
5299                continue
5300            payload = json.dumps(mask_input_state(state, mask_policy), indent=2)
5301
5302        if payload.strip():
5303            blocks.append(build_json_block(title, payload))
5304
5305    return tuple(blocks)
5306
5307
5308# The order commands are presented in, matching how a sync runs them. Anything
5309# else found under the artifacts directory follows, sorted.
5310_REPORT_COMMAND_ORDER = ("spec", "check", "discover", "read")
5311
5312
5313@connector_app.command(name="consolidate-regression-reports")
5314def consolidate_regression_reports(
5315    artifacts_dir: Annotated[
5316        str,
5317        Parameter(
5318            help="Directory holding one subdirectory per command, each with the "
5319            "`report.html` that `regression-test --output-dir` wrote."
5320        ),
5321    ],
5322    output_path: Annotated[
5323        str | None,
5324        Parameter(
5325            name=["--output"],
5326            help="Where to write the consolidated report. "
5327            "Defaults to `report.html` in `--artifacts-dir`.",
5328        ),
5329    ] = None,
5330) -> None:
5331    """Fold one run's per-command regression reports into a single page.
5332
5333    The regression test runs the CLI once per Airbyte command, so each command
5334    writes its own report into its own artifact. This produces the run-level
5335    view -- a verdict table across commands plus every report inlined -- so a
5336    reviewer downloads one file instead of four.
5337
5338    Never fails the run: a command whose report is missing or unreadable is
5339    skipped, because this is a reporting convenience and the per-command
5340    artifacts remain the source of truth.
5341    """
5342    artifacts_path = Path(artifacts_dir)
5343    ordered = [artifacts_path / command for command in _REPORT_COMMAND_ORDER]
5344    extra = sorted(
5345        path
5346        for path in artifacts_path.glob("*")
5347        if path.is_dir() and path not in ordered
5348    )
5349    report_paths = [
5350        directory / HTML_REPORT_FILENAME for directory in [*ordered, *extra]
5351    ]
5352
5353    consolidated = consolidate_reports(report_paths)
5354    if consolidated is None:
5355        print_warning(
5356            f"No per-command reports found under {artifacts_path}; "
5357            "nothing to consolidate."
5358        )
5359        return
5360
5361    destination = Path(output_path) if output_path else artifacts_path / "report.html"
5362    destination.parent.mkdir(parents=True, exist_ok=True)
5363    destination.write_text(consolidated)
5364    print_success(f"Consolidated regression report: {destination}")
5365
5366
5367@connector_app.command(name="fetch-connection-config")
5368def fetch_connection_config_cmd(
5369    connection_id: Annotated[
5370        str,
5371        Parameter(help="The UUID of the Airbyte Cloud connection."),
5372    ],
5373    output_path: Annotated[
5374        str | None,
5375        Parameter(
5376            help="Path to output file or directory. "
5377            "If directory, writes connection-<id>-config.json inside it. "
5378            "Default: platform temp directory (e.g. /tmp/connection-<id>-config.json)"
5379        ),
5380    ] = None,
5381    with_secrets: Annotated[
5382        bool,
5383        Parameter(
5384            name="--with-secrets",
5385            negative="--no-secrets",
5386            help="If set, fetches unmasked secrets from the internal database. "
5387            "Requires GCP_PROD_DB_ACCESS_CREDENTIALS env var or `gcloud auth application-default login`. "
5388            "Must be used with --oc-issue-url.",
5389        ),
5390    ] = False,
5391    oc_issue_url: Annotated[
5392        str | None,
5393        Parameter(
5394            help="OC issue URL for audit logging. Required when using --with-secrets."
5395        ),
5396    ] = None,
5397) -> None:
5398    """Fetch connection configuration from Airbyte Cloud to a local file.
5399
5400    This command retrieves the source configuration for a given connection ID
5401    and writes it to a JSON file. When `--output-path` is omitted the file is
5402    written to the platform temp directory to avoid accidentally committing
5403    secrets to a git repository.
5404
5405    Requires authentication via AIRBYTE_CLOUD_CLIENT_ID and
5406    AIRBYTE_CLOUD_CLIENT_SECRET environment variables.
5407
5408    When --with-secrets is specified, the command fetches unmasked secrets from
5409    the internal database using the connection-retriever. This additionally requires:
5410    - An OC issue URL for audit logging (--oc-issue-url)
5411    - GCP credentials via `GCP_PROD_DB_ACCESS_CREDENTIALS` env var or `gcloud auth application-default login`
5412    - Cloud SQL Python Connector access to the Prod DB replica.
5413    """
5414    path = Path(output_path) if output_path else None
5415    result = fetch_connection_config(
5416        connection_id=connection_id,
5417        output_path=path,
5418        with_secrets=with_secrets,
5419        oc_issue_url=oc_issue_url,
5420    )
5421    if result.success:
5422        print_success(result.message)
5423    else:
5424        print_error(result.message)
5425    print_json(result.model_dump())
5426
5427
5428def _read_json_input(
5429    json_str: str | None,
5430    input_file: str | None,
5431) -> dict:
5432    """Read JSON from a positional arg, --input file, or STDIN (in that priority)."""
5433    if json_str is not None:
5434        return json.loads(json_str)
5435    if input_file is not None:
5436        return json.loads(Path(input_file).read_text())
5437    if not sys.stdin.isatty():
5438        return json.loads(sys.stdin.read())
5439    exit_with_error(
5440        "No JSON input provided. Pass it as a positional argument, "
5441        "via --input <file>, or pipe to STDIN."
5442    )
5443    raise SystemExit(1)  # unreachable, but satisfies type checker
5444
5445
5446# Every way the residency lookup can fail before it has an answer. Callers turn
5447# these into one line and exit, so a lookup failure never routes a connection.
5448_RESIDENCY_LOOKUP_ERRORS: tuple[type[Exception], ...] = (
5449    ConnectionNotFoundError,
5450    ValueError,
5451    google.auth.exceptions.GoogleAuthError,
5452    ProdDbSecretAccessError,
5453    ProdDbCloudSqlIamError,
5454    CloudRunProdDbAccessNotConfiguredError,
5455    sqlalchemy.exc.SQLAlchemyError,
5456    google_api_exceptions.GoogleAPIError,
5457    OSError,
5458)
5459
5460
5461def _describe_residency_lookup_error(connection_id: str, exc: Exception) -> str:
5462    """Describe a failed residency lookup in one line, without a traceback."""
5463    if isinstance(
5464        exc,
5465        (
5466            google.auth.exceptions.DefaultCredentialsError,
5467            google.auth.exceptions.RefreshError,
5468        ),
5469    ):
5470        return (
5471            "No usable GCP credentials for the Prod DB replica "
5472            f"({type(exc).__name__}). Set {ENV_GCP_PROD_DB_ACCESS_CREDENTIALS} "
5473            "or configure Application Default Credentials."
5474        )
5475    if isinstance(
5476        exc,
5477        (
5478            ConnectionNotFoundError,
5479            ValueError,
5480            ProdDbSecretAccessError,
5481            ProdDbCloudSqlIamError,
5482        ),
5483    ):
5484        return str(exc)
5485    return (
5486        "Could not reach the Prod DB replica to read the dataplane group of "
5487        f"connection {connection_id} ({type(exc).__name__})."
5488    )
5489
5490
5491def _refuse_eu_connection_outside_eu_region(connection_id: str, residency: str) -> None:
5492    """Exit before any fetch if an EU connection is used outside the EU region.
5493
5494    This check runs before the public API returns config or catalog, keeping
5495    them off a US runner.
5496    """
5497    if residency == "eu" and not is_running_in_eu_region():
5498        message = (
5499            f"{EU_REGION_REQUIRED_MESSAGE}; connection ID {connection_id} was "
5500            "refused before its config or catalog was fetched."
5501        )
5502        write_github_output("success", False)
5503        write_github_output("error", message)
5504        exit_with_error(
5505            f"{message}\n\n"
5506            "Use a connection from a non-EU workspace, or use GSM-based "
5507            "integration test credentials instead (by omitting --connection-id)."
5508        )
5509
5510
5511def _refuse_non_eu_connection_in_eu_region(connection_id: str) -> None:
5512    """Exit before any fetch if a non-EU connection is used in the EU region.
5513
5514    The mirror of `_refuse_eu_connection_outside_eu_region`, so each runtime reads
5515    only connections in the data region their customer chose.
5516    """
5517    if is_running_in_eu_region():
5518        message = (
5519            f"{EU_REGION_EU_ONLY_MESSAGE}; connection ID {connection_id} was "
5520            "refused before its config or catalog was fetched."
5521        )
5522        write_github_output("success", False)
5523        write_github_output("error", message)
5524        exit_with_error(message)
5525
5526
5527@connection_app.command(name="get-region")
5528def get_connection_region(
5529    connection_id: Annotated[
5530        str,
5531        Parameter(help="The connection ID (UUID) to check."),
5532    ],
5533) -> None:
5534    """Get a connection's dataplane group and derived residency.
5535
5536    Reads only the dataplane group, not connection or actor secrets. Requires
5537    `GCP_PROD_DB_ACCESS_CREDENTIALS` or Application Default Credentials. On any
5538    lookup failure it exits non-zero and writes no GitHub output.
5539    """
5540    try:
5541        dataplane_group = get_connection_dataplane_group(connection_id)
5542    except _RESIDENCY_LOOKUP_ERRORS as exc:
5543        exit_with_error(_describe_residency_lookup_error(connection_id, exc))
5544
5545    residency = residency_for_dataplane_group(dataplane_group)
5546    write_github_output("dataplane_group", dataplane_group)
5547    write_github_output("residency", residency)
5548    print_json(
5549        {
5550            "connection_id": connection_id,
5551            "dataplane_group": dataplane_group,
5552            "residency": residency,
5553        }
5554    )
5555
5556
5557@state_app.command(name="get")
5558def get_connection_state(
5559    connection_id: Annotated[
5560        str,
5561        Parameter(help="The connection ID (UUID) to fetch state for."),
5562    ],
5563    stream_name: Annotated[
5564        str | None,
5565        Parameter(
5566            help="Optional stream name to filter state for a single stream.",
5567            name="--stream",
5568        ),
5569    ] = None,
5570    stream_namespace: Annotated[
5571        str | None,
5572        Parameter(
5573            help="Optional stream namespace to narrow the stream filter.",
5574            name="--namespace",
5575        ),
5576    ] = None,
5577    output: Annotated[
5578        str | None,
5579        Parameter(
5580            help="Path to write the state JSON output to a file.",
5581            name="--output",
5582        ),
5583    ] = None,
5584) -> None:
5585    """Get the current state for an Airbyte Cloud connection."""
5586    workspace = CloudWorkspace.from_env()
5587    conn = workspace.get_connection(connection_id)
5588
5589    if stream_name is not None:
5590        stream_state = conn.get_stream_state(
5591            stream_name=stream_name,
5592            stream_namespace=stream_namespace,
5593        )
5594        result = {
5595            "connection_id": connection_id,
5596            "stream_name": stream_name,
5597            "stream_namespace": stream_namespace,
5598            "stream_state": stream_state,
5599        }
5600    else:
5601        result = conn.dump_raw_state()
5602
5603    json_output = json.dumps(result, indent=2, default=str)
5604    if output is not None:
5605        Path(output).write_text(json_output + "\n")
5606        print(f"State written to {output}", file=sys.stderr)
5607    else:
5608        print(json_output)
5609
5610
5611@state_app.command(name="set")
5612def set_connection_state(
5613    connection_id: Annotated[
5614        str,
5615        Parameter(help="The connection ID (UUID) to update state for."),
5616    ],
5617    state_json: Annotated[
5618        str | None,
5619        Parameter(
5620            help="The connection state as a JSON string. When --stream is used, "
5621            'this is just the stream\'s state blob (e.g., \'{"cursor": "2024-01-01"}\').'
5622            " Otherwise, must include 'stateType', 'connectionId', and the appropriate "
5623            "state field. Can also be provided via --input or STDIN."
5624        ),
5625    ] = None,
5626    stream_name: Annotated[
5627        str | None,
5628        Parameter(
5629            help="Optional stream name to update state for a single stream only.",
5630            name="--stream",
5631        ),
5632    ] = None,
5633    stream_namespace: Annotated[
5634        str | None,
5635        Parameter(
5636            help="Optional stream namespace to identify the stream.",
5637            name="--namespace",
5638        ),
5639    ] = None,
5640    input_file: Annotated[
5641        str | None,
5642        Parameter(
5643            help="Path to a JSON file containing the state to set.",
5644            name="--input",
5645        ),
5646    ] = None,
5647) -> None:
5648    """Set the state for an Airbyte Cloud connection.
5649
5650    State JSON can be provided as a positional argument, via --input <file>,
5651    or piped through STDIN.
5652
5653    Uses the safe variant that prevents updates while a sync is running.
5654    When --stream is provided, only that stream's state is updated within
5655    the existing connection state.
5656    """
5657    parsed_state = _read_json_input(state_json, input_file)
5658    workspace = CloudWorkspace.from_env()
5659    conn = workspace.get_connection(connection_id)
5660
5661    if stream_name is not None:
5662        conn.set_stream_state(
5663            stream_name=stream_name,
5664            state_blob_dict=parsed_state,
5665            stream_namespace=stream_namespace,
5666        )
5667    else:
5668        conn.import_raw_state(parsed_state)
5669
5670    result = conn.dump_raw_state()
5671    json_output = json.dumps(result, indent=2, default=str)
5672    print(json_output)
5673
5674
5675@state_app.command(name="reset")
5676def reset_stream_state(
5677    connection_id: Annotated[
5678        str,
5679        Parameter(help="The connection ID (UUID) to update state for."),
5680    ],
5681    stream_name: Annotated[
5682        str,
5683        Parameter(
5684            help="The configured stream name whose state should be reset.",
5685            name="--stream",
5686        ),
5687    ],
5688    stream_namespace: Annotated[
5689        str | None,
5690        Parameter(
5691            help="Optional stream namespace to identify the stream.",
5692            name="--namespace",
5693        ),
5694    ] = None,
5695) -> None:
5696    """Reset a configured stream's state so the next sync full-refreshes it.
5697
5698    Uses the safe variant that prevents updates while a sync is running.
5699    Returns a restorable `previous_state_backup` in raw Config API format.
5700    """
5701    workspace = CloudWorkspace.from_env()
5702    conn = workspace.get_connection(connection_id)
5703
5704    try:
5705        result = reset_cloud_stream_state(
5706            conn,
5707            stream_name=stream_name,
5708            stream_namespace=stream_namespace,
5709        )
5710    except PyAirbyteInputError as e:
5711        exit_with_error(str(e))
5712
5713    print(json.dumps(result.model_dump(), indent=2, default=str))
5714
5715
5716@catalog_app.command(name="get")
5717def get_connection_catalog(
5718    connection_id: Annotated[
5719        str,
5720        Parameter(help="The connection ID (UUID) to fetch catalog for."),
5721    ],
5722    output: Annotated[
5723        str | None,
5724        Parameter(
5725            help="Path to write the catalog JSON output to a file.",
5726            name="--output",
5727        ),
5728    ] = None,
5729) -> None:
5730    """Get the configured catalog for an Airbyte Cloud connection."""
5731    workspace = CloudWorkspace.from_env()
5732    conn = workspace.get_connection(connection_id)
5733    result = conn.dump_raw_catalog()
5734    if result is None:
5735        exit_with_error("No configured catalog found for this connection.")
5736        raise SystemExit(1)  # unreachable, but satisfies type checker
5737
5738    json_output = json.dumps(result, indent=2, default=str)
5739    if output is not None:
5740        Path(output).write_text(json_output + "\n")
5741        print(f"Catalog written to {output}", file=sys.stderr)
5742    else:
5743        print(json_output)
5744
5745
5746@catalog_app.command(name="set")
5747def set_connection_catalog(
5748    connection_id: Annotated[
5749        str,
5750        Parameter(help="The connection ID (UUID) to update catalog for."),
5751    ],
5752    catalog_json: Annotated[
5753        str | None,
5754        Parameter(
5755            help="The configured catalog as a JSON string. "
5756            "Can also be provided via --input or STDIN."
5757        ),
5758    ] = None,
5759    input_file: Annotated[
5760        str | None,
5761        Parameter(
5762            help="Path to a JSON file containing the catalog to set.",
5763            name="--input",
5764        ),
5765    ] = None,
5766) -> None:
5767    """Set the configured catalog for an Airbyte Cloud connection.
5768
5769    Catalog JSON can be provided as a positional argument, via --input <file>,
5770    or piped through STDIN.
5771
5772    WARNING: This replaces the entire configured catalog.
5773    """
5774    parsed_catalog = _read_json_input(catalog_json, input_file)
5775    workspace = CloudWorkspace.from_env()
5776    conn = workspace.get_connection(connection_id)
5777    conn.import_raw_catalog(parsed_catalog)
5778
5779    result = conn.dump_raw_catalog()
5780    if result is None:
5781        exit_with_error("Failed to retrieve catalog after import.")
5782        raise SystemExit(1)  # unreachable, but satisfies type checker
5783
5784    json_output = json.dumps(result, indent=2, default=str)
5785    print(json_output)
5786
5787
5788@logs_app.command(name="lookup-cloud-backend-error")
5789def lookup_cloud_backend_error(
5790    error_id: Annotated[
5791        str,
5792        Parameter(
5793            help=(
5794                "The error ID (UUID) to search for. This is typically returned "
5795                "in API error responses as {'errorId': '...'}"
5796            )
5797        ),
5798    ],
5799    lookback_days: Annotated[
5800        int,
5801        Parameter(help="Number of days to look back in logs."),
5802    ] = 7,
5803    min_severity_filter: Annotated[
5804        GCPSeverity | None,
5805        Parameter(
5806            help="Optional minimum severity level to filter logs.",
5807        ),
5808    ] = None,
5809    raw: Annotated[
5810        bool,
5811        Parameter(help="Output raw JSON instead of formatted text."),
5812    ] = False,
5813) -> None:
5814    """Look up error details from GCP Cloud Logging by error ID.
5815
5816    When an Airbyte Cloud API returns an error response with only an error ID
5817    (e.g., {"errorId": "3173452e-8f22-4286-a1ec-b0f16c1e078a"}), this command
5818    fetches the full stack trace and error details from GCP Cloud Logging.
5819
5820    Requires GCP credentials with Logs Viewer role on the target project.
5821    Set up credentials with: gcloud auth application-default login
5822    """
5823    print(f"Searching for error ID: {error_id}", file=sys.stderr)
5824    print(f"Lookback days: {lookback_days}", file=sys.stderr)
5825    if min_severity_filter:
5826        print(f"Severity filter: {min_severity_filter}", file=sys.stderr)
5827    print(file=sys.stderr)
5828
5829    result = fetch_error_logs(
5830        error_id=error_id,
5831        lookback_days=lookback_days,
5832        min_severity_filter=min_severity_filter,
5833    )
5834
5835    if raw:
5836        print_json(result.model_dump())
5837        return
5838
5839    print(f"Found {result.total_entries_found} log entries", file=sys.stderr)
5840    print(file=sys.stderr)
5841
5842    if result.payloads:
5843        for i, payload in enumerate(result.payloads):
5844            print(f"=== Log Group {i + 1} ===")
5845            print(f"Timestamp: {payload.timestamp}")
5846            print(f"Severity: {payload.severity}")
5847            if payload.resource.labels.pod_name:
5848                print(f"Pod: {payload.resource.labels.pod_name}")
5849            print(f"Lines: {payload.num_log_lines}")
5850            print()
5851            print(payload.message)
5852            print()
5853    elif result.entries:
5854        print("No grouped payloads, showing raw entries:", file=sys.stderr)
5855        for entry in result.entries:
5856            print(f"[{entry.timestamp}] {entry.severity}: {entry.payload}")
5857    else:
5858        print_error("No log entries found for this error ID.")
5859
5860
5861# =============================================================================
5862# Organization commands
5863# =============================================================================
5864
5865
5866@organization_app.command(name="search")
5867def organization_search(
5868    name_contains: Annotated[
5869        str,
5870        Parameter(
5871            name="--name-contains",
5872            help="Case-insensitive substring to match against organization name or email.",
5873        ),
5874    ],
5875    limit: Annotated[
5876        int,
5877        Parameter(
5878            name="--limit",
5879            help="Maximum number of results to return.",
5880        ),
5881    ] = 20,
5882) -> None:
5883    """Search organizations by name or email substring."""
5884    rows = _search_organizations(name_contains=name_contains, limit=limit)
5885    if not rows:
5886        print_error(f"No organizations found matching '{name_contains}'.")
5887        return
5888
5889    print_json([dict(row) for row in rows])
5890
5891
5892@organization_app.command(name="info")
5893def organization_info(
5894    organization_id: Annotated[
5895        str,
5896        Parameter(help="The organization UUID."),
5897    ],
5898) -> None:
5899    """Get information about an organization (stub — not yet implemented)."""
5900    exit_with_error(
5901        "organization info is not yet implemented. "
5902        "Use 'airbyte-ops cloud organization search' to find organizations by name."
5903    )
5904
5905
5906# =============================================================================
5907# Workspace commands
5908# =============================================================================
5909
5910
5911@workspace_app.command(name="search")
5912def workspace_search(
5913    name_contains: Annotated[
5914        str | None,
5915        Parameter(
5916            name="--name-contains",
5917            help="Case-insensitive substring to match against workspace name or slug.",
5918        ),
5919    ] = None,
5920    email_domain: Annotated[
5921        str | None,
5922        Parameter(
5923            name="--email-domain",
5924            help="Email domain to search for (e.g. 'motherduck.com').",
5925        ),
5926    ] = None,
5927    limit: Annotated[
5928        int,
5929        Parameter(
5930            name="--limit",
5931            help="Maximum number of results to return.",
5932        ),
5933    ] = 100,
5934) -> None:
5935    """Search workspaces by name substring or email domain."""
5936    if not name_contains and not email_domain:
5937        exit_with_error(
5938            "At least one of --name-contains or --email-domain must be provided."
5939        )
5940
5941    if name_contains:
5942        rows = _search_workspaces(name_contains=name_contains, limit=limit)
5943    else:
5944        rows = query_workspaces_by_email_domain(
5945            email_domain=email_domain.lstrip("@"),  # ty: ignore[unresolved-attribute]
5946            limit=limit,
5947        )
5948
5949    if not rows:
5950        search_term = name_contains or email_domain
5951        print_error(f"No workspaces found matching '{search_term}'.")
5952        return
5953
5954    print_json([dict(row) for row in rows])
5955
5956
5957@workspace_app.command(name="info")
5958def workspace_info_cmd(
5959    workspace_id: Annotated[
5960        str,
5961        Parameter(help="The workspace UUID."),
5962    ],
5963) -> None:
5964    """Get information about a workspace."""
5965    result = query_workspace_info(workspace_id=workspace_id)
5966    if not result:
5967        print_error(f"No workspace found with ID '{workspace_id}'.")
5968        return
5969
5970    print_json(dict(result))