airbyte_ops_mcp.cli.registry

CLI commands for connector registry operations.

This module provides CLI wrappers for registry operations. The core logic lives in the airbyte_ops_mcp.registry capability module.

Command groups:

airbyte-ops registry connector list - List all connectors in registry airbyte-ops registry connector-version list - List versions for a connector airbyte-ops registry connector-version next - Compute next version tag (prerelease/RC) airbyte-ops registry connector-version yank - Mark a connector version as yanked airbyte-ops registry connector-version unyank - Remove yank marker from a connector version airbyte-ops registry connector-version metadata get - Read connector metadata from GCS airbyte-ops registry connector-version artifacts generate - Generate version artifacts locally via docker airbyte-ops registry connector-version artifacts publish - Publish version artifacts to GCS airbyte-ops registry store mirror --local|--gcs-bucket|--s3-bucket - Mirror entire registry for testing airbyte-ops registry store compile --store coral:dev|coral:prod - Compile registry indexes and sync latest/ dirs airbyte-ops registry marketing-stubs sync --store coral:prod - Sync connector_stubs.json to GCS airbyte-ops registry marketing-stubs check --store coral:prod - Compare local file with GCS

CLI reference

The commands below are regenerated by poe docs-generate via cyclopts's programmatic docs API; see docs/generate_cli.py.

airbyte-ops registry COMMAND

Connector registry operations (GCS metadata service).

Commands:

airbyte-ops registry connector

Connector listing operations.

airbyte-ops registry connector list

airbyte-ops registry connector list [OPTIONS] STORE

List connectors in the registry.

When filters are applied, reads the compiled cloud_registry.json index for fast lookups. Without filters, falls back to scanning individual metadata blobs (captures all connectors including OSS-only).

Requires GCS_CREDENTIALS environment variable to be set.

Parameters:

  • STORE, --store: Store target (e.g. 'coral:dev', 'coral:prod'). [required]
  • --certified-only, --no-certified-only: Include only certified connectors. [default: False]
  • --support-level: Exact support level to match (e.g. certified, community, archived). [choices: archived, community, certified]
  • --min-support-level: Minimum support level (inclusive). Levels from lowest to highest: archived, community, certified. [choices: archived, community, certified]
  • --connector-type: Filter by connector type: source or destination. [choices: source, destination]
  • --language: Filter by implementation language (e.g. python, java, manifest-only). [choices: python, java, low-code, manifest-only]
  • --format: Output format: 'json' for JSON array, 'text' for newline-separated. [choices: json, text] [default: json]

airbyte-ops registry connector-version

Connector version operations (list, yank, unyank, artifacts).

airbyte-ops registry connector-version metadata

Connector metadata inspection.

Commands:

  • get: Read a connector version's metadata from the registry.
airbyte-ops registry connector-version metadata get
airbyte-ops registry connector-version metadata get [OPTIONS] NAME STORE

Read a connector version's metadata from the registry.

Returns the full metadata.yaml content for a connector at the specified version.

Requires GCS_CREDENTIALS environment variable to be set.

Parameters:

  • NAME, --name: Connector name (e.g., 'source-faker', 'destination-postgres'). [required]
  • STORE, --store: Store target (e.g. 'coral:dev', 'coral:prod'). [required]
  • --version: Version to read (e.g., 'latest', '1.2.3'). [default: latest]
  • --format: Output format: 'json' for JSON, 'raw' for YAML. [choices: json, raw] [default: json]

airbyte-ops registry connector-version artifacts

Version artifact generation and publishing.

Commands:

  • generate: Generate version artifacts for a connector locally.
  • publish: Publish version artifacts to GCS using fsspec rsync.
airbyte-ops registry connector-version artifacts generate
airbyte-ops registry connector-version artifacts generate METADATA-FILE DOCKER-IMAGE [ARGS]

Generate version artifacts for a connector locally.

Runs the connector's docker image with DEPLOYMENT_MODE=cloud and DEPLOYMENT_MODE=oss to obtain both spec variants, then generates the registry entries (cloud.json, oss.json) by applying registryOverrides from the metadata.

The generated metadata.yaml is enriched with git commit info, SBOM URL, and (when applicable) components SHA before writing. Validation is run after generation by default; pass --no-validate to skip.

This is a local-only operation -- no files are uploaded to GCS. Use artifacts publish to upload generated artifacts to GCS.

Parameters:

  • METADATA-FILE, --metadata-file: Path to the connector's metadata.yaml file. [required]
  • DOCKER-IMAGE, --docker-image: Docker image to run spec against (e.g., 'airbyte/source-faker:6.2.38'). [required]
  • OUTPUT-DIR, --output-dir: Directory to write artifacts to. If not specified, a temp directory is created.
  • REPO-ROOT, --repo-root: Root of the Airbyte repo checkout (for resolving doc.md). If not specified, inferred by walking up from metadata-file.
  • DRY-RUN, --dry-run, --no-dry-run: Show what would be generated without running docker or writing files. [default: False]
  • WITH-VALIDATE, --with-validate, --no-validate: Run metadata validators after generation (default: enabled). Use --no-validate to skip. [default: True]
  • WITH-SBOM, --with-sbom, --no-sbom: Generate spdx.json (SBOM) for connectors (default: enabled). Use --no-sbom to skip. [default: True]
  • WITH-DEPENDENCY-DUMP, --with-dependency-dump, --no-dependency-dump: Generate dependencies.json for Python connectors (default: enabled). Use --no-dependency-dump to skip. [default: True]
airbyte-ops registry connector-version artifacts publish
airbyte-ops registry connector-version artifacts publish [OPTIONS] NAME VERSION ARTIFACTS-DIR STORE

Publish version artifacts to GCS using fsspec rsync.

Uploads locally generated artifacts (from artifacts generate) to the versioned path in GCS. By default, metadata is validated before upload; pass --no-validate to skip.

Uses --store to select the destination store and environment:

  • coral:dev → coral dev bucket at root
  • coral:prod → coral prod bucket at root
  • coral:dev/aj-test100 → coral dev bucket under aj-test100/ prefix

Requires GCS_CREDENTIALS environment variable to be set.

Parameters:

  • NAME, --name: Connector name (e.g., 'source-faker'). [required]
  • VERSION, --version: Version to publish artifacts for (e.g., '1.2.3'). [required]
  • ARTIFACTS-DIR, --artifacts-dir: Directory containing generated artifacts to publish (from 'artifacts generate'). [required]
  • STORE, --store: Store target (e.g. 'coral:dev', 'coral:prod', 'coral:dev/prefix'). [required]
  • --dry-run, --no-dry-run: Show what would be published without writing to GCS. [default: False]
  • --with-validate, --no-validate: Validate metadata before uploading (default: enabled). Use --no-validate to skip. [default: True]

airbyte-ops registry connector-version next

airbyte-ops registry connector-version next NAME SHA [ARGS]

Compute the next version tag for a connector.

Outputs the version tag to stdout for easy capture in shell scripts. This is the single source of truth for pre-release version format.

The command fetches the connector's metadata.yaml from GitHub at the given SHA to determine the base version. It also compares against the master branch and prints a warning to stderr if no version bump is detected.

If --base-version is provided, it is used directly instead of fetching from GitHub.

Parameters:

  • NAME, --name: Connector name (e.g., 'source-github'). [required]
  • SHA, --sha: Git commit SHA (full or at least 7 characters). [required]
  • BASE-VERSION, --base-version: Base version override. If not provided, fetched from metadata.yaml at the given SHA.

airbyte-ops registry connector-version list

airbyte-ops registry connector-version list [OPTIONS] NAME STORE

List all versions of a connector in the registry.

Scans the registry bucket to find all versions of a specific connector.

Requires GCS_CREDENTIALS environment variable to be set.

Parameters:

  • NAME, --name: Connector name (e.g., 'source-faker'). [required]
  • STORE, --store: Store target (e.g. 'coral:dev', 'coral:prod'). [required]
  • --format: Output format: 'json' for JSON array, 'text' for newline-separated. [choices: json, text] [default: json]

airbyte-ops registry connector-version yank

airbyte-ops registry connector-version yank [OPTIONS] NAME VERSION STORE

Mark a connector version as yanked.

Writes a version-yank.yml marker file to the version's directory in GCS. Yanked versions are excluded when determining the latest version of a connector.

Requires GCS_CREDENTIALS environment variable to be set.

Parameters:

  • NAME, --name: Connector name (e.g., 'source-faker'). [required]
  • VERSION, --version: Version to yank (e.g., '1.2.3'). [required]
  • STORE, --store: Store target (e.g. 'coral:dev', 'coral:prod'). [required]
  • --reason: Reason for yanking this version. [default: ""]
  • --approval-url: Approval evidence URL to record in the yank marker. [default: ""]
  • --dry-run, --no-dry-run: Show what would be done without making changes. [default: False]

airbyte-ops registry connector-version unyank

airbyte-ops registry connector-version unyank [OPTIONS] NAME VERSION STORE

Rename the active yank marker to a dated audit marker.

Moves version-yank.yml to version-unyanked-yyyymmdd.yml, making the version eligible again when determining the latest version.

Requires GCS_CREDENTIALS environment variable to be set.

Parameters:

  • NAME, --name: Connector name (e.g., 'source-faker'). [required]
  • VERSION, --version: Version to unyank (e.g., '1.2.3'). [required]
  • STORE, --store: Store target (e.g. 'coral:dev', 'coral:prod'). [required]
  • --dry-run, --no-dry-run: Show what would be done without making changes. [default: False]

airbyte-ops registry store

Whole-registry store operations (mirror, compile).

airbyte-ops registry store mirror

airbyte-ops registry store mirror [ARGS]

Create a mirror of the connector registry from the source store.

Reads all connector metadata from the source store and copies it to the specified output target. Supports local filesystem, GCS, and S3 as output targets via fsspec.

Output targets are mutually exclusive: specify exactly one of --local, --gcs-bucket, or --s3-bucket.

The production bucket is categorically disallowed as an output target.

To clean up legacy artifacts (e.g. disabled strict-encrypt connectors) after mirroring, run compile with --with-legacy-migration v1::

airbyte-ops registry store compile --store coral:dev/my-prefix \
    --with-legacy-migration v1

Parameters:

  • LOCAL, --local: Write output to a local directory. Mutually exclusive with --gcs-bucket and --s3-bucket. [default: False]
  • GCS-BUCKET, --gcs-bucket: Write output to a GCS bucket. Must not be the prod bucket. Mutually exclusive with --local and --s3-bucket.
  • S3-BUCKET, --s3-bucket: Write output to an S3 bucket. Mutually exclusive with --local and --gcs-bucket.
  • OUTPUT-PATH-ROOT, --output-path-root: Root path/prefix for the output. For --local, this is a directory path (defaults to a new temp dir if omitted). For --gcs-bucket/--s3-bucket, this prefix is prepended to all blob paths.
  • DRY-RUN, --dry-run, --no-dry-run: Show what would be rebuilt without writing any files. [default: False]
  • SOURCE-STORE, --source-store: Source store to read from (e.g. 'coral:prod', 'coral:dev'). [default: coral:prod]
  • CONNECTOR-NAME, --connector-name, --empty-connector-name: Only rebuild these connectors (by name). Can be specified multiple times, e.g. --connector-name source-faker --connector-name destination-bigquery.

airbyte-ops registry store compile

airbyte-ops registry store compile [OPTIONS] STORE

Compile the registry: sync latest/ dirs, write global and per-connector indexes.

Scans all version directories in the target store, determines the latest GA semver per connector (excluding yanked and pre-release versions), ensures each latest/ directory matches the computed latest, and writes:

  • registries/v0/cloud_registry.json -- global cloud registry index
  • registries/v0/oss_registry.json -- global OSS registry index
  • metadata/airbyte/<connector>/versions.json -- per-connector version index

With --with-secrets-mask, also regenerates:

  • registries/v0/specs_secrets_mask.yaml -- properties marked as secrets

With --with-legacy-migration=v1, deletes cloud.json / oss.json files for connectors whose registryOverrides.cloud.enabled or registryOverrides.oss.enabled is false.

By default, injects connector quality metrics from the latest analytics JSONL export into generated.metrics. Use --no-metrics for offline scenarios.

With --force, resyncs all latest/ directories even if the version marker matches the computed latest version.

Uses efficient glob patterns for scanning (no file downloads during discovery).

Requires GCS_CREDENTIALS environment variable to be set.

Parameters:

  • STORE, --store: Store target (e.g. 'coral:dev', 'coral:prod', 'coral:dev/prefix'). [required]
  • --connector-name, --empty-connector-name: Only compile these connectors (can be repeated).
  • --dry-run, --no-dry-run: Show what would be done without writing. [default: False]
  • --with-secrets-mask, --no-with-secrets-mask: Also regenerate specs_secrets_mask.yaml by scanning all connector specs for airbyte_secret properties. [default: False]
  • --with-legacy-migration: Run a one-time legacy migration step during compile. Currently supported: 'v1' — delete cloud.json / oss.json files for connectors whose registryOverrides.cloud.enabled or registryOverrides.oss.enabled is false. This cleans up artifacts produced by the legacy pipeline that did not respect the enabled flag.
  • --with-metrics, --no-metrics: Inject latest connector quality metrics from the analytics JSONL export into generated.metrics. [default: True]
  • --force, --no-force: Force resync of latest/ directories even if version markers are current. Useful when metadata changes without a version bump. [default: False]

airbyte-ops registry store delete-dev-latest

airbyte-ops registry store delete-dev-latest [OPTIONS] STORE

Delete all latest/ directories from a dev registry store.

Discovers every connector that has a latest/ directory and deletes each one in parallel using a thread pool.

This is useful before a full re-compile to prove that latest/ directories can be correctly regenerated from versioned data.

Only dev stores are allowed (store must begin with 'coral:dev').

Requires GCS_CREDENTIALS environment variable to be set.

Parameters:

  • STORE, --store: Store target (must begin with 'coral:dev'). [required]
  • --connector-name, --empty-connector-name: Only delete latest/ for these connectors (can be repeated).
  • --dry-run, --no-dry-run: Show what would be done without deleting. [default: False]

airbyte-ops registry store compare

airbyte-ops registry store compare [OPTIONS] STORE REFERENCE-STORE

Compare a store against a reference store and report differences.

Evaluates the --store target against --reference-store and reports per-connector artifact diffs and global index diffs.

Requires GCS_CREDENTIALS environment variable to be set.

Parameters:

  • STORE, --store: Store target being evaluated (e.g. 'coral:dev/20260306-mirror-compile'). [required]
  • REFERENCE-STORE, --reference-store: Known-good reference store to compare against. [required]
  • --connector-name, --empty-connector-name: Only compare these connectors (can be repeated).
  • --with-artifacts, --no-artifacts: Compare per-connector artifact files (metadata.yaml, cloud.json, oss.json, spec.json). [default: True]
  • --with-indexes, --no-indexes: Compare global registry index files (cloud_registry.json, oss_registry.json). [default: True]

airbyte-ops registry progressive-rollout

Progressive rollout lifecycle operations.

airbyte-ops registry progressive-rollout status

airbyte-ops registry progressive-rollout status [OPTIONS] NAME

Get progressive rollout status for a connector.

Parameters:

  • NAME, --name: Connector technical name (e.g., source-github). [required]
  • --repo-path: Path to the Airbyte monorepo. Defaults to current directory. [default: /home/runner/work/airbyte-ops-mcp/airbyte-ops-mcp]
  • --active-only, --with-terminal: Only return active non-terminal rollouts. [default: True]
  • --limit: Maximum number of rollout records to return. [default: 100]

airbyte-ops registry progressive-rollout finalize-marker

airbyte-ops registry progressive-rollout finalize-marker [OPTIONS] NAME STORE OUTCOME

Rename an active progressive-rollout.yml marker to an audit marker.

Parameters:

  • NAME, --name: Connector technical name (e.g., source-github). [required]
  • STORE, --store: Store target (e.g. 'coral:dev', 'coral:prod'). [required]
  • OUTCOME, --outcome: Marker outcome used in the dated audit filename. [required] [choices: promoted, aborted]
  • --version: Version to finalize. If omitted, exactly one active marker must exist.
  • --dry-run, --no-dry-run: Show what would be done without making changes. [default: False]

airbyte-ops registry marketing-stubs

Marketing connector stubs GCS operations (whole-file sync).

airbyte-ops registry marketing-stubs check

airbyte-ops registry marketing-stubs check [OPTIONS] STORE

Compare local connector_stubs.json with the version in GCS.

This command reads the entire local connector_stubs.json file and compares it with the version currently published in GCS.

Exit codes:

0: Local file matches GCS (check passed) 1: Differences found (check failed)

Output:

STDOUT: JSON representation of the comparison result STDERR: Informational messages and comparison details

Parameters:

  • STORE, --store: Store target (e.g. 'coral:dev', 'coral:prod'). [required]
  • --repo-root: Path to the airbyte-enterprise repository root. Defaults to current directory. [default: /home/runner/work/airbyte-ops-mcp/airbyte-ops-mcp]

airbyte-ops registry marketing-stubs sync

airbyte-ops registry marketing-stubs sync [OPTIONS] STORE

Sync local connector_stubs.json to GCS.

This command uploads the entire local connector_stubs.json file to GCS, replacing the existing file. Use this after merging changes to master in the airbyte-enterprise repository.

Exit codes:

0: Sync successful (or dry-run completed) 1: Error (file not found, validation failed, etc.)

Output:

STDOUT: JSON representation of the sync result STDERR: Informational messages and status updates

Parameters:

  • STORE, --store: Store target (e.g. 'coral:dev', 'coral:prod'). [required]
  • --repo-root: Path to the airbyte-enterprise repository root. Defaults to current directory. [default: /home/runner/work/airbyte-ops-mcp/airbyte-ops-mcp]
  • --dry-run, --no-dry-run: Show what would be uploaded without making changes. [default: False]
   1# Copyright (c) 2025 Airbyte, Inc., all rights reserved.
   2"""CLI commands for connector registry operations.
   3
   4This module provides CLI wrappers for registry operations. The core logic
   5lives in the `airbyte_ops_mcp.registry` capability module.
   6
   7Command groups:
   8    airbyte-ops registry connector list - List all connectors in registry
   9    airbyte-ops registry connector-version list - List versions for a connector
  10    airbyte-ops registry connector-version next - Compute next version tag (prerelease/RC)
  11    airbyte-ops registry connector-version yank - Mark a connector version as yanked
  12    airbyte-ops registry connector-version unyank - Remove yank marker from a connector version
  13    airbyte-ops registry connector-version metadata get - Read connector metadata from GCS
  14    airbyte-ops registry connector-version artifacts generate - Generate version artifacts locally via docker
  15    airbyte-ops registry connector-version artifacts publish - Publish version artifacts to GCS
  16    airbyte-ops registry store mirror --local|--gcs-bucket|--s3-bucket - Mirror entire registry for testing
  17    airbyte-ops registry store compile --store coral:dev|coral:prod - Compile registry indexes and sync latest/ dirs
  18    airbyte-ops registry marketing-stubs sync --store coral:prod - Sync connector_stubs.json to GCS
  19    airbyte-ops registry marketing-stubs check --store coral:prod - Compare local file with GCS
  20
  21## CLI reference
  22
  23The commands below are regenerated by `poe docs-generate` via cyclopts's
  24programmatic docs API; see `docs/generate_cli.py`.
  25
  26.. include:: ../../../docs/generated/cli/registry.md
  27   :start-line: 2
  28"""
  29
  30from __future__ import annotations
  31
  32# Hide Python-level members from the pdoc page for this module; the rendered
  33# docs for this CLI group come entirely from the grafted `.. include::` in
  34# the module docstring above.
  35__all__: list[str] = []
  36
  37import sys
  38from pathlib import Path
  39from typing import Annotated, Literal
  40
  41import yaml
  42from cyclopts import Parameter
  43
  44from airbyte_ops_mcp.cli._base import App, app
  45from airbyte_ops_mcp.cli._shared import (
  46    error_console,
  47    exit_with_error,
  48    print_error,
  49    print_json,
  50    print_success,
  51)
  52from airbyte_ops_mcp.github_api import (
  53    get_file_contents_at_ref,
  54    resolve_default_github_token,
  55)
  56from airbyte_ops_mcp.mcp.prerelease import (
  57    compute_prerelease_docker_image_tag,
  58)
  59from airbyte_ops_mcp.registry import (
  60    resolve_registry_store,
  61)
  62from airbyte_ops_mcp.registry._enums import (
  63    ConnectorLanguage,
  64    ConnectorType,
  65    SupportLevel,
  66)
  67from airbyte_ops_mcp.registry.compare import compare_stores
  68from airbyte_ops_mcp.registry.connector_stubs import (
  69    CONNECTOR_STUBS_FILE,
  70)
  71from airbyte_ops_mcp.registry.generate import generate_version_artifacts
  72from airbyte_ops_mcp.registry.progressive_rollout_status import (
  73    get_connector_rollout_status,
  74)
  75from airbyte_ops_mcp.registry.registry_store_base import (
  76    Registry,
  77    get_registry,
  78)
  79
  80# Create the registry sub-app
  81registry_app = App(
  82    name="registry", help="Connector registry operations (GCS metadata service)."
  83)
  84app.command(registry_app)
  85
  86# Create the connector sub-app under registry
  87connector_app = App(name="connector", help="Connector listing operations.")
  88registry_app.command(connector_app)
  89
  90# Create the connector-version sub-app under registry
  91connector_version_app = App(
  92    name="connector-version",
  93    help="Connector version operations (list, yank, unyank, artifacts).",
  94)
  95registry_app.command(connector_version_app)
  96
  97# Create the metadata sub-app under connector-version
  98metadata_app = App(
  99    name="metadata",
 100    help="Connector metadata inspection.",
 101)
 102connector_version_app.command(metadata_app)
 103
 104# Create the artifacts sub-app under connector-version
 105artifacts_app = App(
 106    name="artifacts",
 107    help="Version artifact generation and publishing.",
 108)
 109connector_version_app.command(artifacts_app)
 110
 111# Create the store sub-app under registry (whole-registry operations)
 112store_app = App(
 113    name="store",
 114    help="Whole-registry store operations (mirror, compile).",
 115)
 116registry_app.command(store_app)
 117
 118# Create the progressive-rollout sub-app under registry
 119progressive_rollout_app = App(
 120    name="progressive-rollout",
 121    help="Progressive rollout lifecycle operations.",
 122)
 123registry_app.command(progressive_rollout_app)
 124
 125# Create the marketing-stubs sub-app under registry (for whole-file GCS operations)
 126marketing_stubs_app = App(
 127    name="marketing-stubs",
 128    help="Marketing connector stubs GCS operations (whole-file sync).",
 129)
 130registry_app.command(marketing_stubs_app)
 131
 132
 133AIRBYTE_REPO_OWNER = "airbytehq"
 134AIRBYTE_ENTERPRISE_REPO_NAME = "airbyte-enterprise"
 135AIRBYTE_REPO_NAME = "airbyte"
 136CONNECTOR_PATH_PREFIX = "airbyte-integrations/connectors"
 137
 138
 139def _resolve_store(store: str) -> Registry:
 140    """Resolve `--store` to a `Registry` instance.
 141
 142    Wraps `resolve_registry_store` and `get_registry`, converting
 143    `ValueError` into a user-friendly CLI error via `exit_with_error`.
 144    """
 145    try:
 146        resolved = resolve_registry_store(store=store)
 147    except ValueError as e:
 148        exit_with_error(str(e))
 149        raise  # unreachable; satisfies type checker
 150    return get_registry(resolved)
 151
 152
 153def _get_connector_version_from_github(
 154    connector_name: str,
 155    ref: str,
 156    token: str | None = None,
 157) -> str | None:
 158    """Fetch connector version from metadata.yaml via GitHub API.
 159
 160    Args:
 161        connector_name: Connector name (e.g., "source-github")
 162        ref: Git ref (commit SHA, branch name, or tag)
 163        token: GitHub API token (optional for public repos)
 164
 165    Returns:
 166        Version string from metadata.yaml, or None if not found.
 167    """
 168    path = f"{CONNECTOR_PATH_PREFIX}/{connector_name}/metadata.yaml"
 169    contents = get_file_contents_at_ref(
 170        owner=AIRBYTE_REPO_OWNER,
 171        repo=AIRBYTE_REPO_NAME,
 172        path=path,
 173        ref=ref,
 174        token=token,
 175    )
 176    if contents is None:
 177        return None
 178
 179    metadata = yaml.safe_load(contents)
 180    return metadata.get("data", {}).get("dockerImageTag")
 181
 182
 183@connector_version_app.command(name="next")
 184def compute_next_version(
 185    name: Annotated[
 186        str,
 187        Parameter(help="Connector name (e.g., 'source-github')."),
 188    ],
 189    sha: Annotated[
 190        str,
 191        Parameter(help="Git commit SHA (full or at least 7 characters)."),
 192    ],
 193    base_version: Annotated[
 194        str | None,
 195        Parameter(
 196            help="Base version override. If not provided, fetched from metadata.yaml at the given SHA."
 197        ),
 198    ] = None,
 199) -> None:
 200    """Compute the next version tag for a connector.
 201
 202    Outputs the version tag to stdout for easy capture in shell scripts.
 203    This is the single source of truth for pre-release version format.
 204
 205    The command fetches the connector's metadata.yaml from GitHub at the given SHA
 206    to determine the base version. It also compares against the master branch and
 207    prints a warning to stderr if no version bump is detected.
 208
 209    If --base-version is provided, it is used directly instead of fetching from GitHub.
 210
 211    Examples:
 212        airbyte-ops registry connector-version next --name source-github --sha abcdef1234567
 213        # Output: 1.2.3-preview.abcdef1
 214
 215        airbyte-ops registry connector-version next --name source-github --sha abcdef1234567 --base-version 1.2.3
 216        # Output: 1.2.3-preview.abcdef1 (uses provided version, skips GitHub API)
 217    """
 218    # Try to get a GitHub token (optional, but helps avoid rate limiting)
 219    # Token resolution may fail if no token is configured, which is fine for public repos
 220    token: str | None = None
 221    token = resolve_default_github_token(allow_none=True)
 222
 223    # Determine base version
 224    version: str
 225    if base_version:
 226        version = base_version
 227    else:
 228        # Fetch version from metadata.yaml at the given SHA
 229        fetched_version = _get_connector_version_from_github(name, sha, token)
 230        if fetched_version is None:
 231            print(
 232                f"Error: Could not fetch metadata.yaml for {name} at ref {sha}",
 233                file=sys.stderr,
 234            )
 235            sys.exit(1)
 236        version = fetched_version
 237
 238    # Compare with master branch version and warn if no bump detected
 239    master_version = _get_connector_version_from_github(name, "master", token)
 240    if master_version and master_version == version:
 241        print(
 242            f"Warning: No version bump detected for {name}. "
 243            f"Version {version} matches master branch.",
 244            file=sys.stderr,
 245        )
 246
 247    # Compute and output the prerelease tag
 248    tag = compute_prerelease_docker_image_tag(version, sha)
 249    print(tag)
 250
 251
 252@progressive_rollout_app.command(name="status")
 253def progressive_rollout_status(
 254    name: Annotated[
 255        str,
 256        Parameter(help="Connector technical name (e.g., source-github)."),
 257    ],
 258    *,
 259    repo_path: Annotated[
 260        Path,
 261        Parameter(help="Path to the Airbyte monorepo. Defaults to current directory."),
 262    ] = Path.cwd(),
 263    active_only: Annotated[
 264        bool,
 265        Parameter(
 266            help="Only return active non-terminal rollouts.",
 267            negative="--with-terminal",
 268        ),
 269    ] = True,
 270    limit: Annotated[
 271        int,
 272        Parameter(help="Maximum number of rollout records to return."),
 273    ] = 100,
 274) -> None:
 275    """Get progressive rollout status for a connector."""
 276    if not repo_path.exists():
 277        exit_with_error(f"Repository path not found: {repo_path}")
 278    try:
 279        result = get_connector_rollout_status(
 280            repo_path=repo_path,
 281            connector_name=name,
 282            active_only=active_only,
 283            limit=limit,
 284        )
 285    except (FileNotFoundError, ValueError) as e:
 286        exit_with_error(str(e))
 287
 288    print_json(result.model_dump())
 289
 290
 291@progressive_rollout_app.command(name="finalize-marker")
 292def progressive_rollout_finalize_marker(
 293    name: Annotated[
 294        str,
 295        Parameter(help="Connector technical name (e.g., source-github)."),
 296    ],
 297    store: Annotated[
 298        str,
 299        Parameter(help="Store target (e.g. 'coral:dev', 'coral:prod')."),
 300    ],
 301    outcome: Annotated[
 302        Literal["promoted", "aborted"],
 303        Parameter(help="Marker outcome used in the dated audit filename."),
 304    ],
 305    *,
 306    version: Annotated[
 307        str | None,
 308        Parameter(
 309            help="Version to finalize. If omitted, exactly one active marker must exist."
 310        ),
 311    ] = None,
 312    dry_run: Annotated[
 313        bool,
 314        Parameter(help="Show what would be done without making changes."),
 315    ] = False,
 316) -> None:
 317    """Rename an active `progressive-rollout.yml` marker to an audit marker."""
 318    registry = _resolve_store(store)
 319    result = registry.finalize_progressive_rollout_marker(
 320        connector_name=name,
 321        outcome=outcome,
 322        version=version,
 323        dry_run=dry_run,
 324    )
 325    print_json(result.to_dict())
 326    if result.success:
 327        print_success(result.message)
 328    else:
 329        exit_with_error(result.message, code=1)
 330
 331
 332# =============================================================================
 333# REGISTRY I/O - READ COMMANDS
 334# =============================================================================
 335
 336
 337@metadata_app.command(name="get")
 338def get_connector_version_metadata_cmd(
 339    name: Annotated[
 340        str,
 341        Parameter(
 342            help="Connector name (e.g., 'source-faker', 'destination-postgres')."
 343        ),
 344    ],
 345    store: Annotated[
 346        str,
 347        Parameter(
 348            help="Store target (e.g. 'coral:dev', 'coral:prod').",
 349        ),
 350    ],
 351    *,
 352    version: Annotated[
 353        str,
 354        Parameter(help="Version to read (e.g., 'latest', '1.2.3')."),
 355    ] = "latest",
 356    format: Annotated[
 357        Literal["json", "raw"],
 358        Parameter(help="Output format: 'json' for JSON, 'raw' for YAML."),
 359    ] = "json",
 360) -> None:
 361    """Read a connector version's metadata from the registry.
 362
 363    Returns the full metadata.yaml content for a connector at the specified version.
 364
 365    Requires GCS_CREDENTIALS environment variable to be set.
 366
 367    Examples:
 368        airbyte-ops registry connector-version metadata get --name source-faker --store coral:dev
 369        airbyte-ops registry connector-version metadata get --name source-faker --store coral:dev --version 6.2.38
 370        airbyte-ops registry connector-version metadata get --name source-faker --store coral:prod
 371    """
 372    registry = _resolve_store(store)
 373
 374    try:
 375        metadata = registry.get_connector_metadata(
 376            connector_name=name,
 377            version=version,
 378        )
 379    except FileNotFoundError as e:
 380        exit_with_error(str(e), code=1)
 381    except Exception as e:
 382        exit_with_error(f"Error reading metadata: {e}", code=1)
 383
 384    if format == "json":
 385        print_json(metadata)
 386    else:
 387        print(yaml.dump(metadata, default_flow_style=False))
 388
 389
 390@connector_app.command(name="list")
 391def list_connectors_cmd(
 392    store: Annotated[
 393        str,
 394        Parameter(
 395            help="Store target (e.g. 'coral:dev', 'coral:prod').",
 396        ),
 397    ],
 398    *,
 399    certified_only: Annotated[
 400        bool,
 401        Parameter(help="Include only certified connectors."),
 402    ] = False,
 403    support_level: Annotated[
 404        SupportLevel | None,
 405        Parameter(
 406            help=(
 407                "Exact support level to match "
 408                "(e.g. `certified`, `community`, `archived`)."
 409            )
 410        ),
 411    ] = None,
 412    min_support_level: Annotated[
 413        SupportLevel | None,
 414        Parameter(
 415            help=(
 416                "Minimum support level (inclusive). "
 417                "Levels from lowest to highest: `archived`, `community`, `certified`."
 418            )
 419        ),
 420    ] = None,
 421    connector_type: Annotated[
 422        ConnectorType | None,
 423        Parameter(help="Filter by connector type: `source` or `destination`."),
 424    ] = None,
 425    language: Annotated[
 426        ConnectorLanguage | None,
 427        Parameter(
 428            help=(
 429                "Filter by implementation language "
 430                "(e.g. `python`, `java`, `manifest-only`)."
 431            )
 432        ),
 433    ] = None,
 434    format: Annotated[
 435        Literal["json", "text"],
 436        Parameter(
 437            help="Output format: 'json' for JSON array, 'text' for newline-separated."
 438        ),
 439    ] = "json",
 440) -> None:
 441    """List connectors in the registry.
 442
 443    When filters are applied, reads the compiled `cloud_registry.json` index
 444    for fast lookups. Without filters, falls back to scanning individual
 445    metadata blobs (captures all connectors including OSS-only).
 446
 447    Requires GCS_CREDENTIALS environment variable to be set.
 448    """
 449    registry = _resolve_store(store)
 450
 451    # `--certified-only` is sugar for `--support-level certified`.
 452    effective_support_level = support_level
 453    if certified_only:
 454        if support_level and support_level != SupportLevel.CERTIFIED:
 455            exit_with_error(
 456                "`--certified-only` conflicts with `--support-level "
 457                f"{support_level}`. Use one or the other.",
 458                code=1,
 459            )
 460        effective_support_level = SupportLevel.CERTIFIED
 461
 462    try:
 463        connectors = registry.list_connectors(
 464            support_level=effective_support_level,
 465            min_support_level=min_support_level,
 466            connector_type=connector_type,
 467            language=language,
 468        )
 469    except Exception as e:
 470        exit_with_error(f"Error listing connectors: {e}", code=1)
 471
 472    if format == "json":
 473        print_json({"connectors": connectors, "count": len(connectors)})
 474    else:
 475        for connector in connectors:
 476            print(connector)
 477
 478
 479@connector_version_app.command(name="list")
 480def list_connector_versions_cmd(
 481    name: Annotated[
 482        str,
 483        Parameter(help="Connector name (e.g., 'source-faker')."),
 484    ],
 485    store: Annotated[
 486        str,
 487        Parameter(
 488            help="Store target (e.g. 'coral:dev', 'coral:prod').",
 489        ),
 490    ],
 491    *,
 492    format: Annotated[
 493        Literal["json", "text"],
 494        Parameter(
 495            help="Output format: 'json' for JSON array, 'text' for newline-separated."
 496        ),
 497    ] = "json",
 498) -> None:
 499    """List all versions of a connector in the registry.
 500
 501    Scans the registry bucket to find all versions of a specific connector.
 502
 503    Requires GCS_CREDENTIALS environment variable to be set.
 504    """
 505    registry = _resolve_store(store)
 506
 507    try:
 508        versions = registry.list_connector_versions(
 509            connector_name=name,
 510        )
 511    except Exception as e:
 512        exit_with_error(f"Error listing versions: {e}", code=1)
 513
 514    if format == "json":
 515        print_json({"connector": name, "versions": versions, "count": len(versions)})
 516    else:
 517        for v in versions:
 518            print(v)
 519
 520
 521@marketing_stubs_app.command(name="check")
 522def marketing_stubs_check(
 523    store: Annotated[
 524        str,
 525        Parameter(
 526            help="Store target (e.g. 'coral:dev', 'coral:prod').",
 527        ),
 528    ],
 529    *,
 530    repo_root: Annotated[
 531        Path,
 532        Parameter(
 533            help="Path to the airbyte-enterprise repository root. Defaults to current directory."
 534        ),
 535    ] = Path.cwd(),
 536) -> None:
 537    """Compare local connector_stubs.json with the version in GCS.
 538
 539    This command reads the entire local connector_stubs.json file and compares it
 540    with the version currently published in GCS.
 541
 542    Exit codes:
 543        0: Local file matches GCS (check passed)
 544        1: Differences found (check failed)
 545
 546    Output:
 547        STDOUT: JSON representation of the comparison result
 548        STDERR: Informational messages and comparison details
 549
 550    Example:
 551        airbyte-ops registry marketing-stubs check --store coral:prod --repo-root /path/to/airbyte-enterprise
 552        airbyte-ops registry marketing-stubs check --store coral:dev
 553    """
 554    registry = _resolve_store(store)
 555
 556    try:
 557        result = registry.marketing_stubs_check(repo_root=repo_root)
 558    except FileNotFoundError as e:
 559        exit_with_error(str(e))
 560    except ValueError as e:
 561        exit_with_error(str(e))
 562
 563    error_console.print(
 564        f"Comparing local {CONNECTOR_STUBS_FILE} with {result.get('bucket', '')}/{result.get('path', '')}"
 565    )
 566
 567    differences = result.get("differences", [])
 568    if differences:
 569        error_console.print(
 570            f"[yellow]Warning:[/yellow] {len(differences)} difference(s) found:"
 571        )
 572        for diff in differences:
 573            error_console.print(f"  {diff['id']}: {diff['status']}")
 574        print_json(result)
 575        sys.exit(1)
 576
 577    error_console.print(
 578        f"[green]Local file is in sync with GCS ({result.get('local_count', 0)} stubs)[/green]"
 579    )
 580    print_json(result)
 581
 582
 583@marketing_stubs_app.command(name="sync")
 584def marketing_stubs_sync(
 585    store: Annotated[
 586        str,
 587        Parameter(
 588            help="Store target (e.g. 'coral:dev', 'coral:prod').",
 589        ),
 590    ],
 591    *,
 592    repo_root: Annotated[
 593        Path,
 594        Parameter(
 595            help="Path to the airbyte-enterprise repository root. Defaults to current directory."
 596        ),
 597    ] = Path.cwd(),
 598    dry_run: Annotated[
 599        bool,
 600        Parameter(help="Show what would be uploaded without making changes."),
 601    ] = False,
 602) -> None:
 603    """Sync local connector_stubs.json to GCS.
 604
 605    This command uploads the entire local connector_stubs.json file to GCS,
 606    replacing the existing file. Use this after merging changes to master
 607    in the airbyte-enterprise repository.
 608
 609    Exit codes:
 610        0: Sync successful (or dry-run completed)
 611        1: Error (file not found, validation failed, etc.)
 612
 613    Output:
 614        STDOUT: JSON representation of the sync result
 615        STDERR: Informational messages and status updates
 616
 617    Example:
 618        airbyte-ops registry marketing-stubs sync --store coral:prod --repo-root /path/to/airbyte-enterprise
 619        airbyte-ops registry marketing-stubs sync --store coral:dev
 620        airbyte-ops registry marketing-stubs sync --store coral:dev --dry-run
 621    """
 622    registry = _resolve_store(store)
 623
 624    try:
 625        result = registry.marketing_stubs_sync(
 626            repo_root=repo_root,
 627            dry_run=dry_run,
 628        )
 629    except FileNotFoundError as e:
 630        exit_with_error(str(e))
 631    except ValueError as e:
 632        exit_with_error(str(e))
 633
 634    bucket_name = result.get("bucket", "")
 635    path = result.get("path", "")
 636    stub_count = result.get("stub_count", 0)
 637
 638    if dry_run:
 639        error_console.print(
 640            f"[DRY RUN] Would upload {stub_count} stubs to {bucket_name}/{path}"
 641        )
 642    else:
 643        error_console.print(
 644            f"[green]Synced {stub_count} stubs to {bucket_name}/{path}[/green]"
 645        )
 646    print_json(result)
 647
 648
 649# =============================================================================
 650# REGISTRY REBUILD COMMANDS
 651# =============================================================================
 652
 653
 654@store_app.command(name="mirror")
 655def mirror_cmd(
 656    local: Annotated[
 657        bool,
 658        Parameter(
 659            help="Write output to a local directory. Mutually exclusive with --gcs-bucket and --s3-bucket.",
 660            negative="",
 661        ),
 662    ] = False,
 663    gcs_bucket: Annotated[
 664        str | None,
 665        Parameter(
 666            help="Write output to a GCS bucket. Must not be the prod bucket. "
 667            "Mutually exclusive with --local and --s3-bucket.",
 668        ),
 669    ] = None,
 670    s3_bucket: Annotated[
 671        str | None,
 672        Parameter(
 673            help="Write output to an S3 bucket. "
 674            "Mutually exclusive with --local and --gcs-bucket.",
 675        ),
 676    ] = None,
 677    output_path_root: Annotated[
 678        str | None,
 679        Parameter(
 680            help="Root path/prefix for the output. For --local, this is a directory path "
 681            "(defaults to a new temp dir if omitted). For --gcs-bucket/--s3-bucket, "
 682            "this prefix is prepended to all blob paths.",
 683        ),
 684    ] = None,
 685    dry_run: Annotated[
 686        bool,
 687        Parameter(help="Show what would be rebuilt without writing any files."),
 688    ] = False,
 689    source_store: Annotated[
 690        str,
 691        Parameter(
 692            help="Source store to read from (e.g. 'coral:prod', 'coral:dev').",
 693        ),
 694    ] = "coral:prod",
 695    connector_name: Annotated[
 696        tuple[str, ...] | None,
 697        Parameter(
 698            help="Only rebuild these connectors (by name). "
 699            "Can be specified multiple times, e.g. "
 700            "--connector-name source-faker --connector-name destination-bigquery.",
 701        ),
 702    ] = None,
 703) -> None:
 704    """Create a mirror of the connector registry from the source store.
 705
 706    Reads all connector metadata from the source store and copies it
 707    to the specified output target. Supports local filesystem, GCS, and S3
 708    as output targets via fsspec.
 709
 710    Output targets are mutually exclusive: specify exactly one of
 711    --local, --gcs-bucket, or --s3-bucket.
 712
 713    The production bucket is categorically disallowed as an output target.
 714
 715    To clean up legacy artifacts (e.g. disabled strict-encrypt connectors)
 716    after mirroring, run compile with `--with-legacy-migration v1`::
 717
 718        airbyte-ops registry store compile --store coral:dev/my-prefix \\
 719            --with-legacy-migration v1
 720
 721    Examples:
 722        airbyte-ops registry store mirror --local
 723        airbyte-ops registry store mirror --local --source-store coral:dev
 724        airbyte-ops registry store mirror --gcs-bucket dev-airbyte-cloud-connector-metadata-service-2 \\
 725            --output-path-root test-run-123
 726        airbyte-ops registry store mirror --s3-bucket my-test-bucket --dry-run
 727    """
 728    # Validate mutually exclusive output targets
 729    targets = [local, gcs_bucket is not None, s3_bucket is not None]
 730    if sum(targets) != 1:
 731        exit_with_error(
 732            "Specify exactly one output target: --local, --gcs-bucket, or --s3-bucket."
 733        )
 734
 735    if local:
 736        output_mode = "local"
 737    elif gcs_bucket is not None:
 738        output_mode = "gcs"
 739    else:
 740        output_mode = "s3"
 741
 742    registry = _resolve_store(source_store)
 743
 744    result = registry.mirror(
 745        output_mode=output_mode,
 746        output_path_root=output_path_root,
 747        gcs_bucket=gcs_bucket,
 748        s3_bucket=s3_bucket,
 749        dry_run=dry_run,
 750        connector_name=list(connector_name) if connector_name else None,
 751    )
 752
 753    print_json(
 754        {
 755            "status": result.status,
 756            "source_bucket": result.source_bucket,
 757            "output_mode": result.output_mode,
 758            "output_root": result.output_root,
 759            "connectors_processed": result.connectors_processed,
 760            "blobs_copied": result.blobs_copied,
 761            "blobs_skipped": result.blobs_skipped,
 762            "error_count": len(result.errors),
 763            "dry_run": result.dry_run,
 764        }
 765    )
 766
 767    if result.errors:
 768        for err in result.errors[:10]:
 769            print_error(err)
 770        if len(result.errors) > 10:
 771            print_error(f"... and {len(result.errors) - 10} more errors")
 772
 773    if result.status == "success":
 774        print_success(result.summary())
 775    elif result.status == "dry-run":
 776        print_success(f"[DRY RUN] {result.summary()}")
 777    else:
 778        error_console.print(f"[yellow]{result.summary()}[/yellow]")
 779
 780
 781# =============================================================================
 782# VERSION YANK COMMANDS
 783# =============================================================================
 784
 785
 786@connector_version_app.command(name="yank")
 787def yank_cmd(
 788    name: Annotated[
 789        str,
 790        Parameter(help="Connector name (e.g., 'source-faker')."),
 791    ],
 792    version: Annotated[
 793        str,
 794        Parameter(help="Version to yank (e.g., '1.2.3')."),
 795    ],
 796    store: Annotated[
 797        str,
 798        Parameter(
 799            help="Store target (e.g. 'coral:dev', 'coral:prod').",
 800        ),
 801    ],
 802    *,
 803    reason: Annotated[
 804        str,
 805        Parameter(help="Reason for yanking this version."),
 806    ] = "",
 807    approval_url: Annotated[
 808        str,
 809        Parameter(help="Approval evidence URL to record in the yank marker."),
 810    ] = "",
 811    dry_run: Annotated[
 812        bool,
 813        Parameter(help="Show what would be done without making changes."),
 814    ] = False,
 815) -> None:
 816    """Mark a connector version as yanked.
 817
 818    Writes a version-yank.yml marker file to the version's directory in GCS.
 819    Yanked versions are excluded when determining the latest version of a
 820    connector.
 821
 822    Requires GCS_CREDENTIALS environment variable to be set.
 823
 824    Examples:
 825        airbyte-ops registry connector-version yank --name source-faker --version 1.2.3 --store coral:dev
 826        airbyte-ops registry connector-version yank --name source-faker --version 1.2.3 --store coral:dev --reason "Critical bug"
 827        airbyte-ops registry connector-version yank --name source-faker --version 1.2.3 --store coral:prod
 828    """
 829    registry = _resolve_store(store)
 830
 831    result = registry.yank(
 832        connector_name=name,
 833        version=version,
 834        reason=reason,
 835        approval_url=approval_url,
 836        dry_run=dry_run,
 837    )
 838
 839    print_json(result.to_dict())
 840
 841    if result.success:
 842        print_success(result.message)
 843        if not dry_run:
 844            error_console.print(
 845                "\n[yellow]Note:[/yellow] The registry indexes are now stale. "
 846                "To update them, run:\n\n"
 847                f"    airbyte-ops registry store compile --store {store}\n\n"
 848                "Or wait for the next scheduled compile operation."
 849            )
 850    else:
 851        exit_with_error(result.message, code=1)
 852
 853
 854@connector_version_app.command(name="unyank")
 855def unyank_cmd(
 856    name: Annotated[
 857        str,
 858        Parameter(help="Connector name (e.g., 'source-faker')."),
 859    ],
 860    version: Annotated[
 861        str,
 862        Parameter(help="Version to unyank (e.g., '1.2.3')."),
 863    ],
 864    store: Annotated[
 865        str,
 866        Parameter(
 867            help="Store target (e.g. 'coral:dev', 'coral:prod').",
 868        ),
 869    ],
 870    *,
 871    dry_run: Annotated[
 872        bool,
 873        Parameter(help="Show what would be done without making changes."),
 874    ] = False,
 875) -> None:
 876    """Rename the active yank marker to a dated audit marker.
 877
 878    Moves `version-yank.yml` to `version-unyanked-yyyymmdd.yml`, making the
 879    version eligible again when determining the latest version.
 880
 881    Requires GCS_CREDENTIALS environment variable to be set.
 882
 883    Examples:
 884        airbyte-ops registry connector-version unyank --name source-faker --version 1.2.3 --store coral:dev
 885        airbyte-ops registry connector-version unyank --name source-faker --version 1.2.3 --store coral:prod
 886    """
 887    registry = _resolve_store(store)
 888
 889    result = registry.unyank(
 890        connector_name=name,
 891        version=version,
 892        dry_run=dry_run,
 893    )
 894
 895    print_json(result.to_dict())
 896
 897    if result.success:
 898        print_success(result.message)
 899        if not dry_run:
 900            error_console.print(
 901                "\n[yellow]Note:[/yellow] The registry indexes are now stale. "
 902                "To update them, run:\n\n"
 903                f"    airbyte-ops registry store compile --store {store}\n\n"
 904                "Or wait for the next scheduled compile operation."
 905            )
 906    else:
 907        exit_with_error(result.message, code=1)
 908
 909
 910# =============================================================================
 911# ARTIFACT GENERATION COMMANDS
 912# =============================================================================
 913
 914
 915@artifacts_app.command(name="generate")
 916def generate_version_artifacts_cmd(
 917    metadata_file: Annotated[
 918        Path,
 919        Parameter(help="Path to the connector's metadata.yaml file."),
 920    ],
 921    docker_image: Annotated[
 922        str,
 923        Parameter(
 924            help="Docker image to run spec against (e.g., 'airbyte/source-faker:6.2.38')."
 925        ),
 926    ],
 927    output_dir: Annotated[
 928        Path | None,
 929        Parameter(
 930            help="Directory to write artifacts to. If not specified, a temp directory is created."
 931        ),
 932    ] = None,
 933    repo_root: Annotated[
 934        Path | None,
 935        Parameter(
 936            help=(
 937                "Root of the Airbyte repo checkout (for resolving doc.md). "
 938                "If not specified, inferred by walking up from metadata-file."
 939            ),
 940        ),
 941    ] = None,
 942    dry_run: Annotated[
 943        bool,
 944        Parameter(
 945            help="Show what would be generated without running docker or writing files."
 946        ),
 947    ] = False,
 948    with_validate: Annotated[
 949        bool,
 950        Parameter(
 951            help=(
 952                "Run metadata validators after generation (default: enabled). "
 953                "Use --no-validate to skip."
 954            ),
 955            negative="--no-validate",
 956        ),
 957    ] = True,
 958    with_sbom: Annotated[
 959        bool,
 960        Parameter(
 961            help=(
 962                "Generate spdx.json (SBOM) for connectors "
 963                "(default: enabled). Use --no-sbom to skip."
 964            ),
 965            negative="--no-sbom",
 966        ),
 967    ] = True,
 968    with_dependency_dump: Annotated[
 969        bool,
 970        Parameter(
 971            help=(
 972                "Generate dependencies.json for Python connectors "
 973                "(default: enabled). Use --no-dependency-dump to skip."
 974            ),
 975            negative="--no-dependency-dump",
 976        ),
 977    ] = True,
 978) -> None:
 979    """Generate version artifacts for a connector locally.
 980
 981    Runs the connector's docker image with DEPLOYMENT_MODE=cloud and
 982    DEPLOYMENT_MODE=oss to obtain both spec variants, then generates
 983    the registry entries (cloud.json, oss.json) by applying
 984    registryOverrides from the metadata.
 985
 986    The generated metadata.yaml is enriched with git commit info, SBOM URL,
 987    and (when applicable) components SHA before writing.  Validation is run
 988    after generation by default; pass `--no-validate` to skip.
 989
 990    This is a local-only operation -- no files are uploaded to GCS.
 991    Use `artifacts publish` to upload generated artifacts to GCS.
 992
 993    Examples:
 994        airbyte-ops registry connector-version artifacts generate \\
 995            --metadata-file path/to/metadata.yaml \\
 996            --docker-image airbyte/source-faker:6.2.38
 997
 998        airbyte-ops registry connector-version artifacts generate \\
 999            --metadata-file path/to/metadata.yaml \\
1000            --docker-image airbyte/source-faker:6.2.38 \\
1001            --output-dir ./artifacts --with-validate
1002    """
1003    result = generate_version_artifacts(
1004        metadata_file=metadata_file,
1005        docker_image=docker_image,
1006        output_dir=output_dir,
1007        repo_root=repo_root,
1008        dry_run=dry_run,
1009        with_validate=with_validate,
1010        with_dependency_dump=with_dependency_dump,
1011        with_sbom=with_sbom,
1012    )
1013
1014    print_json(result.to_dict())
1015
1016    if result.success:
1017        print_success(
1018            f"Generated {len(result.artifacts_written)} artifacts to {result.output_dir}"
1019        )
1020    else:
1021        all_errors = result.errors + result.validation_errors
1022        exit_with_error(
1023            f"Generation completed with {len(all_errors)} error(s): "
1024            + "; ".join(all_errors),
1025            code=1,
1026        )
1027
1028
1029@artifacts_app.command(name="publish")
1030def publish_version_artifacts_cmd(
1031    name: Annotated[
1032        str,
1033        Parameter(help="Connector name (e.g., 'source-faker')."),
1034    ],
1035    version: Annotated[
1036        str,
1037        Parameter(help="Version to publish artifacts for (e.g., '1.2.3')."),
1038    ],
1039    artifacts_dir: Annotated[
1040        Path,
1041        Parameter(
1042            help="Directory containing generated artifacts to publish (from 'artifacts generate')."
1043        ),
1044    ],
1045    store: Annotated[
1046        str,
1047        Parameter(
1048            help="Store target (e.g. 'coral:dev', 'coral:prod', 'coral:dev/prefix').",
1049        ),
1050    ],
1051    *,
1052    dry_run: Annotated[
1053        bool,
1054        Parameter(help="Show what would be published without writing to GCS."),
1055    ] = False,
1056    with_validate: Annotated[
1057        bool,
1058        Parameter(
1059            help=(
1060                "Validate metadata before uploading (default: enabled). "
1061                "Use --no-validate to skip."
1062            ),
1063            negative="--no-validate",
1064        ),
1065    ] = True,
1066) -> None:
1067    """Publish version artifacts to GCS using fsspec rsync.
1068
1069    Uploads locally generated artifacts (from `artifacts generate`) to the
1070    versioned path in GCS.  By default, metadata is validated before upload;
1071    pass `--no-validate` to skip.
1072
1073    Uses `--store` to select the destination store and environment:
1074
1075    * `coral:dev`              → coral dev bucket at root
1076    * `coral:prod`             → coral prod bucket at root
1077    * `coral:dev/aj-test100`   → coral dev bucket under `aj-test100/` prefix
1078
1079    Requires GCS_CREDENTIALS environment variable to be set.
1080
1081    Examples:
1082        airbyte-ops registry connector-version artifacts publish \\
1083            --name source-faker --version 6.2.38 \\
1084            --artifacts-dir ./artifacts --store coral:dev --with-validate
1085
1086        airbyte-ops registry connector-version artifacts publish \\
1087            --name source-faker --version 6.2.38 \\
1088            --artifacts-dir ./artifacts --store coral:prod
1089
1090        airbyte-ops registry connector-version artifacts publish \\
1091            --name source-faker --version 6.2.38 \\
1092            --artifacts-dir ./artifacts --store coral:dev/aj-test100
1093    """
1094    registry = _resolve_store(store)
1095
1096    result = registry.publish_version_artifacts(
1097        connector_name=name,
1098        version=version,
1099        artifacts_dir=artifacts_dir,
1100        dry_run=dry_run,
1101        with_validate=with_validate,
1102    )
1103
1104    print_json(
1105        {
1106            "status": result.status,
1107            "connector_name": result.connector_name,
1108            "version": result.version,
1109            "target": result.target,
1110            "gcs_destination": result.gcs_destination,
1111            "files_uploaded": result.files_uploaded,
1112            "errors": result.errors,
1113            "validation_errors": result.validation_errors,
1114            "dry_run": result.dry_run,
1115        }
1116    )
1117
1118    if result.success:
1119        print_success(
1120            f"Published {len(result.files_uploaded)} artifacts for "
1121            f"{result.connector_name}@{result.version}{result.gcs_destination}"
1122        )
1123    else:
1124        all_errors = result.errors + result.validation_errors
1125        exit_with_error(
1126            f"Publish completed with {len(all_errors)} error(s): "
1127            + "; ".join(all_errors),
1128            code=1,
1129        )
1130
1131
1132# =============================================================================
1133# COMPILE COMMAND
1134# =============================================================================
1135
1136
1137@store_app.command(name="compile")
1138def compile_cmd(
1139    store: Annotated[
1140        str,
1141        Parameter(
1142            help="Store target (e.g. 'coral:dev', 'coral:prod', 'coral:dev/prefix').",
1143        ),
1144    ],
1145    *,
1146    connector_name: Annotated[
1147        tuple[str, ...] | None,
1148        Parameter(
1149            help="Only compile these connectors (can be repeated).",
1150        ),
1151    ] = None,
1152    dry_run: Annotated[
1153        bool,
1154        Parameter(help="Show what would be done without writing."),
1155    ] = False,
1156    with_secrets_mask: Annotated[
1157        bool,
1158        Parameter(
1159            help=(
1160                "Also regenerate specs_secrets_mask.yaml by scanning all "
1161                "connector specs for airbyte_secret properties."
1162            ),
1163        ),
1164    ] = False,
1165    with_legacy_migration: Annotated[
1166        str | None,
1167        Parameter(
1168            help=(
1169                "Run a one-time legacy migration step during compile. "
1170                "Currently supported: 'v1' — delete cloud.json / oss.json "
1171                "files for connectors whose registryOverrides.cloud.enabled "
1172                "or registryOverrides.oss.enabled is false. This cleans up "
1173                "artifacts produced by the legacy pipeline that did not "
1174                "respect the enabled flag."
1175            ),
1176        ),
1177    ] = None,
1178    with_metrics: Annotated[
1179        bool,
1180        Parameter(
1181            help=(
1182                "Inject latest connector quality metrics from the analytics "
1183                "JSONL export into generated.metrics."
1184            ),
1185            negative="--no-metrics",
1186        ),
1187    ] = True,
1188    force: Annotated[
1189        bool,
1190        Parameter(
1191            help=(
1192                "Force resync of latest/ directories even if version markers are current. "
1193                "Useful when metadata changes without a version bump."
1194            ),
1195        ),
1196    ] = False,
1197) -> None:
1198    """Compile the registry: sync latest/ dirs, write global and per-connector indexes.
1199
1200    Scans all version directories in the target store, determines the latest GA
1201    semver per connector (excluding yanked and pre-release versions), ensures
1202    each `latest/` directory matches the computed latest, and writes:
1203
1204    * `registries/v0/cloud_registry.json` -- global cloud registry index
1205    * `registries/v0/oss_registry.json`   -- global OSS registry index
1206    * `metadata/airbyte/<connector>/versions.json` -- per-connector version index
1207
1208    With `--with-secrets-mask`, also regenerates:
1209
1210    * `registries/v0/specs_secrets_mask.yaml` -- properties marked as secrets
1211
1212    With `--with-legacy-migration=v1`, deletes `cloud.json` / `oss.json`
1213    files for connectors whose `registryOverrides.cloud.enabled` or
1214    `registryOverrides.oss.enabled` is `false`.
1215
1216    By default, injects connector quality metrics from the latest analytics
1217    JSONL export into `generated.metrics`. Use `--no-metrics` for offline
1218    scenarios.
1219
1220    With `--force`, resyncs all latest/ directories even if the version marker
1221    matches the computed latest version.
1222
1223    Uses efficient glob patterns for scanning (no file downloads during discovery).
1224
1225    Requires GCS_CREDENTIALS environment variable to be set.
1226
1227    Examples:
1228        airbyte-ops registry store compile --store coral:dev --dry-run
1229
1230        airbyte-ops registry store compile --store coral:dev/aj-test100 \\
1231            --connector-name source-faker --connector-name destination-bigquery
1232
1233        airbyte-ops registry store compile --store coral:prod --with-secrets-mask
1234
1235        airbyte-ops registry store compile --store coral:dev \\
1236            --with-legacy-migration v1
1237    """
1238    registry = _resolve_store(store)
1239
1240    result = registry.compile(
1241        connector_name=list(connector_name) if connector_name else None,
1242        dry_run=dry_run,
1243        with_secrets_mask=with_secrets_mask,
1244        with_legacy_migration=with_legacy_migration,
1245        with_metrics=with_metrics,
1246        force=force,
1247    )
1248
1249    print_json(
1250        {
1251            "status": result.status,
1252            "target": result.target,
1253            "connectors_scanned": result.connectors_scanned,
1254            "versions_found": result.versions_found,
1255            "yanked_versions": result.yanked_versions,
1256            "latest_updated": result.latest_updated,
1257            "latest_already_current": result.latest_already_current,
1258            "cloud_registry_entries": result.cloud_registry_entries,
1259            "oss_registry_entries": result.oss_registry_entries,
1260            "composite_registry_entries": result.composite_registry_entries,
1261            "metrics_connector_count": result.metrics_connector_count,
1262            "metrics_registry_entries": result.metrics_registry_entries,
1263            "metrics_source": result.metrics_source,
1264            "metrics_error": result.metrics_error,
1265            "version_indexes_written": result.version_indexes_written,
1266            "specs_secrets_mask_properties": result.specs_secrets_mask_properties,
1267            "errors": result.errors,
1268            "dry_run": result.dry_run,
1269        }
1270    )
1271
1272    if result.status == "success" or result.status == "dry-run":
1273        print_success(result.summary())
1274    else:
1275        exit_with_error(
1276            f"Compile completed with {len(result.errors)} error(s): "
1277            + "; ".join(result.errors),
1278            code=1,
1279        )
1280
1281
1282# =============================================================================
1283# DELETE-DEV-LATEST COMMAND
1284# =============================================================================
1285
1286
1287@store_app.command(name="delete-dev-latest")
1288def delete_dev_latest_cmd(
1289    store: Annotated[
1290        str,
1291        Parameter(
1292            help="Store target (must begin with 'coral:dev').",
1293        ),
1294    ],
1295    *,
1296    connector_name: Annotated[
1297        tuple[str, ...] | None,
1298        Parameter(
1299            help="Only delete latest/ for these connectors (can be repeated).",
1300        ),
1301    ] = None,
1302    dry_run: Annotated[
1303        bool,
1304        Parameter(help="Show what would be done without deleting."),
1305    ] = False,
1306) -> None:
1307    """Delete all latest/ directories from a dev registry store.
1308
1309    Discovers every connector that has a `latest/` directory and
1310    deletes each one in parallel using a thread pool.
1311
1312    This is useful before a full re-compile to prove that latest/
1313    directories can be correctly regenerated from versioned data.
1314
1315    Only dev stores are allowed (store must begin with 'coral:dev').
1316
1317    Requires GCS_CREDENTIALS environment variable to be set.
1318
1319    Examples:
1320        airbyte-ops registry store delete-dev-latest --store coral:dev --dry-run
1321
1322        airbyte-ops registry store delete-dev-latest --store coral:dev/aj-test100
1323
1324        airbyte-ops registry store delete-dev-latest --store coral:dev \\
1325            --connector-name source-faker --connector-name destination-bigquery
1326    """
1327    if not store.startswith("coral:dev"):
1328        exit_with_error(
1329            "delete-dev-latest only supports dev stores "
1330            f"(store must begin with 'coral:dev', got '{store}').",
1331            code=1,
1332        )
1333
1334    registry = _resolve_store(store)
1335
1336    result = registry.delete_dev_latest(
1337        connector_name=list(connector_name) if connector_name else None,
1338        dry_run=dry_run,
1339    )
1340
1341    print_json(
1342        {
1343            "status": result.status,
1344            "target": result.target,
1345            "connectors_found": result.connectors_found,
1346            "latest_dirs_deleted": result.latest_dirs_deleted,
1347            "errors": result.errors,
1348            "dry_run": result.dry_run,
1349        }
1350    )
1351
1352    if result.status in ("success", "dry-run"):
1353        print_success(result.summary())
1354    else:
1355        exit_with_error(
1356            f"Delete completed with {len(result.errors)} error(s): "
1357            + "; ".join(result.errors[:5]),
1358            code=1,
1359        )
1360
1361
1362# =============================================================================
1363# STORE COMPARE COMMAND
1364# =============================================================================
1365
1366
1367@store_app.command(name="compare")
1368def compare_cmd(
1369    store: Annotated[
1370        str,
1371        Parameter(
1372            help="Store target being evaluated (e.g. 'coral:dev/20260306-mirror-compile').",
1373        ),
1374    ],
1375    reference_store: Annotated[
1376        str,
1377        Parameter(
1378            help="Known-good reference store to compare against.",
1379        ),
1380    ],
1381    *,
1382    connector_name: Annotated[
1383        tuple[str, ...] | None,
1384        Parameter(
1385            help="Only compare these connectors (can be repeated).",
1386        ),
1387    ] = None,
1388    with_artifacts: Annotated[
1389        bool,
1390        Parameter(
1391            help="Compare per-connector artifact files "
1392            "(metadata.yaml, cloud.json, oss.json, spec.json).",
1393            negative="--no-artifacts",
1394        ),
1395    ] = True,
1396    with_indexes: Annotated[
1397        bool,
1398        Parameter(
1399            help="Compare global registry index files "
1400            "(cloud_registry.json, oss_registry.json).",
1401            negative="--no-indexes",
1402        ),
1403    ] = True,
1404) -> None:
1405    """Compare a store against a reference store and report differences.
1406
1407    Evaluates the `--store` target against `--reference-store` and reports
1408    per-connector artifact diffs and global index diffs.
1409
1410    Requires GCS_CREDENTIALS environment variable to be set.
1411
1412    Examples:
1413        airbyte-ops registry store compare --store coral:dev/20260306-mirror \\
1414            --reference-store coral:prod
1415
1416        airbyte-ops registry store compare --store coral:dev/my-test \\
1417            --connector-name source-faker --no-indexes
1418
1419        airbyte-ops registry store compare --store coral:dev/my-test \\
1420            --no-artifacts
1421    """
1422    store_target = _resolve_store(store)
1423    ref_target = _resolve_store(reference_store)
1424
1425    store_prefix = f"{store_target.prefix}/" if store_target.prefix else ""
1426    ref_prefix = f"{ref_target.prefix}/" if ref_target.prefix else ""
1427
1428    result = compare_stores(
1429        store_bucket=store_target.bucket_name,
1430        store_prefix=store_prefix,
1431        reference_bucket=ref_target.bucket_name,
1432        reference_prefix=ref_prefix,
1433        connector_name=list(connector_name) if connector_name else None,
1434        with_artifacts=with_artifacts,
1435        with_indexes=with_indexes,
1436    )
1437
1438    print_json(result.to_dict())
1439
1440    if result.status == "match":
1441        print_success(result.summary())
1442    elif result.status == "differences-found":
1443        error_console.print(f"[yellow]{result.summary()}[/yellow]")
1444
1445        # Print a concise per-connector diff summary
1446        for diff in result.connector_diffs:
1447            if diff.status in ("only_in_store", "only_in_reference"):
1448                error_console.print(f"  {diff.connector}: {diff.status}")
1449            else:
1450                for ad in diff.artifact_diffs:
1451                    error_console.print(
1452                        f"  {diff.connector}/{ad.file}: {ad.status}"
1453                        + (f" ({ad.details})" if ad.details else "")
1454                    )
1455
1456        for idx_diff in result.index_diffs:
1457            if idx_diff.status != "match":
1458                error_console.print(
1459                    f"  [index] {idx_diff.file}: {idx_diff.status}"
1460                    + (
1461                        f" (store={idx_diff.entry_count_store},"
1462                        f" ref={idx_diff.entry_count_reference})"
1463                        if idx_diff.entry_count_store or idx_diff.entry_count_reference
1464                        else ""
1465                    )
1466                )
1467
1468        sys.exit(1)
1469    else:
1470        exit_with_error(
1471            f"Compare completed with {len(result.errors)} error(s): "
1472            + "; ".join(result.errors[:5]),
1473            code=1,
1474        )