airbyte_ops_mcp.cli.local
CLI commands for local Airbyte monorepo operations.
Commands:
airbyte-ops local connector list - List connectors in the monorepo airbyte-ops local connector info - Get metadata for a single connector airbyte-ops local connector get-version - Get connector version (current or next) airbyte-ops local connector bump-version - Bump connector version airbyte-ops local connector rollouts enable - Enable autopilot rollouts airbyte-ops local connector rollouts disable - Disable autopilot rollouts airbyte-ops local connector qa - Run QA checks on a connector airbyte-ops local connector qa-docs-generate - Generate QA checks documentation airbyte-ops local connector changelog check - Check changelog entries for issues airbyte-ops local connector changelog fix - Fix changelog entry dates airbyte-ops local connector bump-deps - Update Poetry-managed dependencies airbyte-ops local connector marketing-stub check - Validate marketing stub entries airbyte-ops local connector marketing-stub sync - Sync stub from connector metadata airbyte-ops local connector release-block add - Add a connector release block marker airbyte-ops local connector release-block clear - Clear a connector release block marker airbyte-ops local connector release-block list - List connector release block markers
CLI reference
The commands below are regenerated by poe docs-generate via cyclopts's
programmatic docs API; see docs/generate_cli.py.
airbyte-ops local COMMAND
Local Airbyte monorepo operations.
Commands:
connector: Connector operations in the monorepo.
airbyte-ops local connector
Connector operations in the monorepo.
airbyte-ops local connector list
airbyte-ops local connector list REPO-PATH [ARGS]
List connectors in the Airbyte monorepo with filtering options.
Parameters:
REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]CERTIFIED-ONLY, --certified-only, --no-certified-only: Include only certified connectors. [default: False]MODIFIED-ONLY, --modified-only, --no-modified-only: Include only modified connectors (requires PR context). [default: False]LOCAL-CDK, --local-cdk, --no-local-cdk: Include connectors using local CDK reference. When combined with --modified-only, adds local-CDK connectors to the modified set. [default: False]LANGUAGE, --language, --empty-language: Languages to include (python, java, low-code, manifest-only).EXCLUDE-LANGUAGE, --exclude-language, --empty-exclude-language: Languages to exclude.CONNECTOR-TYPE, --connector-type: Connector types to include (source, destination). Accepts CSV or newline-delimited values.MIN-SUPPORT-LEVEL, --min-support-level: Minimum support level (inclusive). Accepts integer (100, 200, 300) or keyword (archived, community, certified).MAX-SUPPORT-LEVEL, --max-support-level: Maximum support level (inclusive). Accepts integer (100, 200, 300) or keyword (archived, community, certified).AUTOPILOT-ENABLED, --autopilot-enabled, --no-autopilot-enabled: Filter by autopilot progressive rollout status. Pass--autopilot-enabled=trueto keep only connectors that have autopilot rollouts enabled (defaultRolloutMode: autopilotandenableProgressiveRollout: true), or--autopilot-enabled=falsefor those that do not. When omitted, connectors are not filtered on rollout status.PR, --pr: PR number or GitHub URL for modification detection.GH-TOKEN, --gh-token: GitHub API token. When provided together with --pr, the GitHub API is used to detect modified files instead of local git diff (avoids shallow-clone issues).EXCLUDE-CONNECTORS, --exclude-connectors, --empty-exclude-connectors: Connectors to exclude from results. Accepts CSV or newline-delimited values. Can be specified multiple times.FORCE-INCLUDE-CONNECTORS, --force-include-connectors, --empty-force-include-connectors: Connectors to force-include regardless of other filters. Accepts CSV or newline-delimited values. Can be specified multiple times.CONNECTORS-FILTER, --connectors-filter, --empty-connectors-filter: Intersect results with this explicit set of connector names. Only connectors present in both the filtered results and this set are returned. Useful for composing multiple filter passes (e.g. combining separate source and destination lists). Accepts CSV or newline-delimited values. Can be specified multiple times.OUTPUT-FORMAT, --output-format: Output format: "csv" (comma-separated), "lines" (one connector per line), "json-gh-matrix" (GitHub Actions matrix JSON). [choices: csv, lines, json-gh-matrix] [default: lines]UNPUBLISHED, --unpublished, --no-unpublished: Filter to only connectors whose local dockerImageTag has not been published to the GCS registry. Requires --store. [default: False]STORE, --store: Store target for unpublished check (e.g. 'coral:prod', 'coral:dev'). Required when --unpublished is set.ASSERT-NONE, --assert-none, --no-assert-none: Exit with non-zero status if any connectors match the filters. Useful for audit/CI checks (e.g. --unpublished --assert-none). [default: False]SORT-BY, --sort-by: Sort connectors by the given key. Only 'name' is supported for now. [choices: name] [default: name]SORT-DIRECTION, --sort-direction: Sort direction: 'asc' (ascending, default) or 'desc' (descending). [choices: asc, desc] [default: asc]LIMIT, --limit: Maximum number of connectors to return. Applied after sorting. Useful for batched processing.OFFSET, --offset: Number of connectors to skip from the start of the (sorted) list. Applied after sorting, before --limit. Useful for batched processing (e.g. --offset=200 --limit=200 for batch 2). [default: 0]
airbyte-ops local connector info
airbyte-ops local connector info CONNECTOR-NAME [ARGS]
Get metadata for a single connector.
Prints JSON output with connector metadata. When running in GitHub Actions (CI env var set), also writes each field to GitHub step outputs.
Parameters:
CONNECTOR-NAME, --connector-name: Name of the connector (e.g., source-github). [required]REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.DPATH, --dpath: Evaluate this dpath expression against the parsed metadata.yaml object and print only that value (e.g., data/dockerImageTag).
airbyte-ops local connector get-version
airbyte-ops local connector get-version NAME REPO-PATH [ARGS]
Get the current or next version for a connector.
By default, prints the current version from metadata.yaml.
This is analogous to local connector info --dpath data/dockerImageTag and
uses the same dpath evaluation internally.
With --next, computes and prints the next version. Requires either --bump-type or --prerelease to be specified.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]NEXT, --next, --no-next: Compute the next version instead of the current version. [default: False]BUMP-TYPE, --bump-type: Version bump type (requires --next). Standard: patch, minor, major. RC: patch_rc, minor_rc, major_rc, rc, promote. [choices: patch, minor, major, patch_rc, minor_rc, major_rc, rc, promote]PRERELEASE, --prerelease, --no-prerelease: Compute a prerelease (preview) tag using the repo HEAD SHA (requires --next). [default: False]
airbyte-ops local connector bump-version
airbyte-ops local connector bump-version NAME REPO-PATH [ARGS]
Bump a connector's version across all relevant files.
Updates version in metadata.yaml (always), pyproject.toml (if exists), and documentation changelog (if --changelog-message provided).
Note: --changelog-message is ignored when --no-changelog is set.
Either --bump-type or --new-version must be provided.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]BUMP-TYPE, --bump-type: Version bump type. Standard: patch, minor, major. RC: patch_rc, minor_rc, major_rc, rc, promote. [choices: patch, minor, major, patch_rc, minor_rc, major_rc, rc, promote]NEW-VERSION, --new-version: Explicit new version (overrides --bump-type if provided).CHANGELOG-MESSAGE, --changelog-message: Message to add to changelog. Ignored if --no-changelog is set.PR-NUMBER, --pr-number: PR number for changelog entry.DRY-RUN, --dry-run, --no-dry-run: Show what would be changed without modifying files. [default: False]NO-CHANGELOG, --no-changelog, --no-no-changelog: Skip changelog updates even if --changelog-message is provided. Useful for ephemeral version bumps (e.g. pre-release artifact generation). [default: False]PROGRESSIVE-ROLLOUT-ENABLED, --progressive-rollout-enabled, --no-progressive-rollout-enabled: Explicitly setenableProgressiveRolloutin metadata.yaml. Passfalseto disable progressive rollout (e.g. for preview builds). When omitted, the automatic behaviour based on --bump-type is used.
airbyte-ops local connector rollouts
Manage a connector's progressive rollout configuration.
Commands:
disable: Disable autopilot rollouts in a connector's metadata.yaml.enable: Enable autopilot rollouts in a connector's metadata.yaml.
airbyte-ops local connector rollouts enable
airbyte-ops local connector rollouts enable NAME REPO-PATH [ARGS]
Enable autopilot rollouts in a connector's metadata.yaml.
Always sets releases.rolloutConfiguration.defaultRolloutMode: autopilot
(autopilot is the only actively-supported mode, so it is implicit here),
fills in autopilotConfig (autoStart, autoPromoteStages, strategy),
and sets enableProgressiveRollout: true — the toggle that turns automatic
progressive rollout on. Defaults only fill in autopilotConfig fields that
are absent, so existing settings are preserved.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]STRATEGY, --strategy: Autopilot pacing strategy: 'fast', 'slow', or 'default'. 'default' is a server-side alias for 'fast'; prefer the explicit 'fast'. Omitted keeps any existing value, or 'fast' when none is set. [choices: fast, slow, default]DRY-RUN, --dry-run, --no-dry-run: Show what would be changed without modifying files. [default: False]
airbyte-ops local connector rollouts disable
airbyte-ops local connector rollouts disable NAME REPO-PATH [ARGS]
Disable autopilot rollouts in a connector's metadata.yaml.
Only sets releases.rolloutConfiguration.enableProgressiveRollout: false —
the toggle that turns automatic progressive rollout off. defaultRolloutMode
and any autopilotConfig are retained (inert until re-enabled), so the
change is lossless. Connectors with no rollout config, or with the flag
already off, are a no-op.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]DRY-RUN, --dry-run, --no-dry-run: Show what would be changed without modifying files. [default: False]
airbyte-ops local connector bump-base-image
airbyte-ops local connector bump-base-image NAME REPO-PATH [ARGS]
Update a connector's base image.
Two modes:
- Default: bump to the latest stable tag within the same major version. Major version changes are treated as breaking-change boundaries.
- --force-latest: bump to the absolute latest stable tag regardless of semver.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]FORCE-LATEST, --force-latest, --no-force-latest: Bump to the absolute latest stable base image, ignoring major-version boundaries. Without this flag the bump stays within the current major version. [default: False]DRY-RUN, --dry-run, --no-dry-run: Show what would be changed without modifying files. [default: False]
airbyte-ops local connector bump-cdk
airbyte-ops local connector bump-cdk NAME REPO-PATH [ARGS]
Bump a connector's CDK dependency.
Two modes:
- Default: refresh the lock file so it resolves the newest CDK that satisfies the existing constraint. The constraint is NOT changed.
- --force-latest: rewrite the constraint to >=LATEST,
For Java connectors, updates build.gradle to the latest CDK version.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]FORCE-LATEST, --force-latest, --no-force-latest: Rewrite the CDK constraint to >=LATEST,[default: False] DRY-RUN, --dry-run, --no-dry-run: Show what would be changed without modifying files. [default: False]
airbyte-ops local connector bump-deps
airbyte-ops local connector bump-deps NAME REPO-PATH [ARGS]
Update a connector's dependencies.
For Python / low-code connectors using Poetry, this runs
poetry update --lock to refresh the lock file with the latest
versions allowed by existing constraints.
For connectors that do not use Poetry (manifest-only, Java, etc.), this is a no-op.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]DRY-RUN, --dry-run, --no-dry-run: Show what would be changed without modifying files. [default: False]
airbyte-ops local connector qa
airbyte-ops local connector qa [ARGS]
Run QA checks on connector(s).
Validates connector metadata, documentation, packaging, security, and versioning. Exit code is non-zero if any checks fail.
Parameters:
NAME, --name, --empty-name: Connector technical name(s) (e.g., source-github). Can be specified multiple times.CONNECTOR-DIRECTORY, --connector-directory: Directory containing connectors to run checks on all connectors in this directory.CHECK, --check, --empty-check: Specific check(s) to run. Can be specified multiple times.REPORT-PATH, --report-path: Path to write the JSON report file.
airbyte-ops local connector qa-docs-generate
airbyte-ops local connector qa-docs-generate OUTPUT-FILE
Generate documentation for QA checks.
Creates a markdown file documenting all available QA checks organized by category.
Parameters:
OUTPUT-FILE, --output-file: Path to write the generated documentation file. [required]
airbyte-ops local connector changelog
Changelog operations for connectors.
Commands:
add: Add a changelog entry for a connector using its current version.check: Check changelog entries for issues.fix: Fix changelog entry dates to match PR merge dates.
airbyte-ops local connector changelog check
airbyte-ops local connector changelog check [ARGS]
Check changelog entries for issues.
Validates changelog dates match PR merge dates and checks for PR number mismatches.
Parameters:
CONNECTOR-NAME, --connector-name: Connector technical name (e.g., source-github).ALL, --all, --no-all: Check all connectors in the repository. [default: False]REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.LOOKBACK-DAYS, --lookback-days: Only check entries with dates within this many days.STRICT, --strict, --no-strict: Exit with error code if any issues are found. [default: False]
airbyte-ops local connector changelog fix
airbyte-ops local connector changelog fix [ARGS]
Fix changelog entry dates to match PR merge dates.
Looks up the actual merge date for each PR referenced in the changelog and updates the date column to match.
Parameters:
CONNECTOR-NAME, --connector-name: Connector technical name (e.g., source-github).ALL, --all, --no-all: Fix all connectors in the repository. [default: False]REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.LOOKBACK-DAYS, --lookback-days: Only fix entries with dates within this many days.DRY-RUN, --dry-run, --no-dry-run: Print changes without modifying files. [default: False]
airbyte-ops local connector changelog add
airbyte-ops local connector changelog add CONNECTOR-NAME PR-NUMBER MESSAGE [ARGS]
Add a changelog entry for a connector using its current version.
Reads the version from metadata.yaml and writes a single changelog entry to the connector's documentation file. Does not modify any version files.
Parameters:
CONNECTOR-NAME, --connector-name: Connector technical name (e.g., source-github). [required]PR-NUMBER, --pr-number: PR number for the changelog entry. [required]MESSAGE, --message: Changelog entry message. [required]REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.DRY-RUN, --dry-run, --no-dry-run: Print changes without modifying files. [default: False]
airbyte-ops local connector marketing-stub
Marketing connector stub operations (local file validation and updates).
Commands:
check: Validate marketing connector stub entries.sync: Sync connector stub(s) from connector metadata.yaml file(s).
airbyte-ops local connector marketing-stub check
airbyte-ops local connector marketing-stub check [ARGS]
Validate marketing connector stub entries.
Checks that stub entries have valid required fields (id, name, url, icon) and optionally validates that the stub matches the connector's metadata.yaml.
Exit codes:
0: All checks passed 1: Validation errors found
Output:
STDOUT: JSON validation result STDERR: Informational messages
Parameters:
CONNECTOR, --connector: Connector name to check (e.g., 'source-oracle-enterprise').ALL, --all, --no-all: Check all stubs in the file. [default: False]REPO-ROOT, --repo-root: Path to the airbyte-enterprise repository root. Defaults to current directory.
airbyte-ops local connector marketing-stub sync
airbyte-ops local connector marketing-stub sync [ARGS]
Sync connector stub(s) from connector metadata.yaml file(s).
Reads the connector's metadata.yaml file and updates the corresponding entry in connector_stubs.json with the current values.
Exit codes:
0: Sync successful (or dry-run completed) 1: Error (connector not found, no metadata, etc.)
Output:
STDOUT: JSON representation of the synced stub(s) STDERR: Informational messages
Parameters:
CONNECTOR, --connector: Connector name to sync (e.g., 'source-oracle-enterprise').ALL, --all, --no-all: Sync all connectors that have metadata.yaml files. [default: False]REPO-ROOT, --repo-root: Path to the airbyte-enterprise repository root. Defaults to current directory.DRY-RUN, --dry-run, --no-dry-run: Show what would be synced without making changes. [default: False]
airbyte-ops local connector release-block
Manage release block markers for connectors.
Commands:
add: Add ablock-release.yamlmarker to prevent publishing a connector.clear: Remove theblock-release.yamlmarker to allow publishing a connector.list: List all connectors that have ablock-release.yamlmarker.
airbyte-ops local connector release-block add
airbyte-ops local connector release-block add NAME REASON [ARGS]
Add a block-release.yaml marker to prevent publishing a connector.
Creates a marker file in the connector's directory that causes the publish pipeline to skip the connector with a warning.
Parameters:
NAME, --name: Connector technical name (e.g., source-faker). [required]REASON, --reason: Human-readable reason for blocking the release. [required]REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.YANKED-VERSION, --yanked-version: Version that was yanked (for reference).BLOCKED-BY, --blocked-by: Email or identifier of the person requesting the block.
airbyte-ops local connector release-block clear
airbyte-ops local connector release-block clear NAME [ARGS]
Remove the block-release.yaml marker to allow publishing a connector.
Parameters:
NAME, --name: Connector technical name (e.g., source-faker). [required]REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.
airbyte-ops local connector release-block list
airbyte-ops local connector release-block list [ARGS]
List all connectors that have a block-release.yaml marker.
Parameters:
REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.OUTPUT-FORMAT, --output-format: Output format: "text" (human-readable), "json" (blocked connectors with block file contents), "csv" (comma-delimited connector names). [choices: text, json, csv] [default: text]
1# Copyright (c) 2025 Airbyte, Inc., all rights reserved. 2"""CLI commands for local Airbyte monorepo operations. 3 4Commands: 5 airbyte-ops local connector list - List connectors in the monorepo 6 airbyte-ops local connector info - Get metadata for a single connector 7 airbyte-ops local connector get-version - Get connector version (current or next) 8 airbyte-ops local connector bump-version - Bump connector version 9 airbyte-ops local connector rollouts enable - Enable autopilot rollouts 10 airbyte-ops local connector rollouts disable - Disable autopilot rollouts 11 airbyte-ops local connector qa - Run QA checks on a connector 12 airbyte-ops local connector qa-docs-generate - Generate QA checks documentation 13 airbyte-ops local connector changelog check - Check changelog entries for issues 14 airbyte-ops local connector changelog fix - Fix changelog entry dates 15 airbyte-ops local connector bump-deps - Update Poetry-managed dependencies 16 airbyte-ops local connector marketing-stub check - Validate marketing stub entries 17 airbyte-ops local connector marketing-stub sync - Sync stub from connector metadata 18 airbyte-ops local connector release-block add - Add a connector release block marker 19 airbyte-ops local connector release-block clear - Clear a connector release block marker 20 airbyte-ops local connector release-block list - List connector release block markers 21 22## CLI reference 23 24The commands below are regenerated by `poe docs-generate` via cyclopts's 25programmatic docs API; see `docs/generate_cli.py`. 26 27.. include:: ../../../docs/generated/cli/local.md 28 :start-line: 2 29""" 30 31from __future__ import annotations 32 33# Hide Python-level members from the pdoc page for this module; the rendered 34# docs for this CLI group come entirely from the grafted `.. include::` in 35# the module docstring above. 36__all__: list[str] = [] 37 38import json 39import os 40import subprocess 41import sys 42from pathlib import Path 43from typing import Annotated, Literal 44 45import yaml 46from airbyte_connector_models.metadata.v0.connector_metadata_definition_v0 import ( 47 ConnectorMetadataDefinitionV0DataConnectorReleasesRolloutConfigurationAutopilotConfigStrategy as AutopilotStrategy, 48) 49from cyclopts import Parameter 50from fastmcp_extensions.cli import exit_with_error, print_json 51from jinja2 import Environment, PackageLoader, select_autoescape 52from rich.console import Console 53 54from airbyte_ops_mcp.airbyte_repo.bump_base_image import ( 55 BaseImageError, 56 bump_base_image, 57) 58from airbyte_ops_mcp.airbyte_repo.bump_cdk import ( 59 CdkBumpError, 60 bump_cdk, 61) 62from airbyte_ops_mcp.airbyte_repo.bump_deps import ( 63 DepsError, 64 bump_deps, 65) 66from airbyte_ops_mcp.airbyte_repo.bump_version import ( 67 ConnectorNotFoundError, 68 InvalidVersionError, 69 VersionNotFoundError, 70 bump_connector_version, 71 calculate_new_version, 72 get_connector_doc_path, 73 get_connector_path, 74 get_current_version, 75 update_changelog, 76) 77from airbyte_ops_mcp.airbyte_repo.changelog_fix import ( 78 ChangelogCheckResult, 79 ChangelogFixResult, 80 check_all_changelogs, 81 check_changelog, 82 fix_all_changelog_dates, 83 fix_changelog_dates, 84) 85from airbyte_ops_mcp.airbyte_repo.list_connectors import ( 86 CONNECTOR_PATH_PREFIX, 87 METADATA_FILE_NAME, 88 _detect_connector_language, 89 get_connectors_with_local_cdk, 90) 91from airbyte_ops_mcp.airbyte_repo.progressive_rollout import ( 92 disable_autopilot_rollout, 93 enable_autopilot_rollout, 94 is_autopilot_rollout_enabled, 95) 96from airbyte_ops_mcp.airbyte_repo.release_block import ( 97 add_release_block, 98 clear_release_block, 99 list_release_blocks, 100) 101from airbyte_ops_mcp.cli._base import App, app 102from airbyte_ops_mcp.connector_metadata import ( 103 ConnectorMetadataDpathError, 104 ConnectorMetadataDpathNotFoundError, 105 format_metadata_dpath_value, 106 get_connector_version_from_metadata, 107 load_raw_connector_metadata_from_local, 108) 109from airbyte_ops_mcp.connector_ops.utils import Connector 110from airbyte_ops_mcp.connector_qa.checks import ENABLED_CHECKS 111from airbyte_ops_mcp.connector_qa.consts import CONNECTORS_QA_DOC_TEMPLATE_NAME 112from airbyte_ops_mcp.connector_qa.models import ( 113 Check, 114 CheckCategory, 115 CheckStatus, 116 Report, 117) 118from airbyte_ops_mcp.connector_qa.utils import ( 119 get_all_connectors_in_directory, 120 remove_strict_encrypt_suffix, 121) 122from airbyte_ops_mcp.mcp.connector_registry import list_connectors_in_repo 123from airbyte_ops_mcp.mcp.connector_versions import compute_prerelease_docker_image_tag 124from airbyte_ops_mcp.registry._enums import ConnectorType, SupportLevel 125from airbyte_ops_mcp.registry.audit import ( 126 AuditResult, 127 find_unpublished_connectors, 128 generate_connector_list_summary, 129) 130from airbyte_ops_mcp.registry.connector_stubs import ( 131 CONNECTOR_STUBS_FILE, 132 ConnectorStub, 133 find_stub_by_connector, 134 load_local_stubs, 135 save_local_stubs, 136) 137from airbyte_ops_mcp.registry.store import resolve_registry_store 138from airbyte_ops_mcp.regression_tests.ci_output import write_github_summary 139 140console = Console() 141error_console = Console(stderr=True) 142 143OutputFormat = Literal["csv", "lines", "json-gh-matrix"] 144SortBy = Literal["name"] 145 146 147def _parse_support_level(value: str) -> SupportLevel: 148 """Parse a support level string into a `SupportLevel` enum member. 149 150 Accepts a keyword ("certified", "community", "archived") or a legacy 151 integer string ("100", "200", "300"). 152 """ 153 return SupportLevel.parse(value.strip().lower()) 154 155 156def _get_connector_support_level(connector_dir: Path) -> SupportLevel | None: 157 """Read support level from connector's metadata.yaml.""" 158 metadata_file = connector_dir / METADATA_FILE_NAME 159 if not metadata_file.exists(): 160 return None 161 metadata = yaml.safe_load(metadata_file.read_text()) 162 support_level_str = metadata.get("data", {}).get("supportLevel") 163 if not support_level_str: 164 return None 165 try: 166 return SupportLevel(support_level_str.lower()) 167 except ValueError: 168 return None 169 170 171def _parse_connector_types(value: str) -> set[ConnectorType]: 172 """Parse connector types from CSV or newline-delimited string.""" 173 types: set[ConnectorType] = set() 174 for item in value.replace(",", "\n").split("\n"): 175 item = item.strip().lower() 176 if item: 177 types.add(ConnectorType.parse(item)) 178 return types 179 180 181def _get_connector_type(connector_name: str) -> str: 182 """Derive connector type from name prefix.""" 183 if connector_name.startswith("source-"): 184 return ConnectorType.SOURCE 185 elif connector_name.startswith("destination-"): 186 return ConnectorType.DESTINATION 187 return "unknown" 188 189 190def _parse_connector_names(value: str) -> set[str]: 191 """Parse connector names from CSV or newline-delimited string.""" 192 names = set() 193 for item in value.replace(",", "\n").split("\n"): 194 item = item.strip() 195 if item: 196 names.add(item) 197 return names 198 199 200def _get_connector_version(connector_dir: Path) -> str | None: 201 """Read connector version (dockerImageTag) from metadata.yaml.""" 202 metadata_file = connector_dir / METADATA_FILE_NAME 203 if not metadata_file.exists(): 204 return None 205 metadata = yaml.safe_load(metadata_file.read_text()) 206 return metadata.get("data", {}).get("dockerImageTag") 207 208 209def _get_connector_autopilot_enabled(connector_dir: Path) -> bool: 210 """Return whether a connector has autopilot progressive rollout enabled. 211 212 Reads the connector's `metadata.yaml` and delegates to 213 `is_autopilot_rollout_enabled`. A connector with no metadata file, or with 214 no (or incomplete) `releases.rolloutConfiguration`, is treated as not 215 enabled. 216 """ 217 metadata_file = connector_dir / METADATA_FILE_NAME 218 if not metadata_file.exists(): 219 return False 220 metadata = yaml.safe_load(metadata_file.read_text()) 221 data = metadata.get("data") if isinstance(metadata, dict) else None 222 if not isinstance(data, dict): 223 return False 224 return is_autopilot_rollout_enabled(data) 225 226 227def _support_level_to_int(level: SupportLevel | None) -> int | None: 228 """Convert a `SupportLevel` to its legacy integer representation for JSON output.""" 229 if level is None: 230 return None 231 return level.precedence 232 233 234def _get_connector_info( 235 connector_name: str, connector_dir: Path 236) -> dict[str, str | int | None]: 237 """Get full connector metadata as a dict with connector_ prefixed keys. 238 239 This is shared between the `list --output-format json-gh-matrix` and `info` commands. 240 """ 241 return { 242 "connector": connector_name, 243 "connector_type": _get_connector_type(connector_name), 244 "connector_language": _detect_connector_language(connector_dir, connector_name) 245 or "unknown", 246 "connector_support_level": _support_level_to_int( 247 _get_connector_support_level(connector_dir) 248 ), 249 "connector_version": _get_connector_version(connector_dir), 250 "connector_dir": f"{CONNECTOR_PATH_PREFIX}/{connector_name}", 251 } 252 253 254# Create the local sub-app 255local_app = App(name="local", help="Local Airbyte monorepo operations.") 256app.command(local_app) 257 258# Create the connector sub-app under local 259connector_app = App(name="connector", help="Connector operations in the monorepo.") 260local_app.command(connector_app) 261 262 263@connector_app.command(name="list") 264def list_connectors( 265 repo_path: Annotated[ 266 str, 267 Parameter(help="Absolute path to the Airbyte monorepo."), 268 ], 269 certified_only: Annotated[ 270 bool, 271 Parameter(help="Include only certified connectors."), 272 ] = False, 273 modified_only: Annotated[ 274 bool, 275 Parameter(help="Include only modified connectors (requires PR context)."), 276 ] = False, 277 local_cdk: Annotated[ 278 bool, 279 Parameter( 280 help=( 281 "Include connectors using local CDK reference. " 282 "When combined with --modified-only, adds local-CDK connectors to the modified set." 283 ) 284 ), 285 ] = False, 286 language: Annotated[ 287 list[str] | None, 288 Parameter(help="Languages to include (python, java, low-code, manifest-only)."), 289 ] = None, 290 exclude_language: Annotated[ 291 list[str] | None, 292 Parameter(help="Languages to exclude."), 293 ] = None, 294 connector_type: Annotated[ 295 str | None, 296 Parameter( 297 help=( 298 "Connector types to include (source, destination). " 299 "Accepts CSV or newline-delimited values." 300 ) 301 ), 302 ] = None, 303 min_support_level: Annotated[ 304 str | None, 305 Parameter( 306 help=( 307 "Minimum support level (inclusive). " 308 "Accepts integer (100, 200, 300) or keyword (archived, community, certified)." 309 ) 310 ), 311 ] = None, 312 max_support_level: Annotated[ 313 str | None, 314 Parameter( 315 help=( 316 "Maximum support level (inclusive). " 317 "Accepts integer (100, 200, 300) or keyword (archived, community, certified)." 318 ) 319 ), 320 ] = None, 321 autopilot_enabled: Annotated[ 322 bool | None, 323 Parameter( 324 help=( 325 "Filter by autopilot progressive rollout status. " 326 "Pass `--autopilot-enabled=true` to keep only connectors that " 327 "have autopilot rollouts enabled (`defaultRolloutMode: autopilot` " 328 "and `enableProgressiveRollout: true`), or " 329 "`--autopilot-enabled=false` for those that do not. " 330 "When omitted, connectors are not filtered on rollout status." 331 ) 332 ), 333 ] = None, 334 pr: Annotated[ 335 str | None, 336 Parameter(help="PR number or GitHub URL for modification detection."), 337 ] = None, 338 gh_token: Annotated[ 339 str | None, 340 Parameter( 341 help=( 342 "GitHub API token. When provided together with --pr, the GitHub API " 343 "is used to detect modified files instead of local git diff " 344 "(avoids shallow-clone issues)." 345 ) 346 ), 347 ] = None, 348 exclude_connectors: Annotated[ 349 list[str] | None, 350 Parameter( 351 help=( 352 "Connectors to exclude from results. " 353 "Accepts CSV or newline-delimited values. Can be specified multiple times." 354 ) 355 ), 356 ] = None, 357 force_include_connectors: Annotated[ 358 list[str] | None, 359 Parameter( 360 help=( 361 "Connectors to force-include regardless of other filters. " 362 "Accepts CSV or newline-delimited values. Can be specified multiple times." 363 ) 364 ), 365 ] = None, 366 connectors_filter: Annotated[ 367 list[str] | None, 368 Parameter( 369 help=( 370 "Intersect results with this explicit set of connector names. " 371 "Only connectors present in both the filtered results and this set are returned. " 372 "Useful for composing multiple filter passes (e.g. combining separate source " 373 "and destination lists). " 374 "Accepts CSV or newline-delimited values. Can be specified multiple times." 375 ) 376 ), 377 ] = None, 378 output_format: Annotated[ 379 OutputFormat, 380 Parameter( 381 help=( 382 'Output format: "csv" (comma-separated), ' 383 '"lines" (one connector per line), ' 384 '"json-gh-matrix" (GitHub Actions matrix JSON).' 385 ) 386 ), 387 ] = "lines", 388 unpublished: Annotated[ 389 bool, 390 Parameter( 391 help=( 392 "Filter to only connectors whose local dockerImageTag " 393 "has not been published to the GCS registry. Requires --store." 394 ) 395 ), 396 ] = False, 397 store: Annotated[ 398 str | None, 399 Parameter( 400 help=( 401 "Store target for unpublished check (e.g. 'coral:prod', 'coral:dev'). " 402 "Required when --unpublished is set." 403 ) 404 ), 405 ] = None, 406 assert_none: Annotated[ 407 bool, 408 Parameter( 409 help=( 410 "Exit with non-zero status if any connectors match the filters. " 411 "Useful for audit/CI checks (e.g. --unpublished --assert-none)." 412 ) 413 ), 414 ] = False, 415 sort_by: Annotated[ 416 SortBy, 417 Parameter( 418 help="Sort connectors by the given key. Only 'name' is supported for now." 419 ), 420 ] = "name", 421 sort_direction: Annotated[ 422 Literal["asc", "desc"], 423 Parameter( 424 help="Sort direction: 'asc' (ascending, default) or 'desc' (descending)." 425 ), 426 ] = "asc", 427 limit: Annotated[ 428 int | None, 429 Parameter( 430 help=( 431 "Maximum number of connectors to return. " 432 "Applied after sorting. Useful for batched processing." 433 ) 434 ), 435 ] = None, 436 offset: Annotated[ 437 int, 438 Parameter( 439 help=( 440 "Number of connectors to skip from the start of the (sorted) list. " 441 "Applied after sorting, before --limit. " 442 "Useful for batched processing (e.g. --offset=200 --limit=200 for batch 2)." 443 ) 444 ), 445 ] = 0, 446) -> None: 447 """List connectors in the Airbyte monorepo with filtering options.""" 448 # Validate mutually exclusive flags 449 if language and exclude_language: 450 exit_with_error("Cannot specify both --language and --exclude-language.") 451 if assert_none and (offset or limit is not None): 452 exit_with_error( 453 "Cannot combine --assert-none with --offset or --limit. " 454 "--assert-none checks all connectors matching the filters, " 455 "which is incompatible with pagination." 456 ) 457 458 # Map CLI flags to MCP tool parameters 459 certified: bool | None = True if certified_only else None 460 modified: bool | None = True if modified_only else None 461 462 language_filter: set[str] | None = set(language) if language else None 463 language_exclude: set[str] | None = ( 464 set(exclude_language) if exclude_language else None 465 ) 466 467 # Parse connector type filter 468 connector_type_filter: set[str] | None = None 469 if connector_type: 470 try: 471 connector_type_filter = _parse_connector_types(connector_type) 472 except ValueError as e: 473 exit_with_error(str(e)) 474 475 # Parse support level filters 476 min_level: SupportLevel | None = None 477 max_level: SupportLevel | None = None 478 if min_support_level: 479 try: 480 min_level = _parse_support_level(min_support_level) 481 except ValueError as e: 482 exit_with_error(str(e)) 483 if max_support_level: 484 try: 485 max_level = _parse_support_level(max_support_level) 486 except ValueError as e: 487 exit_with_error(str(e)) 488 489 # Parse exclude/force-include connector lists (merge multiple flag values) 490 exclude_set: set[str] = set() 491 if exclude_connectors: 492 for value in exclude_connectors: 493 exclude_set.update(_parse_connector_names(value)) 494 495 force_include_set: set[str] = set() 496 if force_include_connectors: 497 for value in force_include_connectors: 498 force_include_set.update(_parse_connector_names(value)) 499 500 connectors_filter_set: set[str] | None = None 501 if connectors_filter: 502 connectors_filter_set = set() 503 for value in connectors_filter: 504 connectors_filter_set.update(_parse_connector_names(value)) 505 506 result = list_connectors_in_repo( 507 repo_path=repo_path, 508 certified=certified, 509 modified=modified, 510 language_filter=language_filter, 511 language_exclude=language_exclude, 512 pr_num_or_url=pr, 513 gh_token=gh_token, 514 ) 515 connectors = list(result.connectors) 516 repo_path_obj = Path(repo_path) 517 518 # Add connectors with local CDK reference if --local-cdk flag is set 519 if local_cdk: 520 local_cdk_connectors = get_connectors_with_local_cdk(repo_path) 521 connectors = sorted(set(connectors) | local_cdk_connectors) 522 523 # Apply connector type filter 524 if connector_type_filter: 525 connectors = [ 526 name 527 for name in connectors 528 if _get_connector_type(name) in connector_type_filter 529 ] 530 531 # Apply support level filters (requires reading metadata) 532 if min_level is not None or max_level is not None: 533 filtered_connectors = [] 534 for name in connectors: 535 connector_dir = repo_path_obj / CONNECTOR_PATH_PREFIX / name 536 level = _get_connector_support_level(connector_dir) 537 if level is None: 538 continue # Skip connectors without support level 539 if min_level is not None and level.precedence < min_level.precedence: 540 continue 541 if max_level is not None and level.precedence > max_level.precedence: 542 continue 543 filtered_connectors.append(name) 544 connectors = filtered_connectors 545 546 # Apply exclude filter 547 if exclude_set: 548 connectors = [name for name in connectors if name not in exclude_set] 549 550 # Apply connectors filter (intersection, narrows the set) 551 if connectors_filter_set is not None: 552 connectors = [name for name in connectors if name in connectors_filter_set] 553 554 # Apply autopilot rollout status filter last among narrowing filters: it reads 555 # and parses each connector's metadata.yaml, so run it on the already 556 # name-filtered set to avoid unnecessary reads. 557 if autopilot_enabled is not None: 558 connectors = [ 559 name 560 for name in connectors 561 if _get_connector_autopilot_enabled( 562 repo_path_obj / CONNECTOR_PATH_PREFIX / name 563 ) 564 == autopilot_enabled 565 ] 566 567 # Apply force-include (union, overrides all other filters) 568 if force_include_set: 569 connectors_set = set(connectors) 570 connectors_set.update(force_include_set) 571 connectors = sorted(connectors_set) 572 573 # Apply unpublished filter (requires GCS check) 574 audit_result: AuditResult | None = None 575 if unpublished: 576 if not store: 577 exit_with_error("--store is required when --unpublished is set.") 578 579 target = resolve_registry_store(store=store) 580 audit_result = find_unpublished_connectors( 581 repo_path=repo_path, 582 bucket_name=target.bucket, 583 connector_names=connectors, 584 ) 585 unpublished_names = {entry.connector_name for entry in audit_result.unpublished} 586 connectors = [name for name in connectors if name in unpublished_names] 587 588 # Log audit summary to stderr 589 error_console.print( 590 f"Audit: {audit_result.checked_count} checked, " 591 f"{len(audit_result.unpublished)} unpublished, " 592 f"{len(audit_result.skipped_archived)} archived-skipped, " 593 f"{len(audit_result.skipped_disabled)} disabled-skipped, " 594 f"{len(audit_result.skipped_rc)} rc-skipped" 595 ) 596 if audit_result.errors: 597 for err in audit_result.errors: 598 error_console.print(f" Warning: {err}") 599 600 # --- Sort, offset, limit (applied after all filters) --- 601 if offset < 0: 602 exit_with_error("--offset must be non-negative.") 603 if limit is not None and limit < 0: 604 exit_with_error("--limit must be non-negative.") 605 606 total_before_slice = len(connectors) 607 reverse = sort_direction == "desc" 608 if sort_by == "name": 609 connectors = sorted(connectors, reverse=reverse) 610 611 if offset: 612 connectors = connectors[offset:] 613 if limit is not None: 614 connectors = connectors[:limit] 615 616 if offset or limit is not None: 617 error_console.print( 618 f"Slice: showing {len(connectors)} of {total_before_slice} " 619 f"(offset={offset}, limit={limit})" 620 ) 621 622 if output_format == "csv": 623 sys.stdout.write(",".join(connectors) + "\n") 624 elif output_format == "lines": 625 for name in connectors: 626 sys.stdout.write(name + "\n") 627 elif output_format == "json-gh-matrix": 628 # Build matrix with full connector metadata 629 include_list = [] 630 for name in connectors: 631 connector_dir = repo_path_obj / CONNECTOR_PATH_PREFIX / name 632 include_list.append(_get_connector_info(name, connector_dir)) 633 matrix = {"include": include_list} 634 sys.stdout.write( 635 json.dumps( 636 matrix, 637 indent=2, 638 default=str, 639 ) 640 + "\n" 641 ) 642 643 # Write GitHub Step Summary when GITHUB_STEP_SUMMARY is set 644 _write_connector_list_summary( 645 connectors=connectors, 646 assert_none=assert_none, 647 is_unpublished=unpublished, 648 audit_result=audit_result, 649 ) 650 651 # Exit non-zero if --assert-none and any connectors matched 652 if assert_none and connectors: 653 error_console.print(f"FAIL: {len(connectors)} connector(s) matched.") 654 sys.exit(1) 655 656 657def _write_github_step_outputs(outputs: dict[str, str | int | None]) -> None: 658 """Write outputs to GitHub Actions step output file if running in CI.""" 659 github_output = os.getenv("GITHUB_OUTPUT") 660 if not (os.getenv("CI") and github_output): 661 return 662 663 with open(github_output, "a", encoding="utf-8") as f: 664 for key, value in outputs.items(): 665 if value is None: 666 continue 667 f.write(f"{key}={value}\n") 668 669 670def _write_connector_list_summary( 671 connectors: list[str], 672 *, 673 assert_none: bool, 674 is_unpublished: bool, 675 audit_result: AuditResult | None, 676) -> None: 677 """Write a GitHub Step Summary for the connector list command. 678 679 This is a no-op when `GITHUB_STEP_SUMMARY` is not set. 680 """ 681 github_summary = os.getenv("GITHUB_STEP_SUMMARY") 682 if not github_summary: 683 return 684 685 summary = generate_connector_list_summary( 686 connectors, 687 assert_none=assert_none, 688 unpublished=is_unpublished, 689 audit_result=audit_result, 690 ) 691 write_github_summary(summary) 692 693 694@connector_app.command(name="info") 695def connector_info( 696 connector_name: Annotated[ 697 str, 698 Parameter(help="Name of the connector (e.g., source-github)."), 699 ], 700 repo_path: Annotated[ 701 str | None, 702 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 703 ] = None, 704 dpath_expression: Annotated[ 705 str | None, 706 Parameter( 707 name="--dpath", 708 help=( 709 "Evaluate this dpath expression against the parsed metadata.yaml " 710 "object and print only that value (e.g., data/dockerImageTag)." 711 ), 712 ), 713 ] = None, 714) -> None: 715 """Get metadata for a single connector. 716 717 Prints JSON output with connector metadata. When running in GitHub Actions 718 (CI env var set), also writes each field to GitHub step outputs. 719 """ 720 # Infer repo_path from current directory if not provided 721 if repo_path is None: 722 # Check if we're in an airbyte repo by looking for the connectors directory 723 cwd = Path.cwd() 724 # Walk up to find airbyte-integrations/connectors 725 for parent in [cwd, *cwd.parents]: 726 if (parent / CONNECTOR_PATH_PREFIX).exists(): 727 repo_path = str(parent) 728 break 729 if repo_path is None: 730 exit_with_error( 731 "Could not infer repo path. Please provide --repo-path or run from within the Airbyte monorepo." 732 ) 733 734 repo_path_obj = Path(repo_path) 735 connector_dir = repo_path_obj / CONNECTOR_PATH_PREFIX / connector_name 736 737 if not connector_dir.exists(): 738 exit_with_error(f"Connector directory not found: {connector_dir}") 739 740 try: 741 value = load_raw_connector_metadata_from_local( 742 repo_path_obj, 743 connector_name, 744 dpath_expression=dpath_expression, 745 ) 746 except ( 747 ConnectorMetadataDpathError, 748 ConnectorMetadataDpathNotFoundError, 749 FileNotFoundError, 750 ValueError, 751 ) as e: 752 exit_with_error(str(e)) 753 754 if dpath_expression is None: 755 print_json(value) 756 return 757 758 sys.stdout.write(format_metadata_dpath_value(value) + "\n") 759 760 761BumpType = Literal[ 762 "patch", 763 "minor", 764 "major", 765 "patch_rc", 766 "minor_rc", 767 "major_rc", 768 "rc", 769 "promote", 770] 771 772 773def _get_git_head_sha(repo_path: Path) -> str: 774 """Get the HEAD commit SHA from a git repository. 775 776 Args: 777 repo_path: Path to the git repository. 778 779 Returns: 780 Full commit SHA string. 781 782 Raises: 783 SystemExit: If the git command fails (e.g., not a git repo). 784 """ 785 try: 786 result = subprocess.run( 787 ["git", "rev-parse", "HEAD"], 788 cwd=repo_path, 789 capture_output=True, 790 text=True, 791 check=True, 792 ) 793 except subprocess.CalledProcessError: 794 exit_with_error( 795 f"Failed to get HEAD SHA from {repo_path}. Is it a git repository?" 796 ) 797 except OSError: 798 exit_with_error("Could not run 'git'. Is git installed and on PATH?") 799 return result.stdout.strip() 800 801 802@connector_app.command(name="get-version") 803def get_version( 804 name: Annotated[ 805 str, 806 Parameter(help="Connector technical name (e.g., source-github)."), 807 ], 808 repo_path: Annotated[ 809 str, 810 Parameter(help="Absolute path to the Airbyte monorepo."), 811 ], 812 compute_next: Annotated[ 813 bool, 814 Parameter( 815 name="--next", 816 help="Compute the next version instead of the current version.", 817 ), 818 ] = False, 819 bump_type: Annotated[ 820 BumpType | None, 821 Parameter( 822 help="Version bump type (requires --next). " 823 "Standard: patch, minor, major. RC: patch_rc, minor_rc, major_rc, rc, promote." 824 ), 825 ] = None, 826 prerelease: Annotated[ 827 bool, 828 Parameter( 829 help="Compute a prerelease (preview) tag using the repo HEAD SHA (requires --next)." 830 ), 831 ] = False, 832) -> None: 833 """Get the current or next version for a connector. 834 835 By default, prints the current version from metadata.yaml. 836 This is analogous to `local connector info --dpath data/dockerImageTag` and 837 uses the same dpath evaluation internally. 838 839 With --next, computes and prints the next version. Requires either 840 --bump-type or --prerelease to be specified. 841 842 Examples: 843 # Current version 844 airbyte-ops local connector get-version --name source-github --repo-path /path/to/airbyte 845 846 # Next patch version 847 VERSION=$(airbyte-ops local connector get-version --name source-github --repo-path /path/to/airbyte --next --bump-type patch) 848 849 # Next prerelease (preview) tag based on HEAD SHA 850 TAG=$(airbyte-ops local connector get-version --name source-github --repo-path /path/to/airbyte --next --prerelease) 851 """ 852 repo = Path(repo_path) 853 854 # Validate flag combinations 855 if bump_type and not compute_next: 856 exit_with_error("--bump-type requires --next.") 857 if prerelease and not compute_next: 858 exit_with_error("--prerelease requires --next.") 859 if bump_type and prerelease: 860 exit_with_error("--bump-type and --prerelease are mutually exclusive.") 861 if compute_next and not bump_type and not prerelease: 862 exit_with_error( 863 "--next requires either --bump-type or --prerelease. " 864 "Automatic detection of bump type is not yet supported." 865 ) 866 867 try: 868 get_connector_path(repo, name) 869 metadata = load_raw_connector_metadata_from_local(repo, name) 870 current_version = get_connector_version_from_metadata(metadata) 871 except ( 872 ConnectorMetadataDpathError, 873 ConnectorMetadataDpathNotFoundError, 874 ConnectorNotFoundError, 875 FileNotFoundError, 876 ValueError, 877 ) as e: 878 exit_with_error(str(e)) 879 880 if not compute_next: 881 sys.stdout.write(current_version + "\n") 882 return 883 884 if prerelease: 885 sha = _get_git_head_sha(repo) 886 try: 887 tag = compute_prerelease_docker_image_tag(current_version, sha) 888 except InvalidVersionError as e: 889 exit_with_error(str(e)) 890 sys.stdout.write(tag + "\n") 891 return 892 893 # bump_type is set (validated above) 894 try: 895 new_version = calculate_new_version( 896 current_version=current_version, 897 bump_type=bump_type, 898 ) 899 except (InvalidVersionError, ValueError) as e: 900 exit_with_error(str(e)) 901 902 sys.stdout.write(new_version + "\n") 903 904 905@connector_app.command(name="bump-version") 906def bump_version( 907 name: Annotated[ 908 str, 909 Parameter(help="Connector technical name (e.g., source-github)."), 910 ], 911 repo_path: Annotated[ 912 str, 913 Parameter(help="Absolute path to the Airbyte monorepo."), 914 ], 915 bump_type: Annotated[ 916 BumpType | None, 917 Parameter( 918 help="Version bump type. Standard: patch, minor, major. RC: patch_rc, minor_rc, major_rc, rc, promote." 919 ), 920 ] = None, 921 new_version: Annotated[ 922 str | None, 923 Parameter(help="Explicit new version (overrides --bump-type if provided)."), 924 ] = None, 925 changelog_message: Annotated[ 926 str | None, 927 Parameter( 928 help="Message to add to changelog. Ignored if --no-changelog is set." 929 ), 930 ] = None, 931 pr_number: Annotated[ 932 int | None, 933 Parameter(help="PR number for changelog entry."), 934 ] = None, 935 dry_run: Annotated[ 936 bool, 937 Parameter(help="Show what would be changed without modifying files."), 938 ] = False, 939 no_changelog: Annotated[ 940 bool, 941 Parameter( 942 help="Skip changelog updates even if --changelog-message is provided. " 943 "Useful for ephemeral version bumps (e.g. pre-release artifact generation)." 944 ), 945 ] = False, 946 progressive_rollout_enabled: Annotated[ 947 bool | None, 948 Parameter( 949 help="Explicitly set `enableProgressiveRollout` in metadata.yaml. " 950 "Pass `false` to disable progressive rollout (e.g. for preview builds). " 951 "When omitted, the automatic behaviour based on --bump-type is used.", 952 ), 953 ] = None, 954) -> None: 955 """Bump a connector's version across all relevant files. 956 957 Updates version in metadata.yaml (always), pyproject.toml (if exists), 958 and documentation changelog (if --changelog-message provided). 959 960 Note: --changelog-message is ignored when --no-changelog is set. 961 962 Either --bump-type or --new-version must be provided. 963 964 Examples: 965 airbyte-ops local connector bump-version --name source-github --repo-path /path/to/airbyte --bump-type patch 966 airbyte-ops local connector bump-version --name source-github --repo-path /path/to/airbyte --new-version 1.2.3-preview.abc1234 --no-changelog 967 airbyte-ops local connector bump-version --name source-github --repo-path /path/to/airbyte --new-version 1.2.3-preview.abc1234 --no-changelog --progressive-rollout-enabled=false 968 """ 969 try: 970 result = bump_connector_version( 971 repo_path=repo_path, 972 connector_name=name, 973 bump_type=bump_type, 974 new_version=new_version, 975 changelog_message=changelog_message, 976 pr_number=pr_number, 977 dry_run=dry_run, 978 no_changelog=no_changelog, 979 progressive_rollout_enabled=progressive_rollout_enabled, 980 ) 981 except ConnectorNotFoundError as e: 982 exit_with_error(str(e)) 983 except VersionNotFoundError as e: 984 exit_with_error(str(e)) 985 except InvalidVersionError as e: 986 exit_with_error(str(e)) 987 except ValueError as e: 988 exit_with_error(str(e)) 989 990 # Build output matching the issue spec 991 output = { 992 "connector": result.connector, 993 "previous_version": result.previous_version, 994 "new_version": result.new_version, 995 "files_modified": result.files_modified, 996 "dry_run": result.dry_run, 997 } 998 print_json(output) 999 1000 # Write to GitHub step outputs if in CI 1001 _write_github_step_outputs( 1002 { 1003 "connector": result.connector, 1004 "previous_version": result.previous_version, 1005 "new_version": result.new_version, 1006 } 1007 ) 1008 1009 1010rollouts_app = App( 1011 name="rollouts", 1012 help="Manage a connector's progressive rollout configuration.", 1013) 1014connector_app.command(rollouts_app) 1015 1016 1017@rollouts_app.command(name="enable") 1018def rollouts_enable( 1019 name: Annotated[ 1020 str, 1021 Parameter(help="Connector technical name (e.g., source-github)."), 1022 ], 1023 repo_path: Annotated[ 1024 str, 1025 Parameter(help="Absolute path to the Airbyte monorepo."), 1026 ], 1027 strategy: Annotated[ 1028 AutopilotStrategy | None, 1029 Parameter( 1030 help="Autopilot pacing strategy: 'fast', 'slow', or 'default'. " 1031 "'default' is a server-side alias for 'fast'; prefer the explicit 'fast'. " 1032 "Omitted keeps any existing value, or 'fast' when none is set." 1033 ), 1034 ] = None, 1035 dry_run: Annotated[ 1036 bool, 1037 Parameter(help="Show what would be changed without modifying files."), 1038 ] = False, 1039) -> None: 1040 """Enable autopilot rollouts in a connector's metadata.yaml. 1041 1042 Always sets `releases.rolloutConfiguration.defaultRolloutMode: autopilot` 1043 (autopilot is the only actively-supported mode, so it is implicit here), 1044 fills in `autopilotConfig` (`autoStart`, `autoPromoteStages`, `strategy`), 1045 and sets `enableProgressiveRollout: true` — the toggle that turns automatic 1046 progressive rollout on. Defaults only fill in `autopilotConfig` fields that 1047 are absent, so existing settings are preserved. 1048 1049 Examples: 1050 airbyte-ops local connector rollouts enable --name source-github --repo-path /path/to/airbyte 1051 airbyte-ops local connector rollouts enable --name source-github --repo-path /path/to/airbyte --strategy slow 1052 """ 1053 try: 1054 result = enable_autopilot_rollout( 1055 repo_path=repo_path, 1056 connector_name=name, 1057 strategy=strategy.value if strategy is not None else None, 1058 dry_run=dry_run, 1059 ) 1060 except ConnectorNotFoundError as e: 1061 exit_with_error(str(e)) 1062 except FileNotFoundError as e: 1063 exit_with_error(str(e)) 1064 except ValueError as e: 1065 exit_with_error(str(e)) 1066 1067 print_json( 1068 { 1069 "connector": result.connector, 1070 "modified": result.modified, 1071 "dry_run": result.dry_run, 1072 "default_rollout_mode": result.default_rollout_mode, 1073 "progressive_rollout_enabled": result.progressive_rollout_enabled, 1074 "strategy": result.strategy, 1075 "auto_start": result.auto_start, 1076 "auto_promote_stages": result.auto_promote_stages, 1077 } 1078 ) 1079 1080 _write_github_step_outputs( 1081 { 1082 "connector": result.connector, 1083 "default_rollout_mode": result.default_rollout_mode, 1084 "progressive_rollout_enabled": str(result.progressive_rollout_enabled), 1085 "strategy": result.strategy, 1086 } 1087 ) 1088 1089 1090@rollouts_app.command(name="disable") 1091def rollouts_disable( 1092 name: Annotated[ 1093 str, 1094 Parameter(help="Connector technical name (e.g., source-github)."), 1095 ], 1096 repo_path: Annotated[ 1097 str, 1098 Parameter(help="Absolute path to the Airbyte monorepo."), 1099 ], 1100 dry_run: Annotated[ 1101 bool, 1102 Parameter(help="Show what would be changed without modifying files."), 1103 ] = False, 1104) -> None: 1105 """Disable autopilot rollouts in a connector's metadata.yaml. 1106 1107 Only sets `releases.rolloutConfiguration.enableProgressiveRollout: false` — 1108 the toggle that turns automatic progressive rollout off. `defaultRolloutMode` 1109 and any `autopilotConfig` are retained (inert until re-enabled), so the 1110 change is lossless. Connectors with no rollout config, or with the flag 1111 already off, are a no-op. 1112 1113 Examples: 1114 airbyte-ops local connector rollouts disable --name source-github --repo-path /path/to/airbyte 1115 """ 1116 try: 1117 result = disable_autopilot_rollout( 1118 repo_path=repo_path, 1119 connector_name=name, 1120 dry_run=dry_run, 1121 ) 1122 except ConnectorNotFoundError as e: 1123 exit_with_error(str(e)) 1124 except FileNotFoundError as e: 1125 exit_with_error(str(e)) 1126 1127 print_json( 1128 { 1129 "connector": result.connector, 1130 "modified": result.modified, 1131 "dry_run": result.dry_run, 1132 "progressive_rollout_enabled": result.progressive_rollout_enabled, 1133 } 1134 ) 1135 1136 _write_github_step_outputs( 1137 { 1138 "connector": result.connector, 1139 "progressive_rollout_enabled": str(result.progressive_rollout_enabled), 1140 } 1141 ) 1142 1143 1144@connector_app.command(name="bump-base-image") 1145def bump_base_image_cmd( 1146 name: Annotated[ 1147 str, 1148 Parameter(help="Connector technical name (e.g., source-github)."), 1149 ], 1150 repo_path: Annotated[ 1151 str, 1152 Parameter(help="Absolute path to the Airbyte monorepo."), 1153 ], 1154 force_latest: Annotated[ 1155 bool, 1156 Parameter( 1157 help=( 1158 "Bump to the absolute latest stable base image, ignoring " 1159 "major-version boundaries. Without this flag the bump stays " 1160 "within the current major version." 1161 ) 1162 ), 1163 ] = False, 1164 dry_run: Annotated[ 1165 bool, 1166 Parameter(help="Show what would be changed without modifying files."), 1167 ] = False, 1168) -> None: 1169 """Update a connector's base image. 1170 1171 Two modes: 1172 1173 * Default: bump to the latest stable tag within the same major version. 1174 Major version changes are treated as breaking-change boundaries. 1175 * --force-latest: bump to the absolute latest stable tag regardless of semver. 1176 1177 Examples: 1178 airbyte-ops local connector bump-base-image --name source-github --repo-path /path/to/airbyte 1179 airbyte-ops local connector bump-base-image --name source-github --repo-path /path/to/airbyte --force-latest 1180 """ 1181 try: 1182 result = bump_base_image( 1183 repo_path=repo_path, 1184 connector_name=name, 1185 force_latest=force_latest, 1186 dry_run=dry_run, 1187 ) 1188 except (ConnectorNotFoundError, BaseImageError) as e: 1189 exit_with_error(str(e)) 1190 except ValueError as e: 1191 exit_with_error(str(e)) 1192 1193 output = { 1194 "connector": result.connector, 1195 "previous_base_image": result.previous_base_image, 1196 "new_base_image": result.new_base_image, 1197 "updated": result.updated, 1198 "dry_run": result.dry_run, 1199 "files_modified": result.files_modified, 1200 "message": result.message, 1201 } 1202 print_json(output) 1203 1204 1205@connector_app.command(name="bump-cdk") 1206def bump_cdk_cmd( 1207 name: Annotated[ 1208 str, 1209 Parameter(help="Connector technical name (e.g., source-github)."), 1210 ], 1211 repo_path: Annotated[ 1212 str, 1213 Parameter(help="Absolute path to the Airbyte monorepo."), 1214 ], 1215 force_latest: Annotated[ 1216 bool, 1217 Parameter( 1218 help=( 1219 "Rewrite the CDK constraint to >=LATEST,<NEXT_MAJOR and refresh " 1220 "the lock file. Without this flag only the lock file is refreshed " 1221 "(constraint unchanged)." 1222 ) 1223 ), 1224 ] = False, 1225 dry_run: Annotated[ 1226 bool, 1227 Parameter(help="Show what would be changed without modifying files."), 1228 ] = False, 1229) -> None: 1230 """Bump a connector's CDK dependency. 1231 1232 Two modes: 1233 1234 * Default: refresh the lock file so it resolves the newest CDK that 1235 satisfies the existing constraint. The constraint is NOT changed. 1236 * --force-latest: rewrite the constraint to >=LATEST,<NEXT_MAJOR and refresh 1237 the lock file. Extras (e.g. file-based) are preserved. 1238 1239 For Java connectors, updates `build.gradle` to the latest CDK version. 1240 1241 Examples: 1242 airbyte-ops local connector bump-cdk --name source-github --repo-path /path/to/airbyte 1243 airbyte-ops local connector bump-cdk --name source-github --repo-path /path/to/airbyte --force-latest 1244 """ 1245 try: 1246 result = bump_cdk( 1247 repo_path=repo_path, 1248 connector_name=name, 1249 force_latest=force_latest, 1250 dry_run=dry_run, 1251 ) 1252 except (ConnectorNotFoundError, CdkBumpError) as e: 1253 exit_with_error(str(e)) 1254 except ValueError as e: 1255 exit_with_error(str(e)) 1256 1257 output = { 1258 "connector": result.connector, 1259 "language": result.language, 1260 "previous_version": result.previous_version, 1261 "new_version": result.new_version, 1262 "updated": result.updated, 1263 "dry_run": result.dry_run, 1264 "files_modified": result.files_modified, 1265 "message": result.message, 1266 } 1267 print_json(output) 1268 1269 1270@connector_app.command(name="bump-deps") 1271def bump_deps_cmd( 1272 name: Annotated[ 1273 str, 1274 Parameter(help="Connector technical name (e.g., source-github)."), 1275 ], 1276 repo_path: Annotated[ 1277 str, 1278 Parameter(help="Absolute path to the Airbyte monorepo."), 1279 ], 1280 dry_run: Annotated[ 1281 bool, 1282 Parameter(help="Show what would be changed without modifying files."), 1283 ] = False, 1284) -> None: 1285 """Update a connector's dependencies. 1286 1287 For Python / low-code connectors using Poetry, this runs 1288 `poetry update --lock` to refresh the lock file with the latest 1289 versions allowed by existing constraints. 1290 1291 For connectors that do not use Poetry (manifest-only, Java, etc.), 1292 this is a no-op. 1293 1294 Examples: 1295 airbyte-ops local connector bump-deps --name source-github --repo-path /path/to/airbyte 1296 airbyte-ops local connector bump-deps --name source-github --repo-path /path/to/airbyte --dry-run 1297 """ 1298 try: 1299 result = bump_deps( 1300 repo_path=repo_path, 1301 connector_name=name, 1302 dry_run=dry_run, 1303 ) 1304 except (ConnectorNotFoundError, DepsError) as e: 1305 exit_with_error(str(e)) 1306 except ValueError as e: 1307 exit_with_error(str(e)) 1308 1309 output = { 1310 "connector": result.connector, 1311 "language": result.language, 1312 "updated": result.updated, 1313 "dry_run": result.dry_run, 1314 "files_modified": result.files_modified, 1315 "outdated_packages": result.outdated_packages, 1316 "message": result.message, 1317 } 1318 print_json(output) 1319 1320 1321@connector_app.command(name="qa") 1322def run_qa_checks( 1323 name: Annotated[ 1324 list[str] | None, 1325 Parameter( 1326 help="Connector technical name(s) (e.g., source-github). Can be specified multiple times." 1327 ), 1328 ] = None, 1329 connector_directory: Annotated[ 1330 str | None, 1331 Parameter( 1332 help="Directory containing connectors to run checks on all connectors in this directory." 1333 ), 1334 ] = None, 1335 check: Annotated[ 1336 list[str] | None, 1337 Parameter(help="Specific check(s) to run. Can be specified multiple times."), 1338 ] = None, 1339 report_path: Annotated[ 1340 str | None, 1341 Parameter(help="Path to write the JSON report file."), 1342 ] = None, 1343) -> None: 1344 """Run QA checks on connector(s). 1345 1346 Validates connector metadata, documentation, packaging, security, and versioning. 1347 Exit code is non-zero if any checks fail. 1348 """ 1349 # Determine which checks to run 1350 checks_to_run = ENABLED_CHECKS 1351 if check: 1352 check_names = set(check) 1353 checks_to_run = [c for c in ENABLED_CHECKS if type(c).__name__ in check_names] 1354 if not checks_to_run: 1355 exit_with_error( 1356 f"No matching checks found. Available checks: {[type(c).__name__ for c in ENABLED_CHECKS]}" 1357 ) 1358 1359 # Collect connectors to check 1360 connectors: list[Connector] = [] 1361 if name: 1362 connectors.extend(Connector(remove_strict_encrypt_suffix(n)) for n in name) 1363 if connector_directory: 1364 connectors.extend(get_all_connectors_in_directory(Path(connector_directory))) 1365 1366 if not connectors: 1367 exit_with_error("No connectors specified. Use --name or --connector-directory.") 1368 1369 connectors = sorted(connectors, key=lambda c: c.technical_name) 1370 1371 # Run checks synchronously (simpler than async for CLI) 1372 all_results = [] 1373 for connector in connectors: 1374 for qa_check in checks_to_run: 1375 result = qa_check.run(connector) 1376 if result.status == CheckStatus.PASSED: 1377 status_icon = "[green]✅ PASS[/green]" 1378 elif result.status == CheckStatus.SKIPPED: 1379 status_icon = "[yellow]🔶 SKIP[/yellow]" 1380 else: 1381 status_icon = "[red]❌ FAIL[/red]" 1382 console.print( 1383 f"{status_icon} {connector.technical_name}: {result.check.name}" 1384 ) 1385 if result.message: 1386 console.print(f" {result.message}") 1387 all_results.append(result) 1388 1389 # Write report if requested 1390 if report_path: 1391 Report(check_results=all_results).write(Path(report_path)) 1392 console.print(f"Report written to {report_path}") 1393 1394 # Exit with error if any checks failed 1395 failed = [r for r in all_results if r.status == CheckStatus.FAILED] 1396 if failed: 1397 exit_with_error(f"{len(failed)} check(s) failed") 1398 1399 1400@connector_app.command(name="qa-docs-generate") 1401def generate_qa_docs( 1402 output_file: Annotated[ 1403 str, 1404 Parameter(help="Path to write the generated documentation file."), 1405 ], 1406) -> None: 1407 """Generate documentation for QA checks. 1408 1409 Creates a markdown file documenting all available QA checks organized by category. 1410 """ 1411 checks_by_category: dict[CheckCategory, list[Check]] = {} 1412 for qa_check in ENABLED_CHECKS: 1413 checks_by_category.setdefault(qa_check.category, []).append(qa_check) 1414 1415 jinja_env = Environment( 1416 loader=PackageLoader("airbyte_ops_mcp.connector_qa", "templates"), 1417 autoescape=select_autoescape(), 1418 trim_blocks=False, 1419 lstrip_blocks=True, 1420 ) 1421 template = jinja_env.get_template(CONNECTORS_QA_DOC_TEMPLATE_NAME) 1422 documentation = template.render(checks_by_category=checks_by_category) 1423 1424 output_path = Path(output_file) 1425 output_path.write_text(documentation) 1426 console.print(f"Documentation written to {output_file}") 1427 1428 1429# Create the changelog sub-app under connector 1430changelog_app = App(name="changelog", help="Changelog operations for connectors.") 1431connector_app.command(changelog_app) 1432 1433 1434@changelog_app.command(name="check") 1435def changelog_check( 1436 connector_name: Annotated[ 1437 str | None, 1438 Parameter(help="Connector technical name (e.g., source-github)."), 1439 ] = None, 1440 all_connectors: Annotated[ 1441 bool, 1442 Parameter("--all", help="Check all connectors in the repository."), 1443 ] = False, 1444 repo_path: Annotated[ 1445 str | None, 1446 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 1447 ] = None, 1448 lookback_days: Annotated[ 1449 int | None, 1450 Parameter(help="Only check entries with dates within this many days."), 1451 ] = None, 1452 strict: Annotated[ 1453 bool, 1454 Parameter(help="Exit with error code if any issues are found."), 1455 ] = False, 1456) -> None: 1457 """Check changelog entries for issues. 1458 1459 Validates changelog dates match PR merge dates and checks for PR number mismatches. 1460 """ 1461 if not connector_name and not all_connectors: 1462 exit_with_error("Either --connector-name or --all must be specified.") 1463 1464 if connector_name and all_connectors: 1465 exit_with_error("Cannot specify both --connector-name and --all.") 1466 1467 if repo_path is None: 1468 cwd = Path.cwd() 1469 for parent in [cwd, *cwd.parents]: 1470 if (parent / CONNECTOR_PATH_PREFIX).exists(): 1471 repo_path = str(parent) 1472 break 1473 if repo_path is None: 1474 exit_with_error( 1475 "Could not infer repo path. Please provide --repo-path or run from within the Airbyte monorepo." 1476 ) 1477 1478 total_issues = 0 1479 1480 if all_connectors: 1481 results = check_all_changelogs(repo_path=repo_path, lookback_days=lookback_days) 1482 for result in results: 1483 if result.has_issues or result.errors: 1484 _print_check_result(result) 1485 total_issues += result.issue_count 1486 else: 1487 result = check_changelog( 1488 repo_path=repo_path, 1489 connector_name=connector_name, 1490 lookback_days=lookback_days, 1491 ) 1492 _print_check_result(result) 1493 total_issues = result.issue_count 1494 1495 if total_issues > 0: 1496 console.print(f"\n[bold]Total issues found: {total_issues}[/bold]") 1497 if strict: 1498 exit_with_error(f"Found {total_issues} issue(s) in changelog(s).") 1499 else: 1500 console.print("[green]No issues found.[/green]") 1501 1502 1503def _print_check_result(result: ChangelogCheckResult) -> None: 1504 """Print a changelog check result.""" 1505 if not result.has_issues and not result.errors: 1506 return 1507 1508 console.print(f"\n[bold]{result.connector}[/bold]") 1509 1510 for warning in result.pr_mismatch_warnings: 1511 console.print( 1512 f" [yellow]WARNING[/yellow] Line {warning.line_number} (v{warning.version}): {warning.message}" 1513 ) 1514 1515 for fix in result.date_issues: 1516 if fix.changed: 1517 console.print( 1518 f" [red]DATE MISMATCH[/red] Line {fix.line_number} (v{fix.version}): " 1519 f"changelog has {fix.old_date}, PR merged on {fix.new_date}" 1520 ) 1521 1522 for error in result.errors: 1523 console.print(f" [red]ERROR[/red] {error}") 1524 1525 1526@changelog_app.command(name="fix") 1527def changelog_fix( 1528 connector_name: Annotated[ 1529 str | None, 1530 Parameter(help="Connector technical name (e.g., source-github)."), 1531 ] = None, 1532 all_connectors: Annotated[ 1533 bool, 1534 Parameter("--all", help="Fix all connectors in the repository."), 1535 ] = False, 1536 repo_path: Annotated[ 1537 str | None, 1538 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 1539 ] = None, 1540 lookback_days: Annotated[ 1541 int | None, 1542 Parameter(help="Only fix entries with dates within this many days."), 1543 ] = None, 1544 dry_run: Annotated[ 1545 bool, 1546 Parameter(help="Print changes without modifying files."), 1547 ] = False, 1548) -> None: 1549 """Fix changelog entry dates to match PR merge dates. 1550 1551 Looks up the actual merge date for each PR referenced in the changelog 1552 and updates the date column to match. 1553 """ 1554 if not connector_name and not all_connectors: 1555 exit_with_error("Either --connector-name or --all must be specified.") 1556 1557 if connector_name and all_connectors: 1558 exit_with_error("Cannot specify both --connector-name and --all.") 1559 1560 if repo_path is None: 1561 cwd = Path.cwd() 1562 for parent in [cwd, *cwd.parents]: 1563 if (parent / CONNECTOR_PATH_PREFIX).exists(): 1564 repo_path = str(parent) 1565 break 1566 if repo_path is None: 1567 exit_with_error( 1568 "Could not infer repo path. Please provide --repo-path or run from within the Airbyte monorepo." 1569 ) 1570 1571 total_fixed = 0 1572 total_warnings = 0 1573 1574 if all_connectors: 1575 results = fix_all_changelog_dates( 1576 repo_path=repo_path, dry_run=dry_run, lookback_days=lookback_days 1577 ) 1578 for result in results: 1579 if result.has_changes or result.warnings or result.errors: 1580 _print_fix_result(result) 1581 total_fixed += result.changed_count 1582 total_warnings += len(result.warnings) 1583 else: 1584 result = fix_changelog_dates( 1585 repo_path=repo_path, 1586 connector_name=connector_name, 1587 dry_run=dry_run, 1588 lookback_days=lookback_days, 1589 ) 1590 _print_fix_result(result) 1591 total_fixed = result.changed_count 1592 total_warnings = len(result.warnings) 1593 1594 action = "Would fix" if dry_run else "Fixed" 1595 console.print(f"\n[bold]{action} {total_fixed} date(s).[/bold]") 1596 if total_warnings > 0: 1597 console.print( 1598 f"[yellow]{total_warnings} warning(s) about PR number mismatches.[/yellow]" 1599 ) 1600 1601 1602def _print_fix_result(result: ChangelogFixResult) -> None: 1603 """Print a changelog fix result.""" 1604 if not result.has_changes and not result.warnings and not result.errors: 1605 return 1606 1607 console.print(f"\n[bold]{result.connector}[/bold]") 1608 1609 for warning in result.warnings: 1610 console.print( 1611 f" [yellow]WARNING[/yellow] Line {warning.line_number} (v{warning.version}): {warning.message}" 1612 ) 1613 1614 for fix in result.fixes: 1615 if fix.changed: 1616 action = "Would fix" if result.dry_run else "Fixed" 1617 console.print( 1618 f" [green]{action}[/green] Line {fix.line_number} (v{fix.version}): " 1619 f"{fix.old_date} -> {fix.new_date}" 1620 ) 1621 1622 for error in result.errors: 1623 console.print(f" [red]ERROR[/red] {error}") 1624 1625 1626@changelog_app.command(name="add") 1627def changelog_add( 1628 connector_name: Annotated[ 1629 str, 1630 Parameter(help="Connector technical name (e.g., source-github)."), 1631 ], 1632 pr_number: Annotated[ 1633 int, 1634 Parameter(help="PR number for the changelog entry."), 1635 ], 1636 message: Annotated[ 1637 str, 1638 Parameter(help="Changelog entry message."), 1639 ], 1640 repo_path: Annotated[ 1641 str | None, 1642 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 1643 ] = None, 1644 dry_run: Annotated[ 1645 bool, 1646 Parameter(help="Print changes without modifying files."), 1647 ] = False, 1648) -> None: 1649 """Add a changelog entry for a connector using its current version. 1650 1651 Reads the version from metadata.yaml and writes a single changelog 1652 entry to the connector's documentation file. Does not modify any 1653 version files. 1654 """ 1655 if repo_path is None: 1656 cwd = Path.cwd() 1657 for parent in [cwd, *cwd.parents]: 1658 if (parent / CONNECTOR_PATH_PREFIX).exists(): 1659 repo_path = str(parent) 1660 break 1661 if repo_path is None: 1662 exit_with_error( 1663 "Could not infer repo path. Please provide --repo-path or run from within the Airbyte monorepo." 1664 ) 1665 1666 try: 1667 connector_path = get_connector_path(Path(repo_path), connector_name) 1668 version = get_current_version(connector_path) 1669 except (ConnectorNotFoundError, VersionNotFoundError) as e: 1670 exit_with_error(str(e)) 1671 1672 doc_path = get_connector_doc_path(Path(repo_path), connector_name) 1673 if doc_path is None or not doc_path.exists(): 1674 exit_with_error(f"Documentation file not found for {connector_name}.") 1675 1676 modified = update_changelog( 1677 doc_path=doc_path, 1678 new_version=version, 1679 changelog_message=message, 1680 pr_number=pr_number, 1681 dry_run=dry_run, 1682 ) 1683 1684 action = "Would add" if dry_run else "Added" 1685 if modified: 1686 console.print( 1687 f"[green]{action} changelog entry for {connector_name} v{version}[/green]" 1688 ) 1689 else: 1690 console.print( 1691 f"[yellow]No changes needed for {connector_name} v{version}[/yellow]" 1692 ) 1693 1694 1695# Create the marketing-stub sub-app under connector 1696marketing_stub_app = App( 1697 name="marketing-stub", 1698 help="Marketing connector stub operations (local file validation and updates).", 1699) 1700connector_app.command(marketing_stub_app) 1701 1702# Path to connectors in the airbyte-enterprise repo 1703ENTERPRISE_CONNECTOR_PATH_PREFIX = "airbyte-integrations/connectors" 1704 1705 1706def _build_stub_from_metadata( 1707 connector_name: str, 1708 metadata: dict, 1709 existing_stub: dict | None = None, 1710) -> dict: 1711 """Build a connector stub from metadata.yaml. 1712 1713 Args: 1714 connector_name: The connector name (e.g., 'source-oracle-enterprise'). 1715 metadata: The parsed metadata.yaml content. 1716 existing_stub: Optional existing stub to preserve extra fields from. 1717 1718 Returns: 1719 A connector stub dictionary. 1720 """ 1721 data = metadata.get("data", {}) 1722 1723 # Determine connector type for the stub 1724 connector_type = data.get("connectorType", "source") 1725 stub_type = f"enterprise_{connector_type}" 1726 1727 # Preserve existing stub ID if available, otherwise use connector name 1728 stub_id = (existing_stub.get("id") if existing_stub else None) or connector_name 1729 1730 # Get the icon URL - construct from icon filename if available 1731 icon_filename = data.get("icon", "") 1732 if icon_filename and not icon_filename.startswith("http"): 1733 # Construct icon URL from the standard GCS path 1734 icon_url = f"https://storage.googleapis.com/prod-airbyte-cloud-connector-metadata-service/resources/connector_stubs/v0/icons/{icon_filename}" 1735 else: 1736 icon_url = icon_filename or "" 1737 1738 # Build the stub 1739 stub: dict = { 1740 "id": stub_id, 1741 "name": data.get("name", connector_name.replace("-", " ").title()), 1742 "label": "enterprise", 1743 "icon": icon_url, 1744 "url": data.get("documentationUrl", ""), 1745 "type": stub_type, 1746 } 1747 1748 # Add definitionId if available 1749 definition_id = data.get("definitionId") 1750 if definition_id: 1751 stub["definitionId"] = definition_id 1752 1753 # Preserve extra fields from existing stub (like codename) 1754 if existing_stub: 1755 for key in existing_stub: 1756 if key not in stub: 1757 stub[key] = existing_stub[key] 1758 1759 return stub 1760 1761 1762@marketing_stub_app.command(name="check") 1763def marketing_stub_check( 1764 connector: Annotated[ 1765 str | None, 1766 Parameter(help="Connector name to check (e.g., 'source-oracle-enterprise')."), 1767 ] = None, 1768 all_connectors: Annotated[ 1769 bool, 1770 Parameter("--all", help="Check all stubs in the file."), 1771 ] = False, 1772 repo_root: Annotated[ 1773 Path | None, 1774 Parameter( 1775 help="Path to the airbyte-enterprise repository root. Defaults to current directory." 1776 ), 1777 ] = None, 1778) -> None: 1779 """Validate marketing connector stub entries. 1780 1781 Checks that stub entries have valid required fields (id, name, url, icon) 1782 and optionally validates that the stub matches the connector's metadata.yaml. 1783 1784 Exit codes: 1785 0: All checks passed 1786 1: Validation errors found 1787 1788 Output: 1789 STDOUT: JSON validation result 1790 STDERR: Informational messages 1791 1792 Example: 1793 airbyte-ops local connector marketing-stub check --connector source-oracle-enterprise --repo-root /path/to/airbyte-enterprise 1794 airbyte-ops local connector marketing-stub check --all --repo-root /path/to/airbyte-enterprise 1795 """ 1796 if not connector and not all_connectors: 1797 exit_with_error("Either --connector or --all must be specified.") 1798 1799 if connector and all_connectors: 1800 exit_with_error("Cannot specify both --connector and --all.") 1801 1802 if repo_root is None: 1803 repo_root = Path.cwd() 1804 1805 # Load local stubs 1806 try: 1807 stubs = load_local_stubs(repo_root) 1808 except FileNotFoundError as e: 1809 exit_with_error(str(e)) 1810 except ValueError as e: 1811 exit_with_error(str(e)) 1812 1813 stubs_to_check = stubs if all_connectors else [] 1814 if connector: 1815 stub = find_stub_by_connector(stubs, connector) 1816 if stub is None: 1817 exit_with_error( 1818 f"Connector stub '{connector}' not found in {CONNECTOR_STUBS_FILE}" 1819 ) 1820 stubs_to_check = [stub] 1821 1822 errors: list[dict] = [] 1823 warnings: list[dict] = [] 1824 placeholders: list[dict] = [] 1825 1826 for stub in stubs_to_check: 1827 stub_id = stub.get("id", "<unknown>") 1828 stub_name = stub.get("name", stub_id) 1829 1830 # Check required fields 1831 required_fields = ["id", "name", "url", "icon"] 1832 for field in required_fields: 1833 if not stub.get(field): 1834 errors.append( 1835 {"stub_id": stub_id, "error": f"Missing required field: {field}"} 1836 ) 1837 1838 # Check if corresponding connector exists and validate against metadata 1839 connector_dir = repo_root / ENTERPRISE_CONNECTOR_PATH_PREFIX / stub_id 1840 metadata_file = connector_dir / METADATA_FILE_NAME 1841 1842 if metadata_file.exists(): 1843 metadata = yaml.safe_load(metadata_file.read_text()) 1844 data = metadata.get("data", {}) 1845 1846 # Check if definitionId matches 1847 metadata_def_id = data.get("definitionId") 1848 stub_def_id = stub.get("definitionId") 1849 if metadata_def_id and stub_def_id and metadata_def_id != stub_def_id: 1850 errors.append( 1851 { 1852 "stub_id": stub_id, 1853 "error": f"definitionId mismatch: stub has '{stub_def_id}', metadata has '{metadata_def_id}'", 1854 } 1855 ) 1856 1857 # Check if name matches 1858 metadata_name = data.get("name") 1859 if metadata_name and stub_name and metadata_name != stub_name: 1860 warnings.append( 1861 { 1862 "stub_id": stub_id, 1863 "warning": f"name mismatch: stub has '{stub_name}', metadata has '{metadata_name}'", 1864 } 1865 ) 1866 else: 1867 # No connector directory - this is a registry placeholder for a future connector 1868 placeholders.append( 1869 { 1870 "stub_id": stub_id, 1871 "name": stub_name, 1872 } 1873 ) 1874 1875 result = { 1876 "checked_count": len(stubs_to_check), 1877 "error_count": len(errors), 1878 "warning_count": len(warnings), 1879 "placeholder_count": len(placeholders), 1880 "valid": len(errors) == 0, 1881 "errors": errors, 1882 "warnings": warnings, 1883 "placeholders": placeholders, 1884 } 1885 1886 # Print placeholders as info (not warnings - these are valid registry placeholders) 1887 if placeholders: 1888 error_console.print( 1889 f"[blue]Found {len(placeholders)} registry placeholder(s) (no local directory):[/blue]" 1890 ) 1891 for placeholder in placeholders: 1892 error_console.print( 1893 f" Found Connector Registry Placeholder (no local directory): {placeholder['name']}" 1894 ) 1895 1896 if errors: 1897 error_console.print(f"[red]Found {len(errors)} error(s):[/red]") 1898 for err in errors: 1899 error_console.print(f" {err['stub_id']}: {err['error']}") 1900 1901 if warnings: 1902 error_console.print(f"[yellow]Found {len(warnings)} warning(s):[/yellow]") 1903 for warn in warnings: 1904 error_console.print(f" {warn['stub_id']}: {warn['warning']}") 1905 1906 if not errors and not warnings: 1907 error_console.print( 1908 f"[green]All {len(stubs_to_check)} stub(s) passed validation[/green]" 1909 ) 1910 1911 print_json(result) 1912 1913 if errors: 1914 exit_with_error("Validation failed", code=1) 1915 1916 1917@marketing_stub_app.command(name="sync") 1918def marketing_stub_sync( 1919 connector: Annotated[ 1920 str | None, 1921 Parameter(help="Connector name to sync (e.g., 'source-oracle-enterprise')."), 1922 ] = None, 1923 all_connectors: Annotated[ 1924 bool, 1925 Parameter("--all", help="Sync all connectors that have metadata.yaml files."), 1926 ] = False, 1927 repo_root: Annotated[ 1928 Path | None, 1929 Parameter( 1930 help="Path to the airbyte-enterprise repository root. Defaults to current directory." 1931 ), 1932 ] = None, 1933 dry_run: Annotated[ 1934 bool, 1935 Parameter(help="Show what would be synced without making changes."), 1936 ] = False, 1937) -> None: 1938 """Sync connector stub(s) from connector metadata.yaml file(s). 1939 1940 Reads the connector's metadata.yaml file and updates the corresponding 1941 entry in connector_stubs.json with the current values. 1942 1943 Exit codes: 1944 0: Sync successful (or dry-run completed) 1945 1: Error (connector not found, no metadata, etc.) 1946 1947 Output: 1948 STDOUT: JSON representation of the synced stub(s) 1949 STDERR: Informational messages 1950 1951 Example: 1952 airbyte-ops local connector marketing-stub sync --connector source-oracle-enterprise --repo-root /path/to/airbyte-enterprise 1953 airbyte-ops local connector marketing-stub sync --all --repo-root /path/to/airbyte-enterprise 1954 airbyte-ops local connector marketing-stub sync --connector source-oracle-enterprise --dry-run 1955 """ 1956 if not connector and not all_connectors: 1957 exit_with_error("Either --connector or --all must be specified.") 1958 1959 if connector and all_connectors: 1960 exit_with_error("Cannot specify both --connector and --all.") 1961 1962 if repo_root is None: 1963 repo_root = Path.cwd() 1964 1965 # Load existing stubs 1966 try: 1967 stubs = load_local_stubs(repo_root) 1968 except FileNotFoundError: 1969 stubs = [] 1970 except ValueError as e: 1971 exit_with_error(str(e)) 1972 1973 # Determine which connectors to sync 1974 connectors_to_sync: list[str] = [] 1975 if connector: 1976 connectors_to_sync = [connector] 1977 else: 1978 # Find all connectors with metadata.yaml in the enterprise connectors directory 1979 connectors_dir = repo_root / ENTERPRISE_CONNECTOR_PATH_PREFIX 1980 if connectors_dir.exists(): 1981 for item in connectors_dir.iterdir(): 1982 if item.is_dir() and (item / METADATA_FILE_NAME).exists(): 1983 connectors_to_sync.append(item.name) 1984 connectors_to_sync.sort() 1985 1986 if not connectors_to_sync: 1987 exit_with_error("No connectors found to sync.") 1988 1989 synced_stubs: list[dict] = [] 1990 updated_count = 0 1991 added_count = 0 1992 1993 for conn_name in connectors_to_sync: 1994 connector_dir = repo_root / ENTERPRISE_CONNECTOR_PATH_PREFIX / conn_name 1995 metadata_file = connector_dir / METADATA_FILE_NAME 1996 1997 if not connector_dir.exists(): 1998 if connector: 1999 exit_with_error(f"Connector directory not found: {connector_dir}") 2000 continue 2001 2002 if not metadata_file.exists(): 2003 if connector: 2004 exit_with_error(f"Metadata file not found: {metadata_file}") 2005 continue 2006 2007 # Load metadata 2008 metadata = yaml.safe_load(metadata_file.read_text()) 2009 2010 # Find existing stub if any 2011 existing_stub = find_stub_by_connector(stubs, conn_name) 2012 2013 # Build new stub from metadata 2014 new_stub = _build_stub_from_metadata(conn_name, metadata, existing_stub) 2015 2016 # Validate the new stub 2017 ConnectorStub(**new_stub) 2018 2019 if dry_run: 2020 action = "update" if existing_stub else "create" 2021 error_console.print(f"[DRY RUN] Would {action} stub for '{conn_name}'") 2022 synced_stubs.append(new_stub) 2023 continue 2024 2025 # Update or add the stub 2026 if existing_stub: 2027 # Find and replace 2028 for i, stub in enumerate(stubs): 2029 if stub.get("id") == existing_stub.get("id"): 2030 stubs[i] = new_stub 2031 break 2032 updated_count += 1 2033 else: 2034 stubs.append(new_stub) 2035 added_count += 1 2036 2037 synced_stubs.append(new_stub) 2038 2039 if not dry_run: 2040 # Save the updated stubs 2041 save_local_stubs(repo_root, stubs) 2042 error_console.print( 2043 f"[green]Synced {len(synced_stubs)} stub(s) to {CONNECTOR_STUBS_FILE} " 2044 f"({added_count} added, {updated_count} updated)[/green]" 2045 ) 2046 else: 2047 error_console.print( 2048 f"[DRY RUN] Would sync {len(synced_stubs)} stub(s) to {CONNECTOR_STUBS_FILE}" 2049 ) 2050 2051 print_json( 2052 synced_stubs if all_connectors else synced_stubs[0] if synced_stubs else {} 2053 ) 2054 2055 2056# Create the release-block sub-app under connector 2057release_block_app = App( 2058 name="release-block", 2059 help="Manage release block markers for connectors.", 2060) 2061connector_app.command(release_block_app) 2062 2063 2064@release_block_app.command(name="add") 2065def release_block_add( 2066 name: Annotated[ 2067 str, 2068 Parameter(help="Connector technical name (e.g., source-faker)."), 2069 ], 2070 reason: Annotated[ 2071 str, 2072 Parameter(help="Human-readable reason for blocking the release."), 2073 ], 2074 repo_path: Annotated[ 2075 str | None, 2076 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 2077 ] = None, 2078 yanked_version: Annotated[ 2079 str | None, 2080 Parameter(help="Version that was yanked (for reference)."), 2081 ] = None, 2082 blocked_by: Annotated[ 2083 str | None, 2084 Parameter(help="Email or identifier of the person requesting the block."), 2085 ] = None, 2086) -> None: 2087 """Add a `block-release.yaml` marker to prevent publishing a connector. 2088 2089 Creates a marker file in the connector's directory that causes the publish 2090 pipeline to skip the connector with a warning. 2091 2092 Example: 2093 2094 airbyte-ops local connector release-block add \\ 2095 --name source-faker \\ 2096 --reason "Version 5.0.1 yanked due to regression" \\ 2097 --repo-path /path/to/airbyte 2098 """ 2099 resolved_path = repo_path or os.environ.get("AIRBYTE_REPO_PATH", ".") 2100 result = add_release_block( 2101 repo_path=resolved_path, 2102 connector_name=name, 2103 reason=reason, 2104 yanked_version=yanked_version, 2105 blocked_by=blocked_by, 2106 ) 2107 if result.success: 2108 error_console.print(f"[green]{result.message}[/green]") 2109 else: 2110 exit_with_error(result.message) 2111 print_json(result.to_dict()) 2112 2113 2114@release_block_app.command(name="clear") 2115def release_block_clear( 2116 name: Annotated[ 2117 str, 2118 Parameter(help="Connector technical name (e.g., source-faker)."), 2119 ], 2120 repo_path: Annotated[ 2121 str | None, 2122 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 2123 ] = None, 2124) -> None: 2125 """Remove the `block-release.yaml` marker to allow publishing a connector. 2126 2127 Example: 2128 2129 airbyte-ops local connector release-block clear \\ 2130 --name source-faker \\ 2131 --repo-path /path/to/airbyte 2132 """ 2133 resolved_path = repo_path or os.environ.get("AIRBYTE_REPO_PATH", ".") 2134 result = clear_release_block( 2135 repo_path=resolved_path, 2136 connector_name=name, 2137 ) 2138 if result.success: 2139 error_console.print(f"[green]{result.message}[/green]") 2140 else: 2141 exit_with_error(result.message) 2142 print_json(result.to_dict()) 2143 2144 2145ReleaseBlockListFormat = Literal["text", "json", "csv"] 2146 2147 2148@release_block_app.command(name="list") 2149def release_block_list( 2150 repo_path: Annotated[ 2151 str | None, 2152 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 2153 ] = None, 2154 output_format: Annotated[ 2155 ReleaseBlockListFormat, 2156 Parameter( 2157 help=( 2158 'Output format: "text" (human-readable), ' 2159 '"json" (blocked connectors with block file contents), ' 2160 '"csv" (comma-delimited connector names).' 2161 ) 2162 ), 2163 ] = "text", 2164) -> None: 2165 """List all connectors that have a `block-release.yaml` marker. 2166 2167 Example: 2168 2169 airbyte-ops local connector release-block list --repo-path /path/to/airbyte 2170 airbyte-ops local connector release-block list --output-format json 2171 airbyte-ops local connector release-block list --output-format csv 2172 """ 2173 resolved_path = repo_path or os.environ.get("AIRBYTE_REPO_PATH", ".") 2174 result = list_release_blocks(repo_path=resolved_path) 2175 2176 if output_format == "json": 2177 print_json(result.to_dict()) 2178 elif output_format == "csv": 2179 names = [info.connector_name for info in result.blocked_connectors] 2180 sys.stdout.write(",".join(names) + "\n") 2181 else: 2182 # text format (default) 2183 if result.count == 0: 2184 console.print( 2185 "[green]No connectors are currently blocked from release.[/green]" 2186 ) 2187 else: 2188 console.print( 2189 f"[yellow]{result.count} connector(s) blocked from release:[/yellow]" 2190 ) 2191 for info in result.blocked_connectors: 2192 version_str = ( 2193 f" (yanked: {info.yanked_version})" if info.yanked_version else "" 2194 ) 2195 console.print(f" - {info.connector_name}: {info.reason}{version_str}")