airbyte_ops_mcp.cli.local
CLI commands for local Airbyte monorepo operations.
Commands:
airbyte-ops local connector list - List connectors in the monorepo airbyte-ops local connector info - Get metadata for a single connector airbyte-ops local connector get-version - Get connector version (current or next) airbyte-ops local connector bump-version - Bump connector version airbyte-ops local connector rollouts enable - Enable autopilot rollouts airbyte-ops local connector rollouts disable - Disable autopilot rollouts airbyte-ops local connector qa - Run QA checks on a connector airbyte-ops local connector qa-docs-generate - Generate QA checks documentation airbyte-ops local connector changelog check - Check changelog entries for issues airbyte-ops local connector changelog fix - Fix changelog entry dates airbyte-ops local connector bump-deps - Update Poetry-managed dependencies airbyte-ops local connector marketing-stub check - Validate marketing stub entries airbyte-ops local connector marketing-stub sync - Sync stub from connector metadata airbyte-ops local connector release-block add - Add a connector release block marker airbyte-ops local connector release-block clear - Clear a connector release block marker airbyte-ops local connector release-block list - List connector release block markers
CLI reference
The commands below are regenerated by poe docs-generate via cyclopts's
programmatic docs API; see docs/generate_cli.py.
airbyte-ops local COMMAND
Local Airbyte monorepo operations.
Commands:
connector: Connector operations in the monorepo.
airbyte-ops local connector
Connector operations in the monorepo.
airbyte-ops local connector list
airbyte-ops local connector list REPO-PATH [ARGS]
List connectors in the Airbyte monorepo with filtering options.
Parameters:
REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]CERTIFIED-ONLY, --certified-only, --no-certified-only: Include only certified connectors. [default: False]MODIFIED-ONLY, --modified-only, --no-modified-only: Include only modified connectors (requires PR context). [default: False]LOCAL-CDK, --local-cdk, --no-local-cdk: Include connectors using local CDK reference. When combined with --modified-only, adds local-CDK connectors to the modified set. [default: False]LANGUAGE, --language, --empty-language: Languages to include (python, java, low-code, manifest-only).EXCLUDE-LANGUAGE, --exclude-language, --empty-exclude-language: Languages to exclude.CONNECTOR-TYPE, --connector-type: Connector types to include (source, destination). Accepts CSV or newline-delimited values.MIN-SUPPORT-LEVEL, --min-support-level: Minimum support level (inclusive). Accepts integer (100, 200, 300) or keyword (archived, community, certified).MAX-SUPPORT-LEVEL, --max-support-level: Maximum support level (inclusive). Accepts integer (100, 200, 300) or keyword (archived, community, certified).AUTOPILOT-ENABLED, --autopilot-enabled, --no-autopilot-enabled: Filter by autopilot progressive rollout status. Pass--autopilot-enabled=trueto keep only connectors that have autopilot rollouts enabled (defaultRolloutMode: autopilotandenableProgressiveRollout: true), or--autopilot-enabled=falsefor those that do not. When omitted, connectors are not filtered on rollout status.PR, --pr: PR number or GitHub URL for modification detection.GH-TOKEN, --gh-token: GitHub API token. When provided together with --pr, the GitHub API is used to detect modified files instead of local git diff (avoids shallow-clone issues).EXCLUDE-CONNECTORS, --exclude-connectors, --empty-exclude-connectors: Connectors to exclude from results. Accepts CSV or newline-delimited values. Can be specified multiple times.FORCE-INCLUDE-CONNECTORS, --force-include-connectors, --empty-force-include-connectors: Connectors to force-include regardless of other filters. Accepts CSV or newline-delimited values. Can be specified multiple times.CONNECTORS-FILTER, --connectors-filter, --empty-connectors-filter: Intersect results with this explicit set of connector names. Only connectors present in both the filtered results and this set are returned. Useful for composing multiple filter passes (e.g. combining separate source and destination lists). Accepts CSV or newline-delimited values. Can be specified multiple times.OUTPUT-FORMAT, --output-format: Output format: "csv" (comma-separated), "lines" (one connector per line), "json-gh-matrix" (GitHub Actions matrix JSON). [choices: csv, lines, json-gh-matrix] [default: lines]UNPUBLISHED, --unpublished, --no-unpublished: Filter to only connectors whose local dockerImageTag has not been published to the GCS registry. Requires --store. [default: False]STORE, --store: Store target for unpublished check (e.g. 'coral:prod', 'coral:dev'). Required when --unpublished is set.ASSERT-NONE, --assert-none, --no-assert-none: Exit with non-zero status if any connectors match the filters. Useful for audit/CI checks (e.g. --unpublished --assert-none). [default: False]SORT-BY, --sort-by: Sort connectors by the given key. Only 'name' is supported for now. [choices: name] [default: name]SORT-DIRECTION, --sort-direction: Sort direction: 'asc' (ascending, default) or 'desc' (descending). [choices: asc, desc] [default: asc]LIMIT, --limit: Maximum number of connectors to return. Applied after sorting. Useful for batched processing.OFFSET, --offset: Number of connectors to skip from the start of the (sorted) list. Applied after sorting, before --limit. Useful for batched processing (e.g. --offset=200 --limit=200 for batch 2). [default: 0]
airbyte-ops local connector info
airbyte-ops local connector info CONNECTOR-NAME [ARGS]
Get metadata for a single connector.
Prints JSON output with connector metadata. When running in GitHub Actions (CI env var set), also writes each field to GitHub step outputs.
Parameters:
CONNECTOR-NAME, --connector-name: Name of the connector (e.g., source-github). [required]REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.DPATH, --dpath: Evaluate this dpath expression against the parsed metadata.yaml object and print only that value (e.g., data/dockerImageTag).
airbyte-ops local connector get-version
airbyte-ops local connector get-version NAME REPO-PATH [ARGS]
Get the current or next version for a connector.
By default, prints the current version from metadata.yaml.
This is analogous to local connector info --dpath data/dockerImageTag and
uses the same dpath evaluation internally.
With --next, computes and prints the next version. Requires either --bump-type or --prerelease to be specified.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]NEXT, --next, --no-next: Compute the next version instead of the current version. [default: False]BUMP-TYPE, --bump-type: Version bump type (requires --next). Standard: patch, minor, major. RC: patch_rc, minor_rc, major_rc, rc, promote. [choices: patch, minor, major, patch_rc, minor_rc, major_rc, rc, promote]PRERELEASE, --prerelease, --no-prerelease: Compute a prerelease (preview) tag using the repo HEAD SHA (requires --next). [default: False]
airbyte-ops local connector bump-version
airbyte-ops local connector bump-version NAME REPO-PATH [ARGS]
Bump a connector's version across all relevant files.
Updates version in metadata.yaml (always), pyproject.toml (if exists), and documentation changelog (if --changelog-message provided).
Note: --changelog-message is ignored when --no-changelog is set.
Either --bump-type or --new-version must be provided.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]BUMP-TYPE, --bump-type: Version bump type. Standard: patch, minor, major. RC: patch_rc, minor_rc, major_rc, rc, promote. [choices: patch, minor, major, patch_rc, minor_rc, major_rc, rc, promote]NEW-VERSION, --new-version: Explicit new version (overrides --bump-type if provided).CHANGELOG-MESSAGE, --changelog-message: Message to add to changelog. Ignored if --no-changelog is set.PR-NUMBER, --pr-number: PR number for changelog entry.DRY-RUN, --dry-run, --no-dry-run: Show what would be changed without modifying files. [default: False]NO-CHANGELOG, --no-changelog, --no-no-changelog: Skip changelog updates even if --changelog-message is provided. Useful for ephemeral version bumps (e.g. pre-release artifact generation). [default: False]PROGRESSIVE-ROLLOUT-ENABLED, --progressive-rollout-enabled, --no-progressive-rollout-enabled: Explicitly setenableProgressiveRolloutin metadata.yaml. Passfalseto disable progressive rollout (e.g. for preview builds). When omitted, the automatic behaviour based on --bump-type is used.
airbyte-ops local connector rollouts
Manage a connector's progressive rollout configuration.
Commands:
disable: Disable autopilot rollouts in a connector's metadata.yaml.enable: Enable autopilot rollouts in a connector's metadata.yaml.
airbyte-ops local connector rollouts enable
airbyte-ops local connector rollouts enable NAME REPO-PATH [ARGS]
Enable autopilot rollouts in a connector's metadata.yaml.
Always sets releases.rolloutConfiguration.defaultRolloutMode: autopilot
(autopilot is the only actively-supported mode, so it is implicit here),
fills in autopilotConfig (autoStart, autoPromoteStages, strategy),
and sets enableProgressiveRollout: true — the toggle that turns automatic
progressive rollout on. Defaults only fill in autopilotConfig fields that
are absent, so existing settings are preserved.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]STRATEGY, --strategy: Autopilot pacing strategy: 'fast', 'slow', or 'default'. 'default' is a server-side alias for 'fast'; prefer the explicit 'fast'. Omitted keeps any existing value, or 'fast' when none is set. [choices: fast, slow, default]DRY-RUN, --dry-run, --no-dry-run: Show what would be changed without modifying files. [default: False]
airbyte-ops local connector rollouts disable
airbyte-ops local connector rollouts disable NAME REPO-PATH [ARGS]
Disable autopilot rollouts in a connector's metadata.yaml.
Only sets releases.rolloutConfiguration.enableProgressiveRollout: false —
the toggle that turns automatic progressive rollout off. defaultRolloutMode
and any autopilotConfig are retained (inert until re-enabled), so the
change is lossless. Connectors with no rollout config, or with the flag
already off, are a no-op.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]DRY-RUN, --dry-run, --no-dry-run: Show what would be changed without modifying files. [default: False]
airbyte-ops local connector bump-base-image
airbyte-ops local connector bump-base-image NAME REPO-PATH [ARGS]
Update a connector's base image.
Two modes:
- Default: bump to the latest stable tag within the same major version. Major version changes are treated as breaking-change boundaries.
- --force-latest: bump to the absolute latest stable tag regardless of semver.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]FORCE-LATEST, --force-latest, --no-force-latest: Bump to the absolute latest stable base image, ignoring major-version boundaries. Without this flag the bump stays within the current major version. [default: False]DRY-RUN, --dry-run, --no-dry-run: Show what would be changed without modifying files. [default: False]
airbyte-ops local connector bump-cdk
airbyte-ops local connector bump-cdk NAME REPO-PATH [ARGS]
Bump a connector's CDK dependency.
Two modes:
- Default: refresh the lock file so it resolves the newest CDK that satisfies the existing constraint. The constraint is NOT changed.
- --force-latest: rewrite the constraint to >=LATEST,
For Java connectors, updates build.gradle to the latest CDK version.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]FORCE-LATEST, --force-latest, --no-force-latest: Rewrite the CDK constraint to >=LATEST,[default: False] DRY-RUN, --dry-run, --no-dry-run: Show what would be changed without modifying files. [default: False]
airbyte-ops local connector bump-deps
airbyte-ops local connector bump-deps NAME REPO-PATH [ARGS]
Update a connector's dependencies.
For Python / low-code connectors using Poetry, this runs
poetry update --lock to refresh the lock file with the latest
versions allowed by existing constraints.
For connectors that do not use Poetry (manifest-only, Java, etc.), this is a no-op.
Parameters:
NAME, --name: Connector technical name (e.g., source-github). [required]REPO-PATH, --repo-path: Absolute path to the Airbyte monorepo. [required]DRY-RUN, --dry-run, --no-dry-run: Show what would be changed without modifying files. [default: False]
airbyte-ops local connector qa
airbyte-ops local connector qa [ARGS]
Run QA checks on connector(s).
Validates connector metadata, documentation, packaging, security, and versioning. Exit code is non-zero if any blocking checks fail.
Parameters:
NAME, --name, --empty-name: Connector technical name(s) (e.g., source-github). Can be specified multiple times.CONNECTOR-DIRECTORY, --connector-directory: Directory containing connectors to run checks on all connectors in this directory.CHECK, --check, --empty-check: Specific check(s) to run. Can be specified multiple times.REPORT-PATH, --report-path: Path to write the JSON report file.
airbyte-ops local connector qa-docs-generate
airbyte-ops local connector qa-docs-generate OUTPUT-FILE
Generate documentation for QA checks.
Creates a markdown file documenting all available QA checks organized by category.
Parameters:
OUTPUT-FILE, --output-file: Path to write the generated documentation file. [required]
airbyte-ops local connector changelog
Changelog operations for connectors.
Commands:
add: Add a changelog entry for a connector using its current version.check: Check changelog entries for issues.fix: Fix changelog entry dates to match PR merge dates.
airbyte-ops local connector changelog check
airbyte-ops local connector changelog check [ARGS]
Check changelog entries for issues.
Validates changelog dates match PR merge dates and checks for PR number mismatches.
Parameters:
CONNECTOR-NAME, --connector-name: Connector technical name (e.g., source-github).ALL, --all, --no-all: Check all connectors in the repository. [default: False]REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.LOOKBACK-DAYS, --lookback-days: Only check entries with dates within this many days.STRICT, --strict, --no-strict: Exit with error code if any issues are found. [default: False]
airbyte-ops local connector changelog fix
airbyte-ops local connector changelog fix [ARGS]
Fix changelog entry dates to match PR merge dates.
Looks up the actual merge date for each PR referenced in the changelog and updates the date column to match.
Parameters:
CONNECTOR-NAME, --connector-name: Connector technical name (e.g., source-github).ALL, --all, --no-all: Fix all connectors in the repository. [default: False]REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.LOOKBACK-DAYS, --lookback-days: Only fix entries with dates within this many days.DRY-RUN, --dry-run, --no-dry-run: Print changes without modifying files. [default: False]
airbyte-ops local connector changelog add
airbyte-ops local connector changelog add CONNECTOR-NAME PR-NUMBER MESSAGE [ARGS]
Add a changelog entry for a connector using its current version.
Reads the version from metadata.yaml and writes a single changelog entry to the connector's documentation file. Does not modify any version files.
Parameters:
CONNECTOR-NAME, --connector-name: Connector technical name (e.g., source-github). [required]PR-NUMBER, --pr-number: PR number for the changelog entry. [required]MESSAGE, --message: Changelog entry message. [required]REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.DRY-RUN, --dry-run, --no-dry-run: Print changes without modifying files. [default: False]
airbyte-ops local connector marketing-stub
Marketing connector stub operations (local file validation and updates).
Commands:
check: Validate marketing connector stub entries.sync: Sync connector stub(s) from connector metadata.yaml file(s).
airbyte-ops local connector marketing-stub check
airbyte-ops local connector marketing-stub check [ARGS]
Validate marketing connector stub entries.
Checks that stub entries have valid required fields (id, name, url, icon) and optionally validates that the stub matches the connector's metadata.yaml.
Exit codes:
0: All checks passed 1: Validation errors found
Output:
STDOUT: JSON validation result STDERR: Informational messages
Parameters:
CONNECTOR, --connector: Connector name to check (e.g., 'source-oracle-enterprise').ALL, --all, --no-all: Check all stubs in the file. [default: False]REPO-ROOT, --repo-root: Path to the airbyte-enterprise repository root. Defaults to current directory.
airbyte-ops local connector marketing-stub sync
airbyte-ops local connector marketing-stub sync [ARGS]
Sync connector stub(s) from connector metadata.yaml file(s).
Reads the connector's metadata.yaml file and updates the corresponding entry in connector_stubs.json with the current values.
Exit codes:
0: Sync successful (or dry-run completed) 1: Error (connector not found, no metadata, etc.)
Output:
STDOUT: JSON representation of the synced stub(s) STDERR: Informational messages
Parameters:
CONNECTOR, --connector: Connector name to sync (e.g., 'source-oracle-enterprise').ALL, --all, --no-all: Sync all connectors that have metadata.yaml files. [default: False]REPO-ROOT, --repo-root: Path to the airbyte-enterprise repository root. Defaults to current directory.DRY-RUN, --dry-run, --no-dry-run: Show what would be synced without making changes. [default: False]
airbyte-ops local connector release-block
Manage release block markers for connectors.
Commands:
add: Add ablock-release.yamlmarker to prevent publishing a connector.clear: Remove theblock-release.yamlmarker to allow publishing a connector.list: List all connectors that have ablock-release.yamlmarker.
airbyte-ops local connector release-block add
airbyte-ops local connector release-block add NAME REASON [ARGS]
Add a block-release.yaml marker to prevent publishing a connector.
Creates a marker file in the connector's directory that causes the publish pipeline to skip the connector with a warning.
Parameters:
NAME, --name: Connector technical name (e.g., source-faker). [required]REASON, --reason: Human-readable reason for blocking the release. [required]REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.YANKED-VERSION, --yanked-version: Version that was yanked (for reference).BLOCKED-BY, --blocked-by: Email or identifier of the person requesting the block.
airbyte-ops local connector release-block clear
airbyte-ops local connector release-block clear NAME [ARGS]
Remove the block-release.yaml marker to allow publishing a connector.
Parameters:
NAME, --name: Connector technical name (e.g., source-faker). [required]REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.
airbyte-ops local connector release-block list
airbyte-ops local connector release-block list [ARGS]
List all connectors that have a block-release.yaml marker.
Parameters:
REPO-PATH, --repo-path: Path to the Airbyte monorepo. Can be inferred from context.OUTPUT-FORMAT, --output-format: Output format: "text" (human-readable), "json" (blocked connectors with block file contents), "csv" (comma-delimited connector names). [choices: text, json, csv] [default: text]
1# Copyright (c) 2025 Airbyte, Inc., all rights reserved. 2"""CLI commands for local Airbyte monorepo operations. 3 4Commands: 5 airbyte-ops local connector list - List connectors in the monorepo 6 airbyte-ops local connector info - Get metadata for a single connector 7 airbyte-ops local connector get-version - Get connector version (current or next) 8 airbyte-ops local connector bump-version - Bump connector version 9 airbyte-ops local connector rollouts enable - Enable autopilot rollouts 10 airbyte-ops local connector rollouts disable - Disable autopilot rollouts 11 airbyte-ops local connector qa - Run QA checks on a connector 12 airbyte-ops local connector qa-docs-generate - Generate QA checks documentation 13 airbyte-ops local connector changelog check - Check changelog entries for issues 14 airbyte-ops local connector changelog fix - Fix changelog entry dates 15 airbyte-ops local connector bump-deps - Update Poetry-managed dependencies 16 airbyte-ops local connector marketing-stub check - Validate marketing stub entries 17 airbyte-ops local connector marketing-stub sync - Sync stub from connector metadata 18 airbyte-ops local connector release-block add - Add a connector release block marker 19 airbyte-ops local connector release-block clear - Clear a connector release block marker 20 airbyte-ops local connector release-block list - List connector release block markers 21 22## CLI reference 23 24The commands below are regenerated by `poe docs-generate` via cyclopts's 25programmatic docs API; see `docs/generate_cli.py`. 26 27.. include:: ../../../docs/generated/cli/local.md 28 :start-line: 2 29""" 30 31from __future__ import annotations 32 33# Hide Python-level members from the pdoc page for this module; the rendered 34# docs for this CLI group come entirely from the grafted `.. include::` in 35# the module docstring above. 36__all__: list[str] = [] 37 38import json 39import os 40import subprocess 41import sys 42from pathlib import Path 43from typing import Annotated, Literal 44 45import yaml 46from airbyte_connector_models.metadata.v0.connector_metadata_definition_v0 import ( 47 ConnectorMetadataDefinitionV0DataConnectorReleasesRolloutConfigurationAutopilotConfigStrategy as AutopilotStrategy, 48) 49from cyclopts import Parameter 50from fastmcp_extensions.cli import exit_with_error, print_json 51from jinja2 import Environment, PackageLoader, select_autoescape 52from rich.console import Console 53from rich.markdown import Markdown 54 55from airbyte_ops_mcp.airbyte_repo.bump_base_image import ( 56 BaseImageError, 57 bump_base_image, 58) 59from airbyte_ops_mcp.airbyte_repo.bump_cdk import ( 60 CdkBumpError, 61 bump_cdk, 62) 63from airbyte_ops_mcp.airbyte_repo.bump_deps import ( 64 DepsError, 65 bump_deps, 66) 67from airbyte_ops_mcp.airbyte_repo.bump_version import ( 68 ConnectorNotFoundError, 69 InvalidVersionError, 70 VersionNotFoundError, 71 bump_connector_version, 72 calculate_new_version, 73 get_connector_doc_path, 74 get_connector_path, 75 get_current_version, 76 update_changelog, 77) 78from airbyte_ops_mcp.airbyte_repo.changelog_fix import ( 79 ChangelogCheckResult, 80 ChangelogFixResult, 81 check_all_changelogs, 82 check_changelog, 83 fix_all_changelog_dates, 84 fix_changelog_dates, 85) 86from airbyte_ops_mcp.airbyte_repo.list_connectors import ( 87 CONNECTOR_PATH_PREFIX, 88 METADATA_FILE_NAME, 89 _detect_connector_language, 90 get_connectors_with_local_cdk, 91) 92from airbyte_ops_mcp.airbyte_repo.progressive_rollout import ( 93 disable_autopilot_rollout, 94 enable_autopilot_rollout, 95 is_autopilot_rollout_enabled, 96) 97from airbyte_ops_mcp.airbyte_repo.release_block import ( 98 add_release_block, 99 clear_release_block, 100 list_release_blocks, 101) 102from airbyte_ops_mcp.cli._base import App, app 103from airbyte_ops_mcp.connector_metadata import ( 104 ConnectorMetadataDpathError, 105 ConnectorMetadataDpathNotFoundError, 106 format_metadata_dpath_value, 107 get_connector_version_from_metadata, 108 load_raw_connector_metadata_from_local, 109) 110from airbyte_ops_mcp.connector_ops.utils import Connector 111from airbyte_ops_mcp.connector_qa.checks import ENABLED_CHECKS 112from airbyte_ops_mcp.connector_qa.checks.form_ux import render_form_ux_summary 113from airbyte_ops_mcp.connector_qa.consts import CONNECTORS_QA_DOC_TEMPLATE_NAME 114from airbyte_ops_mcp.connector_qa.models import ( 115 Check, 116 CheckCategory, 117 CheckStatus, 118 Report, 119) 120from airbyte_ops_mcp.connector_qa.utils import ( 121 get_all_connectors_in_directory, 122 remove_strict_encrypt_suffix, 123) 124from airbyte_ops_mcp.mcp.connector_registry import list_connectors_in_repo 125from airbyte_ops_mcp.mcp.connector_versions import compute_prerelease_docker_image_tag 126from airbyte_ops_mcp.registry._enums import ConnectorType, SupportLevel 127from airbyte_ops_mcp.registry.audit import ( 128 AuditResult, 129 find_unpublished_connectors, 130 generate_connector_list_summary, 131) 132from airbyte_ops_mcp.registry.connector_stubs import ( 133 CONNECTOR_STUBS_FILE, 134 ConnectorStub, 135 find_stub_by_connector, 136 load_local_stubs, 137 save_local_stubs, 138) 139from airbyte_ops_mcp.registry.store import resolve_registry_store 140from airbyte_ops_mcp.regression_tests.ci_output import write_github_summary 141 142console = Console() 143error_console = Console(stderr=True) 144 145OutputFormat = Literal["csv", "lines", "json-gh-matrix"] 146SortBy = Literal["name"] 147 148 149def _parse_support_level(value: str) -> SupportLevel: 150 """Parse a support level string into a `SupportLevel` enum member. 151 152 Accepts a keyword ("certified", "community", "archived") or a legacy 153 integer string ("100", "200", "300"). 154 """ 155 return SupportLevel.parse(value.strip().lower()) 156 157 158def _get_connector_support_level(connector_dir: Path) -> SupportLevel | None: 159 """Read support level from connector's metadata.yaml.""" 160 metadata_file = connector_dir / METADATA_FILE_NAME 161 if not metadata_file.exists(): 162 return None 163 metadata = yaml.safe_load(metadata_file.read_text()) 164 support_level_str = metadata.get("data", {}).get("supportLevel") 165 if not support_level_str: 166 return None 167 try: 168 return SupportLevel(support_level_str.lower()) 169 except ValueError: 170 return None 171 172 173def _parse_connector_types(value: str) -> set[ConnectorType]: 174 """Parse connector types from CSV or newline-delimited string.""" 175 types: set[ConnectorType] = set() 176 for item in value.replace(",", "\n").split("\n"): 177 item = item.strip().lower() 178 if item: 179 types.add(ConnectorType.parse(item)) 180 return types 181 182 183def _get_connector_type(connector_name: str) -> str: 184 """Derive connector type from name prefix.""" 185 if connector_name.startswith("source-"): 186 return ConnectorType.SOURCE 187 elif connector_name.startswith("destination-"): 188 return ConnectorType.DESTINATION 189 return "unknown" 190 191 192def _parse_connector_names(value: str) -> set[str]: 193 """Parse connector names from CSV or newline-delimited string.""" 194 names = set() 195 for item in value.replace(",", "\n").split("\n"): 196 item = item.strip() 197 if item: 198 names.add(item) 199 return names 200 201 202def _get_connector_version(connector_dir: Path) -> str | None: 203 """Read connector version (dockerImageTag) from metadata.yaml.""" 204 metadata_file = connector_dir / METADATA_FILE_NAME 205 if not metadata_file.exists(): 206 return None 207 metadata = yaml.safe_load(metadata_file.read_text()) 208 return metadata.get("data", {}).get("dockerImageTag") 209 210 211def _get_connector_autopilot_enabled(connector_dir: Path) -> bool: 212 """Return whether a connector has autopilot progressive rollout enabled. 213 214 Reads the connector's `metadata.yaml` and delegates to 215 `is_autopilot_rollout_enabled`. A connector with no metadata file, or with 216 no (or incomplete) `releases.rolloutConfiguration`, is treated as not 217 enabled. 218 """ 219 metadata_file = connector_dir / METADATA_FILE_NAME 220 if not metadata_file.exists(): 221 return False 222 metadata = yaml.safe_load(metadata_file.read_text()) 223 data = metadata.get("data") if isinstance(metadata, dict) else None 224 if not isinstance(data, dict): 225 return False 226 return is_autopilot_rollout_enabled(data) 227 228 229def _support_level_to_int(level: SupportLevel | None) -> int | None: 230 """Convert a `SupportLevel` to its legacy integer representation for JSON output.""" 231 if level is None: 232 return None 233 return level.precedence 234 235 236def _get_connector_info( 237 connector_name: str, connector_dir: Path 238) -> dict[str, str | int | None]: 239 """Get full connector metadata as a dict with connector_ prefixed keys. 240 241 This is shared between the `list --output-format json-gh-matrix` and `info` commands. 242 """ 243 return { 244 "connector": connector_name, 245 "connector_type": _get_connector_type(connector_name), 246 "connector_language": _detect_connector_language(connector_dir, connector_name) 247 or "unknown", 248 "connector_support_level": _support_level_to_int( 249 _get_connector_support_level(connector_dir) 250 ), 251 "connector_version": _get_connector_version(connector_dir), 252 "connector_dir": f"{CONNECTOR_PATH_PREFIX}/{connector_name}", 253 } 254 255 256# Create the local sub-app 257local_app = App(name="local", help="Local Airbyte monorepo operations.") 258app.command(local_app) 259 260# Create the connector sub-app under local 261connector_app = App(name="connector", help="Connector operations in the monorepo.") 262local_app.command(connector_app) 263 264 265@connector_app.command(name="list") 266def list_connectors( 267 repo_path: Annotated[ 268 str, 269 Parameter(help="Absolute path to the Airbyte monorepo."), 270 ], 271 certified_only: Annotated[ 272 bool, 273 Parameter(help="Include only certified connectors."), 274 ] = False, 275 modified_only: Annotated[ 276 bool, 277 Parameter(help="Include only modified connectors (requires PR context)."), 278 ] = False, 279 local_cdk: Annotated[ 280 bool, 281 Parameter( 282 help=( 283 "Include connectors using local CDK reference. " 284 "When combined with --modified-only, adds local-CDK connectors to the modified set." 285 ) 286 ), 287 ] = False, 288 language: Annotated[ 289 list[str] | None, 290 Parameter(help="Languages to include (python, java, low-code, manifest-only)."), 291 ] = None, 292 exclude_language: Annotated[ 293 list[str] | None, 294 Parameter(help="Languages to exclude."), 295 ] = None, 296 connector_type: Annotated[ 297 str | None, 298 Parameter( 299 help=( 300 "Connector types to include (source, destination). " 301 "Accepts CSV or newline-delimited values." 302 ) 303 ), 304 ] = None, 305 min_support_level: Annotated[ 306 str | None, 307 Parameter( 308 help=( 309 "Minimum support level (inclusive). " 310 "Accepts integer (100, 200, 300) or keyword (archived, community, certified)." 311 ) 312 ), 313 ] = None, 314 max_support_level: Annotated[ 315 str | None, 316 Parameter( 317 help=( 318 "Maximum support level (inclusive). " 319 "Accepts integer (100, 200, 300) or keyword (archived, community, certified)." 320 ) 321 ), 322 ] = None, 323 autopilot_enabled: Annotated[ 324 bool | None, 325 Parameter( 326 help=( 327 "Filter by autopilot progressive rollout status. " 328 "Pass `--autopilot-enabled=true` to keep only connectors that " 329 "have autopilot rollouts enabled (`defaultRolloutMode: autopilot` " 330 "and `enableProgressiveRollout: true`), or " 331 "`--autopilot-enabled=false` for those that do not. " 332 "When omitted, connectors are not filtered on rollout status." 333 ) 334 ), 335 ] = None, 336 pr: Annotated[ 337 str | None, 338 Parameter(help="PR number or GitHub URL for modification detection."), 339 ] = None, 340 gh_token: Annotated[ 341 str | None, 342 Parameter( 343 help=( 344 "GitHub API token. When provided together with --pr, the GitHub API " 345 "is used to detect modified files instead of local git diff " 346 "(avoids shallow-clone issues)." 347 ) 348 ), 349 ] = None, 350 exclude_connectors: Annotated[ 351 list[str] | None, 352 Parameter( 353 help=( 354 "Connectors to exclude from results. " 355 "Accepts CSV or newline-delimited values. Can be specified multiple times." 356 ) 357 ), 358 ] = None, 359 force_include_connectors: Annotated[ 360 list[str] | None, 361 Parameter( 362 help=( 363 "Connectors to force-include regardless of other filters. " 364 "Accepts CSV or newline-delimited values. Can be specified multiple times." 365 ) 366 ), 367 ] = None, 368 connectors_filter: Annotated[ 369 list[str] | None, 370 Parameter( 371 help=( 372 "Intersect results with this explicit set of connector names. " 373 "Only connectors present in both the filtered results and this set are returned. " 374 "Useful for composing multiple filter passes (e.g. combining separate source " 375 "and destination lists). " 376 "Accepts CSV or newline-delimited values. Can be specified multiple times." 377 ) 378 ), 379 ] = None, 380 output_format: Annotated[ 381 OutputFormat, 382 Parameter( 383 help=( 384 'Output format: "csv" (comma-separated), ' 385 '"lines" (one connector per line), ' 386 '"json-gh-matrix" (GitHub Actions matrix JSON).' 387 ) 388 ), 389 ] = "lines", 390 unpublished: Annotated[ 391 bool, 392 Parameter( 393 help=( 394 "Filter to only connectors whose local dockerImageTag " 395 "has not been published to the GCS registry. Requires --store." 396 ) 397 ), 398 ] = False, 399 store: Annotated[ 400 str | None, 401 Parameter( 402 help=( 403 "Store target for unpublished check (e.g. 'coral:prod', 'coral:dev'). " 404 "Required when --unpublished is set." 405 ) 406 ), 407 ] = None, 408 assert_none: Annotated[ 409 bool, 410 Parameter( 411 help=( 412 "Exit with non-zero status if any connectors match the filters. " 413 "Useful for audit/CI checks (e.g. --unpublished --assert-none)." 414 ) 415 ), 416 ] = False, 417 sort_by: Annotated[ 418 SortBy, 419 Parameter( 420 help="Sort connectors by the given key. Only 'name' is supported for now." 421 ), 422 ] = "name", 423 sort_direction: Annotated[ 424 Literal["asc", "desc"], 425 Parameter( 426 help="Sort direction: 'asc' (ascending, default) or 'desc' (descending)." 427 ), 428 ] = "asc", 429 limit: Annotated[ 430 int | None, 431 Parameter( 432 help=( 433 "Maximum number of connectors to return. " 434 "Applied after sorting. Useful for batched processing." 435 ) 436 ), 437 ] = None, 438 offset: Annotated[ 439 int, 440 Parameter( 441 help=( 442 "Number of connectors to skip from the start of the (sorted) list. " 443 "Applied after sorting, before --limit. " 444 "Useful for batched processing (e.g. --offset=200 --limit=200 for batch 2)." 445 ) 446 ), 447 ] = 0, 448) -> None: 449 """List connectors in the Airbyte monorepo with filtering options.""" 450 # Validate mutually exclusive flags 451 if language and exclude_language: 452 exit_with_error("Cannot specify both --language and --exclude-language.") 453 if assert_none and (offset or limit is not None): 454 exit_with_error( 455 "Cannot combine --assert-none with --offset or --limit. " 456 "--assert-none checks all connectors matching the filters, " 457 "which is incompatible with pagination." 458 ) 459 460 # Map CLI flags to MCP tool parameters 461 certified: bool | None = True if certified_only else None 462 modified: bool | None = True if modified_only else None 463 464 language_filter: set[str] | None = set(language) if language else None 465 language_exclude: set[str] | None = ( 466 set(exclude_language) if exclude_language else None 467 ) 468 469 # Parse connector type filter 470 connector_type_filter: set[str] | None = None 471 if connector_type: 472 try: 473 connector_type_filter = _parse_connector_types(connector_type) # ty: ignore[invalid-assignment] 474 except ValueError as e: 475 exit_with_error(str(e)) 476 477 # Parse support level filters 478 min_level: SupportLevel | None = None 479 max_level: SupportLevel | None = None 480 if min_support_level: 481 try: 482 min_level = _parse_support_level(min_support_level) 483 except ValueError as e: 484 exit_with_error(str(e)) 485 if max_support_level: 486 try: 487 max_level = _parse_support_level(max_support_level) 488 except ValueError as e: 489 exit_with_error(str(e)) 490 491 # Parse exclude/force-include connector lists (merge multiple flag values) 492 exclude_set: set[str] = set() 493 if exclude_connectors: 494 for value in exclude_connectors: 495 exclude_set.update(_parse_connector_names(value)) 496 497 force_include_set: set[str] = set() 498 if force_include_connectors: 499 for value in force_include_connectors: 500 force_include_set.update(_parse_connector_names(value)) 501 502 connectors_filter_set: set[str] | None = None 503 if connectors_filter: 504 connectors_filter_set = set() 505 for value in connectors_filter: 506 connectors_filter_set.update(_parse_connector_names(value)) 507 508 result = list_connectors_in_repo( 509 repo_path=repo_path, 510 certified=certified, 511 modified=modified, 512 language_filter=language_filter, 513 language_exclude=language_exclude, 514 pr_num_or_url=pr, 515 gh_token=gh_token, 516 ) 517 connectors = list(result.connectors) 518 repo_path_obj = Path(repo_path) 519 520 # Add connectors with local CDK reference if --local-cdk flag is set 521 if local_cdk: 522 local_cdk_connectors = get_connectors_with_local_cdk(repo_path) 523 connectors = sorted(set(connectors) | local_cdk_connectors) 524 525 # Apply connector type filter 526 if connector_type_filter: 527 connectors = [ 528 name 529 for name in connectors 530 if _get_connector_type(name) in connector_type_filter 531 ] 532 533 # Apply support level filters (requires reading metadata) 534 if min_level is not None or max_level is not None: 535 filtered_connectors = [] 536 for name in connectors: 537 connector_dir = repo_path_obj / CONNECTOR_PATH_PREFIX / name 538 level = _get_connector_support_level(connector_dir) 539 if level is None: 540 continue # Skip connectors without support level 541 if min_level is not None and level.precedence < min_level.precedence: 542 continue 543 if max_level is not None and level.precedence > max_level.precedence: 544 continue 545 filtered_connectors.append(name) 546 connectors = filtered_connectors 547 548 # Apply exclude filter 549 if exclude_set: 550 connectors = [name for name in connectors if name not in exclude_set] 551 552 # Apply connectors filter (intersection, narrows the set) 553 if connectors_filter_set is not None: 554 connectors = [name for name in connectors if name in connectors_filter_set] 555 556 # Apply autopilot rollout status filter last among narrowing filters: it reads 557 # and parses each connector's metadata.yaml, so run it on the already 558 # name-filtered set to avoid unnecessary reads. 559 if autopilot_enabled is not None: 560 connectors = [ 561 name 562 for name in connectors 563 if _get_connector_autopilot_enabled( 564 repo_path_obj / CONNECTOR_PATH_PREFIX / name 565 ) 566 == autopilot_enabled 567 ] 568 569 # Apply force-include (union, overrides all other filters) 570 if force_include_set: 571 connectors_set = set(connectors) 572 connectors_set.update(force_include_set) 573 connectors = sorted(connectors_set) 574 575 # Apply unpublished filter (requires GCS check) 576 audit_result: AuditResult | None = None 577 if unpublished: 578 if not store: 579 exit_with_error("--store is required when --unpublished is set.") 580 581 target = resolve_registry_store(store=store) 582 audit_result = find_unpublished_connectors( 583 repo_path=repo_path, 584 bucket_name=target.bucket, 585 connector_names=connectors, 586 ) 587 unpublished_names = {entry.connector_name for entry in audit_result.unpublished} 588 connectors = [name for name in connectors if name in unpublished_names] 589 590 # Log audit summary to stderr 591 error_console.print( 592 f"Audit: {audit_result.checked_count} checked, " 593 f"{len(audit_result.unpublished)} unpublished, " 594 f"{len(audit_result.skipped_archived)} archived-skipped, " 595 f"{len(audit_result.skipped_disabled)} disabled-skipped, " 596 f"{len(audit_result.skipped_rc)} rc-skipped" 597 ) 598 if audit_result.errors: 599 for err in audit_result.errors: 600 error_console.print(f" Warning: {err}") 601 602 # --- Sort, offset, limit (applied after all filters) --- 603 if offset < 0: 604 exit_with_error("--offset must be non-negative.") 605 if limit is not None and limit < 0: 606 exit_with_error("--limit must be non-negative.") 607 608 total_before_slice = len(connectors) 609 reverse = sort_direction == "desc" 610 if sort_by == "name": 611 connectors = sorted(connectors, reverse=reverse) 612 613 if offset: 614 connectors = connectors[offset:] 615 if limit is not None: 616 connectors = connectors[:limit] 617 618 if offset or limit is not None: 619 error_console.print( 620 f"Slice: showing {len(connectors)} of {total_before_slice} " 621 f"(offset={offset}, limit={limit})" 622 ) 623 624 if output_format == "csv": 625 sys.stdout.write(",".join(connectors) + "\n") 626 elif output_format == "lines": 627 for name in connectors: 628 sys.stdout.write(name + "\n") 629 elif output_format == "json-gh-matrix": 630 # Build matrix with full connector metadata 631 include_list = [] 632 for name in connectors: 633 connector_dir = repo_path_obj / CONNECTOR_PATH_PREFIX / name 634 include_list.append(_get_connector_info(name, connector_dir)) 635 matrix = {"include": include_list} 636 sys.stdout.write( 637 json.dumps( 638 matrix, 639 indent=2, 640 default=str, 641 ) 642 + "\n" 643 ) 644 645 # Write GitHub Step Summary when GITHUB_STEP_SUMMARY is set 646 _write_connector_list_summary( 647 connectors=connectors, 648 assert_none=assert_none, 649 is_unpublished=unpublished, 650 audit_result=audit_result, 651 ) 652 653 # Exit non-zero if --assert-none and any connectors matched 654 if assert_none and connectors: 655 error_console.print(f"FAIL: {len(connectors)} connector(s) matched.") 656 sys.exit(1) 657 658 659def _write_github_step_outputs(outputs: dict[str, str | int | None]) -> None: 660 """Write outputs to GitHub Actions step output file if running in CI.""" 661 github_output = os.getenv("GITHUB_OUTPUT") 662 if not (os.getenv("CI") and github_output): 663 return 664 665 with open(github_output, "a", encoding="utf-8") as f: 666 for key, value in outputs.items(): 667 if value is None: 668 continue 669 f.write(f"{key}={value}\n") 670 671 672def _write_connector_list_summary( 673 connectors: list[str], 674 *, 675 assert_none: bool, 676 is_unpublished: bool, 677 audit_result: AuditResult | None, 678) -> None: 679 """Write a GitHub Step Summary for the connector list command. 680 681 This is a no-op when `GITHUB_STEP_SUMMARY` is not set. 682 """ 683 github_summary = os.getenv("GITHUB_STEP_SUMMARY") 684 if not github_summary: 685 return 686 687 summary = generate_connector_list_summary( 688 connectors, 689 assert_none=assert_none, 690 unpublished=is_unpublished, 691 audit_result=audit_result, 692 ) 693 write_github_summary(summary) 694 695 696@connector_app.command(name="info") 697def connector_info( 698 connector_name: Annotated[ 699 str, 700 Parameter(help="Name of the connector (e.g., source-github)."), 701 ], 702 repo_path: Annotated[ 703 str | None, 704 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 705 ] = None, 706 dpath_expression: Annotated[ 707 str | None, 708 Parameter( 709 name="--dpath", 710 help=( 711 "Evaluate this dpath expression against the parsed metadata.yaml " 712 "object and print only that value (e.g., data/dockerImageTag)." 713 ), 714 ), 715 ] = None, 716) -> None: 717 """Get metadata for a single connector. 718 719 Prints JSON output with connector metadata. When running in GitHub Actions 720 (CI env var set), also writes each field to GitHub step outputs. 721 """ 722 # Infer repo_path from current directory if not provided 723 if repo_path is None: 724 # Check if we're in an airbyte repo by looking for the connectors directory 725 cwd = Path.cwd() 726 # Walk up to find airbyte-integrations/connectors 727 for parent in [cwd, *cwd.parents]: 728 if (parent / CONNECTOR_PATH_PREFIX).exists(): 729 repo_path = str(parent) 730 break 731 if repo_path is None: 732 exit_with_error( 733 "Could not infer repo path. Please provide --repo-path or run from within the Airbyte monorepo." 734 ) 735 736 repo_path_obj = Path(repo_path) 737 connector_dir = repo_path_obj / CONNECTOR_PATH_PREFIX / connector_name 738 739 if not connector_dir.exists(): 740 exit_with_error(f"Connector directory not found: {connector_dir}") 741 742 try: 743 value = load_raw_connector_metadata_from_local( 744 repo_path_obj, 745 connector_name, 746 dpath_expression=dpath_expression, 747 ) 748 except ( 749 ConnectorMetadataDpathError, 750 ConnectorMetadataDpathNotFoundError, 751 FileNotFoundError, 752 ValueError, 753 ) as e: 754 exit_with_error(str(e)) 755 756 if dpath_expression is None: 757 print_json(value) 758 return 759 760 sys.stdout.write(format_metadata_dpath_value(value) + "\n") 761 762 763BumpType = Literal[ 764 "patch", 765 "minor", 766 "major", 767 "patch_rc", 768 "minor_rc", 769 "major_rc", 770 "rc", 771 "promote", 772] 773 774 775def _get_git_head_sha(repo_path: Path) -> str: 776 """Get the HEAD commit SHA from a git repository. 777 778 Args: 779 repo_path: Path to the git repository. 780 781 Returns: 782 Full commit SHA string. 783 784 Raises: 785 SystemExit: If the git command fails (e.g., not a git repo). 786 """ 787 try: 788 result = subprocess.run( 789 ["git", "rev-parse", "HEAD"], 790 cwd=repo_path, 791 capture_output=True, 792 text=True, 793 check=True, 794 ) 795 except subprocess.CalledProcessError: 796 exit_with_error( 797 f"Failed to get HEAD SHA from {repo_path}. Is it a git repository?" 798 ) 799 except OSError: 800 exit_with_error("Could not run 'git'. Is git installed and on PATH?") 801 return result.stdout.strip() 802 803 804@connector_app.command(name="get-version") 805def get_version( 806 name: Annotated[ 807 str, 808 Parameter(help="Connector technical name (e.g., source-github)."), 809 ], 810 repo_path: Annotated[ 811 str, 812 Parameter(help="Absolute path to the Airbyte monorepo."), 813 ], 814 compute_next: Annotated[ 815 bool, 816 Parameter( 817 name="--next", 818 help="Compute the next version instead of the current version.", 819 ), 820 ] = False, 821 bump_type: Annotated[ 822 BumpType | None, 823 Parameter( 824 help="Version bump type (requires --next). " 825 "Standard: patch, minor, major. RC: patch_rc, minor_rc, major_rc, rc, promote." 826 ), 827 ] = None, 828 prerelease: Annotated[ 829 bool, 830 Parameter( 831 help="Compute a prerelease (preview) tag using the repo HEAD SHA (requires --next)." 832 ), 833 ] = False, 834) -> None: 835 """Get the current or next version for a connector. 836 837 By default, prints the current version from metadata.yaml. 838 This is analogous to `local connector info --dpath data/dockerImageTag` and 839 uses the same dpath evaluation internally. 840 841 With --next, computes and prints the next version. Requires either 842 --bump-type or --prerelease to be specified. 843 844 Examples: 845 # Current version 846 airbyte-ops local connector get-version --name source-github --repo-path /path/to/airbyte 847 848 # Next patch version 849 VERSION=$(airbyte-ops local connector get-version --name source-github --repo-path /path/to/airbyte --next --bump-type patch) 850 851 # Next prerelease (preview) tag based on HEAD SHA 852 TAG=$(airbyte-ops local connector get-version --name source-github --repo-path /path/to/airbyte --next --prerelease) 853 """ 854 repo = Path(repo_path) 855 856 # Validate flag combinations 857 if bump_type and not compute_next: 858 exit_with_error("--bump-type requires --next.") 859 if prerelease and not compute_next: 860 exit_with_error("--prerelease requires --next.") 861 if bump_type and prerelease: 862 exit_with_error("--bump-type and --prerelease are mutually exclusive.") 863 if compute_next and not bump_type and not prerelease: 864 exit_with_error( 865 "--next requires either --bump-type or --prerelease. " 866 "Automatic detection of bump type is not yet supported." 867 ) 868 869 try: 870 get_connector_path(repo, name) 871 metadata = load_raw_connector_metadata_from_local(repo, name) 872 current_version = get_connector_version_from_metadata(metadata) 873 except ( 874 ConnectorMetadataDpathError, 875 ConnectorMetadataDpathNotFoundError, 876 ConnectorNotFoundError, 877 FileNotFoundError, 878 ValueError, 879 ) as e: 880 exit_with_error(str(e)) 881 882 if not compute_next: 883 sys.stdout.write(current_version + "\n") 884 return 885 886 if prerelease: 887 sha = _get_git_head_sha(repo) 888 try: 889 tag = compute_prerelease_docker_image_tag(current_version, sha) 890 except InvalidVersionError as e: 891 exit_with_error(str(e)) 892 sys.stdout.write(tag + "\n") 893 return 894 895 # bump_type is set (validated above) 896 try: 897 new_version = calculate_new_version( 898 current_version=current_version, 899 bump_type=bump_type, # ty: ignore[invalid-argument-type] 900 ) 901 except (InvalidVersionError, ValueError) as e: 902 exit_with_error(str(e)) 903 904 sys.stdout.write(new_version + "\n") 905 906 907@connector_app.command(name="bump-version") 908def bump_version( 909 name: Annotated[ 910 str, 911 Parameter(help="Connector technical name (e.g., source-github)."), 912 ], 913 repo_path: Annotated[ 914 str, 915 Parameter(help="Absolute path to the Airbyte monorepo."), 916 ], 917 bump_type: Annotated[ 918 BumpType | None, 919 Parameter( 920 help="Version bump type. Standard: patch, minor, major. RC: patch_rc, minor_rc, major_rc, rc, promote." 921 ), 922 ] = None, 923 new_version: Annotated[ 924 str | None, 925 Parameter(help="Explicit new version (overrides --bump-type if provided)."), 926 ] = None, 927 changelog_message: Annotated[ 928 str | None, 929 Parameter( 930 help="Message to add to changelog. Ignored if --no-changelog is set." 931 ), 932 ] = None, 933 pr_number: Annotated[ 934 int | None, 935 Parameter(help="PR number for changelog entry."), 936 ] = None, 937 dry_run: Annotated[ 938 bool, 939 Parameter(help="Show what would be changed without modifying files."), 940 ] = False, 941 no_changelog: Annotated[ 942 bool, 943 Parameter( 944 help="Skip changelog updates even if --changelog-message is provided. " 945 "Useful for ephemeral version bumps (e.g. pre-release artifact generation)." 946 ), 947 ] = False, 948 progressive_rollout_enabled: Annotated[ 949 bool | None, 950 Parameter( 951 help="Explicitly set `enableProgressiveRollout` in metadata.yaml. " 952 "Pass `false` to disable progressive rollout (e.g. for preview builds). " 953 "When omitted, the automatic behaviour based on --bump-type is used.", 954 ), 955 ] = None, 956) -> None: 957 """Bump a connector's version across all relevant files. 958 959 Updates version in metadata.yaml (always), pyproject.toml (if exists), 960 and documentation changelog (if --changelog-message provided). 961 962 Note: --changelog-message is ignored when --no-changelog is set. 963 964 Either --bump-type or --new-version must be provided. 965 966 Examples: 967 airbyte-ops local connector bump-version --name source-github --repo-path /path/to/airbyte --bump-type patch 968 airbyte-ops local connector bump-version --name source-github --repo-path /path/to/airbyte --new-version 1.2.3-preview.abc1234 --no-changelog 969 airbyte-ops local connector bump-version --name source-github --repo-path /path/to/airbyte --new-version 1.2.3-preview.abc1234 --no-changelog --progressive-rollout-enabled=false 970 """ 971 try: 972 result = bump_connector_version( 973 repo_path=repo_path, 974 connector_name=name, 975 bump_type=bump_type, 976 new_version=new_version, 977 changelog_message=changelog_message, 978 pr_number=pr_number, 979 dry_run=dry_run, 980 no_changelog=no_changelog, 981 progressive_rollout_enabled=progressive_rollout_enabled, 982 ) 983 except ConnectorNotFoundError as e: 984 exit_with_error(str(e)) 985 except VersionNotFoundError as e: 986 exit_with_error(str(e)) 987 except InvalidVersionError as e: 988 exit_with_error(str(e)) 989 except ValueError as e: 990 exit_with_error(str(e)) 991 992 # Build output matching the issue spec 993 output = { 994 "connector": result.connector, 995 "previous_version": result.previous_version, 996 "new_version": result.new_version, 997 "files_modified": result.files_modified, 998 "dry_run": result.dry_run, 999 } 1000 print_json(output) 1001 1002 # Write to GitHub step outputs if in CI 1003 _write_github_step_outputs( 1004 { 1005 "connector": result.connector, 1006 "previous_version": result.previous_version, 1007 "new_version": result.new_version, 1008 } 1009 ) 1010 1011 1012rollouts_app = App( 1013 name="rollouts", 1014 help="Manage a connector's progressive rollout configuration.", 1015) 1016connector_app.command(rollouts_app) 1017 1018 1019@rollouts_app.command(name="enable") 1020def rollouts_enable( 1021 name: Annotated[ 1022 str, 1023 Parameter(help="Connector technical name (e.g., source-github)."), 1024 ], 1025 repo_path: Annotated[ 1026 str, 1027 Parameter(help="Absolute path to the Airbyte monorepo."), 1028 ], 1029 strategy: Annotated[ 1030 AutopilotStrategy | None, 1031 Parameter( 1032 help="Autopilot pacing strategy: 'fast', 'slow', or 'default'. " 1033 "'default' is a server-side alias for 'fast'; prefer the explicit 'fast'. " 1034 "Omitted keeps any existing value, or 'fast' when none is set." 1035 ), 1036 ] = None, 1037 dry_run: Annotated[ 1038 bool, 1039 Parameter(help="Show what would be changed without modifying files."), 1040 ] = False, 1041) -> None: 1042 """Enable autopilot rollouts in a connector's metadata.yaml. 1043 1044 Always sets `releases.rolloutConfiguration.defaultRolloutMode: autopilot` 1045 (autopilot is the only actively-supported mode, so it is implicit here), 1046 fills in `autopilotConfig` (`autoStart`, `autoPromoteStages`, `strategy`), 1047 and sets `enableProgressiveRollout: true` — the toggle that turns automatic 1048 progressive rollout on. Defaults only fill in `autopilotConfig` fields that 1049 are absent, so existing settings are preserved. 1050 1051 Examples: 1052 airbyte-ops local connector rollouts enable --name source-github --repo-path /path/to/airbyte 1053 airbyte-ops local connector rollouts enable --name source-github --repo-path /path/to/airbyte --strategy slow 1054 """ 1055 try: 1056 result = enable_autopilot_rollout( 1057 repo_path=repo_path, 1058 connector_name=name, 1059 strategy=strategy.value if strategy is not None else None, 1060 dry_run=dry_run, 1061 ) 1062 except ConnectorNotFoundError as e: 1063 exit_with_error(str(e)) 1064 except FileNotFoundError as e: 1065 exit_with_error(str(e)) 1066 except ValueError as e: 1067 exit_with_error(str(e)) 1068 1069 print_json( 1070 { 1071 "connector": result.connector, 1072 "modified": result.modified, 1073 "dry_run": result.dry_run, 1074 "default_rollout_mode": result.default_rollout_mode, 1075 "progressive_rollout_enabled": result.progressive_rollout_enabled, 1076 "strategy": result.strategy, 1077 "auto_start": result.auto_start, 1078 "auto_promote_stages": result.auto_promote_stages, 1079 } 1080 ) 1081 1082 _write_github_step_outputs( 1083 { 1084 "connector": result.connector, 1085 "default_rollout_mode": result.default_rollout_mode, 1086 "progressive_rollout_enabled": str(result.progressive_rollout_enabled), 1087 "strategy": result.strategy, 1088 } 1089 ) 1090 1091 1092@rollouts_app.command(name="disable") 1093def rollouts_disable( 1094 name: Annotated[ 1095 str, 1096 Parameter(help="Connector technical name (e.g., source-github)."), 1097 ], 1098 repo_path: Annotated[ 1099 str, 1100 Parameter(help="Absolute path to the Airbyte monorepo."), 1101 ], 1102 dry_run: Annotated[ 1103 bool, 1104 Parameter(help="Show what would be changed without modifying files."), 1105 ] = False, 1106) -> None: 1107 """Disable autopilot rollouts in a connector's metadata.yaml. 1108 1109 Only sets `releases.rolloutConfiguration.enableProgressiveRollout: false` — 1110 the toggle that turns automatic progressive rollout off. `defaultRolloutMode` 1111 and any `autopilotConfig` are retained (inert until re-enabled), so the 1112 change is lossless. Connectors with no rollout config, or with the flag 1113 already off, are a no-op. 1114 1115 Examples: 1116 airbyte-ops local connector rollouts disable --name source-github --repo-path /path/to/airbyte 1117 """ 1118 try: 1119 result = disable_autopilot_rollout( 1120 repo_path=repo_path, 1121 connector_name=name, 1122 dry_run=dry_run, 1123 ) 1124 except ConnectorNotFoundError as e: 1125 exit_with_error(str(e)) 1126 except FileNotFoundError as e: 1127 exit_with_error(str(e)) 1128 1129 print_json( 1130 { 1131 "connector": result.connector, 1132 "modified": result.modified, 1133 "dry_run": result.dry_run, 1134 "progressive_rollout_enabled": result.progressive_rollout_enabled, 1135 } 1136 ) 1137 1138 _write_github_step_outputs( 1139 { 1140 "connector": result.connector, 1141 "progressive_rollout_enabled": str(result.progressive_rollout_enabled), 1142 } 1143 ) 1144 1145 1146@connector_app.command(name="bump-base-image") 1147def bump_base_image_cmd( 1148 name: Annotated[ 1149 str, 1150 Parameter(help="Connector technical name (e.g., source-github)."), 1151 ], 1152 repo_path: Annotated[ 1153 str, 1154 Parameter(help="Absolute path to the Airbyte monorepo."), 1155 ], 1156 force_latest: Annotated[ 1157 bool, 1158 Parameter( 1159 help=( 1160 "Bump to the absolute latest stable base image, ignoring " 1161 "major-version boundaries. Without this flag the bump stays " 1162 "within the current major version." 1163 ) 1164 ), 1165 ] = False, 1166 dry_run: Annotated[ 1167 bool, 1168 Parameter(help="Show what would be changed without modifying files."), 1169 ] = False, 1170) -> None: 1171 """Update a connector's base image. 1172 1173 Two modes: 1174 1175 * Default: bump to the latest stable tag within the same major version. 1176 Major version changes are treated as breaking-change boundaries. 1177 * --force-latest: bump to the absolute latest stable tag regardless of semver. 1178 1179 Examples: 1180 airbyte-ops local connector bump-base-image --name source-github --repo-path /path/to/airbyte 1181 airbyte-ops local connector bump-base-image --name source-github --repo-path /path/to/airbyte --force-latest 1182 """ 1183 try: 1184 result = bump_base_image( 1185 repo_path=repo_path, 1186 connector_name=name, 1187 force_latest=force_latest, 1188 dry_run=dry_run, 1189 ) 1190 except (ConnectorNotFoundError, BaseImageError) as e: 1191 exit_with_error(str(e)) 1192 except ValueError as e: 1193 exit_with_error(str(e)) 1194 1195 output = { 1196 "connector": result.connector, 1197 "previous_base_image": result.previous_base_image, 1198 "new_base_image": result.new_base_image, 1199 "updated": result.updated, 1200 "dry_run": result.dry_run, 1201 "files_modified": result.files_modified, 1202 "message": result.message, 1203 } 1204 print_json(output) 1205 1206 1207@connector_app.command(name="bump-cdk") 1208def bump_cdk_cmd( 1209 name: Annotated[ 1210 str, 1211 Parameter(help="Connector technical name (e.g., source-github)."), 1212 ], 1213 repo_path: Annotated[ 1214 str, 1215 Parameter(help="Absolute path to the Airbyte monorepo."), 1216 ], 1217 force_latest: Annotated[ 1218 bool, 1219 Parameter( 1220 help=( 1221 "Rewrite the CDK constraint to >=LATEST,<NEXT_MAJOR and refresh " 1222 "the lock file. Without this flag only the lock file is refreshed " 1223 "(constraint unchanged)." 1224 ) 1225 ), 1226 ] = False, 1227 dry_run: Annotated[ 1228 bool, 1229 Parameter(help="Show what would be changed without modifying files."), 1230 ] = False, 1231) -> None: 1232 """Bump a connector's CDK dependency. 1233 1234 Two modes: 1235 1236 * Default: refresh the lock file so it resolves the newest CDK that 1237 satisfies the existing constraint. The constraint is NOT changed. 1238 * --force-latest: rewrite the constraint to >=LATEST,<NEXT_MAJOR and refresh 1239 the lock file. Extras (e.g. file-based) are preserved. 1240 1241 For Java connectors, updates `build.gradle` to the latest CDK version. 1242 1243 Examples: 1244 airbyte-ops local connector bump-cdk --name source-github --repo-path /path/to/airbyte 1245 airbyte-ops local connector bump-cdk --name source-github --repo-path /path/to/airbyte --force-latest 1246 """ 1247 try: 1248 result = bump_cdk( 1249 repo_path=repo_path, 1250 connector_name=name, 1251 force_latest=force_latest, 1252 dry_run=dry_run, 1253 ) 1254 except (ConnectorNotFoundError, CdkBumpError) as e: 1255 exit_with_error(str(e)) 1256 except ValueError as e: 1257 exit_with_error(str(e)) 1258 1259 output = { 1260 "connector": result.connector, 1261 "language": result.language, 1262 "previous_version": result.previous_version, 1263 "new_version": result.new_version, 1264 "updated": result.updated, 1265 "dry_run": result.dry_run, 1266 "files_modified": result.files_modified, 1267 "message": result.message, 1268 } 1269 print_json(output) 1270 1271 1272@connector_app.command(name="bump-deps") 1273def bump_deps_cmd( 1274 name: Annotated[ 1275 str, 1276 Parameter(help="Connector technical name (e.g., source-github)."), 1277 ], 1278 repo_path: Annotated[ 1279 str, 1280 Parameter(help="Absolute path to the Airbyte monorepo."), 1281 ], 1282 dry_run: Annotated[ 1283 bool, 1284 Parameter(help="Show what would be changed without modifying files."), 1285 ] = False, 1286) -> None: 1287 """Update a connector's dependencies. 1288 1289 For Python / low-code connectors using Poetry, this runs 1290 `poetry update --lock` to refresh the lock file with the latest 1291 versions allowed by existing constraints. 1292 1293 For connectors that do not use Poetry (manifest-only, Java, etc.), 1294 this is a no-op. 1295 1296 Examples: 1297 airbyte-ops local connector bump-deps --name source-github --repo-path /path/to/airbyte 1298 airbyte-ops local connector bump-deps --name source-github --repo-path /path/to/airbyte --dry-run 1299 """ 1300 try: 1301 result = bump_deps( 1302 repo_path=repo_path, 1303 connector_name=name, 1304 dry_run=dry_run, 1305 ) 1306 except (ConnectorNotFoundError, DepsError) as e: 1307 exit_with_error(str(e)) 1308 except ValueError as e: 1309 exit_with_error(str(e)) 1310 1311 output = { 1312 "connector": result.connector, 1313 "language": result.language, 1314 "updated": result.updated, 1315 "dry_run": result.dry_run, 1316 "files_modified": result.files_modified, 1317 "outdated_packages": result.outdated_packages, 1318 "message": result.message, 1319 } 1320 print_json(output) 1321 1322 1323@connector_app.command(name="qa") 1324def run_qa_checks( 1325 name: Annotated[ 1326 list[str] | None, 1327 Parameter( 1328 help="Connector technical name(s) (e.g., source-github). Can be specified multiple times." 1329 ), 1330 ] = None, 1331 connector_directory: Annotated[ 1332 str | None, 1333 Parameter( 1334 help="Directory containing connectors to run checks on all connectors in this directory." 1335 ), 1336 ] = None, 1337 check: Annotated[ 1338 list[str] | None, 1339 Parameter(help="Specific check(s) to run. Can be specified multiple times."), 1340 ] = None, 1341 report_path: Annotated[ 1342 str | None, 1343 Parameter(help="Path to write the JSON report file."), 1344 ] = None, 1345) -> None: 1346 """Run QA checks on connector(s). 1347 1348 Validates connector metadata, documentation, packaging, security, and versioning. 1349 Exit code is non-zero if any blocking checks fail. 1350 """ 1351 # Determine which checks to run 1352 checks_to_run = ENABLED_CHECKS 1353 if check: 1354 check_names = set(check) 1355 checks_to_run = [c for c in ENABLED_CHECKS if type(c).__name__ in check_names] 1356 if not checks_to_run: 1357 exit_with_error( 1358 f"No matching checks found. Available checks: {[type(c).__name__ for c in ENABLED_CHECKS]}" 1359 ) 1360 1361 # Collect connectors to check 1362 connectors: list[Connector] = [] 1363 if name: 1364 connectors.extend(Connector(remove_strict_encrypt_suffix(n)) for n in name) 1365 if connector_directory: 1366 connectors.extend(get_all_connectors_in_directory(Path(connector_directory))) 1367 1368 if not connectors: 1369 exit_with_error("No connectors specified. Use --name or --connector-directory.") 1370 1371 connectors = sorted(connectors, key=lambda c: c.technical_name) 1372 1373 # Run checks synchronously (simpler than async for CLI) 1374 all_results = [] 1375 for connector in connectors: 1376 for qa_check in checks_to_run: 1377 result = qa_check.run(connector) 1378 if result.status == CheckStatus.PASSED: 1379 status_icon = "[green]✅ PASS[/green]" 1380 elif result.status == CheckStatus.SKIPPED: 1381 status_icon = "[yellow]🔶 SKIP[/yellow]" 1382 elif result.status == CheckStatus.FAILED and not result.check.blocking: 1383 status_icon = "[yellow]⚠️ WARN[/yellow]" 1384 else: 1385 status_icon = "[red]❌ FAIL[/red]" 1386 console.print( 1387 f"{status_icon} {connector.technical_name}: {result.check.name}" 1388 ) 1389 if result.message: 1390 console.print(f" {result.message}") 1391 all_results.append(result) 1392 1393 form_ux_summary = render_form_ux_summary(all_results) 1394 if form_ux_summary: 1395 form_ux_heading = "## Form UX findings" 1396 form_ux_warning = ( 1397 "These are warnings. They don't fail the QA checks or block merge. " 1398 "Fix them in a separate PR, not in an unrelated change such as a version bump." 1399 ) 1400 console.print( 1401 Markdown(f"{form_ux_heading}\n\n{form_ux_warning}\n\n{form_ux_summary}") 1402 ) 1403 github_summary_path = os.getenv("GITHUB_STEP_SUMMARY") 1404 if github_summary_path: 1405 with Path(github_summary_path).open("a") as github_summary_file: 1406 github_summary_file.write( 1407 f"\n{form_ux_heading}\n\n{form_ux_warning}\n\n{form_ux_summary}\n" 1408 ) 1409 1410 # Write report if requested 1411 if report_path: 1412 Report(check_results=all_results).write(Path(report_path)) 1413 console.print(f"Report written to {report_path}") 1414 1415 # Exit with error if any blocking checks failed 1416 failed = [ 1417 r for r in all_results if r.status == CheckStatus.FAILED and r.check.blocking 1418 ] 1419 if failed: 1420 exit_with_error(f"{len(failed)} check(s) failed") 1421 1422 1423@connector_app.command(name="qa-docs-generate") 1424def generate_qa_docs( 1425 output_file: Annotated[ 1426 str, 1427 Parameter(help="Path to write the generated documentation file."), 1428 ], 1429) -> None: 1430 """Generate documentation for QA checks. 1431 1432 Creates a markdown file documenting all available QA checks organized by category. 1433 """ 1434 checks_by_category: dict[CheckCategory, list[Check]] = {} 1435 for qa_check in ENABLED_CHECKS: 1436 checks_by_category.setdefault(qa_check.category, []).append(qa_check) 1437 1438 jinja_env = Environment( 1439 loader=PackageLoader("airbyte_ops_mcp.connector_qa", "templates"), 1440 autoescape=select_autoescape(), 1441 trim_blocks=False, 1442 lstrip_blocks=True, 1443 ) 1444 template = jinja_env.get_template(CONNECTORS_QA_DOC_TEMPLATE_NAME) 1445 documentation = template.render(checks_by_category=checks_by_category) 1446 1447 output_path = Path(output_file) 1448 output_path.write_text(documentation) 1449 console.print(f"Documentation written to {output_file}") 1450 1451 1452# Create the changelog sub-app under connector 1453changelog_app = App(name="changelog", help="Changelog operations for connectors.") 1454connector_app.command(changelog_app) 1455 1456 1457@changelog_app.command(name="check") 1458def changelog_check( 1459 connector_name: Annotated[ 1460 str | None, 1461 Parameter(help="Connector technical name (e.g., source-github)."), 1462 ] = None, 1463 all_connectors: Annotated[ 1464 bool, 1465 Parameter("--all", help="Check all connectors in the repository."), 1466 ] = False, 1467 repo_path: Annotated[ 1468 str | None, 1469 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 1470 ] = None, 1471 lookback_days: Annotated[ 1472 int | None, 1473 Parameter(help="Only check entries with dates within this many days."), 1474 ] = None, 1475 strict: Annotated[ 1476 bool, 1477 Parameter(help="Exit with error code if any issues are found."), 1478 ] = False, 1479) -> None: 1480 """Check changelog entries for issues. 1481 1482 Validates changelog dates match PR merge dates and checks for PR number mismatches. 1483 """ 1484 if not connector_name and not all_connectors: 1485 exit_with_error("Either --connector-name or --all must be specified.") 1486 1487 if connector_name and all_connectors: 1488 exit_with_error("Cannot specify both --connector-name and --all.") 1489 1490 if repo_path is None: 1491 cwd = Path.cwd() 1492 for parent in [cwd, *cwd.parents]: 1493 if (parent / CONNECTOR_PATH_PREFIX).exists(): 1494 repo_path = str(parent) 1495 break 1496 if repo_path is None: 1497 exit_with_error( 1498 "Could not infer repo path. Please provide --repo-path or run from within the Airbyte monorepo." 1499 ) 1500 1501 total_issues = 0 1502 1503 if all_connectors: 1504 results = check_all_changelogs(repo_path=repo_path, lookback_days=lookback_days) 1505 for result in results: 1506 if result.has_issues or result.errors: 1507 _print_check_result(result) 1508 total_issues += result.issue_count 1509 else: 1510 result = check_changelog( 1511 repo_path=repo_path, 1512 connector_name=connector_name, # ty: ignore[invalid-argument-type] 1513 lookback_days=lookback_days, 1514 ) 1515 _print_check_result(result) 1516 total_issues = result.issue_count 1517 1518 if total_issues > 0: 1519 console.print(f"\n[bold]Total issues found: {total_issues}[/bold]") 1520 if strict: 1521 exit_with_error(f"Found {total_issues} issue(s) in changelog(s).") 1522 else: 1523 console.print("[green]No issues found.[/green]") 1524 1525 1526def _print_check_result(result: ChangelogCheckResult) -> None: 1527 """Print a changelog check result.""" 1528 if not result.has_issues and not result.errors: 1529 return 1530 1531 console.print(f"\n[bold]{result.connector}[/bold]") 1532 1533 for warning in result.pr_mismatch_warnings: 1534 console.print( 1535 f" [yellow]WARNING[/yellow] Line {warning.line_number} (v{warning.version}): {warning.message}" 1536 ) 1537 1538 for fix in result.date_issues: 1539 if fix.changed: 1540 console.print( 1541 f" [red]DATE MISMATCH[/red] Line {fix.line_number} (v{fix.version}): " 1542 f"changelog has {fix.old_date}, PR merged on {fix.new_date}" 1543 ) 1544 1545 for error in result.errors: 1546 console.print(f" [red]ERROR[/red] {error}") 1547 1548 1549@changelog_app.command(name="fix") 1550def changelog_fix( 1551 connector_name: Annotated[ 1552 str | None, 1553 Parameter(help="Connector technical name (e.g., source-github)."), 1554 ] = None, 1555 all_connectors: Annotated[ 1556 bool, 1557 Parameter("--all", help="Fix all connectors in the repository."), 1558 ] = False, 1559 repo_path: Annotated[ 1560 str | None, 1561 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 1562 ] = None, 1563 lookback_days: Annotated[ 1564 int | None, 1565 Parameter(help="Only fix entries with dates within this many days."), 1566 ] = None, 1567 dry_run: Annotated[ 1568 bool, 1569 Parameter(help="Print changes without modifying files."), 1570 ] = False, 1571) -> None: 1572 """Fix changelog entry dates to match PR merge dates. 1573 1574 Looks up the actual merge date for each PR referenced in the changelog 1575 and updates the date column to match. 1576 """ 1577 if not connector_name and not all_connectors: 1578 exit_with_error("Either --connector-name or --all must be specified.") 1579 1580 if connector_name and all_connectors: 1581 exit_with_error("Cannot specify both --connector-name and --all.") 1582 1583 if repo_path is None: 1584 cwd = Path.cwd() 1585 for parent in [cwd, *cwd.parents]: 1586 if (parent / CONNECTOR_PATH_PREFIX).exists(): 1587 repo_path = str(parent) 1588 break 1589 if repo_path is None: 1590 exit_with_error( 1591 "Could not infer repo path. Please provide --repo-path or run from within the Airbyte monorepo." 1592 ) 1593 1594 total_fixed = 0 1595 total_warnings = 0 1596 1597 if all_connectors: 1598 results = fix_all_changelog_dates( 1599 repo_path=repo_path, dry_run=dry_run, lookback_days=lookback_days 1600 ) 1601 for result in results: 1602 if result.has_changes or result.warnings or result.errors: 1603 _print_fix_result(result) 1604 total_fixed += result.changed_count 1605 total_warnings += len(result.warnings) 1606 else: 1607 result = fix_changelog_dates( 1608 repo_path=repo_path, 1609 connector_name=connector_name, # ty: ignore[invalid-argument-type] 1610 dry_run=dry_run, 1611 lookback_days=lookback_days, 1612 ) 1613 _print_fix_result(result) 1614 total_fixed = result.changed_count 1615 total_warnings = len(result.warnings) 1616 1617 action = "Would fix" if dry_run else "Fixed" 1618 console.print(f"\n[bold]{action} {total_fixed} date(s).[/bold]") 1619 if total_warnings > 0: 1620 console.print( 1621 f"[yellow]{total_warnings} warning(s) about PR number mismatches.[/yellow]" 1622 ) 1623 1624 1625def _print_fix_result(result: ChangelogFixResult) -> None: 1626 """Print a changelog fix result.""" 1627 if not result.has_changes and not result.warnings and not result.errors: 1628 return 1629 1630 console.print(f"\n[bold]{result.connector}[/bold]") 1631 1632 for warning in result.warnings: 1633 console.print( 1634 f" [yellow]WARNING[/yellow] Line {warning.line_number} (v{warning.version}): {warning.message}" 1635 ) 1636 1637 for fix in result.fixes: 1638 if fix.changed: 1639 action = "Would fix" if result.dry_run else "Fixed" 1640 console.print( 1641 f" [green]{action}[/green] Line {fix.line_number} (v{fix.version}): " 1642 f"{fix.old_date} -> {fix.new_date}" 1643 ) 1644 1645 for error in result.errors: 1646 console.print(f" [red]ERROR[/red] {error}") 1647 1648 1649@changelog_app.command(name="add") 1650def changelog_add( 1651 connector_name: Annotated[ 1652 str, 1653 Parameter(help="Connector technical name (e.g., source-github)."), 1654 ], 1655 pr_number: Annotated[ 1656 int, 1657 Parameter(help="PR number for the changelog entry."), 1658 ], 1659 message: Annotated[ 1660 str, 1661 Parameter(help="Changelog entry message."), 1662 ], 1663 repo_path: Annotated[ 1664 str | None, 1665 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 1666 ] = None, 1667 dry_run: Annotated[ 1668 bool, 1669 Parameter(help="Print changes without modifying files."), 1670 ] = False, 1671) -> None: 1672 """Add a changelog entry for a connector using its current version. 1673 1674 Reads the version from metadata.yaml and writes a single changelog 1675 entry to the connector's documentation file. Does not modify any 1676 version files. 1677 """ 1678 if repo_path is None: 1679 cwd = Path.cwd() 1680 for parent in [cwd, *cwd.parents]: 1681 if (parent / CONNECTOR_PATH_PREFIX).exists(): 1682 repo_path = str(parent) 1683 break 1684 if repo_path is None: 1685 exit_with_error( 1686 "Could not infer repo path. Please provide --repo-path or run from within the Airbyte monorepo." 1687 ) 1688 1689 try: 1690 connector_path = get_connector_path(Path(repo_path), connector_name) 1691 version = get_current_version(connector_path) 1692 except (ConnectorNotFoundError, VersionNotFoundError) as e: 1693 exit_with_error(str(e)) 1694 1695 doc_path = get_connector_doc_path(Path(repo_path), connector_name) 1696 if doc_path is None or not doc_path.exists(): 1697 exit_with_error(f"Documentation file not found for {connector_name}.") 1698 1699 modified = update_changelog( 1700 doc_path=doc_path, 1701 new_version=version, 1702 changelog_message=message, 1703 pr_number=pr_number, 1704 dry_run=dry_run, 1705 ) 1706 1707 action = "Would add" if dry_run else "Added" 1708 if modified: 1709 console.print( 1710 f"[green]{action} changelog entry for {connector_name} v{version}[/green]" 1711 ) 1712 else: 1713 console.print( 1714 f"[yellow]No changes needed for {connector_name} v{version}[/yellow]" 1715 ) 1716 1717 1718# Create the marketing-stub sub-app under connector 1719marketing_stub_app = App( 1720 name="marketing-stub", 1721 help="Marketing connector stub operations (local file validation and updates).", 1722) 1723connector_app.command(marketing_stub_app) 1724 1725# Path to connectors in the airbyte-enterprise repo 1726ENTERPRISE_CONNECTOR_PATH_PREFIX = "airbyte-integrations/connectors" 1727 1728 1729def _build_stub_from_metadata( 1730 connector_name: str, 1731 metadata: dict, 1732 existing_stub: dict | None = None, 1733) -> dict: 1734 """Build a connector stub from metadata.yaml. 1735 1736 Args: 1737 connector_name: The connector name (e.g., 'source-oracle-enterprise'). 1738 metadata: The parsed metadata.yaml content. 1739 existing_stub: Optional existing stub to preserve extra fields from. 1740 1741 Returns: 1742 A connector stub dictionary. 1743 """ 1744 data = metadata.get("data", {}) 1745 1746 # Determine connector type for the stub 1747 connector_type = data.get("connectorType", "source") 1748 stub_type = f"enterprise_{connector_type}" 1749 1750 # Preserve existing stub ID if available, otherwise use connector name 1751 stub_id = (existing_stub.get("id") if existing_stub else None) or connector_name 1752 1753 # Get the icon URL - construct from icon filename if available 1754 icon_filename = data.get("icon", "") 1755 if icon_filename and not icon_filename.startswith("http"): 1756 # Construct icon URL from the standard GCS path 1757 icon_url = f"https://storage.googleapis.com/prod-airbyte-cloud-connector-metadata-service/resources/connector_stubs/v0/icons/{icon_filename}" 1758 else: 1759 icon_url = icon_filename or "" 1760 1761 # Build the stub 1762 stub: dict = { 1763 "id": stub_id, 1764 "name": data.get("name", connector_name.replace("-", " ").title()), 1765 "label": "enterprise", 1766 "icon": icon_url, 1767 "url": data.get("documentationUrl", ""), 1768 "type": stub_type, 1769 } 1770 1771 # Add definitionId if available 1772 definition_id = data.get("definitionId") 1773 if definition_id: 1774 stub["definitionId"] = definition_id 1775 1776 # Preserve extra fields from existing stub (like codename) 1777 if existing_stub: 1778 for key in existing_stub: 1779 if key not in stub: 1780 stub[key] = existing_stub[key] 1781 1782 return stub 1783 1784 1785@marketing_stub_app.command(name="check") 1786def marketing_stub_check( 1787 connector: Annotated[ 1788 str | None, 1789 Parameter(help="Connector name to check (e.g., 'source-oracle-enterprise')."), 1790 ] = None, 1791 all_connectors: Annotated[ 1792 bool, 1793 Parameter("--all", help="Check all stubs in the file."), 1794 ] = False, 1795 repo_root: Annotated[ 1796 Path | None, 1797 Parameter( 1798 help="Path to the airbyte-enterprise repository root. Defaults to current directory." 1799 ), 1800 ] = None, 1801) -> None: 1802 """Validate marketing connector stub entries. 1803 1804 Checks that stub entries have valid required fields (id, name, url, icon) 1805 and optionally validates that the stub matches the connector's metadata.yaml. 1806 1807 Exit codes: 1808 0: All checks passed 1809 1: Validation errors found 1810 1811 Output: 1812 STDOUT: JSON validation result 1813 STDERR: Informational messages 1814 1815 Example: 1816 airbyte-ops local connector marketing-stub check --connector source-oracle-enterprise --repo-root /path/to/airbyte-enterprise 1817 airbyte-ops local connector marketing-stub check --all --repo-root /path/to/airbyte-enterprise 1818 """ 1819 if not connector and not all_connectors: 1820 exit_with_error("Either --connector or --all must be specified.") 1821 1822 if connector and all_connectors: 1823 exit_with_error("Cannot specify both --connector and --all.") 1824 1825 if repo_root is None: 1826 repo_root = Path.cwd() 1827 1828 # Load local stubs 1829 try: 1830 stubs = load_local_stubs(repo_root) 1831 except FileNotFoundError as e: 1832 exit_with_error(str(e)) 1833 except ValueError as e: 1834 exit_with_error(str(e)) 1835 1836 stubs_to_check = stubs if all_connectors else [] 1837 if connector: 1838 stub = find_stub_by_connector(stubs, connector) 1839 if stub is None: 1840 exit_with_error( 1841 f"Connector stub '{connector}' not found in {CONNECTOR_STUBS_FILE}" 1842 ) 1843 stubs_to_check = [stub] 1844 1845 errors: list[dict] = [] 1846 warnings: list[dict] = [] 1847 placeholders: list[dict] = [] 1848 1849 for stub in stubs_to_check: 1850 stub_id = stub.get("id", "<unknown>") 1851 stub_name = stub.get("name", stub_id) 1852 1853 # Check required fields 1854 required_fields = ["id", "name", "url", "icon"] 1855 for field in required_fields: 1856 if not stub.get(field): 1857 errors.append( 1858 {"stub_id": stub_id, "error": f"Missing required field: {field}"} 1859 ) 1860 1861 # Check if corresponding connector exists and validate against metadata 1862 connector_dir = repo_root / ENTERPRISE_CONNECTOR_PATH_PREFIX / stub_id 1863 metadata_file = connector_dir / METADATA_FILE_NAME 1864 1865 if metadata_file.exists(): 1866 metadata = yaml.safe_load(metadata_file.read_text()) 1867 data = metadata.get("data", {}) 1868 1869 # Check if definitionId matches 1870 metadata_def_id = data.get("definitionId") 1871 stub_def_id = stub.get("definitionId") 1872 if metadata_def_id and stub_def_id and metadata_def_id != stub_def_id: 1873 errors.append( 1874 { 1875 "stub_id": stub_id, 1876 "error": f"definitionId mismatch: stub has '{stub_def_id}', metadata has '{metadata_def_id}'", 1877 } 1878 ) 1879 1880 # Check if name matches 1881 metadata_name = data.get("name") 1882 if metadata_name and stub_name and metadata_name != stub_name: 1883 warnings.append( 1884 { 1885 "stub_id": stub_id, 1886 "warning": f"name mismatch: stub has '{stub_name}', metadata has '{metadata_name}'", 1887 } 1888 ) 1889 else: 1890 # No connector directory - this is a registry placeholder for a future connector 1891 placeholders.append( 1892 { 1893 "stub_id": stub_id, 1894 "name": stub_name, 1895 } 1896 ) 1897 1898 result = { 1899 "checked_count": len(stubs_to_check), 1900 "error_count": len(errors), 1901 "warning_count": len(warnings), 1902 "placeholder_count": len(placeholders), 1903 "valid": len(errors) == 0, 1904 "errors": errors, 1905 "warnings": warnings, 1906 "placeholders": placeholders, 1907 } 1908 1909 # Print placeholders as info (not warnings - these are valid registry placeholders) 1910 if placeholders: 1911 error_console.print( 1912 f"[blue]Found {len(placeholders)} registry placeholder(s) (no local directory):[/blue]" 1913 ) 1914 for placeholder in placeholders: 1915 error_console.print( 1916 f" Found Connector Registry Placeholder (no local directory): {placeholder['name']}" 1917 ) 1918 1919 if errors: 1920 error_console.print(f"[red]Found {len(errors)} error(s):[/red]") 1921 for err in errors: 1922 error_console.print(f" {err['stub_id']}: {err['error']}") 1923 1924 if warnings: 1925 error_console.print(f"[yellow]Found {len(warnings)} warning(s):[/yellow]") 1926 for warn in warnings: 1927 error_console.print(f" {warn['stub_id']}: {warn['warning']}") 1928 1929 if not errors and not warnings: 1930 error_console.print( 1931 f"[green]All {len(stubs_to_check)} stub(s) passed validation[/green]" 1932 ) 1933 1934 print_json(result) 1935 1936 if errors: 1937 exit_with_error("Validation failed", code=1) 1938 1939 1940@marketing_stub_app.command(name="sync") 1941def marketing_stub_sync( 1942 connector: Annotated[ 1943 str | None, 1944 Parameter(help="Connector name to sync (e.g., 'source-oracle-enterprise')."), 1945 ] = None, 1946 all_connectors: Annotated[ 1947 bool, 1948 Parameter("--all", help="Sync all connectors that have metadata.yaml files."), 1949 ] = False, 1950 repo_root: Annotated[ 1951 Path | None, 1952 Parameter( 1953 help="Path to the airbyte-enterprise repository root. Defaults to current directory." 1954 ), 1955 ] = None, 1956 dry_run: Annotated[ 1957 bool, 1958 Parameter(help="Show what would be synced without making changes."), 1959 ] = False, 1960) -> None: 1961 """Sync connector stub(s) from connector metadata.yaml file(s). 1962 1963 Reads the connector's metadata.yaml file and updates the corresponding 1964 entry in connector_stubs.json with the current values. 1965 1966 Exit codes: 1967 0: Sync successful (or dry-run completed) 1968 1: Error (connector not found, no metadata, etc.) 1969 1970 Output: 1971 STDOUT: JSON representation of the synced stub(s) 1972 STDERR: Informational messages 1973 1974 Example: 1975 airbyte-ops local connector marketing-stub sync --connector source-oracle-enterprise --repo-root /path/to/airbyte-enterprise 1976 airbyte-ops local connector marketing-stub sync --all --repo-root /path/to/airbyte-enterprise 1977 airbyte-ops local connector marketing-stub sync --connector source-oracle-enterprise --dry-run 1978 """ 1979 if not connector and not all_connectors: 1980 exit_with_error("Either --connector or --all must be specified.") 1981 1982 if connector and all_connectors: 1983 exit_with_error("Cannot specify both --connector and --all.") 1984 1985 if repo_root is None: 1986 repo_root = Path.cwd() 1987 1988 # Load existing stubs 1989 try: 1990 stubs = load_local_stubs(repo_root) 1991 except FileNotFoundError: 1992 stubs = [] 1993 except ValueError as e: 1994 exit_with_error(str(e)) 1995 1996 # Determine which connectors to sync 1997 connectors_to_sync: list[str] = [] 1998 if connector: 1999 connectors_to_sync = [connector] 2000 else: 2001 # Find all connectors with metadata.yaml in the enterprise connectors directory 2002 connectors_dir = repo_root / ENTERPRISE_CONNECTOR_PATH_PREFIX 2003 if connectors_dir.exists(): 2004 for item in connectors_dir.iterdir(): 2005 if item.is_dir() and (item / METADATA_FILE_NAME).exists(): 2006 connectors_to_sync.append(item.name) 2007 connectors_to_sync.sort() 2008 2009 if not connectors_to_sync: 2010 exit_with_error("No connectors found to sync.") 2011 2012 synced_stubs: list[dict] = [] 2013 updated_count = 0 2014 added_count = 0 2015 2016 for conn_name in connectors_to_sync: 2017 connector_dir = repo_root / ENTERPRISE_CONNECTOR_PATH_PREFIX / conn_name 2018 metadata_file = connector_dir / METADATA_FILE_NAME 2019 2020 if not connector_dir.exists(): 2021 if connector: 2022 exit_with_error(f"Connector directory not found: {connector_dir}") 2023 continue 2024 2025 if not metadata_file.exists(): 2026 if connector: 2027 exit_with_error(f"Metadata file not found: {metadata_file}") 2028 continue 2029 2030 # Load metadata 2031 metadata = yaml.safe_load(metadata_file.read_text()) 2032 2033 # Find existing stub if any 2034 existing_stub = find_stub_by_connector(stubs, conn_name) 2035 2036 # Build new stub from metadata 2037 new_stub = _build_stub_from_metadata(conn_name, metadata, existing_stub) 2038 2039 # Validate the new stub 2040 ConnectorStub(**new_stub) 2041 2042 if dry_run: 2043 action = "update" if existing_stub else "create" 2044 error_console.print(f"[DRY RUN] Would {action} stub for '{conn_name}'") 2045 synced_stubs.append(new_stub) 2046 continue 2047 2048 # Update or add the stub 2049 if existing_stub: 2050 # Find and replace 2051 for i, stub in enumerate(stubs): 2052 if stub.get("id") == existing_stub.get("id"): 2053 stubs[i] = new_stub 2054 break 2055 updated_count += 1 2056 else: 2057 stubs.append(new_stub) 2058 added_count += 1 2059 2060 synced_stubs.append(new_stub) 2061 2062 if not dry_run: 2063 # Save the updated stubs 2064 save_local_stubs(repo_root, stubs) 2065 error_console.print( 2066 f"[green]Synced {len(synced_stubs)} stub(s) to {CONNECTOR_STUBS_FILE} " 2067 f"({added_count} added, {updated_count} updated)[/green]" 2068 ) 2069 else: 2070 error_console.print( 2071 f"[DRY RUN] Would sync {len(synced_stubs)} stub(s) to {CONNECTOR_STUBS_FILE}" 2072 ) 2073 2074 print_json( 2075 synced_stubs if all_connectors else synced_stubs[0] if synced_stubs else {} 2076 ) 2077 2078 2079# Create the release-block sub-app under connector 2080release_block_app = App( 2081 name="release-block", 2082 help="Manage release block markers for connectors.", 2083) 2084connector_app.command(release_block_app) 2085 2086 2087@release_block_app.command(name="add") 2088def release_block_add( 2089 name: Annotated[ 2090 str, 2091 Parameter(help="Connector technical name (e.g., source-faker)."), 2092 ], 2093 reason: Annotated[ 2094 str, 2095 Parameter(help="Human-readable reason for blocking the release."), 2096 ], 2097 repo_path: Annotated[ 2098 str | None, 2099 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 2100 ] = None, 2101 yanked_version: Annotated[ 2102 str | None, 2103 Parameter(help="Version that was yanked (for reference)."), 2104 ] = None, 2105 blocked_by: Annotated[ 2106 str | None, 2107 Parameter(help="Email or identifier of the person requesting the block."), 2108 ] = None, 2109) -> None: 2110 """Add a `block-release.yaml` marker to prevent publishing a connector. 2111 2112 Creates a marker file in the connector's directory that causes the publish 2113 pipeline to skip the connector with a warning. 2114 2115 Example: 2116 2117 airbyte-ops local connector release-block add \\ 2118 --name source-faker \\ 2119 --reason "Version 5.0.1 yanked due to regression" \\ 2120 --repo-path /path/to/airbyte 2121 """ 2122 resolved_path = repo_path or os.environ.get("AIRBYTE_REPO_PATH", ".") 2123 result = add_release_block( 2124 repo_path=resolved_path, 2125 connector_name=name, 2126 reason=reason, 2127 yanked_version=yanked_version, 2128 blocked_by=blocked_by, 2129 ) 2130 if result.success: 2131 error_console.print(f"[green]{result.message}[/green]") 2132 else: 2133 exit_with_error(result.message) 2134 print_json(result.to_dict()) 2135 2136 2137@release_block_app.command(name="clear") 2138def release_block_clear( 2139 name: Annotated[ 2140 str, 2141 Parameter(help="Connector technical name (e.g., source-faker)."), 2142 ], 2143 repo_path: Annotated[ 2144 str | None, 2145 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 2146 ] = None, 2147) -> None: 2148 """Remove the `block-release.yaml` marker to allow publishing a connector. 2149 2150 Example: 2151 2152 airbyte-ops local connector release-block clear \\ 2153 --name source-faker \\ 2154 --repo-path /path/to/airbyte 2155 """ 2156 resolved_path = repo_path or os.environ.get("AIRBYTE_REPO_PATH", ".") 2157 result = clear_release_block( 2158 repo_path=resolved_path, 2159 connector_name=name, 2160 ) 2161 if result.success: 2162 error_console.print(f"[green]{result.message}[/green]") 2163 else: 2164 exit_with_error(result.message) 2165 print_json(result.to_dict()) 2166 2167 2168ReleaseBlockListFormat = Literal["text", "json", "csv"] 2169 2170 2171@release_block_app.command(name="list") 2172def release_block_list( 2173 repo_path: Annotated[ 2174 str | None, 2175 Parameter(help="Path to the Airbyte monorepo. Can be inferred from context."), 2176 ] = None, 2177 output_format: Annotated[ 2178 ReleaseBlockListFormat, 2179 Parameter( 2180 help=( 2181 'Output format: "text" (human-readable), ' 2182 '"json" (blocked connectors with block file contents), ' 2183 '"csv" (comma-delimited connector names).' 2184 ) 2185 ), 2186 ] = "text", 2187) -> None: 2188 """List all connectors that have a `block-release.yaml` marker. 2189 2190 Example: 2191 2192 airbyte-ops local connector release-block list --repo-path /path/to/airbyte 2193 airbyte-ops local connector release-block list --output-format json 2194 airbyte-ops local connector release-block list --output-format csv 2195 """ 2196 resolved_path = repo_path or os.environ.get("AIRBYTE_REPO_PATH", ".") 2197 result = list_release_blocks(repo_path=resolved_path) 2198 2199 if output_format == "json": 2200 print_json(result.to_dict()) 2201 elif output_format == "csv": 2202 names = [info.connector_name for info in result.blocked_connectors] 2203 sys.stdout.write(",".join(names) + "\n") 2204 else: 2205 # text format (default) 2206 if result.count == 0: 2207 console.print( 2208 "[green]No connectors are currently blocked from release.[/green]" 2209 ) 2210 else: 2211 console.print( 2212 f"[yellow]{result.count} connector(s) blocked from release:[/yellow]" 2213 ) 2214 for info in result.blocked_connectors: 2215 version_str = ( 2216 f" (yanked: {info.yanked_version})" if info.yanked_version else "" 2217 ) 2218 console.print(f" - {info.connector_name}: {info.reason}{version_str}")