airbyte_ops_mcp.mcp.session_feedback
MCP tool for Devin session feedback reporting.
This module exposes the session feedback operation as an MCP tool for AI agents. It is a thin wrapper around the shared dispatch function in the human_in_the_loop module, with hardcoded channel, emoji, and formatting for the feedback use case.
Feedback is structured with categories and fields modeled after standard issue/bug report templates to ensure actionable, consistent reports.
MCP reference
MCP primitives registered by the session_feedback module of the airbyte-internal-ops server: 2 tool(s), 0 prompt(s), 0 resource(s).
Tools (2)
devin_session_feedback
Hints: open-world
Report structured feedback about a Devin session experience via Slack.
Posts a formatted feedback message to the #hydra-feedback Slack channel, tagging the reporting user and @AJ Steers. The message includes a clickable button for the Devin session link. For negative feedback, a triage workflow is automatically dispatched to launch a Devin session with v3 analyze mode that can inspect the original session's full conversation history.
IMPORTANT: This feedback will be logged publicly in Slack. Inform the user that their feedback is visible to the team and they may be contacted for additional details.
Use this tool when a user explicitly asks to report a positive or negative experience with their Devin session. Before calling this tool, let the user know:
- Their feedback will be posted publicly in the #hydra-feedback Slack channel
- They may be contacted by the team for more details
- Both the reporting user and @AJ Steers will be tagged in the message
- For negative feedback, a triage session will be automatically launched to inspect the reported session
The Slack message is sent by a GitHub Actions workflow so that Slack credentials are never exposed to the calling agent.
Parameters:
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
feedback_type |
enum("positive", "negative") |
yes | — | Type of feedback: 'positive' for a good experience or 'negative' for a bad experience. Use 'positive' when the user expresses satisfaction, praise, or a success story. Use 'negative' when the user reports a problem, frustration, or failure. |
category |
enum("tool_failure", "missing_guidance", "suspected_hallucination", "bad_approach", "excessive_iteration", "poor_quality", "other_concern", "great_results", "exceeded_expectations", "fast_completion", "good_communication", "other_positive_feedback") |
yes | — | Feedback category. For NEGATIVE feedback, use one of: 'tool_failure' (a specific tool/integration broke), 'missing_guidance' (Devin lacked instructions or context), 'suspected_hallucination' (Devin fabricated information or made incorrect claims), 'bad_approach' (Devin took a fundamentally wrong strategy), 'excessive_iteration' (too many loops/retries before success), 'poor_quality' (output quality below expectations), 'other_concern'. For POSITIVE feedback, use one of: 'great_results' (task completed with high quality), 'exceeded_expectations' (went above and beyond), 'fast_completion' (completed quickly and efficiently), 'good_communication' (kept user well-informed), 'other_positive_feedback'. |
task_description |
string |
yes | — | Brief description of what the user asked Devin to do. This sets the context for the feedback. |
agent_session_url |
string |
yes | — | Your agent session URL so the team can view the full context. Use the session URL from your system prompt. |
reporting_user |
string |
yes | — | The person providing the feedback. Accepts an email address (e.g. 'aj@airbyte.io'), a GitHub handle prefixed with @ (e.g. '@aaronsteers'), or a Slack user ID (e.g. 'U05AKF1BCC9'). |
session_playbook |
string |
yes | — | ID of the Devin playbook associated with the session (e.g. 'devin_feedback_triage'), or 'none' when no playbook is associated. Required so feedback can identify whether playbook instructions may need updates. |
related_skill_name |
string | null |
no | null |
Optional skill ID associated with the feedback (e.g. 'delete-declarative-source-def') when a related skill may need updates or is suspected of having issues. |
expected_behavior |
string | null |
no | null |
What should have happened. REQUIRED for negative feedback. Describe the expected outcome clearly. |
observed_behavior |
string | null |
no | null |
What actually happened. REQUIRED for negative feedback. Describe the actual outcome, including any error messages or unexpected results. |
what_went_well |
string | null |
no | null |
What specifically was good about the experience. REQUIRED for positive feedback. Be specific about what Devin did well. |
severity |
enum("low", "medium", "high", "critical") | null |
no | null |
Severity of the issue. Recommended for negative feedback. 'low' = minor inconvenience, 'medium' = notable impact, 'high' = significant blocker, 'critical' = complete failure. |
steps_to_reproduce |
string | null |
no | null |
Optional steps to reproduce the issue. Helpful for negative feedback to enable the team to investigate. |
session_to_evaluate |
string | null |
no | null |
Optional Devin session URL to evaluate/triage. Use this when reporting feedback about a different session (not your own). If omitted, agent_session_url is used as the session to triage (i.e., the reporter is reporting on itself). |
Show input JSON schema
{
"additionalProperties": false,
"properties": {
"feedback_type": {
"description": "Type of feedback: 'positive' for a good experience or 'negative' for a bad experience. Use 'positive' when the user expresses satisfaction, praise, or a success story. Use 'negative' when the user reports a problem, frustration, or failure.",
"enum": [
"positive",
"negative"
],
"type": "string"
},
"category": {
"description": "Feedback category. For NEGATIVE feedback, use one of: 'tool_failure' (a specific tool/integration broke), 'missing_guidance' (Devin lacked instructions or context), 'suspected_hallucination' (Devin fabricated information or made incorrect claims), 'bad_approach' (Devin took a fundamentally wrong strategy), 'excessive_iteration' (too many loops/retries before success), 'poor_quality' (output quality below expectations), 'other_concern'. For POSITIVE feedback, use one of: 'great_results' (task completed with high quality), 'exceeded_expectations' (went above and beyond), 'fast_completion' (completed quickly and efficiently), 'good_communication' (kept user well-informed), 'other_positive_feedback'.",
"enum": [
"tool_failure",
"missing_guidance",
"suspected_hallucination",
"bad_approach",
"excessive_iteration",
"poor_quality",
"other_concern",
"great_results",
"exceeded_expectations",
"fast_completion",
"good_communication",
"other_positive_feedback"
],
"type": "string"
},
"task_description": {
"description": "Brief description of what the user asked Devin to do. This sets the context for the feedback.",
"type": "string"
},
"agent_session_url": {
"description": "Your agent session URL so the team can view the full context. Use the session URL from your system prompt.",
"type": "string"
},
"reporting_user": {
"description": "The person providing the feedback. Accepts an email address (e.g. 'aj@airbyte.io'), a GitHub handle prefixed with @ (e.g. '@aaronsteers'), or a Slack user ID (e.g. 'U05AKF1BCC9').",
"type": "string"
},
"session_playbook": {
"description": "ID of the Devin playbook associated with the session (e.g. 'devin_feedback_triage'), or 'none' when no playbook is associated. Required so feedback can identify whether playbook instructions may need updates.",
"type": "string"
},
"related_skill_name": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Optional skill ID associated with the feedback (e.g. 'delete-declarative-source-def') when a related skill may need updates or is suspected of having issues."
},
"expected_behavior": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "What should have happened. REQUIRED for negative feedback. Describe the expected outcome clearly."
},
"observed_behavior": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "What actually happened. REQUIRED for negative feedback. Describe the actual outcome, including any error messages or unexpected results."
},
"what_went_well": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "What specifically was good about the experience. REQUIRED for positive feedback. Be specific about what Devin did well."
},
"severity": {
"anyOf": [
{
"enum": [
"low",
"medium",
"high",
"critical"
],
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Severity of the issue. Recommended for negative feedback. 'low' = minor inconvenience, 'medium' = notable impact, 'high' = significant blocker, 'critical' = complete failure."
},
"steps_to_reproduce": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Optional steps to reproduce the issue. Helpful for negative feedback to enable the team to investigate."
},
"session_to_evaluate": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Optional Devin session URL to evaluate/triage. Use this when reporting feedback about a *different* session (not your own). If omitted, agent_session_url is used as the session to triage (i.e., the reporter is reporting on itself)."
}
},
"required": [
"feedback_type",
"category",
"task_description",
"agent_session_url",
"reporting_user",
"session_playbook"
],
"type": "object"
}
Show output JSON schema
{
"description": "Response from the session feedback tool.",
"properties": {
"success": {
"description": "Whether the workflow was triggered successfully",
"type": "boolean"
},
"message": {
"description": "Human-readable status message",
"type": "string"
},
"workflow_url": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "URL to view the GitHub Actions workflow file"
},
"run_id": {
"anyOf": [
{
"type": "integer"
},
{
"type": "null"
}
],
"default": null,
"description": "GitHub Actions workflow run ID"
},
"run_url": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Direct URL to the GitHub Actions workflow run"
},
"triage_run_url": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "URL to the auto-triage workflow run"
}
},
"required": [
"success",
"message"
],
"type": "object"
}
devin_session_feedback_followup
Hints: open-world
Post a follow-up to an existing feedback thread in #hydra-feedback.
This is the "second call" in the feedback workflow: after
devin_session_feedback creates the initial report, this tool appends
triage findings or additional context as a threaded reply.
Each reply is wrapped with a disclaimer clarifying that the thread is non-interactive and not monitored by any agent.
Workspace validation ensures only URLs from the expected Slack workspace are accepted.
Parameters:
| Name | Type | Required | Default | Description |
|---|---|---|---|---|
thread_url |
string |
yes | — | Slack thread URL from the original feedback post in #hydra-feedback. This is the thread where follow-up context will be appended. Example: https://airbytehq-team.slack.com/archives/C0ACUHRP6B1/p1773062711122019 |
message |
string |
yes | — | Follow-up message text in Slack mrkdwn format. Typically a triage report or additional context about the feedback being investigated. Supports bold, _italic_, code, code blocks, > blockquotes, and |
agent_session_url |
string |
yes | — | Your agent session URL for audit trail. Use the session URL from your system prompt. |
Show input JSON schema
{
"additionalProperties": false,
"properties": {
"thread_url": {
"description": "Slack thread URL from the original feedback post in #hydra-feedback. This is the thread where follow-up context will be appended. Example: https://airbytehq-team.slack.com/archives/C0ACUHRP6B1/p1773062711122019",
"type": "string"
},
"message": {
"description": "Follow-up message text in Slack mrkdwn format. Typically a triage report or additional context about the feedback being investigated. Supports *bold*, _italic_, `code`, ```code blocks```, > blockquotes, and <url|label> links.",
"type": "string"
},
"agent_session_url": {
"description": "Your agent session URL for audit trail. Use the session URL from your system prompt.",
"type": "string"
}
},
"required": [
"thread_url",
"message",
"agent_session_url"
],
"type": "object"
}
Show output JSON schema
{
"description": "Response from the session feedback follow-up tool.",
"properties": {
"success": {
"description": "Whether the follow-up was posted successfully",
"type": "boolean"
},
"message": {
"description": "Human-readable status message",
"type": "string"
},
"reply_ts": {
"anyOf": [
{
"type": "string"
},
{
"type": "null"
}
],
"default": null,
"description": "Timestamp of the posted reply (Slack ts format)"
}
},
"required": [
"success",
"message"
],
"type": "object"
}
1# Copyright (c) 2025 Airbyte, Inc., all rights reserved. 2"""MCP tool for Devin session feedback reporting. 3 4This module exposes the session feedback operation as an MCP tool for AI agents. 5It is a thin wrapper around the shared dispatch function in the human_in_the_loop 6module, with hardcoded channel, emoji, and formatting for the feedback use case. 7 8Feedback is structured with categories and fields modeled after standard issue/bug 9report templates to ensure actionable, consistent reports. 10 11## MCP reference 12 13.. include:: ../../../docs/mcp-generated/session_feedback.md 14 :start-line: 2 15""" 16 17from __future__ import annotations 18 19__all__: list[str] = [] 20 21import logging 22import re 23from enum import StrEnum 24from typing import Annotated, Literal 25 26import requests 27from fastmcp import FastMCP 28from fastmcp_extensions import mcp_tool, register_mcp_tools 29from pydantic import BaseModel, Field 30 31from airbyte_ops_mcp.github_actions import ( 32 WorkflowDispatchResult, 33 resolve_default_workflow_branch, 34 trigger_workflow_dispatch, 35) 36from airbyte_ops_mcp.github_api import resolve_ci_trigger_github_token 37from airbyte_ops_mcp.human_in_the_loop import dispatch_escalation 38from airbyte_ops_mcp.slack_api import SlackAPIError, SlackURLParseError 39from airbyte_ops_mcp.slack_posting import parse_slack_thread_url, post_thread_reply 40 41logger = logging.getLogger(__name__) 42 43# Hardcoded settings for the feedback tool — not agent-controllable. 44_FEEDBACK_CHANNEL = "C0ACUHRP6B1" 45_AJ_STEERS_IDENTIFIER = "U05AKF1BCC9" 46 47# Triage workflow settings 48_TRIAGE_REPO_OWNER = "airbytehq" 49_TRIAGE_REPO_NAME = "airbyte-ops-mcp" 50_TRIAGE_WORKFLOW_FILE = "devin-session-triage.yml" 51_TRIAGE_DEFAULT_BRANCH = "main" 52_AI_SKILLS_REPO_URL = "https://github.com/airbytehq/ai-skills" 53_INTERNAL_SKILLS_URL = ( 54 "https://internal.airbyte.ai/docs/internal-docs/ai-engineering/skills" 55) 56_PLAYBOOK_ID_PATTERN = re.compile(r"^[a-z0-9_-]+$") 57_SKILL_ID_PATTERN = re.compile(r"^[a-z0-9-]+$") 58 59# --- Category definitions --- 60 61_CATEGORY_DISPLAY: dict[str, str] = { 62 "tool_failure": "Tool Failure", 63 "missing_guidance": "Missing Guidance", 64 "suspected_hallucination": "Suspected Hallucination", 65 "bad_approach": "Bad Approach", 66 "excessive_iteration": "Excessive Iteration", 67 "poor_quality": "Poor Quality", 68 "other_concern": "Other Concern", 69 "great_results": "Great Results", 70 "exceeded_expectations": "Exceeded Expectations", 71 "fast_completion": "Fast Completion", 72 "good_communication": "Good Communication", 73 "other_positive_feedback": "Other Positive Feedback", 74} 75 76 77class FeedbackCategory(StrEnum): 78 """Feedback categories for Devin session reports.""" 79 80 # Negative categories 81 TOOL_FAILURE = "tool_failure" 82 MISSING_GUIDANCE = "missing_guidance" 83 SUSPECTED_HALLUCINATION = "suspected_hallucination" 84 BAD_APPROACH = "bad_approach" 85 EXCESSIVE_ITERATION = "excessive_iteration" 86 POOR_QUALITY = "poor_quality" 87 OTHER_CONCERN = "other_concern" 88 89 # Positive categories 90 GREAT_RESULTS = "great_results" 91 EXCEEDED_EXPECTATIONS = "exceeded_expectations" 92 FAST_COMPLETION = "fast_completion" 93 GOOD_COMMUNICATION = "good_communication" 94 OTHER_POSITIVE_FEEDBACK = "other_positive_feedback" 95 96 def is_negative(self) -> bool: 97 """Return True if this is a negative feedback category.""" 98 return self in _NEGATIVE_MEMBERS 99 100 def display_name(self) -> str: 101 """Return the human-readable display name for this category.""" 102 return _CATEGORY_DISPLAY.get(self.value, self.value) 103 104 105_NEGATIVE_MEMBERS = frozenset( 106 { 107 FeedbackCategory.TOOL_FAILURE, 108 FeedbackCategory.MISSING_GUIDANCE, 109 FeedbackCategory.SUSPECTED_HALLUCINATION, 110 FeedbackCategory.BAD_APPROACH, 111 FeedbackCategory.EXCESSIVE_ITERATION, 112 FeedbackCategory.POOR_QUALITY, 113 FeedbackCategory.OTHER_CONCERN, 114 } 115) 116 117_SEVERITY_DISPLAY: dict[str, str] = { 118 "low": "Low", 119 "medium": "Medium", 120 "high": "High", 121 "critical": "Critical", 122} 123 124 125def _feedback_emoji(feedback_type: str) -> str: 126 """Return the header emoji for the given feedback type.""" 127 return ":tada:" if feedback_type == "positive" else ":warning:" 128 129 130def _feedback_label(feedback_type: str) -> str: 131 """Return the header label for the given feedback type.""" 132 type_display = "Positive" if feedback_type == "positive" else "Negative" 133 return f"Devin Session Feedback ({type_display})" 134 135 136def _format_playbook_link(playbook_id: str) -> str: 137 """Return Slack mrkdwn for a playbook identifier.""" 138 if playbook_id == "none": 139 return "none" 140 return f"<{_AI_SKILLS_REPO_URL}/blob/main/devin/playbooks/{playbook_id}.md|{playbook_id}>" 141 142 143def _format_skill_link(skill_id: str) -> str: 144 """Return Slack mrkdwn for a skill identifier.""" 145 return f"<{_INTERNAL_SKILLS_URL}/#{skill_id}|{skill_id}>" 146 147 148def _validate_playbook_id(playbook_id: str) -> str | None: 149 """Return an error message if `playbook_id` is not a valid playbook identifier.""" 150 if playbook_id == "none" or _PLAYBOOK_ID_PATTERN.fullmatch(playbook_id): 151 return None 152 return "session_playbook must be 'none' or a lowercase playbook ID using only letters, numbers, '-' and '_'." 153 154 155def _validate_skill_id(skill_id: str | None) -> str | None: 156 """Return an error message if `skill_id` is not a valid skill identifier.""" 157 if skill_id is None or _SKILL_ID_PATTERN.fullmatch(skill_id): 158 return None 159 return "related_skill_name must be a lowercase skill ID using only letters, numbers, and '-'." 160 161 162def _build_feedback_body( 163 *, 164 feedback_type: str, 165 category: str, 166 task_description: str, 167 session_playbook: str, 168 related_skill_name: str | None, 169 expected_behavior: str | None, 170 observed_behavior: str | None, 171 what_went_well: str | None, 172 severity: str | None, 173 steps_to_reproduce: str | None, 174) -> str: 175 """Build a Slack mrkdwn message body from structured feedback fields.""" 176 lines: list[str] = [] 177 178 cat = FeedbackCategory(category) 179 lines.append(f"*Category:* {cat.display_name()}") 180 181 if severity: 182 sev_display = _SEVERITY_DISPLAY.get(severity, severity) 183 lines.append(f"*Severity:* {sev_display}") 184 185 lines.append("") 186 lines.append(f"*Task:* {task_description}") 187 lines.append(f"*Session Playbook:* {_format_playbook_link(session_playbook)}") 188 if related_skill_name: 189 lines.append(f"*Related Skill:* {_format_skill_link(related_skill_name)}") 190 191 if feedback_type == "negative": 192 if expected_behavior: 193 lines.append("") 194 lines.append(f"*Expected Behavior:* {expected_behavior}") 195 if observed_behavior: 196 lines.append("") 197 lines.append(f"*Observed Behavior:* {observed_behavior}") 198 if steps_to_reproduce: 199 lines.append("") 200 lines.append(f"*Steps to Reproduce:* {steps_to_reproduce}") 201 else: 202 if what_went_well: 203 lines.append("") 204 lines.append(f"*What Went Well:* {what_went_well}") 205 206 if feedback_type == "negative": 207 lines.append("") 208 lines.append( 209 "_Auto-triage: a Devin session with v3 analyze mode will inspect this session._" 210 ) 211 212 return "\n".join(lines) 213 214 215def _validate_negative_fields( 216 expected_behavior: str | None, 217 observed_behavior: str | None, 218) -> str | None: 219 """Return an error message if required negative feedback fields are missing.""" 220 missing: list[str] = [] 221 if not expected_behavior: 222 missing.append("expected_behavior") 223 if not observed_behavior: 224 missing.append("observed_behavior") 225 if missing: 226 return f"Negative feedback requires: {', '.join(missing)}." 227 return None 228 229 230def _validate_positive_fields( 231 what_went_well: str | None, 232) -> str | None: 233 """Return an error message if required positive feedback fields are missing.""" 234 if not what_went_well: 235 return "Positive feedback requires: what_went_well." 236 return None 237 238 239def _dispatch_triage_workflow( 240 session_url: str, 241 feedback_context: str, 242 reporting_user: str, 243 session_playbook: str, 244 related_skill_name: str | None = None, 245 cc_persons: str = "", 246 header_emoji: str = "", 247 header_label: str = "", 248) -> WorkflowDispatchResult | None: 249 """Dispatch the v3 session triage workflow. 250 251 The triage workflow launches a Devin session with v3 analyze mode and 252 posts a single Slack notification via the HITL reusable workflow. 253 Formatting params (emoji, header, cc) are passed through to the HITL 254 notification so the caller doesn't need to post separately. 255 256 Returns the dispatch result, or None if dispatch fails. 257 """ 258 token = resolve_ci_trigger_github_token() 259 inputs: dict[str, str] = { 260 "session_url": session_url, 261 "feedback_context": feedback_context, 262 "reporting_user": reporting_user, 263 "session_playbook": session_playbook, 264 } 265 if related_skill_name: 266 inputs["related_skill_name"] = related_skill_name 267 if cc_persons: 268 inputs["cc_persons"] = cc_persons 269 if header_emoji: 270 inputs["header_emoji"] = header_emoji 271 if header_label: 272 inputs["header_label"] = header_label 273 try: 274 return trigger_workflow_dispatch( 275 owner=_TRIAGE_REPO_OWNER, 276 repo=_TRIAGE_REPO_NAME, 277 workflow_file=_TRIAGE_WORKFLOW_FILE, 278 ref=resolve_default_workflow_branch(_TRIAGE_DEFAULT_BRANCH), 279 inputs=inputs, 280 token=token, 281 ) 282 except requests.HTTPError: 283 logger.exception("Failed to dispatch triage workflow") 284 return None 285 286 287class SessionFeedbackResponse(BaseModel): 288 """Response from the session feedback tool.""" 289 290 success: bool = Field(description="Whether the workflow was triggered successfully") 291 message: str = Field(description="Human-readable status message") 292 workflow_url: str | None = Field( 293 default=None, 294 description="URL to view the GitHub Actions workflow file", 295 ) 296 run_id: int | None = Field( 297 default=None, 298 description="GitHub Actions workflow run ID", 299 ) 300 run_url: str | None = Field( 301 default=None, 302 description="Direct URL to the GitHub Actions workflow run", 303 ) 304 triage_run_url: str | None = Field( 305 default=None, 306 description="URL to the auto-triage workflow run", 307 ) 308 309 310@mcp_tool( 311 read_only=False, 312 idempotent=False, 313 open_world=True, 314) 315def devin_session_feedback( 316 feedback_type: Annotated[ 317 Literal["positive", "negative"], 318 Field( 319 description=( 320 "Type of feedback: 'positive' for a good experience or 'negative' for a " 321 "bad experience. Use 'positive' when the user expresses satisfaction, " 322 "praise, or a success story. Use 'negative' when the user reports a problem, " 323 "frustration, or failure." 324 ), 325 ), 326 ], 327 category: Annotated[ 328 FeedbackCategory, 329 Field( 330 description=( 331 "Feedback category. " 332 "For NEGATIVE feedback, use one of: " 333 "'tool_failure' (a specific tool/integration broke), " 334 "'missing_guidance' (Devin lacked instructions or context), " 335 "'suspected_hallucination' (Devin fabricated information or made incorrect claims), " 336 "'bad_approach' (Devin took a fundamentally wrong strategy), " 337 "'excessive_iteration' (too many loops/retries before success), " 338 "'poor_quality' (output quality below expectations), " 339 "'other_concern'. " 340 "For POSITIVE feedback, use one of: " 341 "'great_results' (task completed with high quality), " 342 "'exceeded_expectations' (went above and beyond), " 343 "'fast_completion' (completed quickly and efficiently), " 344 "'good_communication' (kept user well-informed), " 345 "'other_positive_feedback'." 346 ), 347 ), 348 ], 349 task_description: Annotated[ 350 str, 351 Field( 352 description=( 353 "Brief description of what the user asked Devin to do. " 354 "This sets the context for the feedback." 355 ), 356 ), 357 ], 358 agent_session_url: Annotated[ 359 str, 360 Field( 361 description=( 362 "Your agent session URL so the team can view the full context. " 363 "Use the session URL from your system prompt." 364 ), 365 ), 366 ], 367 reporting_user: Annotated[ 368 str, 369 Field( 370 description=( 371 "The person providing the feedback. Accepts an email address " 372 "(e.g. 'aj@airbyte.io'), a GitHub handle prefixed with @ " 373 "(e.g. '@aaronsteers'), or a Slack user ID (e.g. 'U05AKF1BCC9')." 374 ), 375 ), 376 ], 377 session_playbook: Annotated[ 378 str, 379 Field( 380 description=( 381 "ID of the Devin playbook associated with the session (e.g. " 382 "'devin_feedback_triage'), or 'none' when no playbook is associated. " 383 "Required so feedback can identify whether playbook instructions may need updates." 384 ), 385 ), 386 ], 387 related_skill_name: Annotated[ 388 str | None, 389 Field( 390 default=None, 391 description=( 392 "Optional skill ID associated with the feedback (e.g. " 393 "'delete-declarative-source-def') when a related skill may need updates " 394 "or is suspected of having issues." 395 ), 396 ), 397 ], 398 expected_behavior: Annotated[ 399 str | None, 400 Field( 401 default=None, 402 description=( 403 "What should have happened. REQUIRED for negative feedback. " 404 "Describe the expected outcome clearly." 405 ), 406 ), 407 ], 408 observed_behavior: Annotated[ 409 str | None, 410 Field( 411 default=None, 412 description=( 413 "What actually happened. REQUIRED for negative feedback. " 414 "Describe the actual outcome, including any error messages or unexpected results." 415 ), 416 ), 417 ], 418 what_went_well: Annotated[ 419 str | None, 420 Field( 421 default=None, 422 description=( 423 "What specifically was good about the experience. REQUIRED for positive feedback. " 424 "Be specific about what Devin did well." 425 ), 426 ), 427 ], 428 severity: Annotated[ 429 Literal["low", "medium", "high", "critical"] | None, 430 Field( 431 default=None, 432 description=( 433 "Severity of the issue. Recommended for negative feedback. " 434 "'low' = minor inconvenience, 'medium' = notable impact, " 435 "'high' = significant blocker, 'critical' = complete failure." 436 ), 437 ), 438 ], 439 steps_to_reproduce: Annotated[ 440 str | None, 441 Field( 442 default=None, 443 description=( 444 "Optional steps to reproduce the issue. Helpful for negative feedback " 445 "to enable the team to investigate." 446 ), 447 ), 448 ], 449 session_to_evaluate: Annotated[ 450 str | None, 451 Field( 452 default=None, 453 description=( 454 "Optional Devin session URL to evaluate/triage. Use this when reporting " 455 "feedback about a *different* session (not your own). If omitted, " 456 "agent_session_url is used as the session to triage (i.e., the reporter " 457 "is reporting on itself)." 458 ), 459 ), 460 ], 461) -> SessionFeedbackResponse: 462 """Report structured feedback about a Devin session experience via Slack. 463 464 Posts a formatted feedback message to the #hydra-feedback Slack channel, 465 tagging the reporting user and @AJ Steers. The message includes a clickable 466 button for the Devin session link. For negative feedback, a triage workflow 467 is automatically dispatched to launch a Devin session with v3 analyze mode 468 that can inspect the original session's full conversation history. 469 470 IMPORTANT: This feedback will be logged publicly in Slack. Inform the user 471 that their feedback is visible to the team and they may be contacted for 472 additional details. 473 474 Use this tool when a user explicitly asks to report a positive or negative 475 experience with their Devin session. Before calling this tool, let the user 476 know: 477 - Their feedback will be posted publicly in the #hydra-feedback Slack channel 478 - They may be contacted by the team for more details 479 - Both the reporting user and @AJ Steers will be tagged in the message 480 - For negative feedback, a triage session will be automatically launched to inspect the reported session 481 482 The Slack message is sent by a GitHub Actions workflow so that Slack 483 credentials are never exposed to the calling agent. 484 """ 485 # Validate category matches feedback type. 486 cat = FeedbackCategory(category) 487 is_negative_feedback = feedback_type == "negative" 488 id_validation_error = _validate_playbook_id(session_playbook) or _validate_skill_id( 489 related_skill_name 490 ) 491 if id_validation_error: 492 return SessionFeedbackResponse( 493 success=False, 494 message=id_validation_error, 495 ) 496 497 if cat.is_negative() != is_negative_feedback: 498 expected_kind = "negative" if is_negative_feedback else "positive" 499 valid = [ 500 c.value for c in FeedbackCategory if c.is_negative() == is_negative_feedback 501 ] 502 return SessionFeedbackResponse( 503 success=False, 504 message=( 505 f"Invalid category '{category}' for {feedback_type} feedback. " 506 f"Valid {expected_kind} categories: {', '.join(valid)}." 507 ), 508 ) 509 510 # Validate required fields based on feedback type. 511 if feedback_type == "negative": 512 validation_error = _validate_negative_fields( 513 expected_behavior=expected_behavior, 514 observed_behavior=observed_behavior, 515 ) 516 else: 517 validation_error = _validate_positive_fields( 518 what_went_well=what_went_well, 519 ) 520 521 if validation_error: 522 return SessionFeedbackResponse( 523 success=False, 524 message=validation_error, 525 ) 526 527 message_body = _build_feedback_body( 528 feedback_type=feedback_type, 529 category=category, 530 task_description=task_description, 531 session_playbook=session_playbook, 532 related_skill_name=related_skill_name, 533 expected_behavior=expected_behavior, 534 observed_behavior=observed_behavior, 535 what_went_well=what_went_well, 536 severity=severity, 537 steps_to_reproduce=steps_to_reproduce, 538 ) 539 540 # For negative feedback, dispatch triage workflow (which also posts to Slack 541 # via the HITL reusable workflow — single message with triage button). 542 # For positive feedback, dispatch HITL directly (no triage needed). 543 if is_negative_feedback: 544 triage_session_url = session_to_evaluate or agent_session_url 545 # _dispatch_triage_workflow catches exceptions internally and returns None 546 triage_result = _dispatch_triage_workflow( 547 session_url=triage_session_url, 548 feedback_context=message_body, 549 reporting_user=reporting_user, 550 session_playbook=session_playbook, 551 related_skill_name=related_skill_name, 552 cc_persons=_AJ_STEERS_IDENTIFIER, 553 header_emoji=_feedback_emoji(feedback_type), 554 header_label=_feedback_label(feedback_type), 555 ) 556 if triage_result is not None: 557 view_url = triage_result.run_url or triage_result.workflow_url 558 return SessionFeedbackResponse( 559 success=True, 560 message=( 561 "Feedback submitted. Auto-triage workflow launched. " 562 "A Slack notification will be posted to #hydra-feedback " 563 "once the triage session starts. " 564 f"View workflow progress at: {view_url}" 565 ), 566 workflow_url=triage_result.workflow_url, 567 run_id=triage_result.run_id, 568 run_url=triage_result.run_url, 569 triage_run_url=view_url, 570 ) 571 # Triage dispatch failed — fall back to direct HITL notification 572 # so negative feedback is still recorded in Slack. 573 logger.warning( 574 "Triage workflow dispatch failed; falling back to direct HITL dispatch." 575 ) 576 577 # Positive feedback (or negative feedback fallback): dispatch HITL directly 578 result = dispatch_escalation( 579 target_person=reporting_user, 580 message=message_body, 581 agent_session_url=agent_session_url, 582 cc=[_AJ_STEERS_IDENTIFIER], 583 channel_override=_FEEDBACK_CHANNEL, 584 header_emoji=_feedback_emoji(feedback_type), 585 header_label=_feedback_label(feedback_type), 586 ) 587 588 view_url = result.run_url or result.workflow_url 589 return SessionFeedbackResponse( 590 success=True, 591 message=( 592 f"Feedback submitted and posted to #hydra-feedback. " 593 f"The reporting user and @AJ Steers have been tagged. " 594 f"View progress at: {view_url}" 595 ), 596 workflow_url=result.workflow_url, 597 run_id=result.run_id, 598 run_url=result.run_url, 599 ) 600 601 602# --------------------------------------------------------------------------- 603# Non-interactive thread disclaimer 604# --------------------------------------------------------------------------- 605 606_FOLLOWUP_HEADER = "🤖 *Automated Triage Update*" 607_FOLLOWUP_FOOTER_TEMPLATE = ( 608 "_ℹ️ This thread is not monitored by Devin. " 609 "Replies here will not be seen by any agent. " 610 "For follow-up, use the <{agent_session_url}|linked session> or create a new task._" 611) 612 613 614def _wrap_followup_message(message: str, *, agent_session_url: str) -> str: 615 """Wrap a follow-up message with session link and non-interactive disclaimer.""" 616 footer = _FOLLOWUP_FOOTER_TEMPLATE.format(agent_session_url=agent_session_url) 617 return f"{_FOLLOWUP_HEADER}\n\n{message}\n\n{footer}" 618 619 620# --------------------------------------------------------------------------- 621# Feedback follow-up tool 622# --------------------------------------------------------------------------- 623 624 625class SessionFeedbackFollowupResponse(BaseModel): 626 """Response from the session feedback follow-up tool.""" 627 628 success: bool = Field(description="Whether the follow-up was posted successfully") 629 message: str = Field(description="Human-readable status message") 630 reply_ts: str | None = Field( 631 default=None, 632 description="Timestamp of the posted reply (Slack ts format)", 633 ) 634 635 636@mcp_tool( 637 read_only=False, 638 idempotent=False, 639 open_world=True, 640) 641def devin_session_feedback_followup( 642 thread_url: Annotated[ 643 str, 644 Field( 645 description=( 646 "Slack thread URL from the original feedback post in #hydra-feedback. " 647 "This is the thread where follow-up context will be appended. " 648 "Example: https://airbytehq-team.slack.com/archives/C0ACUHRP6B1/p1773062711122019" 649 ), 650 ), 651 ], 652 message: Annotated[ 653 str, 654 Field( 655 description=( 656 "Follow-up message text in Slack mrkdwn format. " 657 "Typically a triage report or additional context about the " 658 "feedback being investigated. " 659 "Supports *bold*, _italic_, `code`, ```code blocks```, " 660 "> blockquotes, and <url|label> links." 661 ), 662 ), 663 ], 664 agent_session_url: Annotated[ 665 str, 666 Field( 667 description=( 668 "Your agent session URL for audit trail. " 669 "Use the session URL from your system prompt." 670 ), 671 ), 672 ], 673) -> SessionFeedbackFollowupResponse: 674 """Post a follow-up to an existing feedback thread in #hydra-feedback. 675 676 This is the "second call" in the feedback workflow: after 677 `devin_session_feedback` creates the initial report, this tool appends 678 triage findings or additional context as a threaded reply. 679 680 Each reply is wrapped with a disclaimer clarifying that the thread is 681 non-interactive and not monitored by any agent. 682 683 Workspace validation ensures only URLs from the expected Slack 684 workspace are accepted. 685 """ 686 try: 687 channel_id, thread_ts = parse_slack_thread_url(thread_url) 688 except SlackURLParseError as exc: 689 return SessionFeedbackFollowupResponse( 690 success=False, 691 message=str(exc), 692 ) 693 694 wrapped_message = _wrap_followup_message( 695 message, agent_session_url=agent_session_url 696 ) 697 698 try: 699 reply_ts = post_thread_reply( 700 channel_id=channel_id, 701 thread_ts=thread_ts, 702 message=wrapped_message, 703 ) 704 except SlackAPIError as exc: 705 return SessionFeedbackFollowupResponse( 706 success=False, 707 message=f"Slack API error: {exc}", 708 ) 709 710 logger.info( 711 "Feedback follow-up posted: channel=%s thread_ts=%s agent=%s", 712 channel_id, 713 thread_ts, 714 agent_session_url, 715 ) 716 return SessionFeedbackFollowupResponse( 717 success=True, 718 message=f"Follow-up posted to feedback thread in channel {channel_id}.", 719 reply_ts=reply_ts, 720 ) 721 722 723def register_session_feedback_tools(app: FastMCP) -> None: 724 """Register session feedback tools with the FastMCP app.""" 725 register_mcp_tools(app, mcp_module=__name__)