feat: harden agent verification and runtime contracts

Introduce two-gate checker settlement, explicit graph visibility and checkpoints, structured LLM reliability, centralized action specifications, and corresponding tests and design documentation.

artemis/agents/checker/checker.json

@@ -1,4 +1,9 @@
 {
-  "checker_system": "You are a precise and reliable verification expert (Checker). Your task is to determine whether the Operator has successfully completed the active subgoal by combining visual evidence from the current screenshot with data stored in persistent notes.",
-    "checker_prompt": "# Inputs\n- Active Subgoal: {{ subgoal }}\n- Execution History:\n{{ plan_and_history }}\n- Dialogue History: {{ dialogue_history }}\n- Visual State: The provided screenshot.\n\n# Guidelines\n- **Visual + Data Verification**: You must combine the screenshot with note contents. If the subgoal involves extracting, saving, or recording information (e.g., \"save contacts\", \"record address\"), you **MUST** verify that the content actually exists in the note. Do not assume success just because the information is visible on the screen. If the signs of success are not obvious, you can check the history, especially the most recent steps, to see if the operator performed their own check and whether that check was reasonable, to determine if the subgoal was truly successful.\n- **Visual Priority (Anti-History-Bias)**: Your primary ground truth is the visual evidence on the **Current Screenshot**. The `Execution History` lists previous successes or failures, which may be outdated due to dynamic background recovery or self-healing. If the screenshot demonstrably shows the active subgoal's target state is achieved, you **MUST** mark `success: true`, regardless of previous failed steps. Trust what you see, not past FAILED texts.\n- **Constraint Verification**: For subgoals containing specific parameters (such as names, brands, topics, events, numbers, or dates), confirm that the visible text precisely reflects these specified details. A successful state is defined by a clear, direct match with the subgoal's constraints.\n- **Iteration & Monitoring Subgoals Validation**: For concrete iteration sub-tasks (e.g., inspecting Candidate #1) or periodic polling subgoals (e.g., Polling Round #1 check), verify whether the Operator correctly executed the step and logged the progress/findings into notes or the task plan. Do NOT mark a monitoring or iteration step as failed simply because the overall monitoring session is ongoing or no new incoming events were detected in that specific polling cycle. As long as the discrete check was performed and recorded, mark `success: true`.\n- **Realistic Expectations**: You cannot scroll the screen. If a subgoal requires saving all items but only some are visible on the screen, verify that at least the **visible items** are recorded in the note. As long as the visible part is recorded and there are no obvious errors, consider it a success. Do not fail the task simply because off-screen content cannot be verified.\n- **Success Criteria**: Set `success` to `true` if the core objective of the subgoal is met and it does not block subsequent steps. Set `success` to `false` if the state is incorrect, required data was not saved, or the operator is stuck/looping.\n\n# Output Protocol\nInvoke the `submit_verification` tool with `success` (boolean) and `reason` (string) to submit your final decision. If you need to inspect note data first, use `read_note`."
+  "system": "You are a precise, evidence-driven verification expert (Checker). You audit declared check items against recorded execution evidence and issue verdicts. You never operate the device, never modify any state, and never negotiate with the executing agent — you only observe, probe read-only state where allowed, and judge.",
+  "base_rules": "# Verdict Rules (always apply)\n- Quote every check item verbatim in `item_text` — never paraphrase.\n- Every verdict MUST carry concrete evidence: a specific probe output, a specific note excerpt, or a specific on-screen/screenshot observation. A `failed` verdict without concrete evidence is invalid and will be downgraded.\n- `inconclusive` means the evidence needed for a judgment was never captured or is genuinely ambiguous. When evidence was never collected, output `inconclusive` and state explicitly that the evidence is missing — do NOT output `failed` for lack of evidence.\n- Judge only what the check item declares. Do not invent additional criteria and do not relax the declared ones.",
+  "verify_semantics": "# `verify` items (completion acceptance criteria)\n- A `verify` item asks: was the anchored subgoal actually accomplished to this standard?\n- `failed` verify verdicts trigger a repair loop, so a failed verdict MUST include a `suggestion`: a concrete, actionable hint telling the executor what is missing or wrong.\n- The release decision (whether execution proceeds) is computed outside of you and is fail-open on errors; your job is only the truthful verdict value.",
+  "assert_semantics": "# `assert` items (test assertions)\n- An `assert` item is a test expectation, not an instruction. A failed assertion is a legitimate, valuable test result: record it truthfully with evidence. Nothing will (or should) try to repair it.\n- Never mark an assert `failed` merely because evidence is absent — absence of captured evidence is `inconclusive` with an explicit note that the evidence is missing.\n- `suggestion` is not needed for assert verdicts.",
+  "anchor_guide": "# Evidence anchoring (checkpoint audit)\n- You are auditing a subgoal at its completion moment. The authoritative evidence is the execution history around the anchor step (the anchor step's pre-action screenshot and the preceding step's post-action state) plus persistently probeable device state.\n- For transient expectations (toasts, dialogs, momentary states) and ordering expectations, you MUST inspect the recorded history near the anchor step — the live device has already moved on and is NOT valid evidence for them.\n- Prefer persistent state probes and recorded notes over inference from summaries.",
+  "final_guide": "# Final review (task exit audit)\n- Audit against the USER'S ORIGINAL GOAL, not merely the plan's checkboxes.\n- `@end` items: judge them using the FINAL device state provided to you.\n- `on_complete` items: cite the verdict ledger — an already-recorded PASS/FAIL from the item's checkpoint attempt is authoritative (an assert that ever failed stays failed). For items never checked, judge from the historical evidence near their subgoal's completion; if that evidence was never captured, output `inconclusive` stating the evidence is missing. NEVER re-judge an `on_complete` item from the current screen: the current screen only proves the present, not the past moment the item was declared for.\n- `unmet_subgoals`: list, verbatim, any plan subgoal marked complete whose verify criteria are demonstrably not met. Leave it empty otherwise; failed asserts alone do NOT make a subgoal unmet.",
+  "probe_guide": "# Device probes (read-only)\n- `probe_device` runs a fixed, enumerated set of read-only state probes. Prefer probing persistent state over inferring it: alarms (`alarms`), battery status (`battery`), foreground activities (`foreground`), notifications (`notifications`), a settings value (`setting` with `namespace` + `key`), a content provider query (`content` with `uri`), installed packages (`packages`), a system property (`prop` with `key`).\n- Probes outside this table do not exist; you cannot run arbitrary commands."
 }

artemis/agents/diagnoser/diagnoser.py

@@ -30,7 +30,7 @@ from artemis.data_engine.trace import (
     trace_langchain_tool,
 )
 from artemis.graph.state import State
-from artemis.services.llm import get_llm, invoke_llm_with_timeout_message
+from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
 from artemis.tools.command_tool import get_run_short_adb_command_tool
 from artemis.tools.diagnoser_submit_answer_tool import get_submit_answer_tool
 from artemis.tools.index import get_tool_by_name
@@ -173,21 +173,11 @@ class Diagnoser:
                 logger.error(f"Failed to build plan and history in Diagnoser: {e}")
                 plan_and_history = f"Error building plan and history: {e}"
 
-        # Get Checker Feedback
+        # Get verification findings injected by the checkpoint harvest, if any
         checker_feedback = ""
-        if self.ctx.data_engine:
-            notes_dir = Path(self.ctx.data_engine.base_dir) / "notes"
-            verification_chat_path = notes_dir / f"verification_chat_{subgoal_hash}.json"
-            if verification_chat_path.exists():
-                try:
-                    turns = json.loads(verification_chat_path.read_text(encoding="utf-8"))
-                    dialogue_lines = []
-                    for t in turns:
-                        role = "Operator" if t["role"] == "operator" else "Checker"
-                        dialogue_lines.append(f"**{role} (Round {t['round']})**:\n{t['content']}")
-                    checker_feedback = "\n\n".join(dialogue_lines)
-                except Exception as e:
-                    logger.error(f"Error reading verification chat in Diagnoser: {e}")
+        findings = getattr(state, "operator_feedback", None)
+        if findings:
+            checker_feedback = "\n".join(f"- {f}" for f in findings)
 
         # Get Current Screenshot and UI Hierarchy
         latest_screenshot_b64 = None
@@ -309,18 +299,6 @@ class Diagnoser:
                         )
                     llm = base_llm.bind_tools(tools=traced_tools)
 
-                async def run_stream():
-                    full_response = None
-                    trace_id = CURRENT_TRACE_ID.get()
-                    async for chunk in llm.astream(current_messages):
-                        if full_response is None:
-                            full_response = chunk
-                        else:
-                            full_response += chunk
-
-                        pass
-                    return full_response
-
                 with TraceSpan(name="gemini_diagnoser_call", ctx=self.ctx) as span:
                     span.payload = {
                         "context_components": {
@@ -331,7 +309,9 @@ class Diagnoser:
                             "diagnostic_query": prompt,
                         }
                     }
-                    response = await invoke_llm_with_timeout_message(run_stream())
+                    response = await invoke_llm_with_timeout_message(
+                        acomplete(llm, current_messages)
+                    )
                     span.result = (
                         response.content if hasattr(response, "content") else str(response)
                     )

artemis/agents/flash/runner.py

@@ -49,9 +49,11 @@ from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.data_engine.trace import trace
 from artemis.graph.perception import _check_injected_instruction_file
 from artemis.graph.state import State
+from artemis.llm.structured import ParseFailure, parse_structured
 from artemis.mcp.action_executor import McpActionExecutor
 from artemis.services.llm import (
     RobustChatModelWrapper,
+    acomplete,
     get_google_llm,
     get_llm,
     invoke_llm_with_timeout_message,
@@ -182,20 +184,23 @@ class FlashRunner:
             # Check for real-time injected instructions
             if self.ctx.data_engine and self.ctx.data_engine.base_dir:
                 try:
-                    injected_instruction = await asyncio.to_thread(
+                    injected_payload = await asyncio.to_thread(
                         _check_injected_instruction_file,
                         str(self.ctx.data_engine.base_dir),
                     )
-                    if injected_instruction:
-                        messages.append(
-                            HumanMessage(
-                                content=(
-                                    "[REAL-TIME INJECTED INSTRUCTION from user]:"
-                                    f" {injected_instruction}\nYou MUST immediately follow"
-                                    " this instruction and adjust your plan/actions."
-                                )
-                            )
+                    if injected_payload and injected_payload.get("instruction"):
+                        injected_text = (
+                            "[REAL-TIME INJECTED INSTRUCTION from user]:"
+                            f" {injected_payload['instruction']}\nYou MUST immediately"
+                            " follow this instruction and adjust your plan/actions."
                         )
+                        if injected_payload.get("release_loop"):
+                            injected_text += (
+                                "\nThe user has explicitly authorized stopping any"
+                                " ongoing monitoring loop; you may now wrap up and"
+                                " complete the task."
+                            )
+                        messages.append(HumanMessage(content=injected_text))
                 except Exception as e:
                     logger.warning(f"Failed to check injected instruction in FlashRunner: {e}")
 
@@ -224,61 +229,12 @@ class FlashRunner:
             # Bind active tools
             bound_llm = llm.bind_tools(current_tools)
 
-            # Invoke Model with Streaming & Retry
-            response = None
-            max_retries = 3
-
-            for attempt in range(max_retries):
-                try:
-                    stream_exec_id = uuid.uuid4()
-
-                    async def run_stream():
-                        full_response = None
-                        async for chunk in bound_llm.astream(messages):
-                            if full_response is None:
-                                full_response = chunk
-                            else:
-                                full_response += chunk
-
-                            # Real-time token streaming to DataEngine for live UI display
-                            if (
-                                getattr(self, "ctx", None)
-                                and getattr(self.ctx, "data_engine", None)
-                                and getattr(chunk, "content", None)
-                            ):
-                                text_to_stream = ""
-                                thinking_to_stream = ""
-                                if isinstance(chunk.content, str):
-                                    text_to_stream = chunk.content
-                                elif isinstance(chunk.content, list):
-                                    for item in chunk.content:
-                                        if isinstance(item, str):
-                                            text_to_stream += item
-                                        elif isinstance(item, dict):
-                                            if item.get("type") == "text":
-                                                text_to_stream += item.get("text", "")
-                                            elif item.get("type") == "thinking":
-                                                thinking_to_stream += item.get("thinking", "")
-                                if text_to_stream:
-                                    self.ctx.data_engine.stream_output(
-                                        stream_exec_id, text_to_stream, is_thinking=False
-                                    )
-                                if thinking_to_stream:
-                                    self.ctx.data_engine.stream_output(
-                                        stream_exec_id, thinking_to_stream, is_thinking=True
-                                    )
-                        return full_response
-
-                    response = await invoke_llm_with_timeout_message(
-                        run_stream(), timeout_seconds=10, hard_timeout=180
-                    )
-                    if response:
-                        break
-                except Exception as e:
-                    logger.warning(f"FlashRunner LLM turn failed on attempt {attempt + 1}: {e}")
-                    if attempt == max_retries - 1:
-                        raise e
-                    await asyncio.sleep(2.0**attempt)
+            # Invoke Model. Streaming, live-token UI deltas, classified
+            # retries, and pause/resume are all owned by the LLM gateway
+            # (acomplete); a typed LLMCallError propagates if it gives up.
+            response = await invoke_llm_with_timeout_message(
+                acomplete(bound_llm, messages), timeout_seconds=10, hard_timeout=180
+            )
 
             if response is None:
                 break
@@ -372,19 +328,20 @@ class FlashRunner:
 
             # Fallback text parsing if tool_calls not parsed natively
             if not tool_calls and "```json" in raw_text:
-                try:
-                    snippet = raw_text.split("```json")[1].split("```")[0].strip()
-                    parsed = json.loads(snippet)
-                    if isinstance(parsed, dict) and "name" in parsed:
-                        tool_calls = [
-                            {
-                                "name": parsed["name"],
-                                "args": parsed.get("args", {}),
-                                "id": str(uuid.uuid4()),
-                            }
-                        ]
-                except Exception:
-                    pass
+                parsed = parse_structured(raw_text)
+                if isinstance(parsed, ParseFailure):
+                    logger.warning(
+                        "FlashRunner response contained a JSON block that could"
+                        f" not be parsed: {parsed.error}"
+                    )
+                elif isinstance(parsed, dict) and "name" in parsed:
+                    tool_calls = [
+                        {
+                            "name": parsed["name"],
+                            "args": parsed.get("args", {}),
+                            "id": str(uuid.uuid4()),
+                        }
+                    ]
 
             if not tool_calls:
                 logger.info(

artemis/agents/history_analyzer/history_analyzer.py

@@ -25,7 +25,7 @@ from langchain_core.messages import (
 from langchain_core.tools import BaseTool, tool
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan, trace
-from artemis.services.llm import get_llm, invoke_llm_with_timeout_message
+from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
 from artemis.tools.scratchpad import (
     get_list_notes_tool_pure,
     get_read_note_tool_pure,
@@ -164,19 +164,7 @@ class HistoryAnalyzer:
         for turn in range(max_turns):
             logger.info(f"HistoryAnalyzer ReAct turn {turn + 1}")
 
-            async def run_stream():
-                full_response = None
-                trace_id = CURRENT_TRACE_ID.get()
-                async for chunk in llm.astream(messages):
-                    if full_response is None:
-                        full_response = chunk
-                    else:
-                        full_response += chunk
-
-                    pass
-                return full_response
-
-            response = await invoke_llm_with_timeout_message(run_stream())
+            response = await invoke_llm_with_timeout_message(acomplete(llm, messages))
             messages.append(response)
 
             if not response.tool_calls:

artemis/agents/log_analyzer/log_analyzer.py

@@ -58,7 +58,7 @@ from artemis.data_engine.trace import (
     trace,
 )
 from artemis.graph.state import State
-from artemis.services.llm import get_llm, invoke_llm_with_timeout_message
+from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
 from artemis.tools.index import get_tool_by_name
 from artemis.tools.mobile.log_utils import fetch_and_filter_logs
 from artemis.tools.mobile.read_logs import get_read_logs_tool
@@ -220,19 +220,7 @@ class LogAnalyzerNode:
                     )
                 llm = base_llm.bind_tools(tools=tools_for_binding)
 
-            async def run_stream():
-                full_response = None
-                trace_id = CURRENT_TRACE_ID.get()
-                async for chunk in llm.astream(current_messages):
-                    if full_response is None:
-                        full_response = chunk
-                    else:
-                        full_response += chunk
-
-                    pass
-                return full_response
-
-            response = await invoke_llm_with_timeout_message(run_stream())
+            response = await invoke_llm_with_timeout_message(acomplete(llm, current_messages))
 
             if response.content:
                 outcome = response.content

artemis/agents/object_detector/object_detector.py

@@ -22,9 +22,8 @@ from pathlib import Path
 import re
 
 from langchain_core.messages import HumanMessage, ToolMessage
-from langgraph.types import Command
 
-from artemis.constants import VALIDATOR_MESSAGES_KEY
+from artemis.llm.structured import ParseFailure, parse_structured
 from artemis.services.llm import get_llm
 from artemis.utils.logger import get_logger
 
@@ -78,22 +77,24 @@ async def _detect_single_label(
                 )
             logger.info(f"Received response for '{label}' (Attempt {i + 1})")
 
-            cleaned_initial = output.strip()
-            code_block_match = re.search(r"```(?:json)?\s*([\s\S]*?)\s*```", cleaned_initial)
-            if code_block_match:
-                cleaned_initial = code_block_match.group(1).strip()
-
-            try:
-                res_json = json.loads(cleaned_initial)
-                if isinstance(res_json, list):
-                    valid_results = []
-                    for item in res_json:
-                        if isinstance(item, dict):
-                            item["label"] = label
-                            valid_results.append(item)
-                    return valid_results
-            except json.JSONDecodeError:
-                pass
+            res_json = parse_structured(output)
+            if isinstance(res_json, list):
+                valid_results = []
+                for item in res_json:
+                    if isinstance(item, dict):
+                        item["label"] = label
+                        valid_results.append(item)
+                return valid_results
+            if isinstance(res_json, ParseFailure):
+                logger.warning(
+                    f"Object detector response for '{label}' was not valid"
+                    f" JSON (attempt {i + 1}): {res_json.error}"
+                )
+            else:
+                logger.warning(
+                    f"Object detector response for '{label}' parsed to"
+                    f" {type(res_json).__name__} instead of a list (attempt {i + 1})."
+                )
 
         except Exception as e:
             logger.warning(f"Detection attempt {i + 1} failed for '{label}': {e}")
@@ -194,35 +195,17 @@ async def _run_object_detection(
 
 
 async def _create_error_command(ctx, state, tool_call_id, error_message, wrapper):
-    tool_message = ToolMessage(
+    return ToolMessage(
         tool_call_id=tool_call_id or "default_tool_call",
         content=wrapper.on_failure_fn(error_message),
         additional_kwargs={"error": error_message},
         status="error",
     )
-    return Command(
-        update=await state.asanitize_update(
-            ctx=ctx,
-            update={
-                VALIDATOR_MESSAGES_KEY: [tool_message],
-            },
-            agent="validator",
-        ),
-    )
 
 
 async def _create_success_command(ctx, state, tool_call_id, output, wrapper):
-    tool_message = ToolMessage(
+    return ToolMessage(
         tool_call_id=tool_call_id or "default_tool_call",
         content=wrapper.on_success_fn(output),
         status="success",
     )
-    return Command(
-        update=await state.asanitize_update(
-            ctx=ctx,
-            update={
-                VALIDATOR_MESSAGES_KEY: [tool_message],
-            },
-            agent="validator",
-        ),
-    )

artemis/agents/operator/operator.json

@@ -1,4 +1,3 @@
 {
-  "main_template": "# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen. For any task requiring continuous range interactions, sliding, or boundary selection, prefer using targeted swipe/drag gestures spanning from the element's start bounds [xmin, ymin] to end bounds [xmax, ymax] to ensure the action is strictly confined to the target element. For drag-and-drop, list reordering, or sliding sliders/SeekBars, set duration >= 1000 (e.g. 1500) and always drag slightly PAST the target position to overcome touch slop and reliably trigger the drop/snap/value update. When setting a slider to Maximum (100%) or Minimum (0%), always swipe fully to the extreme boundary to guarantee reaching the absolute limits.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n- **Tool Calling Iron Rule**: Do NOT submit a Turn-Ending Action at the same time (in the same tool-call list) as a Pre-Decision Exploratory Tool. You must first gather information (e.g., finding coordinates or diagnosing a state), manage ADB background tasks, or update memory notes. You can use pre-decision exploratory tools to complete this task. You will be prompted again with the tool's result, at which point you can output your final physical/turn-ending action.\n- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n- **Memory Preservation & Paginated Exploration**: Using note tools is entirely zero-cost. Actively use this tool to maintain goals and logs. The note tool is the only reliable way to persist memory. Therefore, when you find information on the screen that might be relevant to the task, even if you are not certain, you should first save the information currently visible on the screen before exploring further. If you have already recorded some information, you can use `append_note` to append the new information to the existing content. The note tool can also help you organize lists or information across multiple screens; you can consolidate all information into notes for unified processing.\n- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor. Calibrate your scroll distance so that this anchor remains visible near the top of the subsequent screen, creating a seamless visual overlap that guarantees zero missed items. Terminate exploration when a definitive boundary is reached (e.g., list bottom reached or no new content appears after scrolling).\n  2. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.\n  3. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for \"double-checking\". Only perform incremental re-verification if the list has been refreshed or contentLine truncated
-  "troubleshooter_template": "# ROLE\nYou are the Expert Android Device Troubleshooter. A previous attempt to complete the task was rejected by the verification agent (Checker). Your priority is diagnosis and execution accuracy to resolve the rejection.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen. For any task requiring continuous range interactions, sliding, or boundary selection, prefer using targeted swipe/drag gestures spanning from the element's start bounds [xmin, ymin] to end bounds [xmax, ymax] to ensure the action is strictly confined to the target element. For drag-and-drop, list reordering, or sliding sliders/SeekBars, set duration >= 1000 (e.g. 1500) and always drag slightly PAST the target position to overcome touch slop and reliably trigger the drop/snap/value update. When setting a slider to Maximum (100%) or Minimum (0%), always swipe fully to the extreme boundary to guarantee reaching the absolute limits.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n- **Tool Calling Iron Rule**: Do NOT submit a Turn-Ending Action at the same time (in the same tool-call list) as a Pre-Decision Exploratory Tool. You must first gather information (e.g., finding coordinates or diagnosing a state), manage ADB background tasks, or update memory notes. You can use pre-decision exploratory tools to complete this task. You will be prompted again with the tool's result, at which point you can output your final physical/turn-ending action.\n- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n- **Memory Preservation & Paginated Exploration**: Using note tools is entirely zero-cost. Actively use this tool to maintain goals and logs. The note tool is the only reliable way to persist memory. Therefore, when you find information on the screen that might be relevant to the task, even if you are not certain, you should first save the information currently visible on the screen before exploring further. If you have already recorded some information, you can use `append_note` to append the new information to the existing content. The note tool can also help you organize lists or information across multiple screens; you can consolidate all information into notes for unified processing.\n- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor. Calibrate your scroll distance so that this anchor remains visible near the top of the subsequent screen, creating a seamless visual overlap that guarantees zero missed items. Terminate exploration when a definitive boundary is reached (e.g., list bottom reached or no new content appears after scrolling).\n  2. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.\n  3. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for \"double-checking\". Only perform incremental rLine truncated
+  "main_template": "# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen.[% if \"swipe\" in available_tools %] For any task requiring continuous range interactions, sliding, or boundary selection, prefer using targeted swipe/drag gestures spanning from the element's start bounds [xmin, ymin] to end bounds [xmax, ymax] to ensure the action is strictly confined to the target element. For drag-and-drop, list reordering, or sliding sliders/SeekBars, set duration >= 1000 (e.g. 1500) and always drag slightly PAST the target position to overcome touch slop and reliably trigger the drop/snap/value update. When setting a slider to Maximum (100%) or Minimum (0%), always swipe fully to the extreme boundary to guarantee reaching the absolute limits.[% endif %]\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n- **Tool Calling Contract**: Tools that return information needed for the decision ([[ tool_enum(pre_decision_tools, \"and\") ]]) are result-dependent Pre-Decision Tools. Do not submit them in the same tool-call list as a Turn-Ending Action; inspect their result first. The write-through tools `update_note` and `append_note` may accompany at most one Turn-Ending Action when their content is based entirely on evidence already available before that action. Never write an action's expected outcome as if it had already been observed.\n[% if \"wait_for_delay\" in available_tools %]- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n[% endif %]- **Memory Preservation & Paginated Exploration**: Using note tools is entirely zero-cost. Actively use this tool to maintain goals and logs. The note tool is the only reliable way to persist memory. Therefore, when you find information on the screen that might be relevant to the task, even if you are not certain, you should first save the information currently visible on the screen before exploring further. If you have already recorded some information, you can use `append_note` to append the new information to the existing content. The note tool can also help you organize lists or information across multiple screens; you can consolidate all information into notes for unified processing.\n- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.\n  2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading state. This boundary probe must remain unidLine truncated
 }

artemis/agents/operator/operator.py

@@ -12,21 +12,14 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import asyncio
 import json
-from pathlib import Path
 import re
-from typing import Annotated, Any, Literal
+from typing import Any
 from uuid import uuid4
 
 from langchain_core.messages import SystemMessage, ToolMessage
-from langchain_core.tools import BaseTool, tool
-from langgraph.types import Command
+from langchain_core.tools import BaseTool
 
-from artemis.constants import (
-    OPERATOR_MAX_CHAT_ROUNDS,
-    VALIDATOR_MESSAGES_KEY,
-)
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import (
     TraceSpan,
@@ -34,7 +27,9 @@ from artemis.data_engine.trace import (
     trace_langchain_tool,
 )
 from artemis.graph.state import State
-from artemis.services.llm import get_llm, invoke_llm_with_timeout_message
+from artemis.graph.visibility import strict_state
+from artemis.mcp.action_specs import OPERATOR_SHELL_ORDER, operator_shell_tool
+from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
 from artemis.tools.command_tool import (
     _BACKGROUND_TASKS,
     _FINISHED_TASKS_LOGS,
@@ -51,19 +46,12 @@ from artemis.utils.coordinates import (
     parse_swipe_parameters,
 )
 from artemis.utils.decorators import wrap_with_callbacks
-from artemis.utils.file import create_snapshot
 from artemis.utils.logger import get_logger
-from artemis.utils.notes import get_note_file_path, get_notes_dir
+from artemis.utils.notes import get_note_file_path
 from artemis.utils.task_tree import (
     build_plan_and_history,
     get_active_subgoal_hashes,
 )
-from artemis.utils.verification import (
-    append_verification_chat,
-    get_verification_chat_path,
-    get_verification_chat_rounds,
-    read_verification_chat,
-)
 from artemis.utils.visualization import format_minimal_list_with_elements
 
 logger = get_logger(__name__)
@@ -86,7 +74,8 @@ from artemis.agents.operator.prompts import (
     TemplatePromptComponent,
     ObservationPromptComponent,
     ScreenshotSimilarityPromptComponent,
-    CheckerFeedbackPromptComponent,
+    FeedbackPromptComponent,
+    CheckItemsExplainerPromptComponent,
     BackgroundTasksPromptComponent,
     ShortTermMemoryPromptComponent,
     TaskPlanWarningPromptComponent,
@@ -169,56 +158,6 @@ class OperatorNode:
 
         return steps, task_plan
 
-    def _get_verification_chat_rounds(self) -> tuple[int, int]:
-        if not self.ctx.data_engine:
-            return 0, 0
-
-        subgoal_hash = self._get_active_subgoal_hash()
-        chat_path = get_verification_chat_path(self.ctx.data_engine.base_dir, subgoal_hash)
-        turns = read_verification_chat(chat_path)
-        return get_verification_chat_rounds(turns)
-
-    def _has_checker_feedback(self) -> bool:
-        if not self.ctx.data_engine:
-            return False
-
-        subgoal_hash = self._get_active_subgoal_hash()
-        chat_path = get_verification_chat_path(self.ctx.data_engine.base_dir, subgoal_hash)
-        return chat_path.exists()
-
-    def _get_reply_to_checker_tool(self) -> BaseTool:
-        @tool
-        def reply_to_checker(
-            reasoning: Annotated[
-                str,
-                "Your reasoning or observations regarding the verification feedback.",
-            ],
-        ):
-            """[TERMINAL] Use this tool to reply to the verification feedback with your reasoning or observations.
-
-            This ends your turn.
-            """
-
-            if self.ctx.data_engine:
-                subgoal_hash = self._get_active_subgoal_hash()
-                chat_path = get_verification_chat_path(self.ctx.data_engine.base_dir, subgoal_hash)
-
-                turns = read_verification_chat(chat_path)
-                max_op, max_chk = get_verification_chat_rounds(turns)
-
-                round_num = max(max_op + 1, max_chk)
-                if round_num == 0:
-                    round_num = 1
-
-                if append_verification_chat(chat_path, "operator", reasoning, round_num):
-                    return "Successfully recorded your reply to the Checker."
-                else:
-                    return "Failed to save reply."
-            else:
-                return "Error: DataEngine not available."
-
-        return reply_to_checker
-
     async def _build_prompt(
         self,
         state: State,
@@ -235,39 +174,26 @@ class OperatorNode:
 
         builder = PromptBuilder()
 
-        has_feedback = self._has_checker_feedback()
-
+        # One constant component list: the prompt template is never switched by
+        # verification results. Check-related components are append-only and
+        # render nothing when there is nothing to say.
         components = self.prompt_components
         if not components:
-            if has_feedback:
-                components = [
-                    (
-                        TemplatePromptComponent(),
-                        {"template_name": "troubleshooter_template"},
-                    ),
-                    (ObservationPromptComponent(), {}),
-                    (ScreenshotSimilarityPromptComponent(), {}),
-                    (InjectedInstructionPromptComponent(), {}),
-                    (ShortTermMemoryPromptComponent(), {}),
-                    (BackgroundTasksPromptComponent(), {}),
-                    (CheckerFeedbackPromptComponent(), {}),
-                    (TaskPlanWarningPromptComponent(), {}),
-                    (ToolLimitWarningPromptComponent(), {}),
-                ]
-            else:
-                components = [
-                    (
-                        TemplatePromptComponent(),
-                        {"template_name": "main_template"},
-                    ),
-                    (ObservationPromptComponent(), {}),
-                    (ScreenshotSimilarityPromptComponent(), {}),
-                    (InjectedInstructionPromptComponent(), {}),
-                    (ShortTermMemoryPromptComponent(), {}),
-                    (BackgroundTasksPromptComponent(), {}),
-                    (TaskPlanWarningPromptComponent(), {}),
-                    (ToolLimitWarningPromptComponent(), {}),
-                ]
+            components = [
+                (
+                    TemplatePromptComponent(),
+                    {"template_name": "main_template"},
+                ),
+                (CheckItemsExplainerPromptComponent(), {}),
+                (ObservationPromptComponent(), {}),
+                (ScreenshotSimilarityPromptComponent(), {}),
+                (InjectedInstructionPromptComponent(), {}),
+                (ShortTermMemoryPromptComponent(), {}),
+                (BackgroundTasksPromptComponent(), {}),
+                (FeedbackPromptComponent(), {}),
+                (TaskPlanWarningPromptComponent(), {}),
+                (ToolLimitWarningPromptComponent(), {}),
+            ]
         else:
             components = [(c, {}) for c in components] + [(ToolLimitWarningPromptComponent(), {})]
 
@@ -298,31 +224,13 @@ class OperatorNode:
         traced_tools: list,
         new_subagent_calls: list,
         state: State,
-    ) -> tuple[list[dict] | None, bool, str | None, str | None, bool]:
+    ) -> tuple[list[dict] | None, str | None, str | None, bool]:
         max_iterations = 20
         action_result = None
-        requested_argue = False
         raw_thoughts = []
         native_thoughts = []
         tool_limit_exceeded = False
 
-        async def read_stream_and_accumulate(target_llm, msgs):
-            try:
-                full_response = None
-                has_chunks = False
-                async for chunk in target_llm.astream(msgs):
-                    has_chunks = True
-                    if full_response is None:
-                        full_response = chunk
-                    else:
-                        full_response += chunk
-                if has_chunks and full_response is not None:
-                    return full_response
-            except Exception as e:
-                logger.debug(f"astream failed or not supported: {e}. Falling back to ainvoke.")
-
-            return await target_llm.ainvoke(msgs)
-
         for iteration in range(max_iterations):
             if iteration == max_iterations - 1:
                 logger.warning(
@@ -340,9 +248,7 @@ class OperatorNode:
                 )
 
             bound_llm = base_llm.bind_tools(tools=traced_tools)
-            response = await invoke_llm_with_timeout_message(
-                read_stream_and_accumulate(bound_llm, current_messages)
-            )
+            response = await invoke_llm_with_timeout_message(acomplete(bound_llm, current_messages))
 
             if hasattr(response, "response_metadata") and response.response_metadata:
                 usage = (
@@ -420,23 +326,19 @@ class OperatorNode:
             action_calls = [
                 tc for tc in response.tool_calls if normalize_name(tc["name"]) in action_tool_names
             ]
-            reply_to_checker_calls = [
-                tc for tc in response.tool_calls if normalize_name(tc["name"]) == "reply_to_checker"
-            ]
             other_calls = [
                 tc
                 for tc in response.tool_calls
-                if normalize_name(tc["name"]) not in action_tool_names + ["reply_to_checker"]
+                if normalize_name(tc["name"]) not in action_tool_names
             ]
 
             tool_outputs = []
             validation_errors = False
-            requested_argue = False
 
             other_tool_failed = False
             other_tool_failure_msg = ""
 
-            has_terminal_calls = bool(action_calls or reply_to_checker_calls)
+            has_terminal_calls = bool(action_calls)
 
             if other_calls:
                 results = []
@@ -478,12 +380,8 @@ class OperatorNode:
                                 content = get_tool_result_content(result_obj)
                                 status = "success"
 
-                                if isinstance(result_obj, Command):
-                                    updates = result_obj.update
-                                    if VALIDATOR_MESSAGES_KEY in updates:
-                                        msgs = updates[VALIDATOR_MESSAGES_KEY]
-                                        if msgs and hasattr(msgs[0], "status"):
-                                            status = msgs[0].status
+                                if isinstance(result_obj, ToolMessage) and result_obj.status:
+                                    status = result_obj.status
 
                                 is_err = status == "error"
                                 if not is_err:
@@ -567,29 +465,6 @@ class OperatorNode:
                             )
                         action_calls = []
 
-                    if reply_to_checker_calls:
-                        logger.warning(
-                            "Scratchpad/helper tool call failed. Deferring"
-                            f" reply to checker: {other_tool_failure_msg}"
-                        )
-                        for tc in reply_to_checker_calls:
-                            tool_outputs.append(
-                                ToolMessage(
-                                    tool_call_id=tc["id"],
-                                    content=(
-                                        "Your reply to the checker was"
-                                        " rejected because an accompanying"
-                                        " tool call failed. Error:"
-                                        f" {other_tool_failure_msg}. Currently,"
-                                        " no response has been sent to the"
-                                        " checker. Please correct the error"
-                                        " and try again."
-                                    ),
-                                    status="error",
-                                )
-                            )
-                        reply_to_checker_calls = []
-
                 elif has_terminal_calls:
                     has_deferring_calls = any(
                         normalize_name(tc["name"]) in DEFERRING_TOOLS for tc in other_calls
@@ -623,28 +498,7 @@ class OperatorNode:
                                 )
                             )
 
-                        for tc in reply_to_checker_calls:
-                            tool_outputs.append(
-                                ToolMessage(
-                                    tool_call_id=tc["id"],
-                                    content=(
-                                        "Your pre-decision tools have been"
-                                        " successfully processed. However, your"
-                                        " response to the checker was rejected"
-                                        " because you are not allowed to"
-                                        " simultaneously use result-dependent"
-                                        " pre-decision tools and reply to the"
-                                        " checker. Currently, no response has been"
-                                        " sent to the checker. Please review"
-                                        " your updated context and re-output"
-                                        " your reply."
-                                    ),
-                                    status="success",
-                                )
-                            )
-
                         action_calls = []
-                        reply_to_checker_calls = []
 
             if action_calls:
                 logger.info(
@@ -676,32 +530,10 @@ class OperatorNode:
                 if not validation_errors:
                     action_result = actions_list
 
-            if reply_to_checker_calls:
-                logger.info("Operator replied to checker.")
-                tc = reply_to_checker_calls[0]
-                requested_argue = True
-
-                # Execute it immediately
-                result = await self._run_other_tool(
-                    tc,
-                    traced_tools,
-                    new_subagent_calls,
-                    state,
-                    raw_thoughts,
-                    native_thoughts,
-                )
-                tool_outputs.append(
-                    ToolMessage(
-                        tool_call_id=tc["id"],
-                        content=result,
-                        status="success" if not result.startswith("Error") else "error",
-                    )
-                )
-
             for tm in tool_outputs:
                 current_messages.append(tm)
 
-            if (action_calls and not validation_errors) or requested_argue:
+            if action_calls and not validation_errors:
                 break
 
             if validation_errors:
@@ -714,7 +546,6 @@ class OperatorNode:
         native_thinking = "\n".join(native_thoughts) if native_thoughts else None
         return (
             action_result,
-            requested_argue,
             raw_thinking,
             native_thinking,
             tool_limit_exceeded,
@@ -765,18 +596,10 @@ class OperatorNode:
     )
     @trace(type="agent", name="operator")
     async def __call__(self, state: State):
+        state = strict_state(state, "operator")
         # 1. Check for infinite loops
         self._check_infinite_loop(state)
 
-        # Create snapshot for optimistic execution
-        if self.ctx.data_engine:
-            notes_dir = get_notes_dir(self.ctx.data_engine.base_dir)
-            if notes_dir.exists():
-                snapshot_dir = notes_dir.with_name("notes_snapshot")
-                await asyncio.to_thread(create_snapshot, notes_dir, snapshot_dir)
-                self.ctx.task_plan_snapshot = snapshot_dir
-                logger.info(f"Created snapshot of notes at {snapshot_dir}")
-
         # 2. Get perception data from state (populated by Perception node)
         operator_raw_data = getattr(state, "operator_raw_data", {}) or {}
         latest_screenshot_b64 = operator_raw_data.get("screenshot_b64")
@@ -822,45 +645,16 @@ class OperatorNode:
 
         current_step_num = len(steps) + 1
 
-        # 5. Prepare Tools. Device-action declarations are assembled against the
-        # installed actuator backend's capabilities: an action the backend does not
-        # implement is simply never declared (and the prompt assembly drops its
-        # teaching segments in lockstep).
+        # 5. Prepare Tools. Device-action shells come from the canonical manifest
+        # (artemis/mcp/action_specs.py) and are assembled against the installed
+        # actuator backend's capabilities: an action the backend does not implement
+        # is simply never declared (and the prompt assembly drops its teaching
+        # segments in lockstep).
         available_actions = self._available_device_actions()
-        action_tool_factories = {
-            "click": self._get_click_tool,
-            "input_text": self._get_input_text_tool,
-            "swipe": self._get_swipe_tool,
-            "press_key": self._get_press_key_tool,
-            "manage_app": self._get_manage_app_tool,
-            "wait_for_delay": self._get_wait_for_delay_tool,
-            "long_press": self._get_long_press_tool,
-        }
         all_tools = [
-            factory()
-            for name, factory in action_tool_factories.items()
-            if name in available_actions
+            operator_shell_tool(name) for name in OPERATOR_SHELL_ORDER if name in available_actions
         ] + self.tools
 
-        if self._has_checker_feedback():
-            max_op, max_chk = self._get_verification_chat_rounds()
-            max_rounds = (
-                getattr(
-                    getattr(self.ctx, "execution_setup", None),
-                    "checker_max_chat_rounds",
-                    OPERATOR_MAX_CHAT_ROUNDS,
-                )
-                if self.ctx and getattr(self.ctx, "execution_setup", None)
-                else OPERATOR_MAX_CHAT_ROUNDS
-            )
-            if max_op < max_rounds:
-                reply_to_checker_tool = self._get_reply_to_checker_tool()
-                all_tools.append(reply_to_checker_tool)
-            else:
-                logger.info(
-                    f"Maximum chat rounds ({max_rounds}) reached. Removing reply_to_checker tool."
-                )
-
         # 5. Evaluate dynamic tools and prepare background tasks
         has_long_output = False
         for tinfo in _FINISHED_TASKS_LOGS.values():
@@ -924,7 +718,6 @@ class OperatorNode:
         new_subagent_calls = []
         (
             action_result,
-            requested_argue,
             raw_thinking,
             native_thinking,
             tool_limit_exceeded,
@@ -955,22 +748,17 @@ class OperatorNode:
             if stm_match:
                 short_term_memory = stm_match.group(1).strip()
 
-        return await state.asanitize_update(
-            ctx=self.ctx,
-            update={
-                "structured_decisions": structured_decisions,
-                "operator_raw_thinking": raw_thinking,
-                "operator_native_thinking": native_thinking,
-                "short_term_memory": short_term_memory,
-                "indexed_points": state.indexed_points,
-                "complete_subgoals_by_ids": [],
-                "current_step_id": state.current_step_id,
-                "subagent_calls": ((state.subagent_calls or []) + new_subagent_calls),
-                "operator_replied": requested_argue,
-                "operator_tool_limit_exceeded": tool_limit_exceeded,
-            },
-            agent="operator",
-        )
+        return {
+            "structured_decisions": structured_decisions,
+            "operator_raw_thinking": raw_thinking,
+            "operator_native_thinking": native_thinking,
+            "short_term_memory": short_term_memory,
+            "indexed_points": state.indexed_points,
+            "indexed_elements": state.indexed_elements,
+            "current_step_id": state.current_step_id,
+            "subagent_calls": ((state.subagent_calls or []) + new_subagent_calls),
+            "operator_tool_limit_exceeded": tool_limit_exceeded,
+        }
 
     def _get_active_subgoal_hash(self) -> str:
         """Parses task_plan.md to find the active top-level subgoal hash."""
@@ -1007,198 +795,6 @@ class OperatorNode:
             chronological_last_step=True,
         )
 
-    def _get_long_press_tool(self) -> BaseTool:
-        @tool
-        def long_press(
-            target: Annotated[
-                int | list[int],
-                "Long press target. Can be an element index number (int, e.g."
-                " 3) OR normalized coordinates (list of 2 integers, e.g. [500,"
-                " 600]).",
-            ],
-            duration: Annotated[int, "Long press duration in milliseconds (default 1000)."] = 1000,
-        ):
-            """[ACTION] Long press on the target location on the screen (supports element index or absolute normalized coordinates)."""
-            return "Action Recorded"
-
-        return long_press
-
-    def _get_click_tool(self) -> BaseTool:
-        @tool
-        def click(
-            target: Annotated[
-                int | list[int],
-                "Click target. Can be an element index number (int, e.g. 3) OR"
-                " normalized coordinates (list of 2 integers, e.g. [500,"
-                " 600]).",
-            ],
-            times: Annotated[
-                int,
-                "Number of consecutive clicks on this target. Use this for"
-                " double-clicks or multi-clicks (e.g. 7 to enter developer"
-                " mode). Default is 1.",
-            ] = 1,
-            delay_ms: Annotated[
-                int,
-                "Delay in milliseconds between consecutive clicks. Default is 100.",
-            ] = 100,
-        ):
-            """[ACTION] Click on the target location on the screen (supports element index or absolute normalized coordinates)."""
-            return "Action Recorded"
-
-        return click
-
-    def _get_input_text_tool(self) -> BaseTool:
-        @tool
-        def input_text(
-            text: Annotated[
-                str,
-                "The text content to input. Supports multi-line content with '\\n'.",
-            ],
-            target: Annotated[
-                int | list[int],
-                "Input target field. Can be an input box element index number"
-                " (int, e.g. 3) OR normalized coordinates (list of 2 integers,"
-                " e.g. [500, 600]).",
-            ],
-            clear_exist: Annotated[
-                bool,
-                "Whether to clear existing text before typing. True (default):"
-                " clear/replace entire text. False: append at the end of"
-                " existing content.",
-            ] = True,
-        ):
-            """[ACTION] Type text into the target input field (supports replacing whole text or appending to the end, and multi-line strings with '\\n')."""
-            return "Action Recorded"
-
-        return input_text
-
-    def _get_swipe_tool(self) -> BaseTool:
-        @tool
-        def swipe(
-            direction: Annotated[
-                Literal["up", "down", "left", "right"] | None,
-                "Direction for scrolling and swiping: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
-                " 'down' (drags top-to-bottom, scrolling up to reveal content above),"
-                " 'left' (drags right-to-left, scrolling right),"
-                " 'right' (drags left-to-right, scrolling left).",
-            ] = None,
-            start: Annotated[
-                list[int] | None,
-                "Start normalized coordinates [start_x, start_y] in 0-1000 scale for precise,"
-                " local interactions (e.g. adjusting sliders, SeekBars, fine range selection, or drag-and-drop).",
-            ] = None,
-            end: Annotated[
-                list[int] | None,
-                "End normalized coordinates [end_x, end_y] in 0-1000 scale for precise,"
-                " local interactions (e.g. adjusting sliders, SeekBars, fine range selection, or drag-and-drop).",
-            ] = None,
-            target: Annotated[
-                int | list[int] | str | None,
-                "Optional target element index (e.g. 2) or container bounds [left, top, right, bottom] to scope the directional swipe within.",
-            ] = None,
-            gesture: Annotated[
-                Literal["up", "down", "left", "right"] | list[int] | None,
-                "Backward-compatible swipe gesture: smart direction string ('up', 'down', 'left', 'right')"
-                " OR precise custom coordinates [start_x, start_y, end_x, end_y] in 0-1000 scale.",
-            ] = None,
-            duration: Annotated[
-                int | None,
-                "Optional swipe/drag duration in milliseconds (default 800). For drag-and-drop,"
-                " list reordering, or sliding/adjusting sliders (e.g., volume, brightness, SeekBars),"
-                " set duration >= 1000 (e.g. 1500). If omitted for directional swipe, duration is computed automatically.",
-            ] = None,
-        ):
-            """[ACTION] Perform a swipe, drag, or slider-adjustment gesture on the screen.
-
-            • Directional Scrolling ('direction'): Recommended for general browsing and standard page scrolling in most scenarios. Automatically computes safe swipe vectors and adaptive duration, retains a ~40% visual overlap anchor for zero-omission traversal, and prevents inertial flings. Supports scoping to a sub-container via 'target'. If it fails on certain custom layouts, fall back to specifying exact coordinates ('start' and 'end') directly.
-            • Precise Coordinate Gestures ('start', 'end'): Best for local, fine-grained interactions such as adjusting sliders/SeekBars (e.g., volume, brightness, progress bars), drag-and-drop / list reordering, or as a reliable fallback when directional scrolling fails on specific containers. Always drag slightly PAST the target position to overcome touch slop and reliably trigger the update. When setting a slider to Maximum (100%) or Minimum (0%), swipe fully to the extreme boundary.
-
-            Args:
-                direction: Smart directional scrolling ('up', 'down', 'left', 'right'). Automatically computes safe swipe vectors, retaining 40% visual overlap: 'up' (reveal content below), 'down' (reveal content above), 'left', 'right'.
-                start: Start normalized coordinates [start_x, start_y] in 0-1000 scale.
-                end: End normalized coordinates [end_x, end_y] in 0-1000 scale.
-                target: Optional target element index (e.g. 2) or container bounds [left, top, right, bottom] to scope the directional swipe within.
-                gesture: Backward-compatible parameter: direction string OR custom coordinates list [start_x, start_y, end_x, end_y] in 0-1000 scale.
-                duration: Optional gesture duration in milliseconds (default 800).
-            """
-            return "Action Recorded"
-
-        return swipe
-
-    def _get_press_key_tool(self) -> BaseTool:
-        @tool
-        def press_key(
-            key: Annotated[
-                Literal["ENTER", "BACK", "HOME", "APP_SWITCH"],
-                "Standard Android system button name (ENTER, BACK, HOME, APP_SWITCH).",
-            ],
-        ):
-            """[ACTION] Press a physical or virtual system button (e.g.
-
-            ENTER, BACK, HOME, APP_SWITCH).
-            """
-            return "Action Recorded"
-
-        return press_key
-
-    def _get_manage_app_tool(self) -> BaseTool:
-        @tool
-        def manage_app(
-            action: Annotated[Literal["launch", "stop"], "The action type."],
-            app_name: Annotated[str, "Display name or package name of the application."],
-        ):
-            """[ACTION] Launch or force stop a specified application."""
-            return "Action Recorded"
-
-        return manage_app
-
-    def _get_wait_for_delay_tool(self) -> BaseTool:
-        @tool
-        def wait_for_delay(
-            time_in_ms: Annotated[
-                int,
-                "The exact duration to wait in milliseconds. Accurately convert the"
-                " required time duration into milliseconds based on your objective"
-                " or plan (e.g., 2000 for 2s, 5000 for 5s, 60000 for 1 minute,"
-                " 180000 for 3 minutes, 300000 for 5 minutes).",
-            ],
-        ):
-            """[ACTION] Pause execution and wait for a specified duration in milliseconds.
-
-            Use this whenever you need time to elapse—whether for UI loading, animations,
-            screen transitions, or longer scheduled delays and intervals specified in the task.
-            """
-            return "Action Recorded"
-
-        return wait_for_delay
-
-    def _get_wait_for_text_tool(self) -> BaseTool:
-        @tool
-        def wait_for_text(
-            text: Annotated[
-                str,
-                "A simple, distinct keyword (e.g., 'Success', 'Done',"
-                " 'Loading') to watch on the screen. Try to pass a clean word"
-                " without trailing punctuation (e.g. 'Loading' instead of"
-                " 'Loading...') for optimal matching reliability.",
-            ],
-            state: Annotated[
-                Literal["visible", "hidden"],
-                "Wait for the keyword to appear ('visible') or disappear"
-                " ('hidden') from the screen.",
-            ] = "visible",
-            timeout_ms: Annotated[int, "Maximum timeout in milliseconds (default 5000)."] = 5000,
-        ):
-            """[ACTION] Pause execution and wait intelligently for a specific, distinct text or keyword to appear or disappear on the screen.
-
-            Use this when waiting for a specific loading screen to finish or a
-            specific success/confirmation state to appear.
-            """
-            return "Action Recorded"
-
-        return wait_for_text
-
     def _translate_and_validate_tool(self, tc: dict, state: State) -> tuple[list[dict], str | None]:
         actions, err = self._translate_and_validate_tool_inner(tc, state)
         if err:
@@ -1384,7 +980,7 @@ class OperatorNode:
                     direction=target,
                     target=args.get("target"),
                     indexed_elements=getattr(state, "indexed_elements", None),
-                    ui_hierarchy=getattr(state, "ui_tree", None),
+                    ui_hierarchy=getattr(state, "latest_ui_hierarchy", None),
                     width=width,
                     height=height,
                     duration=duration,

artemis/agents/operator/prompts.py

@@ -14,10 +14,9 @@
 
 import base64
 import io
-import json
 from pathlib import Path
 
-from jinja2 import Template
+from jinja2 import Environment, StrictUndefined, Template
 from langchain_core.messages import HumanMessage, SystemMessage
 from PIL import Image
 
@@ -28,27 +27,27 @@ from artemis.tools.command_tool import (
     _is_output_long,
 )
 from artemis.utils.logger import get_logger
-from artemis.utils.task_tree import get_active_subgoal_hashes
+from artemis.utils.notes import get_note_file_path
+from artemis.utils.plan_grammar import parse_plan, render_plan_grammar_spec
 
 logger = get_logger(__name__)
 
 
-from artemis.agents.prompt_assembly import (
-    gate_segment,
-    render_tool_enum,
-    resolve_available,
-)
-
-_LEGACY_TOOL_CALLING_RULE = """- **Tool Calling Iron Rule**: Do NOT submit a Turn-Ending Action at the same time (in the same tool-call list) as a Pre-Decision Exploratory Tool. You must first gather information (e.g., finding coordinates or diagnosing a state), manage ADB background tasks, or update memory notes. You can use pre-decision exploratory tools to complete this task. You will be prompted again with the tool's result, at which point you can output your final physical/turn-ending action."""
-
-_LEGACY_TOOL_PROTOCOL = """1. **Pre-Decision Exploratory Tools**:
-   - *What they are*: Helper/Subagent tools (such as `ask_explorer`, `ask_diagnoser`, `video_analyzer`), ADB command tools (`run_adb_command`, `manage_task`), and memory note tools (`read_note`, `list_notes`, `save_note`, `update_note`, `append_note`).
-   - *How they work*: These tools gather details or update/read memory. When you invoke these tools, the framework will immediately execute them and return the results, allowing you to continue thinking and make your final decision.
-2. **Turn-Ending Actions**:"""
+from artemis.agents.prompt_assembly import render_tool_enum, resolve_available
+from artemis.mcp.action_specs import OPERATOR_SHELL_ORDER
 
 # --- Assembled tool references -------------------------------------------------------
+# The operator.json templates carry availability slots rendered by
+# ``apply_operator_prompt_contract``: ``[[ tool_enum(...) ]]`` expands an ordered,
+# backticked enumeration limited to the available tool set, and
+# ``[% if "x" in available_tools %]...[% endif %]`` gates a self-contained teaching
+# segment. An unavailable tool therefore leaves no trace in the prompt at all. The
+# slot delimiters are square-bracketed so the standard ``{{ ... }}`` context
+# variables (initial_goal, plan_and_history, plan_grammar, ...) pass through
+# untouched for the later context render.
+#
 # The orderings below reproduce the historical prompt wording exactly; with the full
-# tool set the rendered output is byte-identical to the previous literals. See
+# tool set the rendered output is byte-identical to the pre-assembly literals. See
 # artemis/agents/prompt_assembly.py for the assembly rationale.
 
 _PRE_DECISION_HELPER_TOOLS = ("ask_explorer", "ask_diagnoser", "video_analyzer")
@@ -59,16 +58,9 @@ _PRE_DECISION_ALL_TOOLS = (
 )
 
 # The two device-action enumeration slots in operator.json (identical in both
-# templates), in their historical orders.
-_PHYSICAL_ACTIONS_ORDER = (
-    "click",
-    "input_text",
-    "swipe",
-    "press_key",
-    "manage_app",
-    "wait_for_delay",
-    "long_press",
-)
+# templates), in their historical orders. The physical order is the canonical
+# manifest's shell-binding order; the turn-ending order is a prompt property.
+_PHYSICAL_ACTIONS_ORDER = OPERATOR_SHELL_ORDER
 _TURN_ENDING_ORDER = (
     "click",
     "swipe",
@@ -79,120 +71,31 @@ _TURN_ENDING_ORDER = (
     "wait_for_delay",
 )
 
-_PHYSICAL_ACTIONS_SLOT = (
-    "Physical device actions (`click`, `input_text`, `swipe`, `press_key`,"
-    " `manage_app`, `wait_for_delay`, `long_press`)"
-)
-_TURN_ENDING_SLOT = (
-    "Turn-Ending Action (`click`, `swipe`, `input_text`, `long_press`,"
-    " `press_key`, `manage_app`, or `wait_for_delay`)"
-)
-
-# Self-contained instruction segments that teach a single tool. Removed wholesale when
-# the tool is unavailable, so nothing in the prompt dangles.
-_WAIT_FOR_DELAY_TRANSITIONS_SEGMENT = (
-    "- **Loading & Transitions**: When entering page transitions (e.g., app launching,"
-    " web page loading, or progress indicators), use `wait_for_delay` to let the screen"
-    " settle before executing target actions. Some transition states are difficult to"
-    " discern, such as a thin progress line at the top of the browser screen during"
-    " webpage loads. Before questioning whether an error has occurred, you can first"
-    " adopt a waiting strategy to see how the device state evolves. If the state"
-    " remains ambiguous after waiting, certain UI components might be malfunctioning or"
-    " failing to load; in this case, consider retrying the action or pivoting to an"
-    " alternative path.\n"
-)
-_WAIT_FOR_DELAY_INTERVAL_PARENTHETICAL = (
-    " (convert minutes to milliseconds accurately for `wait_for_delay`)"
-)
-_VIDEO_ANALYZER_TRANSIENT_SEGMENT = (
-    "   - *Missed Transient States*: If the current state jumped ahead and you suspect"
-    " you missed an important initial or intermediate screen (e.g., a number or flash"
-    " message that was overwritten by your last action), consider using the"
-    " `video_analyzer` tool to look back at the recent history.\n"
-)
-_VIDEO_ANALYZER_SNAPSHOT_SENTENCE = (
-    " If you have already missed certain critical information and cannot re-view it by"
-    " going back or other operations, you can use the `video_analyzer` to look back at"
-    " historical information."
-)
-# Tail of the "Action Scope & Interaction Precision" bullet: everything past the two
-# generic gesture-scope sentences is swipe/drag teaching.
-_SWIPE_PRECISION_SEGMENT = (
-    " For any task requiring continuous range interactions, sliding, or boundary"
-    " selection, prefer using targeted swipe/drag gestures spanning from the element's"
-    " start bounds [xmin, ymin] to end bounds [xmax, ymax] to ensure the action is"
-    " strictly confined to the target element. For drag-and-drop, list reordering, or"
-    " sliding sliders/SeekBars, set duration >= 1000 (e.g. 1500) and always drag"
-    " slightly PAST the target position to overcome touch slop and reliably trigger the"
-    " drop/snap/value update. When setting a slider to Maximum (100%) or Minimum (0%),"
-    " always swipe fully to the extreme boundary to guarantee reaching the absolute"
-    " limits."
-)
-
 #: Every tool name the operator prompt slots can reference. ``available_tools=None``
 #: resolves to this set, preserving the historical output.
 OPERATOR_PROMPT_TOOLSET: frozenset[str] = frozenset(
     _PRE_DECISION_ALL_TOOLS + _PHYSICAL_ACTIONS_ORDER
 )
 
-
-def _tool_calling_rule(available: frozenset[str]) -> str:
-    tools = render_tool_enum(_PRE_DECISION_ALL_TOOLS, available, final_sep="and")
-    return (
-        "- **Tool Calling Contract**: Tools that return information needed for the"
-        f" decision ({tools}) are result-dependent Pre-Decision Tools. Do not submit"
-        " them in the same tool-call list as a Turn-Ending Action; inspect their result"
-        " first. The write-through tools `update_note` and `append_note` may accompany"
-        " at most one Turn-Ending Action when their content is based entirely on"
-        " evidence already available before that action. Never write an action's"
-        " expected outcome as if it had already been observed."
-    )
-
-
-def _tool_protocol(available: frozenset[str]) -> str:
-    helper = render_tool_enum(_PRE_DECISION_HELPER_TOOLS, available)
-    adb = render_tool_enum(_PRE_DECISION_ADB_TOOLS, available)
-    memory = render_tool_enum(_PRE_DECISION_MEMORY_TOOLS, available)
-    return (
-        "1. **Result-Dependent Pre-Decision Tools**:\n"
-        f"   - *What they are*: Helper/Subagent tools ({helper}), ADB/task tools"
-        f" ({adb}), and memory tools whose result must be inspected ({memory}).\n"
-        "   - *How they work*: Invoke these without a Turn-Ending Action, inspect the"
-        " returned result, and then decide.\n"
-        "2. **Write-Through Memory Tools**:\n"
-        "   - `update_note` and `append_note` may run alongside at most one Turn-Ending"
-        " Action only when recording facts already observed in the current context. If"
-        " the note content depends on the action's result, wait for the next"
-        " observation before writing it.\n"
-        "3. **Turn-Ending Actions**:"
-    )
-
-_LEGACY_CHECKER_REJECTION_TRIGGER = """   - **Validation/Checker Rejection**: The verification agent (Checker) rejected your subgoal completion (i.e., you are in troubleshooter mode and this is a retry)."""
-
-_AMBIGUOUS_CHECKER_REJECTION_TRIGGER = """   - **Ambiguous Validation/Checker Rejection**: Use `ask_diagnoser` only when the rejection cause is unclear, the evidence conflicts, or no safe local correction is evident. A Checker rejection alone is not a mandatory diagnosis trigger."""
-
-_LEGACY_TROUBLESHOOTER_CHALLENGE = """   - *Challenge Checker*: If you visually confirm the subgoal's target state is *already achieved*, you **MUST** invoke the `reply_to_checker` tool to state your observation."""
-
-_TROUBLESHOOTER_CHALLENGE = """   - *Challenge Checker*: If current evidence clearly proves the subgoal is already achieved, invoke `reply_to_checker` directly. This branch takes precedence over diagnosis; do not call `ask_diagnoser` first.
-   - *Clear Local Correction*: If the rejection cause is explicit and a safe correction is evident from the current screen, perform that correction directly. This branch also takes precedence over generic diagnosis triggers. Use `ask_diagnoser` only for ambiguity, conflicting evidence, or repeated failure."""
-
-_LEGACY_LARGE_LIST_STRATEGY = """- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:
-  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor. Calibrate your scroll distance so that this anchor remains visible near the top of the subsequent screen, creating a seamless visual overlap that guarantees zero missed items. Terminate exploration when a definitive boundary is reached (e.g., list bottom reached or no new content appears after scrolling).
-  2. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.
-  3. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for "double-checking". Only perform incremental re-verification if the list has been refreshed or content has mutated, while maintaining strict single-pass handling of new events."""
-
-_LARGE_LIST_STRATEGY = """- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:
-  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.
-  2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading state. This boundary probe must remain unidirectional; never reverse direction merely to prove completion. If the task declares an earlier exit condition (for example, find one verified match or process N items), that explicit condition may end the traversal without reaching the list boundary.
-  3. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.
-  4. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed with confirmed anchor continuity and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for "double-checking", even if older execution history has been pruned. Resume from the last recorded anchor and direction. Only the single bounded recovery above, or a refreshed/mutated list, permits limited re-verification; otherwise continue the single pass over new content."""
+#: Renders the availability slots in operator.json. Square-bracket delimiters keep
+#: the standard ``{{ ... }}`` context placeholders inert during this phase.
+_CONTRACT_ENV = Environment(
+    block_start_string="[%",
+    block_end_string="%]",
+    variable_start_string="[[",
+    variable_end_string="]]",
+    comment_start_string="[#",
+    comment_end_string="#]",
+    keep_trailing_newline=True,
+    undefined=StrictUndefined,
+)
 
 
 def apply_operator_prompt_contract(
     prompt_template: str,
     available_tools: frozenset[str] | None = None,
 ) -> str:
-    """Align prompt-level tool and recovery rules with Operator runtime semantics.
+    """Renders a template's availability slots against the actually-available tools.
 
     Args:
         prompt_template: One of the raw operator.json templates.
@@ -203,65 +106,19 @@ def apply_operator_prompt_contract(
     """
     available = frozenset(resolve_available(available_tools, OPERATOR_PROMPT_TOOLSET))
 
-    prompt_template = prompt_template.replace(
-        _LEGACY_TOOL_CALLING_RULE,
-        _tool_calling_rule(available),
-    )
-    prompt_template = prompt_template.replace(
-        _LEGACY_TOOL_PROTOCOL,
-        _tool_protocol(available),
-    )
-    prompt_template = prompt_template.replace(
-        _LEGACY_CHECKER_REJECTION_TRIGGER,
-        _AMBIGUOUS_CHECKER_REJECTION_TRIGGER,
-    )
-    prompt_template = prompt_template.replace(
-        _LEGACY_TROUBLESHOOTER_CHALLENGE,
-        _TROUBLESHOOTER_CHALLENGE,
-    )
-    prompt_template = prompt_template.replace(
-        _LEGACY_LARGE_LIST_STRATEGY,
-        _LARGE_LIST_STRATEGY,
-    )
+    def tool_enum(names, final_sep: str | None = None) -> str:
+        return render_tool_enum(tuple(names), available, final_sep=final_sep)
 
-    # Device-action enumeration slots (identical in both templates).
-    prompt_template = prompt_template.replace(
-        _PHYSICAL_ACTIONS_SLOT,
-        "Physical device actions ("
-        + render_tool_enum(_PHYSICAL_ACTIONS_ORDER, available)
-        + ")",
+    return _CONTRACT_ENV.from_string(prompt_template).render(
+        available_tools=available,
+        tool_enum=tool_enum,
+        pre_decision_tools=_PRE_DECISION_ALL_TOOLS,
+        helper_tools=_PRE_DECISION_HELPER_TOOLS,
+        adb_tools=_PRE_DECISION_ADB_TOOLS,
+        memory_tools=_PRE_DECISION_MEMORY_TOOLS,
+        physical_actions=_PHYSICAL_ACTIONS_ORDER,
+        turn_ending_actions=_TURN_ENDING_ORDER,
     )
-    prompt_template = prompt_template.replace(
-        _TURN_ENDING_SLOT,
-        "Turn-Ending Action ("
-        + render_tool_enum(_TURN_ENDING_ORDER, available, final_sep="or")
-        + ")",
-    )
-
-    # Instruction segments that teach a single tool.
-    prompt_template = prompt_template.replace(
-        _WAIT_FOR_DELAY_TRANSITIONS_SEGMENT,
-        gate_segment(_WAIT_FOR_DELAY_TRANSITIONS_SEGMENT, available, "wait_for_delay"),
-    )
-    prompt_template = prompt_template.replace(
-        _WAIT_FOR_DELAY_INTERVAL_PARENTHETICAL,
-        gate_segment(
-            _WAIT_FOR_DELAY_INTERVAL_PARENTHETICAL, available, "wait_for_delay"
-        ),
-    )
-    prompt_template = prompt_template.replace(
-        _VIDEO_ANALYZER_TRANSIENT_SEGMENT,
-        gate_segment(_VIDEO_ANALYZER_TRANSIENT_SEGMENT, available, "video_analyzer"),
-    )
-    prompt_template = prompt_template.replace(
-        _VIDEO_ANALYZER_SNAPSHOT_SENTENCE,
-        gate_segment(_VIDEO_ANALYZER_SNAPSHOT_SENTENCE, available, "video_analyzer"),
-    )
-    prompt_template = prompt_template.replace(
-        _SWIPE_PRECISION_SEGMENT,
-        gate_segment(_SWIPE_PRECISION_SEGMENT, available, "swipe"),
-    )
-    return prompt_template
 
 
 class PromptBuilder:
@@ -340,11 +197,22 @@ class TemplatePromptComponent(PromptComponent):
 
         plan_and_history = kwargs.get("plan_and_history", "No plan or history yet.")
 
+        # Grammar spec assembly is a function of configuration: with both check
+        # gates disabled, the check-line grammar never enters any prompt.
+        include_checks = bool(setup and getattr(setup, "checks_enabled", False))
+        # The rejection/finding diagnosis trigger only exists while a mechanism
+        # that can produce rejections or findings is active.
+        verification_active = include_checks or bool(
+            setup and not getattr(setup, "disable_planner_validation", True)
+        )
+
         full_prompt = Template(prompt_template).render(
             initial_goal=state.initial_goal,
             subgoals_status="",
             plan_and_history=plan_and_history,
             unified_history="",
+            plan_grammar=render_plan_grammar_spec(include_checks),
+            verification_active=verification_active,
         )
 
         parts = full_prompt.split("# CURRENT OBSERVATION")
@@ -369,43 +237,64 @@ class ObservationPromptComponent(PromptComponent):
         builder.add_human_content(f"--- Visible UI Elements ---\n{minimal_list}")
 
 
-class CheckerFeedbackPromptComponent(PromptComponent):
+class FeedbackPromptComponent(PromptComponent):
+    """Append-only injection of source-tagged findings.
+
+    Reads ``state.operator_feedback`` (written by ``execution_check_node`` at
+    harvest / planner rejection and by exit settlement on a bounce-back).
+    Renders nothing when there are no findings — the prompt template itself
+    is never switched.
+    """
+
     async def __call__(self, builder: PromptBuilder, state: State, ctx: ArtemisContext, **kwargs):
-        if not ctx.data_engine:
+        findings = getattr(state, "operator_feedback", None)
+        if not findings:
             return
+        lines = "\n".join(f"- {f}" for f in findings)
+        builder.add_human_content(
+            "--- Verification Findings ---\n"
+            f"{lines}\n"
+            "Each finding is tagged with its source. Checker verdicts"
+            " ([verify failed], [final check]) are independent judgments —"
+            " address any reverted subgoal accordingly; do not re-litigate"
+            " them. [planner] findings explain why a plan change was rejected"
+            " and rolled back — factor the reason into your next strategy."
+        )
 
-        notes_dir = Path(ctx.data_engine.base_dir) / "notes"
 
-        subgoal_hash = "default"
-        task_plan_path = notes_dir / "task_plan.md"
-        if task_plan_path.exists():
-            try:
-                content = task_plan_path.read_text(encoding="utf-8")
+class CheckItemsExplainerPromptComponent(PromptComponent):
+    """Behavioral guidance for check lines, rendered iff the CURRENT plan
+    actually contains check lines (content-driven, not switch-driven: a resumed
+    plan carrying check lines still gets the explanation)."""
 
-                parent_hash, _ = get_active_subgoal_hashes(content)
-                subgoal_hash = parent_hash
-            except Exception as e:
-                logger.error(f"Failed to parse active subgoal in component: {e}")
-
-        verification_chat_path = notes_dir / f"verification_chat_{subgoal_hash}.json"
-        turns = []
-        if verification_chat_path.exists():
-            try:
-                turns = json.loads(verification_chat_path.read_text(encoding="utf-8"))
-                logger.info(f"Read verification chat for {subgoal_hash}: {len(turns)} turns")
-            except Exception as e:
-                logger.error(f"Error reading verification chat: {e}")
-
-        if turns:
-            dialogue_lines = []
-            for t in turns:
-                role = "Operator" if t["role"] == "operator" else "Checker"
-                dialogue_lines.append(f"**{role} (Round {t['round']})**:\n{t['content']}")
-            checker_feedback = "\n\n".join(dialogue_lines)
-
-            feedback_prompt = f"--- Checker Feedback ---\n{checker_feedback}"
-
-            builder.add_human_content(feedback_prompt)
+    async def __call__(self, builder: PromptBuilder, state: State, ctx: ArtemisContext, **kwargs):
+        if not ctx or not getattr(ctx, "data_engine", None):
+            return
+        try:
+            task_plan_path = get_note_file_path(ctx.data_engine.base_dir, "task_plan")
+            if not task_plan_path.exists():
+                return
+            snapshot = parse_plan(task_plan_path.read_text(encoding="utf-8"))
+        except Exception as e:
+            logger.error(f"Failed to parse plan for check-items explainer: {e}")
+            return
+        if not snapshot.all_check_items:
+            return
+        builder.add_human_content(
+            "--- About the plan's check lines ---\n"
+            "The task plan declares `- verify:` / `- assert:` check lines."
+            " `verify:` lines are the acceptance criteria for their subgoal — use"
+            " them to confirm your work is complete, but the judgment is made by"
+            " an independent Checker: never declare a check passed yourself or"
+            " record conclusions on its behalf. `assert:` lines are test"
+            " assertions — take NO extra actions for them and never construct or"
+            " fake state to satisfy one; a failing assertion is a legitimate test"
+            " result. Check lines must not be deleted or reworded (deletions are"
+            " automatically restored by the system); keep them verbatim when"
+            " rewriting the plan — adding new ones is allowed. Simply mark"
+            " completions per the plan grammar as usual; checking runs"
+            " asynchronously in the background and does not block you."
+        )
 
 
 class BackgroundTasksPromptComponent(PromptComponent):

artemis/agents/outputter/outputter.py

@@ -26,6 +26,7 @@ from artemis.config import OutputConfig
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import trace
 from artemis.graph.state import State
+from artemis.llm.structured import ParseFailure, parse_structured
 from artemis.services.llm import get_llm, invoke_llm_with_timeout_message, with_fallback
 from artemis.tools.scratchpad import (
     get_append_note_tool_pure,
@@ -141,6 +142,47 @@ async def outputter(
     )
 
     human_message_content = [{"type": "text", "text": human_message}]
+
+    # Verification material (only present when the run declared check items):
+    # the full append-only verdict sequence plus the four-state test summary.
+    if ctx.data_engine:
+        try:
+            from artemis.graph.checkpoints import read_ledger, read_run_outcome
+
+            ledger_records = read_ledger(ctx.data_engine.base_dir)
+            run_outcome = read_run_outcome(ctx.data_engine.base_dir) or getattr(
+                graph_output, "run_outcome", None
+            )
+            if ledger_records or run_outcome:
+                lines = ["--- Verification Verdict Ledger (append-only) ---"]
+                for r in ledger_records:
+                    lines.append(
+                        f"- attempt {r.get('attempt_id')} [{r.get('kind')}]"
+                        f" '{r.get('item_text')}' -> {r.get('status')}:"
+                        f" {r.get('evidence', '')}"
+                    )
+                if run_outcome:
+                    tests = run_outcome.get("tests") or {}
+                    lines.append(
+                        "--- Test Summary ---\n"
+                        f"task_status={run_outcome.get('task_status')},"
+                        f" passed={tests.get('passed', 0)},"
+                        f" failed={tests.get('failed', 0)},"
+                        f" inconclusive={tests.get('inconclusive', 0)},"
+                        f" unchecked={tests.get('unchecked', 0)}"
+                    )
+                lines.append(
+                    "Report every declared check item with its final state"
+                    " (passed / failed / inconclusive / unchecked) and include"
+                    " the complete verdict sequence above. A failed assertion"
+                    " is a test result to report verbatim, not something to"
+                    " explain away. Items reported unchecked were never"
+                    " judged — never present them as passed."
+                )
+                human_message_content.append({"type": "text", "text": "\n".join(lines)})
+        except Exception as e:
+            logger.error(f"Failed to attach verdict ledger to outputter: {e}")
+
     if screenshot_b64:
         human_message_content.append(
             {
@@ -306,16 +348,15 @@ async def outputter(
     # Fallback to old behavior if it didn't match the template
     if isinstance(raw_answer, str):
         raw_answer_stripped = raw_answer.strip()
-        if raw_answer_stripped.startswith("{") and raw_answer_stripped.endswith("}"):
-            try:
-                return json.loads(raw_answer_stripped)
-            except Exception:
-                pass
-        elif raw_answer_stripped.startswith("```json") and raw_answer_stripped.endswith("```"):
-            try:
-                content = raw_answer_stripped[7:-3].strip()
-                return json.loads(content)
-            except Exception:
-                pass
+        if raw_answer_stripped.startswith(("{", "[", "```")):
+            parsed = parse_structured(raw_answer_stripped)
+            if not isinstance(parsed, ParseFailure):
+                return parsed
+            # The raw text is a legitimate final answer; the miss is only
+            # noteworthy because the answer *looked* like JSON.
+            logger.warning(
+                "Outputter answer looked like JSON but could not be parsed"
+                f" ({parsed.error}); returning it as plain text."
+            )
 
     return raw_answer

artemis/agents/planner/planner.json

@@ -1,12 +1,14 @@
 {
   "blocks": {
     "role": "## Role\nYou are a Senior Test Engineer leading a team of mobile execution agents. Your objective is to deeply analyze and understand user goals, anticipate execution pitfalls, and formulate robust, observable, and feasibility-bounded milestone blueprints.",
-    "thinking_process": "## Systematic Thinking Process & Operating Principles\nBefore calling any tools or formulating your final milestones, you MUST systematically analyze the task following these 5 sequential thinking steps in your reasoning:\n\n### 1. Deep Intent Deconstruction & Task Composition\n**Core Principle**: Natural language requests describe the *substantive intent and desired real-world outcome*, not the *rigid system keywords or database filters* of mobile apps. Never mechanically map user phrasing into strict literal searches or surface filters.\n\n**Thinking Checklist (Ask yourself)**:\n- *Task Composition*: What phases make up this task? Is it a sequence of one-off linear steps (e.g., open app, configure settings), an iterative loop over multiple items (e.g., inspecting a list of search results), or a periodic polling loop (e.g., refreshing to monitor new messages, waiting for battery level)?\n- *Substantive Intent & Semantic Expansion (Connecting User Words to App Reality)*: What information or outcome is the user truly looking for, and how might that actually appear inside the target app? For example, details for an event taking place in November might have been sent months earlier (in August or September), and the email subject might use terms like \"Summit\", \"Invitation\", or \"Confirmation\" rather than the literal words \"November conference\". Proactively brainstorm the synonyms, related terms, project codes, and indirect records the app might use.\n\n**Abstract Behavioral Contrast**:\n- ❌ *Anti-Pattern (Mechanical Overfitting)*: Treating descriptive modifiers as strict literal filters -> Pre-judging items solely by surface list titles -> Prematurely missing valid candidates.\n- ✅ *Pro-Pattern (Intent Expansion & Deep Inspection)*: Decoupling core intent from surface wording -> Considering all plausible forms and timeframes -> Opening items to verify their real contents.\n\n### 2. Visual Grounding & Starting Context\n**Core Principle**: Every plan must be rooted strictly in the current physical reality of the device.\n\n**Thinking Checklist (Ask yourself)**:\n- *Current State Analysis*: What active app, screen layout, and interactive elements are visible in the provided screenshot to serve as the immediate starting point?\n\n### 3. Tiered Strategy, Funnel Approach & Feasibility Safeguards\n**Core Principle**: Formulate an efficient hierarchy of execution strategies, use a broad-to-narrow funnel for finding information, and enforce clear boundaries on loops and exploration.\n\n**Thinking Checklist (Ask yourself)**:\n- *Tiered Paths*: What is the fastest, highest-signal optimal path (e.g., targeted in-app search, direct shortcuts)? If that fails or yields no results, what is the pragmatic fallback path (e.g., browsing categories, chronological scanning)?\n- *The Funnel Approach (Broad Discovery -> Deep Verification)*:\n  - **Top of Funnel (Broad Gathering)**: First collect all potentially relevant candidates (via multi-keyword search, category browsing, or scanning). Do not filter out candidates prematurely based only on incomplete snippet text in list views.\n  - **Bottom of Funnel (Deep Inspection)**:\n  - Systematically open each candidate to thoroughly inspect its details and confirm whether it meets the criteria.\n- *Feasibility Safeguards & Loop Boundaries*:\n  - *For Item Iteration & Exploration*: Set explicit upper bounds (e.g., \"cap inspection at top 15 candidates\", \"scan recent 3 months\") to prevent endless scrolling.\n  - *For Periodic Monitoring / Polling*: Specify the check action (e.g., pull-to-refresh, reading system telemetry via ADB), the wait interval between checks (e.g., check every 5 minutes to conserve resources), and the exit criteria (e.g., stop when condition is met, or continue until manually stopped).\n\n### 4. Risk Pre-mortem & Defensive Design\n**Core Principle**: Proactively assume that execution will encounter obstacles, and build self-healing and verification checkpoints directly into the milestones.\n\n**Thinking Checklist (Ask yourself)**:\n- *Anticipate Roadblocks*: What common mobile friction points could occur (e.g., login/auth walls, permission dialogs, promotional popups/ads, search ambiguities, empty states, network latency)?\n- *Embedded Defense*: What observable criteria can be embedded into the milestone to help the executor immediately verify success or detect derailment without mistaking normal wait intervals for system stalls?\n\n### 5. Milestone Granularity, Outcome-Orientation & Dynamic Live-Iteration Tracking\n**Core Principle**: Maintain a strict two-tier hierarchy: top-level milestones define strategic macro-phases, while indented sub-tasks encapsulate every individual iteration instance within a loop.\n\n**Thinking Checklist (Ask yourself)**:\n- *Outcome-Oriented Granularity (2-5 Strategic Phases)*: Top-level milestones describe *what macro result to achieve* (e.g., \"Search for candidate emails and extract confirmed itiLine truncated
+    "thinking_process": "## Systematic Thinking Process & Operating Principles\nBefore calling any tools or formulating your final milestones, you MUST systematically analyze the task following these 5 sequential thinking steps in your reasoning:\n\n### 1. Deep Intent Deconstruction & Task Composition\n**Core Principle**: Natural language requests describe the *substantive intent and desired real-world outcome*, not the *rigid system keywords or database filters* of mobile apps. Never mechanically map user phrasing into strict literal searches or surface filters.\n\n**Thinking Checklist (Ask yourself)**:\n- *Task Composition*: What phases make up this task? Is it a sequence of one-off linear steps (e.g., open app, configure settings), an iterative loop over multiple items (e.g., inspecting a list of search results), or a periodic polling loop (e.g., refreshing to monitor new messages, waiting for battery level)?\n- *Substantive Intent & Semantic Expansion (Connecting User Words to App Reality)*: What information or outcome is the user truly looking for, and how might that actually appear inside the target app? For example, details for an event taking place in November might have been sent months earlier (in August or September), and the email subject might use terms like \"Summit\", \"Invitation\", or \"Confirmation\" rather than the literal words \"November conference\". Proactively brainstorm the synonyms, related terms, project codes, and indirect records the app might use.\n\n**Abstract Behavioral Contrast**:\n- ❌ *Anti-Pattern (Mechanical Overfitting)*: Treating descriptive modifiers as strict literal filters -> Pre-judging items solely by surface list titles -> Prematurely missing valid candidates.\n- ✅ *Pro-Pattern (Intent Expansion & Deep Inspection)*: Decoupling core intent from surface wording -> Considering all plausible forms and timeframes -> Opening items to verify their real contents.\n\n### 2. Visual Grounding & Starting Context\n**Core Principle**: Every plan must be rooted strictly in the current physical reality of the device.\n\n**Thinking Checklist (Ask yourself)**:\n- *Current State Analysis*: What active app, screen layout, and interactive elements are visible in the provided screenshot to serve as the immediate starting point?\n\n### 3. Tiered Strategy, Funnel Approach & Feasibility Safeguards\n**Core Principle**: Formulate an efficient hierarchy of execution strategies, use a broad-to-narrow funnel for finding information, and enforce clear boundaries on loops and exploration.\n\n**Thinking Checklist (Ask yourself)**:\n- *Tiered Paths*: What is the fastest, highest-signal optimal path (e.g., targeted in-app search, direct shortcuts)? If that fails or yields no results, what is the pragmatic fallback path (e.g., browsing categories, chronological scanning)?\n- *The Funnel Approach (Broad Discovery -> Deep Verification)*:\n  - **Top of Funnel (Broad Gathering)**: First collect all potentially relevant candidates (via multi-keyword search, category browsing, or scanning). Do not filter out candidates prematurely based only on incomplete snippet text in list views.\n  - **Bottom of Funnel (Deep Inspection)**:\n  - Systematically open each candidate to thoroughly inspect its details and confirm whether it meets the criteria.\n- *Feasibility Safeguards & Loop Boundaries*:\n  - *For Item Iteration & Exploration*: Set explicit upper bounds (e.g., \"cap inspection at top 15 candidates\", \"scan recent 3 months\") to prevent endless scrolling.\n  - *For Periodic Monitoring / Polling*: Specify the check action (e.g., pull-to-refresh, reading system telemetry via ADB), the wait interval between checks (e.g., check every 5 minutes to conserve resources), and the exit criteria (e.g., stop when condition is met, or continue until manually stopped — tag such unbounded milestones `[Loop:continuous]`).\n\n### 4. Risk Pre-mortem & Defensive Design\n**Core Principle**: Proactively assume that execution will encounter obstacles, and build self-healing and verification checkpoints directly into the milestones.\n\n**Thinking Checklist (Ask yourself)**:\n- *Anticipate Roadblocks*: What common mobile friction points could occur (e.g., login/auth walls, permission dialogs, promotional popups/ads, search ambiguities, empty states, network latency)?\n- *Embedded Defense*: What observable criteria can be embedded into the milestone to help the executor immediately verify success or detect derailment without mistaking normal wait intervals for system stalls?\n\n### 5. Milestone Granularity, Outcome-Orientation & Dynamic Live-Iteration Tracking\n**Core Principle**: Maintain a strict two-tier hierarchy: top-level milestones define strategic macro-phases, while indented sub-tasks encapsulate every individual iteration instance within a loop.\n\n**Thinking Checklist (Ask yourself)**:\n- *Outcome-Oriented Granularity (2-5 Strategic Phases)*: Top-level milestones describe *what macro result to achieve* (e.g., \Line truncated
     "capabilities_operator": "## Execution Capabilities\n\nThe execution agent primarily interacts with the mobile device and analyzes screen recordings and logs. It can perform the following types of actions:\n- **UI Interaction**: Tapping, swiping, and entering text on the screen.\n- **App Management**: Launching and stopping specific applications.\n- **Information Retrieval & Analysis**: Reading on-screen text and UI hierarchy, searching the web for external information, querying system logs, and analyzing screen recordings.\n- **Terminal & Background Operations**: Executing arbitrary ADB shell commands on the device for exploration. It is capable of running long-running ADB commands asynchronously in the background as tasks.\n\n### Capability Boundaries & Planning Rules\n- **Cross-App Operations**: When a task requires interacting with multiple applications, split the operations into distinct milestones for each app. Avoid mixing actions for different apps in a single milestone.",
-    "output_protocol_initial": "## Task Plan & Scratchpad Protocol\n\n1. **Articulate Thinking**: Begin by reasoning through the 5-step **Systematic Thinking Process** to establish your strategy, funnel approach, bounds, and pre-mortem checks.\n2. **Designate Notes in Milestones & Pre-generate Answer Cards**: If the task involves collecting findings, logging periodic monitoring events, recording candidates, or analytical summaries, designate clear, descriptive note names directly in the milestone descriptions (e.g., `candidates_list`, `monitoring_log`, `schedule_summary`). You can and should invoke the `save_note` tool to pre-generate these structured template files (answer card / scratchpad skeletons with Markdown headers, sections, or tables) before submitting `task_plan`. This provides downstream executors with a clear, ready-to-fill schema.\n3. **Submit Plan**: Submit your initial plan by invoking the `save_note` tool with `key='task_plan'`. Avoid writing the plan only in your text response, as it will only be recognized by the system if submitted via the tool.\n\n### Plan Formatting Rules\n- Format the plan strictly as a Markdown task list starting with `- [ ]` for uncompleted items.\n- For linear macro-milestones: `- [ ] <Phase Description>`\n- For repetitive, iterative, or periodic polling milestones, tag the item with `[Loop]` and declare its Exit Boundary and Interval (e.g., `- [ ] [Loop] <Overall Loop / Monitoring Goal> (Exit: <Condition or Continuous until manually stopped>; Interval: <e.g., every 5 minutes>)`).\n- **Strict Two-Tier Hierarchy & Loop Subtask Contract**:\n  1. **Subtask Nesting Rule**: ALL individual iteration instances, polling checks, candidate inspections, or baseline setups MUST be formatted as 2-space indented sub-tasks (`  - [ ]`) directly nested under their parent `[Loop]` milestone.\n  2. **Mandatory First-Instance Initialization**: Every `[Loop]` milestone MUST be initialized with its first concrete child sub-task (e.g., `  - [ ] Inspect Candidate #1: ...` or `  - [ ] Polling Check #1 (Baseline & Initial Sync): ...`).\n  3. **No Flattened Cycles**: NEVER elevate individual checks, recurring polling cycles, or candidate inspections to top-level milestones outside the `[Loop]`. Top-level items are reserved exclusively for strategic macro-phases.\n- Ensure the content of the `task_plan` note contains only the task list, excluding introductions, explanations, conversational text, or headers.",
+    "output_protocol_initial": "## Task Plan & Scratchpad Protocol\n\n1. **Articulate Thinking**: Begin by reasoning through the 5-step **Systematic Thinking Process** to establish your strategy, funnel approach, bounds, and pre-mortem checks.\n2. **Designate Notes in Milestones & Pre-generate Answer Cards**: If the task involves collecting findings, logging periodic monitoring events, recording candidates, or analytical summaries, designate clear, descriptive note names directly in the milestone descriptions (e.g., `candidates_list`, `monitoring_log`, `schedule_summary`). You can and should invoke the `save_note` tool to pre-generate these structured template files (answer card / scratchpad skeletons with Markdown headers, sections, or tables) before submitting `task_plan`. This provides downstream executors with a clear, ready-to-fill schema.\n3. **Submit Plan**: Submit your initial plan by invoking the `save_note` tool with `key='task_plan'`. Avoid writing the plan only in your text response, as it will only be recognized by the system if submitted via the tool.\n\n### Plan Formatting Rules\n- Format the plan strictly as a Markdown task list starting with `- [ ]` for uncompleted items.\n{{ plan_grammar }}\n- For linear macro-milestones: `- [ ] <Phase Description>`\n- For bounded repetitive, iterative, or periodic polling milestones, tag the item with `[Loop]` and declare its Exit Boundary and Interval (e.g., `- [ ] [Loop] <Overall Loop / Monitoring Goal> (Exit: <Condition>; Interval: <e.g., every 5 minutes>)`).\n- For unbounded continuous monitoring that only the user can stop, tag the item with `[Loop:continuous]` instead (e.g., `- [ ] [Loop:continuous] <Monitoring Goal> (Interval: <e.g., every 5 minutes>)`); never declare a self-serve exit condition for it.\n- **Strict Two-Tier Hierarchy & Loop Subtask Contract**:\n  1. **Subtask Nesting Rule**: ALL individual iteration instances, polling checks, candidate inspections, or baseline setups MUST be formatted as 2-space indented sub-tasks (`  - [ ]`) directly nested under their parent `[Loop]` milestone.\n  2. **Mandatory First-Instance Initialization**: Every `[Loop]` milestone MUST be initialized with its first concrete child sub-task (e.g., `  - [ ] Inspect Candidate #1: ...` or `  - [ ] Polling Check #1 (Baseline & Initial Sync): ...`).\n  3. **No Flattened Cycles**: NEVER elevate individual checks, recurring polling cycles, or candidate inspections to top-level milestones outside the `[Loop]`. Top-level items are reserved exclusively for strategic macro-phases.\n- Ensure the content of the `task_plan` note contains only the task list, excluding introductions, explanations, conversational text, or headers.",
     "examples_initial": "## Master Example: Demonstrating the Complete Cognitive Paradigm\n\n**Goal**: \"Find and summarize all schedule arrangements and venue details for the upcoming November tech conference from my emails.\"\n\n### Naive / Incorrect Approaches:\n1. *Flawed Semantics & Too Granular*:\n- [ ] Tap search icon in Gmail\n- [ ] Type \"November\"\n- [ ] Open first email and tap back\n*(Fails: Micromanages atomic UI taps; assumes emails for November arrangements must have \"November\" in the title or be received in November)*\n2. *Vague & Unbounded Trap*:\n- [ ] Open email and check all emails for conference details\n*(Fails: Unbounded scan trap; causes infinite scrolling and step budget exhaustion on mobile)*\n\n---\n\n### Correct Approach (Full Cognitive Thinking & Robust Plan):\n\n#### Step-by-Step Thinking:\n1. **Intent & Semantic Expansion**:\n   - *Core Objective*: Extract confirmed schedule dates, sessions, and venue info for the November tech conference.\n   - *Semantic Insight*: Conference communications are often sent months in advance (e.g., ticket confirmations or registration emails in August/September). The subject line may say \"Global Developer Summit\", \"Tech Expo Agenda\", or \"Registration Confirmed\" rather than literally \"November conference\".\n   - *Task Composition*: Linear setup (open email app and search) -> Iterative candidate inspection loop (examine each promising email) -> Final consolidation.\n2. **Visual Grounding**:\n   - The device screenshot shows the Home screen with the Gmail app icon visible. The plan must start by opening the email client.\n3. **Tiered Strategy, Funnel Approach & Safeguards**:\n   - *Tier 1 (Optimal)*: Use in-app search with rich semantic keywords (\"conference\", \"summit\", \"schedule\", \"agenda\", \"venue\", \"November\").\n   - *Tier 2 (Fallback)*: If search is unavailable or yields zero results, fallback to chronological scanning of recent 3 months.\n   - *Funnel (Broad Gathering -> Deep Inspection)*: Gather candidate emails from search results; do not discard candidates based solely on list previews. Open each candidate thread to inspect full body details.\n   - *Feasibility Boundary*: Cap the candidate email inspection pool at top 15 emails to prevent infinite scrolling.\n4. **Risk Pre-mortem**:\n   - *Risks*: Promotional marketing spam; conflicting drafts with preliminary dates; cancelled sessions.\n   - *Defensive Check*: Instruct downstream executor to filter out promotional newsletters, verify session timestamps, and confirm final venue address.\n5. **Milestone Granularity & Scratchpad Blueprint**:\n   - Focus milestones on business outcomes (search, inspect candidates, consolidate), avoiding micromanaged click-by-click steps.\n   - Use indented sub-tasks to guide the loop of inspecting each candidate email, initializing with Item #1 and adding subsequent items as they are identified. This prevents the agent from losing state or re-inspecting the same items.\n   - Designate note `november_conference_summary` as the destination for recording findings and pre-generate the structured template note.\n\n#### Tool Invocation:\n1. Pre-generate designated answer card note template:\n`save_note(key='november_conference_summary', content='# November Tech Conference Summary\\n\\n## Key Details\\n- Conference Name: \\n- Dates: \\n- Venue / Address: \\n\\n## Daily Schedule & Sessions\\n(To be populated during email inspection)\\n')`\n\n2. Submit milestone plan:\n`save_note(key='task_plan', content='- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\\n  - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\\n- [ ] Review and consolidate all extracted information in `november_conference_summary` into a clear, finalized conference itinerary')`\n\n#### Content of `task_plan` Note:\n- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\n  - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\n- [ ] ReviLine truncated
+    "check_generation": "## Declaring Verification Standards (check lines)\nYou may attach check lines to the plan; an independent Checker judges them asynchronously.\n\n- **User-declared expectations become `- assert:` lines.** Whenever the user's goal states an explicit expectation (\"should show X\", \"must not crash\", \"the toast Y should appear\"), declare it as an assert line preserving the user's wording verbatim. Expectations about the whole run or the final state use the `@end` suffix (task-level lines at the end of the plan).\n- **`- verify:` lines only for milestones that genuinely need acceptance checks** — completion points of irreversible operations, or milestones that directly contribute to the final goal. Do NOT attach a verify line to every subgoal: restraint is an explicit requirement (checking cost scales with the number of check lines).\n- **Do not fabricate assertions** for subgoals that have nothing verifiable. Rewrite transient expectations as persistently probeable state wherever possible (e.g., \"alarm exists in the alarm list\" instead of \"a toast flashed\") — evidence for transient moments only exists if the execution history happened to capture it.\n- **Ordering requirements are post-hoc.** If the user demands \"A must be confirmed before B runs\", still generate the assertion preserving their semantics, but annotate in the plan that ordering is verified retroactively from history — the system performs post-hoc audits and CANNOT block B from executing. Never silently promise enforcement that does not exist.",
+    "check_audit": "- **Check Standard Protection**: Check lines (`- verify:` / `- assert:`) declared in the plan are the task's verification standards. REJECT any proposed plan that deletes, weakens, or rewords existing check lines — softening a declared standard is never an acceptable adaptation. Adding new check lines is allowed.",
     "role_validator": "## Role\nYou are the Task Plan Validator. The Operator has attempted to modify the top-level task plan. Your job is to ensure the proposed new plan still aligns with the user's original intent and is viable.",
-    "validator_guidelines": "## Validation Principles\n- **Goal Alignment**: Ensure the new plan ultimately accomplishes the `Initial Goal` requested by the user. If the Operator has abandoned the core objective or fundamentally altered it, you must reject the plan.\n- **Logical Coherence**: Verify that the proposed milestones form a logical, achievable sequence to reach the goal. Support valid ongoing iteration/polling subgoals and trigger branches without misconstruing periodic checks as infinite loops. Reject the plan if it introduces contradictory steps, unhandled deadlocks, or obvious logical fallacies.\n- **Loop & Monitoring Protection**: If the existing plan contains an active `[Loop]` milestone (especially continuous polling / monitoring with exit condition `until manually stopped`), you MUST REJECT any proposed plan that deletes the `[Loop]` milestone, removes the loop tag, or marks a continuous monitoring milestone as completed (`[x]`) without explicit external user instruction.\n- **Adaptive Acceptance**: The Operator operates in a dynamic environment and may encounter unexpected UI states or dynamic event triggers. You should approve alternative strategies, extra intermediate steps, loop status tracking, or necessary workarounds as long as they serve the primary goal effectively.\n- **Constructive Feedback**: If you must reject the plan, provide clear, objective, and actionable feedback explaining the logical flaw or goal deviation, and suggest how the Operator should correct the plan."
+    "validator_guidelines": "## Validation Principles\n- **Goal Alignment**: Ensure the new plan ultimately accomplishes the `Initial Goal` requested by the user. If the Operator has abandoned the core objective or fundamentally altered it, you must reject the plan.\n- **Logical Coherence**: Verify that the proposed milestones form a logical, achievable sequence to reach the goal. Support valid ongoing iteration/polling subgoals and trigger branches without misconstruing periodic checks as infinite loops. Reject the plan if it introduces contradictory steps, unhandled deadlocks, or obvious logical fallacies.\n{{ plan_grammar }}\n- **Loop & Monitoring Protection**: If the existing plan contains an active `[Loop]` or `[Loop:continuous]` milestone, you MUST REJECT any proposed plan that deletes the milestone or removes its loop tag, as well as any plan that marks a `[Loop:continuous]` continuous monitoring milestone as completed (`[x]`) without an explicit external user stop signal.\n- **Adaptive Acceptance**: The Operator operates in a dynamic environment and may encounter unexpected UI states or dynamic event triggers. You should approve alternative strategies, extra intermediate steps, loop status tracking, or necessary workarounds as long as they serve the primary goal effectively.\n- **Constructive Feedback**: If you must reject the plan, provide clear, objective, and actionable feedback explaining the logical flaw or goal deviation, and suggest how the Operator should correct the plan."
   },
   "modes": {
     "initial_plan": {

artemis/agents/planner/planner.py

@@ -25,7 +25,9 @@ from artemis.context import ArtemisContext
 from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan, trace
 from artemis.graph.state import State
+from artemis.graph.visibility import strict_state
 from artemis.services.llm import (
+    acomplete,
     get_llm,
     invoke_llm_with_timeout_message,
     with_fallback,
@@ -44,11 +46,30 @@ from artemis.tools.tool_wrapper import (
 from artemis.utils.decorators import wrap_with_callbacks
 from artemis.utils.logger import get_logger
 from artemis.utils.notes import get_note_file_path
+from artemis.utils.plan_grammar import render_plan_grammar_spec
 from artemis.utils.task_tree import build_plan_and_history
 
 logger = get_logger(__name__)
 
 
+def _checks_enabled(ctx: ArtemisContext) -> bool:
+    """Config-driven prompt assembly gate: with both check gates disabled, no
+    check-related instruction reaches any planner prompt."""
+    setup = getattr(ctx, "execution_setup", None)
+    return bool(setup and getattr(setup, "checks_enabled", False))
+
+
+def build_planner_system_blocks(prompts_data: dict, mode: str, include_checks: bool) -> list[str]:
+    """Assembly point for the planner prompt's component list (§1.3): the
+    check-related block mounts only while the checking feature is active."""
+    blocks = list(prompts_data["modes"][mode]["system"])
+    if include_checks:
+        extra = "check_generation" if mode == "initial_plan" else "check_audit"
+        if extra in prompts_data.get("blocks", {}):
+            blocks.append(extra)
+    return blocks
+
+
 class _CyFunctionDetectorMeta(type):
     def __instancecheck__(self, instance):
         name = type(instance).__name__
@@ -97,10 +118,13 @@ async def run_async_planner_validation(
         with open(prompts_path, encoding="utf-8") as f:
             prompts_data = json.load(f)
 
+        include_checks = _checks_enabled(ctx)
         mode_config = prompts_data["modes"]["validator"]
-        system_blocks = mode_config["system"]
+        system_blocks = build_planner_system_blocks(prompts_data, "validator", include_checks)
         system_content = "\n\n".join(prompts_data["blocks"][block] for block in system_blocks)
-        system_message = Template(system_content).render()
+        system_message = Template(system_content).render(
+            plan_grammar=render_plan_grammar_spec(include_checks)
+        )
 
         history_str = "No history available."
         if ctx.data_engine:
@@ -163,6 +187,7 @@ class PlannerNode:
     )
     @trace(type="agent", name="planner")
     async def __call__(self, state: State):
+        state = strict_state(state, "planner")
         prompts_path = Path(__file__).parent / "planner.json"
         with open(prompts_path, encoding="utf-8") as f:
             prompts_data = json.load(f)
@@ -170,10 +195,16 @@ class PlannerNode:
         mode = "initial_plan"
         mode_config = prompts_data["modes"][mode]
 
-        system_blocks = mode_config["system"]
+        include_checks = _checks_enabled(self.ctx)
+        # Assembly point (not in-template conditionals): check-generation
+        # instructions only mount while the checking feature is active, so a
+        # fully-disabled configuration yields a byte-identical planner prompt.
+        system_blocks = build_planner_system_blocks(prompts_data, mode, include_checks)
         system_content = "\n\n".join(prompts_data["blocks"][block] for block in system_blocks)
 
-        system_message = Template(system_content).render()
+        system_message = Template(system_content).render(
+            plan_grammar=render_plan_grammar_spec(include_checks)
+        )
 
         # Get screenshot
         screenshot_b64 = None
@@ -287,22 +318,14 @@ class PlannerNode:
         for validation_attempt in range(max_validation_attempts):
             tool_called = False
 
-            async def run_stream(llm_model):
-                full_response = None
-                trace_id = CURRENT_TRACE_ID.get()
-                async for chunk in llm_model.astream(current_messages):
-                    if full_response is None:
-                        full_response = chunk
-                    else:
-                        full_response += chunk
-
-                    pass
-                return full_response
-
             for _ in range(max_iterations):
                 response = await with_fallback(
-                    main_call=lambda: invoke_llm_with_timeout_message(run_stream(llm)),
-                    fallback_call=lambda: invoke_llm_with_timeout_message(run_stream(llm_fallback)),
+                    main_call=lambda: invoke_llm_with_timeout_message(
+                        acomplete(llm, current_messages)
+                    ),
+                    fallback_call=lambda: invoke_llm_with_timeout_message(
+                        acomplete(llm_fallback, current_messages)
+                    ),
                 )
 
                 if response is None or not response.tool_calls:
@@ -415,8 +438,4 @@ class PlannerNode:
                 f"Failed to generate a valid plan after {max_validation_attempts} attempts."
             )
 
-        return await state.asanitize_update(
-            ctx=self.ctx,
-            update={},
-            agent="planner",
-        )
+        return {}

artemis/agents/summarizer/summarizer.py

@@ -25,7 +25,8 @@ from langchain_core.messages import (
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import CURRENT_TRACE_ID, trace
 from artemis.graph.state import State
-from artemis.services.llm import get_llm
+from artemis.graph.visibility import strict_state
+from artemis.services.llm import acomplete, get_llm
 from artemis.utils.coordinates import normalize_any_structure
 from artemis.utils.logger import get_logger
 from artemis.utils.task_tree import (
@@ -42,6 +43,7 @@ class SummarizerNode:
 
     @trace(type="agent", name="summarizer")
     async def __call__(self, state: State):
+        state = strict_state(state, "summarizer")
         # 1. Trigger background task for summary and data engine update
         if self.ctx.data_engine and state.current_step_id:
             step_id = UUID(state.current_step_id)
@@ -179,26 +181,16 @@ class SummarizerNode:
                     HumanMessage(content="\n\n".join(content)),
                 ]
 
+                aggregated_chunk = await acomplete(llm, messages)
+
                 generated_text = ""
-                aggregated_chunk = None
-                async for chunk in llm.astream(messages):
-                    if aggregated_chunk is None:
-                        aggregated_chunk = chunk
-                    else:
-                        aggregated_chunk += chunk
-
-                    if chunk.content:
-                        text_to_stream = ""
-                        if isinstance(chunk.content, str):
-                            text_to_stream = chunk.content
-                        elif isinstance(chunk.content, list):
-                            for item in chunk.content:
-                                if isinstance(item, dict) and item.get("type") == "text":
-                                    text_to_stream += item.get("text", "")
-
-                        if text_to_stream:
-                            generated_text += text_to_stream
-                    pass
+                content = getattr(aggregated_chunk, "content", "") if aggregated_chunk else ""
+                if isinstance(content, str):
+                    generated_text = content
+                elif isinstance(content, list):
+                    for item in content:
+                        if isinstance(item, dict) and item.get("type") == "text":
+                            generated_text += item.get("text", "")
 
                 summary = generated_text.strip()
                 logger.info(f"Summary generated: {summary}")

artemis/agents/validator/tool_declarations.py

@@ -30,6 +30,7 @@ from artemis.context import ArtemisContext
 from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.core.tool_declaration import ToolDeclaration
 from artemis.data_engine.trace import trace
+from artemis.mcp.action_specs import tool_declaration
 from artemis.mcp.action_types import ActionCode
 from artemis.mcp.actuators.adb import AdbActuator
 from artemis.mcp.observation import observe
@@ -187,9 +188,7 @@ async def capture_screenshot_and_parse_ui(
     Thin adapter over :func:`artemis.mcp.observation.observe` that adds the LangGraph
     ``State`` write-back (indexed points/elements) the agents rely on.
     """
-    obs, screenshot_bytes = await observe(
-        ctx, controller, settle_ms=0 if skip_settling else 400
-    )
+    obs, screenshot_bytes = await observe(ctx, controller, settle_ms=0 if skip_settling else 400)
     if not obs.ok:
         return None, None, None
 
@@ -473,7 +472,7 @@ class MobileActionExecutor:
                     direction=target,
                     target=swipe_input.get("target"),
                     indexed_elements=getattr(state, "indexed_elements", None) if state else None,
-                    ui_hierarchy=getattr(state, "ui_tree", None) if state else None,
+                    ui_hierarchy=getattr(state, "latest_ui_hierarchy", None) if state else None,
                     width=width,
                     height=height,
                     duration=final_duration,
@@ -753,189 +752,19 @@ class MobileActionExecutor:
         )
 
 
-CLICK_TOOL = ToolDeclaration(
-    name="click",
-    description=(
-        "[ACTION] Tap/click on the target coordinate on the screen. The screen"
-        " after click will be returned automatically. For buttons, checkboxes,"
-        " tabs, icons, items in a list, tap ON the element. For text fields /"
-        " search bars / input boxes, tap INSIDE the input box to focus it."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "target": {
-                "type": "array",
-                "items": {"type": "integer"},
-                "description": "Normalized coordinates [x, y] in 0-1000 scale.",
-            },
-            "times": {
-                "type": "integer",
-                "description": "Number of taps to perform (default 1). Set to 2 for double click.",
-            },
-            "delay_ms": {
-                "type": "integer",
-                "description": "Delay between taps in milliseconds (default 100).",
-            },
-        },
-        "required": ["target"],
-    },
-)
+# Device-action declarations are generated from the canonical manifest
+# (artemis/mcp/action_specs.py); the historical constant names remain the public API.
+CLICK_TOOL = tool_declaration("click")
 
-CLICK_SEQUENCE_TOOL = ToolDeclaration(
-    name="click_sequence",
-    description=(
-        "[ACTION] Executes a sequence of taps one by one in order on the"
-        " specified targets (e.g. [[500, 280], [885, 362]]). The screen will be returned ONLY after all clicks"
-        " in the sequence have completed."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "sequence": {
-                "type": "array",
-                "items": {
-                    "type": "array",
-                    "items": {"type": "integer"},
-                    "description": (
-                        "Normalized coordinates [x, y] in 0-1000 scale (e.g., [500, 280]),"
-                        " or a single integer element index."
-                    ),
-                },
-                "description": "List of targets to tap in sequence, e.g. [[500, 280], [885, 362]].",
-            },
-            "delay_ms": {
-                "type": "integer",
-                "description": "Delay between consecutive taps in milliseconds (default 50ms).",
-            },
-        },
-        "required": ["sequence"],
-    },
-)
+CLICK_SEQUENCE_TOOL = tool_declaration("click_sequence")
 
-LONG_PRESS_TOOL = ToolDeclaration(
-    name="long_press",
-    description=(
-        "[ACTION] Long press on a target coordinate on the screen. The screen"
-        " after long pressing will be returned automatically."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "target": {
-                "type": "array",
-                "items": {"type": "integer"},
-                "description": "Normalized coordinates [x, y] in 0-1000 scale.",
-            },
-            "duration_ms": {
-                "type": "integer",
-                "description": "Duration of press in milliseconds (default 1000ms).",
-            },
-        },
-        "required": ["target"],
-    },
-)
+LONG_PRESS_TOOL = tool_declaration("long_press")
 
-INPUT_TEXT_TOOL = ToolDeclaration(
-    name="input_text",
-    description=(
-        "[ACTION] Type text into an input field on the screen. The screen after"
-        " typing will be returned automatically. Automatically taps inside the"
-        " input box at target [x, y] to focus before typing."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "text": {
-                "type": "string",
-                "description": "Text to type into the focused input field.",
-            },
-            "target": {
-                "type": "array",
-                "items": {"type": "integer"},
-                "description": "Coordinates [x, y] of the input box in 0-1000 scale.",
-            },
-            "clear_exist": {
-                "type": "boolean",
-                "description": (
-                    "Whether to clear existing text in the input box before typing (default True)."
-                ),
-            },
-        },
-        "required": ["text", "target"],
-    },
-)
+INPUT_TEXT_TOOL = tool_declaration("input_text")
 
-SWIPE_TOOL = ToolDeclaration(
-    name="swipe",
-    description=(
-        "[ACTION] Perform a swipe, drag, or slider-adjustment gesture on the screen. The screen and UI hierarchy after swipe will be returned automatically.\n\n"
-        "• Directional Scrolling ('direction' or 'action'): Recommended for general browsing and standard page scrolling in most scenarios. Automatically computes safe swipe vectors and adaptive duration, retains a ~40% visual overlap anchor for zero-omission traversal, and prevents inertial flings. Supports scoping to a sub-container via 'target'. If it fails on certain custom layouts, fall back to specifying exact coordinates ('start' and 'end') directly.\n"
-        "• Precise Coordinate Gestures ('start', 'end' or coordinates list): Best for local, fine-grained interactions such as adjusting sliders/SeekBars (e.g., volume, brightness, progress bars), drag-and-drop / list reordering, or as a reliable fallback when directional scrolling fails on specific containers. Always drag slightly PAST the target position to overcome touch slop and reliably trigger the update. When setting a slider to Maximum (100%) or Minimum (0%), swipe fully to the extreme boundary."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "direction": {
-                "type": "string",
-                "enum": ["up", "down", "left", "right"],
-                "description": (
-                    "Direction for scrolling and swiping: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
-                    " 'down' (drags top-to-bottom, scrolling up to reveal content above),"
-                    " 'left' (drags right-to-left, scrolling right),"
-                    " 'right' (drags left-to-right, scrolling left)."
-                ),
-            },
-            "start": {
-                "type": "array",
-                "items": {"type": "integer"},
-                "description": "Start normalized coordinates [start_x, start_y] in 0-1000 scale.",
-            },
-            "end": {
-                "type": "array",
-                "items": {"type": "integer"},
-                "description": "End normalized coordinates [end_x, end_y] in 0-1000 scale.",
-            },
-            "target": {
-                "description": "Optional target element index (e.g. 2) or container bounds [left, top, right, bottom] to scope the directional swipe within.",
-            },
-            "action": {
-                "description": (
-                    "Backward-compatible swipe gesture: smart direction string ('up', 'down', 'left', 'right')"
-                    " OR precise custom coordinates [start_x, start_y, end_x, end_y] in 0-1000 scale."
-                ),
-            },
-            "duration": {
-                "type": "integer",
-                "description": (
-                    "Optional swipe/drag duration in milliseconds (default 800). For drag-and-drop,"
-                    " list reordering, or sliding/adjusting sliders (e.g., volume, brightness, SeekBars),"
-                    " set duration >= 1000 (e.g. 1500). If omitted for directional swipe, duration is computed automatically."
-                ),
-            },
-        },
-    },
-)
+SWIPE_TOOL = tool_declaration("swipe")
 
-PRESS_KEY_TOOL = ToolDeclaration(
-    name="press_key",
-    description=(
-        "[ACTION] Press a physical or virtual system button. The screen after"
-        " pressing will be returned automatically."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "key": {
-                "type": "string",
-                "description": (
-                    "Standard Android system button name (ENTER, BACK, HOME, APP_SWITCH)."
-                ),
-            }
-        },
-        "required": ["key"],
-    },
-)
+PRESS_KEY_TOOL = tool_declaration("press_key")
 
 READ_NOTE_TOOL = ToolDeclaration(
     name="read_note",
@@ -966,49 +795,9 @@ LIST_NOTES_TOOL = ToolDeclaration(
     parameters={"type": "object", "properties": {}},
 )
 
-MANAGE_APP_TOOL = ToolDeclaration(
-    name="manage_app",
-    description=(
-        "[ACTION] Launch or force stop a specified application. The screen"
-        " after launch/stop will be returned automatically."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "action": {
-                "type": "string",
-                "description": "The action type ('launch' or 'stop').",
-            },
-            "app_name": {
-                "type": "string",
-                "description": "Display name or package name of the application.",
-            },
-        },
-        "required": ["action", "app_name"],
-    },
-)
+MANAGE_APP_TOOL = tool_declaration("manage_app")
 
-WAIT_FOR_DELAY_TOOL = ToolDeclaration(
-    name="wait_for_delay",
-    description=(
-        "[ACTION] Pause execution and wait for a specified duration in milliseconds."
-        " The screen after pause will be returned automatically."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "time_in_ms": {
-                "type": "integer",
-                "description": (
-                    "The exact duration to pause in milliseconds (e.g., 2000 for 2s,"
-                    " 60000 for 1 min, 180000 for 3 mins, 300000 for 5 mins)."
-                    " Convert any required waiting duration into milliseconds."
-                ),
-            }
-        },
-        "required": ["time_in_ms"],
-    },
-)
+WAIT_FOR_DELAY_TOOL = tool_declaration("wait_for_delay")
 
 REPORT_FAILURE_ANALYSIS_TOOL = ToolDeclaration(
     name="report_failure_analysis",

artemis/agents/validator/validator.py

@@ -38,9 +38,10 @@ from artemis.context import ArtemisContext
 from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.data_engine.trace import CURRENT_TRACE_ID, trace
 from artemis.graph.state import State
+from artemis.graph.visibility import strict_state
 from artemis.mcp.action_names import to_canonical_call
 from artemis.mcp.action_session import ActionSession, get_action_session
-from artemis.services.llm import get_llm
+from artemis.services.llm import acomplete_structured, get_llm
 from artemis.utils import image_diff, visualization
 from artemis.utils.decorators import wrap_with_callbacks
 from artemis.utils.logger import get_logger
@@ -154,11 +155,7 @@ class ValidatorNode:
     async def _execute_validation_loop(self, state: State):
         actions, error_msg = self._parse_decisions(state.structured_decisions)
         if error_msg:
-            return await state.asanitize_update(
-                ctx=self.ctx,
-                update={},
-                agent="validator",
-            )
+            return {}
 
         execution = []
         failed_action = None
@@ -176,11 +173,7 @@ class ValidatorNode:
             decision_screenshot_b64 = last_screenshot_b64
             decision_screenshot_name = last_screenshot_name
         except Exception:
-            return await state.asanitize_update(
-                ctx=self.ctx,
-                update={},
-                agent="validator",
-            )
+            return {}
 
         step_id = None
         if state.current_step_id:
@@ -355,7 +348,9 @@ class ValidatorNode:
                 if post_screenshot_b64:
                     decoded_bytes = base64.b64decode(post_screenshot_b64)
                     last_screenshot_b64 = post_screenshot_b64
-                    logger.info(f"Successfully decoded post_screenshot_b64 ({len(decoded_bytes)} bytes)")
+                    logger.info(
+                        f"Successfully decoded post_screenshot_b64 ({len(decoded_bytes)} bytes)"
+                    )
                     if self.ctx.data_engine:
                         post_image_name = self.ctx.data_engine.get_or_create_image(decoded_bytes)
                         last_screenshot_name = post_image_name
@@ -466,11 +461,7 @@ class ValidatorNode:
                 step_id, report, post_image_name=distinct_post_image_name
             )
 
-        return await state.asanitize_update(
-            ctx=self.ctx,
-            update={"last_execution_result": report},
-            agent="validator",
-        )
+        return {"last_execution_result": report}
 
     @wrap_with_callbacks(
         before=lambda: logger.info("Starting Validator Agent..."),
@@ -479,6 +470,7 @@ class ValidatorNode:
     )
     @trace(type="agent", name="validator")
     async def __call__(self, state: State):
+        state = strict_state(state, "validator")
 
         step_id = None
         if state.current_step_id:
@@ -1196,22 +1188,16 @@ class ValidatorNode:
                     HumanMessage(content=user_content),
                 ]
 
-                # D. Invoke Universal VLM
-                response = await llm.ainvoke(messages)
-                output = response.content if isinstance(response.content, str) else ""
-                if isinstance(response.content, list):
-                    output = "".join(
-                        b.get("text", "")
-                        for b in response.content
-                        if isinstance(b, dict) and "text" in b
+                # D. Invoke Universal VLM. Parsing (fences, repair) and one
+                # corrective re-ask on unparseable JSON are handled by the
+                # structured-output layer; a StructuredOutputError lands in
+                # this attempt loop's except like any other attempt failure.
+                res_json = await acomplete_structured(llm, messages)
+                if not isinstance(res_json, dict):
+                    raise ValueError(
+                        "Pixel validation response parsed to"
+                        f" {type(res_json).__name__}, expected a JSON object."
                     )
-
-                if output.startswith("```"):
-                    code_match = re.search(r"```(?:json)?\s*([\s\S]*?)\s*```", output)
-                    if code_match:
-                        output = code_match.group(1).strip()
-
-                res_json = json.loads(output)
                 reasoning = res_json.get("reasoning", "")
                 is_present = res_json.get("is_present", True)
                 confidence = res_json.get("confidence", 1.0)

artemis/agents/video_analyzer/reliability.py

@@ -12,15 +12,24 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-"""Failure classification and circuit breaking for video model calls."""
+"""Failure classification and circuit breaking for video model calls.
+
+Thin media-aware extension of :mod:`artemis.llm.reliability`, which owns the
+generic taxonomy and circuit breaker.  This module adds the media-processing
+category and the ``should_split`` recovery decision (bisecting long clips).
+"""
 
 from __future__ import annotations
 
 from dataclasses import dataclass
 from enum import StrEnum
-import threading
-import time
-from typing import Any
+
+from artemis.llm.reliability import (
+    CircuitBreaker,
+    FailureCategory,
+    classify_failure,
+    extract_status_code,
+)
 
 
 class VideoFailureCategory(StrEnum):
@@ -43,115 +52,37 @@ class VideoFailure:
     should_fallback: bool
 
 
+_MEDIA_MARKERS = ("ffmpeg", "codec", "corrupt", "invalid video", "invalid audio", "media")
+
+
 def classify_video_failure(error: BaseException) -> VideoFailure:
     """Convert provider/transport/media exceptions into recovery decisions."""
-
-    if isinstance(error, KeyboardInterrupt) or type(error).__name__ == "CancelledError":
-        return VideoFailure(VideoFailureCategory.CANCELLED, False, False, False)
-
-    code = getattr(error, "code", None) or getattr(error, "status_code", None)
-    try:
-        code = int(code) if code is not None else None
-    except (TypeError, ValueError):
-        code = None
-
+    generic = classify_failure(error)
     message = str(error).lower()
-    if code == 429 or any(marker in message for marker in ("rate limit", "quota", "resource_exhausted")):
-        return VideoFailure(VideoFailureCategory.RATE_LIMIT, True, False, True)
-    if code in {500, 502, 503, 504} or any(
-        marker in message
-        for marker in ("unavailable", "overloaded", "high demand", "service unavailable")
-    ):
-        return VideoFailure(VideoFailureCategory.PROVIDER_UNAVAILABLE, True, False, True)
-    if isinstance(error, TimeoutError) or "timed out" in message or "timeout" in message:
-        return VideoFailure(VideoFailureCategory.TIMEOUT, True, True, True)
-    if isinstance(error, (ConnectionError, OSError)) and not any(
-        marker in message for marker in ("ffmpeg", "video", "audio", "codec", "media")
-    ):
-        return VideoFailure(VideoFailureCategory.CONNECTION, True, False, True)
-    if code in {401, 403} or any(
-        marker in message for marker in ("unauthorized", "forbidden", "api key", "credential")
-    ):
-        return VideoFailure(VideoFailureCategory.AUTHENTICATION, False, False, True)
-    if any(
-        marker in message
-        for marker in ("ffmpeg", "codec", "corrupt", "invalid video", "invalid audio", "media")
-    ):
+    has_media_marker = any(marker in message for marker in _MEDIA_MARKERS)
+
+    # Media markers take precedence over generic connection / bad-request /
+    # unknown classification (rate-limit, unavailable, timeout, and auth
+    # signals still win, matching the original ordering of checks).
+    if has_media_marker and generic.category in {
+        FailureCategory.CONNECTION,
+        FailureCategory.BAD_REQUEST,
+        FailureCategory.UNKNOWN,
+    }:
         return VideoFailure(VideoFailureCategory.MEDIA_PROCESSING, False, True, True)
-    if code in {400, 404, 413, 415, 422}:
-        return VideoFailure(VideoFailureCategory.BAD_REQUEST, False, code == 413, True)
-    return VideoFailure(VideoFailureCategory.UNKNOWN, True, True, True)
+
+    category = VideoFailureCategory(generic.category.value)
+    should_split = category in {VideoFailureCategory.TIMEOUT, VideoFailureCategory.UNKNOWN} or (
+        category is VideoFailureCategory.BAD_REQUEST and extract_status_code(error) == 413
+    )
+    # The video pipeline keeps its historical behavior of falling back to the
+    # universal analyzer even for bad requests (a different provider may accept
+    # the same media payload).
+    should_fallback = generic.should_fallback or category is VideoFailureCategory.BAD_REQUEST
+    if category is VideoFailureCategory.CANCELLED:
+        should_fallback = False
+    return VideoFailure(category, generic.retryable, should_split, should_fallback)
 
 
-@dataclass
-class _CircuitState:
-    consecutive_failures: int = 0
-    open_until: float = 0.0
-    half_open_in_flight: bool = False
-
-
-class VideoCircuitBreaker:
-    """Small context-scoped circuit breaker keyed by provider/model identity."""
-
-    def __init__(self, threshold: int = 3, cooldown_seconds: float = 60.0) -> None:
-        self.threshold = max(1, int(threshold))
-        self.cooldown_seconds = max(0.1, float(cooldown_seconds))
-        self._states: dict[str, _CircuitState] = {}
-        self._lock = threading.RLock()
-
-    def allow(self, key: str, *, now: float | None = None) -> bool:
-        current = time.time() if now is None else now
-        with self._lock:
-            state = self._states.get(key)
-            if state is None:
-                return True
-            if state.open_until:
-                if current < state.open_until or state.half_open_in_flight:
-                    return False
-                # Half-open: permit exactly one trial. Other concurrent calls
-                # continue to bypass the primary until it succeeds or fails.
-                state.half_open_in_flight = True
-                state.consecutive_failures = max(0, self.threshold - 1)
-                return True
-            return not state.half_open_in_flight
-
-    def record_success(self, key: str) -> None:
-        with self._lock:
-            self._states.pop(key, None)
-
-    def record_failure(
-        self,
-        key: str,
-        failure: VideoFailure,
-        *,
-        now: float | None = None,
-    ) -> None:
-        if failure.category not in {
-            VideoFailureCategory.RATE_LIMIT,
-            VideoFailureCategory.PROVIDER_UNAVAILABLE,
-            VideoFailureCategory.TIMEOUT,
-            VideoFailureCategory.CONNECTION,
-        }:
-            with self._lock:
-                state = self._states.get(key)
-                if state is not None and state.half_open_in_flight:
-                    self._states.pop(key, None)
-            return
-        current = time.time() if now is None else now
-        with self._lock:
-            state = self._states.setdefault(key, _CircuitState())
-            state.half_open_in_flight = False
-            state.consecutive_failures += 1
-            if state.consecutive_failures >= self.threshold:
-                state.open_until = current + self.cooldown_seconds
-
-    def snapshot(self) -> dict[str, dict[str, Any]]:
-        with self._lock:
-            return {
-                key: {
-                    "consecutive_failures": state.consecutive_failures,
-                    "open_until": state.open_until,
-                    "half_open_in_flight": state.half_open_in_flight,
-                }
-                for key, state in self._states.items()
-            }
+class VideoCircuitBreaker(CircuitBreaker):
+    """Context-scoped circuit breaker keyed by provider/model identity."""

artemis/config/agent.py

@@ -16,7 +16,7 @@
 
 import os
 from pathlib import Path
-from typing import Any
+from typing import Any, Literal
 
 from pydantic import BaseModel, Field, model_validator
 
@@ -143,11 +143,22 @@ class CommitteeConfig(BaseModel):
 
 
 class CheckerConfig(BaseModel):
-    """Configuration specific to Checker subgoal verification and rollback."""
+    """Configuration for plan-driven checkpoint verification and the final review."""
 
     enabled: bool = Field(
         default=False,
-        description="Whether to enable visual verification and snapshot rollback by the Checker agent.",
+        description=(
+            "Master switch (compat alias): False disables BOTH midway"
+            " checkpoints and the final check."
+        ),
+    )
+    midway_checks: bool = Field(
+        default=True,
+        description="Whether plan-declared midway checkpoints run (requires enabled=True).",
+    )
+    final_check: bool = Field(
+        default=True,
+        description="Whether the exit final review runs (requires enabled=True).",
     )
     max_iterations: int = Field(
         default=20,
@@ -155,11 +166,41 @@ class CheckerConfig(BaseModel):
         le=50,
         description="Maximum verification iterations for Checker LLM reasoning and inspection.",
     )
-    max_chat_rounds: int = Field(
-        default=4,
+    final_check_max_attempts: int = Field(
+        default=3,
         ge=1,
         le=10,
-        description="Maximum debate/rebuttal rounds between Operator and Checker.",
+        description="Maximum final-review attempts before ending with a blocked outcome.",
+    )
+    checkpoint_max_repairs: int = Field(
+        default=2,
+        ge=0,
+        le=10,
+        description="Per-checkpoint repair quota for failed verify criteria.",
+    )
+    max_concurrent_checkpoints: int = Field(
+        default=3,
+        ge=1,
+        le=10,
+        description="Maximum concurrently running checkpoint attempts.",
+    )
+    checkpoint_timeout: float = Field(
+        default=180.0,
+        gt=0,
+        description="Per-attempt timeout (seconds) for one checkpoint check.",
+    )
+    settlement_timeout: float = Field(
+        default=120.0,
+        gt=0,
+        description="Aggregate timeout (seconds) for the exit settlement barrier.",
+    )
+    assert_failure_policy: Literal["continue", "halt"] = Field(
+        default="continue",
+        description="Whether an assert failure lets execution continue or halts the run.",
+    )
+    device_probes: bool = Field(
+        default=True,
+        description="Whether the Checker's enumerated read-only device probes are registered.",
     )
     model_config = {"extra": "allow"}
 

artemis/config/settings.py

@@ -131,6 +131,16 @@ class Settings(BaseSettings):
         description="Enable context caching when running multi-turn pro/ultra Explorer",
     )
 
+    # LLM Reliability
+    LLM_PAUSE_TIMEOUT_SECONDS: float = Field(
+        default=900.0,
+        description=(
+            "How long a task waits in the paused state after exhausting LLM"
+            " retries before failing with LLMExhaustedError. <= 0 waits"
+            " forever (legacy interactive behavior)."
+        ),
+    )
+
     # Paths & Storage
     TRACES_PATH: Path = Field(default_factory=get_default_traces_path)
     DATA_ENGINE_DB_PATH: Path = Field(default_factory=get_data_engine_db_path)

artemis/constants.py

@@ -14,11 +14,9 @@
 
 RECURSION_LIMIT = 30000
 MAX_MESSAGES_IN_HISTORY = 25
-VALIDATOR_MESSAGES_KEY = "validator_messages"
 
 # Optimization Constants
 CHECKER_MAX_ITERATIONS = 20
-OPERATOR_MAX_CHAT_ROUNDS = 4
 VALIDATOR_POLL_TIMEOUT = 2.0
 VALIDATOR_POLL_INTERVAL = 0.2
 VALIDATOR_OCR_INTERVAL = 1.0

artemis/context.py

@@ -106,8 +106,18 @@ class ExecutionSetup(BaseModel):
     app_lock_status: AppLaunchResult | None = None
     video_recording_tools_enabled: bool = Field(default_factory=detect_video_tools_enabled)
     disable_checker: bool = True
+    """Legacy master switch (compat alias): ``True`` disables BOTH the midway
+    checkpoints and the final check, regardless of the individual gates below."""
+    disable_midway_checks: bool = False
+    disable_final_check: bool = False
     checker_max_iterations: int = 20
-    checker_max_chat_rounds: int = 4
+    final_check_max_attempts: int = 3
+    checkpoint_max_repairs: int = 2
+    max_concurrent_checkpoints: int = 3
+    checkpoint_timeout: float = 180.0
+    settlement_timeout: float = 120.0
+    assert_failure_policy: Literal["continue", "halt"] = "continue"
+    disable_device_probes: bool = False
     disable_planner_validation: bool = True
     planner_validation_threshold: float = 0.85
     enable_committee: bool = False
@@ -117,6 +127,23 @@ class ExecutionSetup(BaseModel):
     explorer: ExplorerConfig = Field(default_factory=ExplorerConfig)
     explorer_versions: dict[str, str] = Field(default_factory=dict)
 
+    @property
+    def midway_checks_enabled(self) -> bool:
+        """Midway checkpoint gate: honors both the individual switch and the
+        legacy ``disable_checker`` master alias."""
+        return not (self.disable_midway_checks or self.disable_checker)
+
+    @property
+    def final_check_enabled(self) -> bool:
+        """Final review gate: honors both the individual switch and the legacy
+        ``disable_checker`` master alias."""
+        return not (self.disable_final_check or self.disable_checker)
+
+    @property
+    def checks_enabled(self) -> bool:
+        """True when any check gate is active (drives conditional prompt assembly)."""
+        return self.midway_checks_enabled or self.final_check_enabled
+
     @property
     def explorer_version(self) -> str:
         return self.explorer.default_version
@@ -160,20 +187,43 @@ class ArtemisContext(BaseModel):
     execution_setup: ExecutionSetup | None = None
 
     data_engine: DataEngine | None = None
-    checker_task: asyncio.Task[Any] | None = None
     planner_task: asyncio.Task[Any] | None = None
     background_tasks: list[asyncio.Task[Any]] = Field(default_factory=list)
     background_task_grace_period_seconds: float = 2.0
     package_cache: dict[str, str | None] = Field(default_factory=dict)
     background_jobs: dict[str, dict[str, Any]] = Field(default_factory=dict)
 
-    task_plan_snapshot: Path | None = None
+    pending_checkpoints: list[Any] = Field(default_factory=list)
+    """Queued :class:`~artemis.graph.checkpoints.PendingCheckpoint` entries.
+    ``_process_plan_write`` only enqueues; ``execution_check_node`` spawns after
+    the turn's step is recorded so the evidence anchor points at a real step."""
+    checkpoint_tasks: dict[str, Any] = Field(default_factory=dict)
+    """checkpoint_id -> (attempt_id, asyncio.Task). One in-flight attempt per
+    checkpoint; supersede harvests/cancels the old entry before replacing it."""
+    checkpoint_attempt_seq: dict[str, int] = Field(default_factory=dict)
+    """checkpoint_id -> monotonically increasing attempt counter."""
+    checkpoint_repairs: dict[str, int] = Field(default_factory=dict)
+    """checkpoint_id -> number of verify-fail repairs already applied."""
+    assert_halt: bool = False
+    """Latched by an assert failure under ``assert_failure_policy='halt'``."""
+    final_check_attempts: int = 0
+    """Number of final-check passes already executed at exit settlement."""
+
     task_plan_content_before: str | None = None
+    last_validated_plan: str | None = None
+    """Ratchet baseline for planner validation: the last plan content whose
+    top-level milestones were validated (or the initial plan). Drift is always
+    judged against this, never against the immediately preceding write."""
+    pending_validated_plan: str | None = None
+    """Plan content currently under async planner validation; becomes the new
+    baseline when the validator approves it."""
 
     _genai_client: Any | None = PrivateAttr(default=None)
     _video_blackboard: Any | None = PrivateAttr(default=None)
     _video_circuit_breaker: Any | None = PrivateAttr(default=None)
     _mobile_controller: Any | None = PrivateAttr(default=None)
+    _active_driver: Any | None = PrivateAttr(default=None)
+    """Cached device driver singleton (artemis.drivers.factory.get_driver)."""
     mcp_client_ctx: Any | None = None
     mcp_session: Any | None = None
     action_session: Any | None = None
@@ -200,9 +250,7 @@ class ArtemisContext(BaseModel):
             tasks = list(self.background_tasks)
             pending_tasks = [task for task in tasks if not task.done()]
             grace_period = (
-                max(0.0, self.background_task_grace_period_seconds)
-                if exc_type is None
-                else 0.0
+                max(0.0, self.background_task_grace_period_seconds) if exc_type is None else 0.0
             )
 
             if pending_tasks and grace_period > 0:

artemis/core/context.py

@@ -50,6 +50,9 @@ class ExecutionContext:
         self.device = DeviceInfoContext(device_id=device_id, platform=platform)
         self.driver: Any | None = None
         self.telemetry: Any | None = None
+        # Fully-wired ArtemisContext (data_engine, adb clients, execution setup).
+        # Required by the live (non-mock) paths of GraphRunner/ReactiveRunner.
+        self.artemis_context: Any | None = None
         self.custom_attributes: dict[str, Any] = {}
 
     def set_attribute(self, key: str, value: Any) -> None:

artemis/data_engine/engine.py

@@ -240,7 +240,9 @@ class DataEngine:
                 except ValueError:
                     pass
         else:
-            env_session_id = os.getenv("ARTEMIS_CLOUD_SESSION_ID") or os.getenv("ARTEMIS_SESSION_ID")
+            env_session_id = os.getenv("ARTEMIS_CLOUD_SESSION_ID") or os.getenv(
+                "ARTEMIS_SESSION_ID"
+            )
             if env_session_id:
                 try:
                     session_id = UUID(env_session_id)
@@ -293,7 +295,9 @@ class DataEngine:
         else:
             self._step_number_cache.clear()
             self.current_step_number = 0
-        logger.info(f"Session started: {session_id} (current step counter: {self.current_step_number})")
+        logger.info(
+            f"Session started: {session_id} (current step counter: {self.current_step_number})"
+        )
         self._publish("session_started", session.model_dump())
         return session_id
 
@@ -511,7 +515,9 @@ class DataEngine:
                     existing_steps = self.storage.get_steps(self.current_session_id) or []
                 except Exception:
                     existing_steps = []
-            max_existing = max([s.step_number for s in existing_steps], default=0) if existing_steps else 0
+            max_existing = (
+                max([s.step_number for s in existing_steps], default=0) if existing_steps else 0
+            )
             self.current_step_number = max(self.current_step_number, max_existing) + 1
             step_number = self.current_step_number
 
@@ -940,6 +946,31 @@ class DataEngine:
                 pass
         return None
 
+    def get_step_record(self, step_number: int):
+        """Read-only lookup of a step record by its 1-based step number."""
+        if not self.current_session_id:
+            return None
+        try:
+            steps = self.storage.get_steps(self.current_session_id) or []
+        except Exception as e:
+            logger.error(f"Failed to fetch steps for step lookup: {e}")
+            return None
+        for s in steps:
+            if s.step_number == step_number:
+                return s
+        return None
+
+    def get_step_image_path(self, step_number: int, which: str = "pre") -> Path | None:
+        """Read-only path of a step's pre/post screenshot, if recorded."""
+        step = self.get_step_record(step_number)
+        if step is None:
+            return None
+        image_name = step.pre_image_name if which == "pre" else step.post_image_name
+        if not image_name:
+            return None
+        path = self.get_image_path(image_name)
+        return path if path.exists() else None
+
     def update_step_summary(self, step_id: UUID | int | str, summary: str):
         """Update the step with a concise summary in background."""
         target_uuid = None

artemis/engine/graph_runner.py

@@ -14,14 +14,11 @@
 
 """Pro Graph Runner orchestrating multi-agent state machines with self-healing."""
 
-from langchain_core.messages import HumanMessage
-
-from artemis.context import ArtemisContext, DeviceContext
 from artemis.core.state import ExecutionContextState, ExecutionStatus, StepRecord
 from artemis.drivers.mock.mock_driver import MockDeviceDriver
 from artemis.engine.base_runner import BaseRunner
 from artemis.graph.graph import get_graph
-from artemis.platform import platform
+from artemis.graph.state import State
 from artemis.utils.logger import get_logger
 
 logger = get_logger(__name__)
@@ -45,16 +42,18 @@ class GraphRunner(BaseRunner):
 
         if not is_mock:
             try:
-                artemis_ctx = getattr(self.ctx, "artemis_context", None)
+                artemis_ctx = self.ctx.artemis_context
                 if artemis_ctx is None:
-                    artemis_ctx = ArtemisContext(
-                        device=DeviceContext(
-                            host_platform=platform.os_type.name,
-                            device_id=self.driver.device_id,
-                        )
+                    # Fail fast with a clear contract violation instead of
+                    # silently building an empty ArtemisContext that drops
+                    # data_engine/llm_config and dies later in perception.
+                    raise RuntimeError(
+                        "GraphRunner requires ExecutionContext.artemis_context"
+                        " to be wired (data_engine, adb clients, execution"
+                        " setup) before running the live graph path."
                     )
                 graph = await get_graph(artemis_ctx)
-                graph_input = {"messages": [HumanMessage(content=self.ctx.task_goal)]}
+                graph_input = State.initial(self.ctx.task_goal)
                 async for _ in graph.astream(
                     input=graph_input, config={"recursion_limit": max_turns}
                 ):

artemis/engine/reactive_runner.py

@@ -18,12 +18,10 @@ import time
 
 from artemis.agents.flash.runner import FlashRunner
 from artemis.config import load_agent_config
-from artemis.context import ArtemisContext, DeviceContext
 from artemis.core.state import ExecutionContextState, ExecutionStatus, StepRecord
 from artemis.drivers.mock.mock_driver import MockDeviceDriver
 from artemis.engine.base_runner import BaseRunner
 from artemis.graph.state import State
-from artemis.platform import platform
 from artemis.utils.logger import get_logger
 
 logger = get_logger(__name__)
@@ -54,21 +52,23 @@ class ReactiveRunner(BaseRunner):
 
         if not is_mock:
             try:
-                artemis_ctx = getattr(self.ctx, "artemis_context", None)
+                artemis_ctx = self.ctx.artemis_context
                 if artemis_ctx is None:
-                    artemis_ctx = ArtemisContext(
-                        device=DeviceContext(
-                            host_platform=platform.os_type.name,
-                            device_id=self.driver.device_id,
-                        )
+                    # Fail fast with a clear contract violation instead of
+                    # silently building an empty ArtemisContext that drops
+                    # data_engine/llm_config and dies later mid-run.
+                    raise RuntimeError(
+                        "ReactiveRunner requires ExecutionContext.artemis_context"
+                        " to be wired (data_engine, adb clients, execution"
+                        " setup) before running the live Flash path."
                     )
                 flash_runner = FlashRunner(
                     artemis_ctx, goal=self.ctx.task_goal, max_turns=max_turns
                 )
-                legacy_state = State(messages=[])
+                legacy_state = State.initial(self.ctx.task_goal)
                 result = await flash_runner.run(legacy_state)
 
-                state.current_turn = len(getattr(legacy_state, "messages", [])) or 1
+                state.current_turn = 1
                 if result.get("status") == "completed":
                     state.status = ExecutionStatus.SUCCESS
                 else:

artemis/graph/checkpoints.py

@@ -0,0 +1,649 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Plan-driven checkpoint scheduling, the append-only verdict ledger, and exit
+settlement bookkeeping.
+
+Three invariants from the redesign govern everything here:
+
+1. **Append-only history**: the Operator's context is never rolled back or
+   switched; verdicts influence the future (status flips forward, findings
+   injected) but never rewrite the past.
+2. **Release / verdict / wrap-up separation**: fail-open only affects the
+   release decision — the verdict value itself (``inconclusive``) is always
+   recorded verbatim; assert failures are never repaired but always produce a
+   machine-readable test result.
+3. **Zero side effects inside check tasks**: state mutation happens only at
+   harvest points (``execution_check_node``) and at exit settlement.
+"""
+
+from __future__ import annotations
+
+import asyncio
+import json
+import re
+import time
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any, Literal
+
+from pydantic import BaseModel, Field
+
+from artemis.utils.logger import get_logger
+from artemis.utils.notes import get_note_file_path
+from artemis.utils.plan_grammar import (
+    CHECKBOX_LINE_RE,
+    CheckItem,
+    PlanSnapshot,
+    parse_plan,
+    subgoal_hash,
+)
+
+logger = get_logger(__name__)
+
+LEDGER_FILENAME = "check_ledger.jsonl"
+RUN_OUTCOME_FILENAME = "run_outcome.json"
+
+# --- Config accessors (all fail-safe against a missing execution_setup) --------------
+
+
+def _setup(ctx) -> Any:
+    return getattr(ctx, "execution_setup", None)
+
+
+def midway_checks_enabled(ctx) -> bool:
+    setup = _setup(ctx)
+    return bool(setup) and bool(getattr(setup, "midway_checks_enabled", False))
+
+
+def final_check_enabled(ctx) -> bool:
+    setup = _setup(ctx)
+    return bool(setup) and bool(getattr(setup, "final_check_enabled", False))
+
+
+def _setting(ctx, name: str, default):
+    setup = _setup(ctx)
+    value = getattr(setup, name, None) if setup else None
+    return default if value is None else value
+
+
+# --- Data contracts ------------------------------------------------------------------
+
+
+@dataclass(frozen=True)
+class EvidenceAnchor:
+    """Anchors a checkpoint to the evidence available at completion time."""
+
+    anchor_step_id: str | None
+    trigger_ts: float
+    plan_text: str
+
+
+@dataclass(frozen=True)
+class PendingCheckpoint:
+    """A queued checkpoint: enqueue happens at plan-write time, spawn happens in
+    ``execution_check_node`` after the turn's step is recorded."""
+
+    checkpoint_id: str
+    subgoal_text: str
+    check_items: tuple[CheckItem, ...]
+    plan_text: str
+    trigger_ts: float
+
+
+@dataclass(frozen=True)
+class CheckpointRun:
+    """One in-flight (or finished) check attempt held in ``ctx.checkpoint_tasks``."""
+
+    attempt_id: str
+    task: asyncio.Task
+    checkpoint: PendingCheckpoint
+
+
+class TestSummary(BaseModel):
+    passed: int = 0
+    failed: int = 0
+    inconclusive: int = 0
+    unchecked: int = 0
+    failed_items: list[dict] = Field(default_factory=list)
+
+    @property
+    def total(self) -> int:
+        return self.passed + self.failed + self.inconclusive + self.unchecked
+
+
+class RunOutcome(BaseModel):
+    """Machine-readable run outcome: goal-achievement axis x test-result axis."""
+
+    task_status: Literal["completed", "partial", "blocked"]
+    tests: TestSummary
+
+
+# --- Ledger (append-only) ------------------------------------------------------------
+
+
+def ledger_path(base_dir: str | Path) -> Path:
+    return Path(base_dir) / LEDGER_FILENAME
+
+
+def append_ledger_record(base_dir: str | Path, record: dict) -> None:
+    """Appends one verdict record. Records are immutable once written; any later
+    result for the same item can only be appended as a new record."""
+    record = dict(record)
+    record.setdefault("ts", time.time())
+    path = ledger_path(base_dir)
+    try:
+        path.parent.mkdir(parents=True, exist_ok=True)
+        with path.open("a", encoding="utf-8") as f:
+            f.write(json.dumps(record, ensure_ascii=False) + "\n")
+    except Exception as e:
+        logger.error(f"Failed to append check ledger record: {e}")
+
+
+def read_ledger(base_dir: str | Path) -> list[dict]:
+    path = ledger_path(base_dir)
+    if not path.exists():
+        return []
+    records: list[dict] = []
+    try:
+        for line in path.read_text(encoding="utf-8").splitlines():
+            line = line.strip()
+            if not line:
+                continue
+            try:
+                records.append(json.loads(line))
+            except Exception:
+                logger.warning(f"Skipping malformed ledger line: {line[:120]}")
+    except Exception as e:
+        logger.error(f"Failed to read check ledger: {e}")
+    return records
+
+
+def has_ledger_records(base_dir: str | Path) -> bool:
+    path = ledger_path(base_dir)
+    try:
+        return path.exists() and path.stat().st_size > 0
+    except Exception:
+        return False
+
+
+# --- Plan helpers --------------------------------------------------------------------
+
+
+def _read_plan_text(ctx) -> str:
+    if not getattr(ctx, "data_engine", None):
+        return ""
+    path = get_note_file_path(ctx.data_engine.base_dir, "task_plan")
+    if not path.exists():
+        return ""
+    try:
+        return path.read_text(encoding="utf-8")
+    except Exception as e:
+        logger.error(f"Failed to read task plan: {e}")
+        return ""
+
+
+def _subgoal_still_current(ctx, checkpoint_id: str) -> bool:
+    """The anchored subgoal's text is unchanged iff an item with the same
+    content hash still exists in the current plan."""
+    snapshot = parse_plan(_read_plan_text(ctx))
+    return any(i.is_top_level and i.key == checkpoint_id for i in snapshot.items)
+
+
+def revert_subgoal_status(ctx, subgoal_key: str) -> bool:
+    """Flips the matching top-level subgoal from ``[x]`` back to ``[/]``.
+
+    This is a forward-looking state change (never a rollback of history) and is
+    written directly to the file, bypassing the wrapped note tools.
+    """
+    if not getattr(ctx, "data_engine", None):
+        return False
+    path = get_note_file_path(ctx.data_engine.base_dir, "task_plan")
+    if not path.exists():
+        return False
+    try:
+        lines = path.read_text(encoding="utf-8").split("\n")
+        changed = False
+        for idx, line in enumerate(lines):
+            match = CHECKBOX_LINE_RE.match(line)
+            if not match or match.group("indent"):
+                continue
+            if subgoal_hash(match.group("text").strip()) == subgoal_key:
+                if match.group("status") == "x":
+                    lines[idx] = re.sub(r"^(\s*-\s*\[)x(\])", r"\g<1>/\g<2>", line)
+                    changed = True
+                break
+        if changed:
+            path.write_text("\n".join(lines), encoding="utf-8")
+        return changed
+    except Exception as e:
+        logger.error(f"Failed to revert subgoal status: {e}")
+        return False
+
+
+# --- Queueing (called from _process_plan_write; never spawns) ------------------------
+
+
+def queue_checkpoints(
+    ctx,
+    state,
+    after: PlanSnapshot,
+    new_completions: list[str],
+    plan_text: str,
+) -> None:
+    """Enqueues a pending checkpoint for every newly completed top-level subgoal
+    that carries ``on_complete`` check items.
+
+    Spawning is deliberately deferred to ``execution_check_node``: at this
+    moment the turn's step is not yet recorded, so an anchor taken here would
+    point at the previous turn.
+    """
+    if not midway_checks_enabled(ctx):
+        return
+    if state is not None and getattr(state, "user_stop_requested", False):
+        return
+    items_by_key = {i.key: i for i in after.top_level}
+    for text in new_completions:
+        key = subgoal_hash(text)
+        item = items_by_key.get(key)
+        if item is None:
+            continue
+        check_items = tuple(ci for ci in after.check_items_of(item) if ci.when == "on_complete")
+        if not check_items:
+            continue
+        ctx.pending_checkpoints.append(
+            PendingCheckpoint(
+                checkpoint_id=key,
+                subgoal_text=text,
+                check_items=check_items,
+                plan_text=plan_text,
+                trigger_ts=time.time(),
+            )
+        )
+        logger.info(f"Queued checkpoint for completed subgoal: {text}")
+
+
+# --- Harvest (§5.2) ------------------------------------------------------------------
+
+
+def _record_verdicts(
+    ctx,
+    run: CheckpointRun,
+    verdicts: list,
+    anchor_step_id: str | None,
+) -> None:
+    base_dir = ctx.data_engine.base_dir
+    when_by_text = {(ci.kind, ci.text): ci.when for ci in run.checkpoint.check_items}
+    for v in verdicts:
+        append_ledger_record(
+            base_dir,
+            {
+                "attempt_id": run.attempt_id,
+                "checkpoint_id": run.checkpoint.checkpoint_id,
+                "item_text": v.item_text,
+                "kind": v.kind,
+                "when": when_by_text.get((v.kind, v.item_text), "on_complete"),
+                "status": v.status,
+                "evidence": v.evidence,
+                "suggestion": getattr(v, "suggestion", ""),
+                "anchor_step_id": anchor_step_id,
+            },
+        )
+
+
+def _record_attempt_failure(ctx, run: CheckpointRun, status: str, evidence: str) -> None:
+    base_dir = ctx.data_engine.base_dir
+    for ci in run.checkpoint.check_items:
+        append_ledger_record(
+            base_dir,
+            {
+                "attempt_id": run.attempt_id,
+                "checkpoint_id": run.checkpoint.checkpoint_id,
+                "item_text": ci.text,
+                "kind": ci.kind,
+                "when": ci.when,
+                "status": status,
+                "evidence": evidence,
+                "anchor_step_id": None,
+            },
+        )
+
+
+def harvest_run(
+    ctx,
+    state,
+    run: CheckpointRun,
+    *,
+    allow_side_effects: bool,
+    cancelled_status: str = "superseded",
+    anchor_step_id: str | None = None,
+) -> list[str]:
+    """Books a finished (done/cancelled) attempt into the ledger and, when the
+    verdict is still applicable, applies its execution-state side effects.
+
+    Returns Operator-facing findings (injected as ``operator_feedback``). The
+    verdict values themselves are never altered by the release decision:
+    an errored verify is *released* (fail-open) but *recorded* inconclusive.
+    """
+    task = run.task
+    findings: list[str] = []
+    checkpoint = run.checkpoint
+
+    if task.cancelled():
+        evidence = (
+            "attempt superseded by a newer completion of the same subgoal"
+            if cancelled_status == "superseded"
+            else "cancelled at exit settlement (settlement timeout)"
+        )
+        _record_attempt_failure(ctx, run, cancelled_status, evidence)
+        return findings
+
+    exc = task.exception()
+    if exc is not None:
+        status = "inconclusive"
+        reason = (
+            f"check timed out after {_setting(ctx, 'checkpoint_timeout', 180.0)}s"
+            if isinstance(exc, (asyncio.TimeoutError, TimeoutError))
+            else f"check raised an exception: {exc}"
+        )
+        logger.warning(
+            f"Checkpoint attempt {run.attempt_id} did not produce a verdict"
+            f" ({reason}); releasing (fail-open) but recording inconclusive."
+        )
+        _record_attempt_failure(ctx, run, status, reason)
+        return findings
+
+    report = task.result()
+    verdicts = list(getattr(report, "verdicts", []) or [])
+    _record_verdicts(ctx, run, verdicts, anchor_step_id)
+
+    # Applicability gate for side effects: the verdict must belong to the
+    # current attempt of its checkpoint AND the anchored subgoal's text must be
+    # unchanged in the current plan. Stale/mismatched verdicts are ledger-only.
+    applicable = allow_side_effects and _subgoal_still_current(ctx, checkpoint.checkpoint_id)
+    user_stopped = bool(state is not None and getattr(state, "user_stop_requested", False))
+
+    verify_failures = [v for v in verdicts if v.kind == "verify" and v.status == "failed"]
+    assert_failures = [v for v in verdicts if v.kind == "assert" and v.status == "failed"]
+
+    if verify_failures and applicable and not user_stopped:
+        max_repairs = int(_setting(ctx, "checkpoint_max_repairs", 2))
+        repairs = ctx.checkpoint_repairs.get(checkpoint.checkpoint_id, 0)
+        if repairs < max_repairs:
+            ctx.checkpoint_repairs[checkpoint.checkpoint_id] = repairs + 1
+            reverted = revert_subgoal_status(ctx, checkpoint.checkpoint_id)
+            for v in verify_failures:
+                suggestion = getattr(v, "suggestion", "") or ""
+                findings.append(
+                    f"[verify failed] Subgoal '{checkpoint.subgoal_text}' —"
+                    f" criterion '{v.item_text}': {v.evidence}"
+                    + (f" Suggestion: {suggestion}" if suggestion else "")
+                )
+            if reverted:
+                findings.append(
+                    f"The subgoal '{checkpoint.subgoal_text}' has been set back to"
+                    " in-progress ([/]). Address the findings above, then complete"
+                    " it again."
+                )
+        else:
+            logger.warning(
+                f"Checkpoint {checkpoint.checkpoint_id} exhausted its repair"
+                f" quota ({max_repairs}); verdict stays failed, no further"
+                " repair loop."
+            )
+
+    if assert_failures:
+        policy = str(_setting(ctx, "assert_failure_policy", "continue"))
+        if policy == "halt":
+            ctx.assert_halt = True
+            logger.warning(
+                "Assert failure under 'halt' policy: latching halt flag for exit settlement."
+            )
+
+    return findings
+
+
+def harvest_finished_checkpoints(ctx, state) -> list[str]:
+    """Non-blocking harvest: books every ``done()`` attempt, never awaits a
+    running one. Returns Operator-facing findings."""
+    if not getattr(ctx, "data_engine", None):
+        return []
+    findings: list[str] = []
+    for cid, run in list(ctx.checkpoint_tasks.items()):
+        if not run.task.done():
+            continue
+        del ctx.checkpoint_tasks[cid]
+        findings.extend(harvest_run(ctx, state, run, allow_side_effects=True))
+    return findings
+
+
+# --- Spawn (§5.1: after record_step) -------------------------------------------------
+
+
+async def spawn_pending_checkpoints(ctx, state, anchor_step_id: str | None) -> list[str]:
+    """Spawns queued checkpoints against the just-recorded step, honoring the
+    concurrency cap and superseding stale attempts without losing verdicts.
+
+    Returns findings harvested from superseded-but-finished attempts.
+    """
+    findings: list[str] = []
+    if not getattr(ctx, "data_engine", None):
+        return findings
+    if not midway_checks_enabled(ctx):
+        return findings
+    if state is not None and getattr(state, "user_stop_requested", False):
+        return findings
+
+    from artemis.agents.checker.checker import run_checkpoint_check
+
+    max_concurrent = int(_setting(ctx, "max_concurrent_checkpoints", 3))
+    timeout = float(_setting(ctx, "checkpoint_timeout", 180.0))
+    goal = getattr(state, "initial_goal", "") if state is not None else ""
+
+    while ctx.pending_checkpoints and len(ctx.checkpoint_tasks) < max_concurrent:
+        pending: PendingCheckpoint = ctx.pending_checkpoints.pop(0)
+        cid = pending.checkpoint_id
+
+        old_run: CheckpointRun | None = ctx.checkpoint_tasks.pop(cid, None)
+        if old_run is not None:
+            if old_run.task.done():
+                # A finished-but-unharvested attempt: book it first (verdicts,
+                # including failures, are never dropped), ledger-only.
+                findings.extend(harvest_run(ctx, state, old_run, allow_side_effects=False))
+            else:
+                old_run.task.cancel()
+                try:
+                    await old_run.task
+                except (asyncio.CancelledError, Exception):
+                    pass
+                harvest_run(ctx, state, old_run, allow_side_effects=False)
+
+        seq = ctx.checkpoint_attempt_seq.get(cid, 0) + 1
+        ctx.checkpoint_attempt_seq[cid] = seq
+        attempt_id = f"{cid}#{seq}"
+        anchor = EvidenceAnchor(
+            anchor_step_id=anchor_step_id,
+            trigger_ts=pending.trigger_ts,
+            plan_text=pending.plan_text,
+        )
+        task = asyncio.create_task(
+            asyncio.wait_for(
+                run_checkpoint_check(
+                    ctx,
+                    check_items=pending.check_items,
+                    anchor=anchor,
+                    goal=goal,
+                    subgoal_text=pending.subgoal_text,
+                ),
+                timeout=timeout,
+            )
+        )
+        ctx.checkpoint_tasks[cid] = CheckpointRun(
+            attempt_id=attempt_id, task=task, checkpoint=pending
+        )
+        logger.info(f"Spawned checkpoint attempt {attempt_id} anchored at step {anchor_step_id}")
+
+    return findings
+
+
+# --- Exit settlement barrier (§5.3 phase 1) ------------------------------------------
+
+
+async def settle_all_checkpoints(ctx, state) -> None:
+    """The one intentional blocking wait of the whole flow: collects every
+    outstanding checkpoint before exit, bounded by ``settlement_timeout``.
+    Attempts still running at the deadline are cancelled and recorded
+    ``unchecked`` (settlement timeout). Runs unconditionally — a started check
+    must be booked before exit regardless of the final-check switch."""
+    if not getattr(ctx, "data_engine", None):
+        return
+    runs = list(ctx.checkpoint_tasks.values())
+    ctx.checkpoint_tasks.clear()
+    if not runs:
+        return
+
+    timeout = float(_setting(ctx, "settlement_timeout", 120.0))
+    tasks = [r.task for r in runs]
+    try:
+        _done, pending = await asyncio.wait(tasks, timeout=timeout)
+    except Exception as e:
+        logger.error(f"Settlement wait failed: {e}")
+        pending = {t for t in tasks if not t.done()}
+
+    for t in pending:
+        t.cancel()
+    for t in pending:
+        try:
+            await t
+        except (asyncio.CancelledError, Exception):
+            pass
+
+    for run in runs:
+        # Settlement is bookkeeping, not a repair point: side effects off.
+        harvest_run(
+            ctx,
+            state,
+            run,
+            allow_side_effects=False,
+            cancelled_status="unchecked",
+        )
+
+
+# --- Run outcome ---------------------------------------------------------------------
+
+_RESOLVABLE = ("passed", "failed", "inconclusive")
+
+
+def resolve_item_status(kind: str, records: list[dict]) -> str:
+    """Resolves one check item's final status from its ledger records.
+
+    Assert: the first failure is permanent — any failed record makes the item
+    failed. Verify: the latest substantive verdict wins (a repaired-then-passed
+    verify is passed). No substantive record at all -> unchecked.
+    """
+    substantive = [r for r in records if r.get("status") in _RESOLVABLE]
+    if kind == "assert" and any(r.get("status") == "failed" for r in substantive):
+        return "failed"
+    if substantive:
+        return substantive[-1]["status"]
+    return "unchecked"
+
+
+def compute_test_summary(check_items: list[CheckItem], records: list[dict]) -> TestSummary:
+    summary = TestSummary()
+    by_item: dict[tuple[str, str], list[dict]] = {}
+    for r in records:
+        by_item.setdefault((str(r.get("kind")), str(r.get("item_text"))), []).append(r)
+
+    seen: set[tuple[str, str]] = set()
+    all_items: list[tuple[str, str]] = []
+    for ci in check_items:
+        sig = (ci.kind, ci.text)
+        if sig not in seen:
+            seen.add(sig)
+            all_items.append(sig)
+    # Items that only ever existed in the ledger (e.g. their plan lines were
+    # legitimately revised by the Planner) still count.
+    for sig in by_item:
+        if sig not in seen:
+            seen.add(sig)
+            all_items.append(sig)
+
+    for kind, text in all_items:
+        status = resolve_item_status(kind, by_item.get((kind, text), []))
+        if status == "passed":
+            summary.passed += 1
+        elif status == "failed":
+            summary.failed += 1
+            failing = [r for r in by_item.get((kind, text), []) if r.get("status") == "failed"]
+            summary.failed_items.append(
+                {
+                    "item_text": text,
+                    "kind": kind,
+                    "evidence": failing[-1].get("evidence", "") if failing else "",
+                }
+            )
+        elif status == "inconclusive":
+            summary.inconclusive += 1
+        else:
+            summary.unchecked += 1
+    return summary
+
+
+def compute_run_outcome(
+    snapshot: PlanSnapshot,
+    records: list[dict],
+    *,
+    verify_blocked: bool,
+) -> RunOutcome:
+    """Assembles the machine-readable run outcome.
+
+    ``verify_blocked`` marks the "verify unmet and budget exhausted" exit —
+    the goal-achievement axis then reads blocked even though the plan claims
+    completion. Assert failures never change ``task_status``: a finished task
+    with failed assertions is ``completed`` with ``tests.failed > 0``.
+    """
+    tests = compute_test_summary(list(snapshot.all_check_items), records)
+    if verify_blocked:
+        task_status: Literal["completed", "partial", "blocked"] = "blocked"
+    elif snapshot.all_top_level_done:
+        task_status = "completed"
+    else:
+        task_status = "partial"
+    return RunOutcome(task_status=task_status, tests=tests)
+
+
+def write_run_outcome(base_dir: str | Path, outcome: RunOutcome, extra: dict | None = None) -> None:
+    try:
+        payload = outcome.model_dump()
+        if extra:
+            payload.update(extra)
+        path = Path(base_dir) / RUN_OUTCOME_FILENAME
+        path.parent.mkdir(parents=True, exist_ok=True)
+        path.write_text(
+            json.dumps(payload, ensure_ascii=False, indent=2),
+            encoding="utf-8",
+        )
+    except Exception as e:
+        logger.error(f"Failed to write run outcome: {e}")
+
+
+def read_run_outcome(base_dir: str | Path) -> dict | None:
+    path = Path(base_dir) / RUN_OUTCOME_FILENAME
+    if not path.exists():
+        return None
+    try:
+        return json.loads(path.read_text(encoding="utf-8"))
+    except Exception:
+        return None

artemis/graph/perception.py

@@ -22,6 +22,7 @@ from artemis.context import ArtemisContext
 from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.data_engine.trace import trace
 from artemis.graph.state import State
+from artemis.graph.visibility import strict_state
 from artemis.utils.logger import get_logger
 from artemis.utils.ocr_api import is_ocr_configured, perform_ocr
 from artemis.utils.ocr_xml_fusion import (
@@ -71,15 +72,23 @@ def _should_skip_settling(state: State) -> bool:
 
 
 @trace(type="agent", name="perception")
-def _check_injected_instruction_file(base_dir: str) -> str | None:
+def _check_injected_instruction_file(base_dir: str) -> dict | None:
+    """Consumes a pending injected-instruction payload, if any.
+
+    Returns the payload dict: `instruction` (str | None) plus the explicit
+    `release_loop` stop signal (bool) declared by the injecting caller.
+    """
     instruction_path = Path(base_dir) / "injected_instruction.json"
     if instruction_path.exists():
         with open(instruction_path, encoding="utf-8") as f:
             data = json.load(f)
-            injected_instruction = data.get("instruction")
         instruction_path.unlink()
-        logger.info(f"Read and deleted injected instruction: '{injected_instruction}'")
-        return injected_instruction
+        payload = {
+            "instruction": data.get("instruction"),
+            "release_loop": bool(data.get("release_loop")),
+        }
+        logger.info(f"Read and deleted injected instruction payload: {payload}")
+        return payload
     return None
 
 
@@ -88,6 +97,7 @@ async def perception_node(state: State, ctx: ArtemisContext) -> dict:
 
     Always executes at the start of an operator cycle.
     """
+    state = strict_state(state, "perception")
     logger.info("Entering Perception Node...")
 
     if not ctx.data_engine.current_step_id:
@@ -96,10 +106,14 @@ async def perception_node(state: State, ctx: ArtemisContext) -> dict:
 
     # Check for injected instruction without blocking event loop
     injected_instruction = None
+    user_stop_requested = False
     if ctx.data_engine and ctx.data_engine.base_dir:
-        injected_instruction = await asyncio.to_thread(
+        injected_payload = await asyncio.to_thread(
             _check_injected_instruction_file, str(ctx.data_engine.base_dir)
         )
+        if injected_payload:
+            injected_instruction = injected_payload.get("instruction")
+            user_stop_requested = bool(injected_payload.get("release_loop"))
 
     controller = UnifiedMobileController(ctx)
 
@@ -171,16 +185,6 @@ async def perception_node(state: State, ctx: ArtemisContext) -> dict:
         )
         logger.info(f"Offloaded perception data saving to background with image_name: {image_name}")
 
-    # Cache latest perception data on ArtemisContext for downstream zero-latency reuse (e.g. Checker)
-    ctx.latest_perception_data = {
-        "screenshot_b64": latest_screenshot_b64,
-        "latest_ui_hierarchy": fused_xml,
-        "xml_hierarchy": xml_hierarchy if xml_hierarchy is not None else [],
-        "ocr_results": ocr_results,
-        "width": device_data.width,
-        "height": device_data.height,
-    }
-
     # Return state update
     update = {
         "latest_screenshot": screenshot_path,
@@ -193,5 +197,7 @@ async def perception_node(state: State, ctx: ArtemisContext) -> dict:
             "height": device_data.height,
         },
         "injected_instruction": injected_instruction,
+        # sticky_or reducer: once latched True, later False updates keep it True
+        "user_stop_requested": user_stop_requested,
     }
     return update

artemis/graph/state.py

@@ -12,14 +12,12 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-from typing import Annotated, Any
-from langchain_core.messages import AnyMessage
-from langgraph.graph import add_messages
-from artemis.config import AgentNode
-from artemis.context import ArtemisContext
-from artemis.utils.logger import get_logger
+from typing import Annotated
+
 from pydantic import BaseModel, ConfigDict
 
+from artemis.utils.logger import get_logger
+
 logger = get_logger(__name__)
 
 
@@ -28,38 +26,91 @@ def take_last(a, b):
     return b
 
 
+def sticky_or(a, b):
+    """Reducer that latches True: once set, the flag survives later updates."""
+    return bool(a) or bool(b)
+
+
 class State(BaseModel):
+    """Graph channel state.
+
+    Every field is explicit (``extra="forbid"``): an undeclared write or an
+    unknown constructor key fails loudly instead of becoming a silent ghost
+    key. Undeclared *reads* (``getattr(state, k, default)``) are guarded by the
+    node visibility manifest in ``artemis.graph.visibility`` instead.
+    """
+
     model_config = ConfigDict(
         arbitrary_types_allowed=True,
-        extra="allow",
+        extra="forbid",
     )
 
-    messages: Annotated[list[AnyMessage], "Sequential messages", add_messages]
-
-    remaining_steps: Annotated[int | None, "Remaining steps before the task is completed"] = None
-
-    # planner related keys
+    # ── Control plane (cross-turn signals & routing) ─────────────────────
     initial_goal: Annotated[str, "Initial goal given by the user"]
     injected_instruction: Annotated[
         str | None,
         "Injected instruction from user for micro-adjustment",
         take_last,
     ] = None
+    user_stop_requested: Annotated[
+        bool,
+        "Latched True once the user externally signals stop (release_loop),"
+        " unlocking completion of [Loop:continuous] milestones",
+        sticky_or,
+    ] = False
+    checker_success: Annotated[
+        bool | None,
+        "True if checker succeeded or not triggered, False if failed",
+        take_last,
+    ] = None
+    operator_feedback: Annotated[
+        list[str] | None,
+        "Source-tagged findings ([checker]/[planner]/[final check]) injected"
+        " append-only into the Operator's next prompt (never switches the"
+        " prompt template)",
+        take_last,
+    ] = None
+    run_outcome: Annotated[
+        dict | None,
+        "Machine-readable run outcome (task_status + test summary), populated"
+        " by exit settlement before END",
+        take_last,
+    ] = None
+    exit_settlement_route: Annotated[
+        str | None,
+        "Routing decision produced by exit_settlement_node ('continue' | 'end')",
+        take_last,
+    ] = None
 
-    # operator related keys (perception)
+    # ── Perception (moves behind PerceptionStore in Phase 2) ─────────────
     latest_ui_hierarchy: Annotated[
         list[dict] | None, "Latest UI hierarchy of the device", take_last
-    ]
-    latest_screenshot: Annotated[str | None, "Path to the latest screenshot", take_last]
-    focused_app_info: Annotated[str | None, "Focused app info", take_last]
-    device_date: Annotated[str | None, "Date of the device", take_last]
+    ] = None
+    latest_screenshot: Annotated[str | None, "Path to the latest screenshot", take_last] = None
+    operator_raw_data: Annotated[
+        dict | None,
+        "Raw perception data from operator (screenshot, xml, ocr)",
+        take_last,
+    ] = None
+    indexed_points: Annotated[
+        list[list[int]] | None,
+        "Active coordinate mappings matching indexed text elements",
+        take_last,
+    ] = None
+    indexed_elements: Annotated[
+        list[dict] | None,
+        "Active interactable elements matching indexed text elements,"
+        " containing text, bounds, class, and center coordinates.",
+        take_last,
+    ] = None
 
-    # operator related keys (decisions)
+    # ── Turn products (previous turn's decisions & results) ──────────────
+    current_step_id: Annotated[str | None, "Current step ID from data engine", take_last] = None
     structured_decisions: Annotated[
         str | None,
         "Structured decisions made by the operator, for the validator to follow",
         take_last,
-    ]
+    ] = None
     operator_raw_thinking: Annotated[
         str | None,
         "Raw thinking process of the operator",
@@ -75,60 +126,23 @@ class State(BaseModel):
         "Short term memory / scratchpad for the operator",
         take_last,
     ] = None
-    complete_subgoals_by_ids: Annotated[
-        list[str],
-        "List of subgoal IDs to complete",
-        take_last,
-    ]
-    current_agent: Annotated[str | None, "Current active agent", take_last] = None
-    operator_replan_reason: Annotated[str | None, "Reason for operator replan", take_last] = None
-    subgoal_plan: Annotated[list[Any] | None, "Subgoal plan", take_last] = None
-    operator_tactical_plan: Annotated[list[Any] | None, "Operator tactical plan", take_last] = None
-
-    # validator related keys
-    validator_messages: Annotated[list[AnyMessage], "Sequential Validator messages", add_messages]
     last_execution_result: Annotated[
         dict | None, "Last execution result from validator", take_last
     ] = None
-    current_step_id: Annotated[str | None, "Current step ID from data engine", take_last] = None
-    operator_raw_data: Annotated[
-        dict | None,
-        "Raw perception data from operator (screenshot, xml, ocr)",
-        take_last,
-    ] = None
-    checker_success: Annotated[
-        bool | None,
-        "True if checker succeeded or not triggered, False if failed",
-        take_last,
-    ] = None
     subagent_calls: Annotated[
         list[str], "List of sub-agent calls in current session", take_last
     ] = []
-    operator_replied: Annotated[bool | None, "True if operator replied to checker", take_last] = (
-        None
-    )
     operator_tool_limit_exceeded: Annotated[
         bool | None,
         "True if operator exceeded tool call limit in the previous turn",
         take_last,
     ] = None
-    indexed_points: Annotated[
-        list[list[int]] | None,
-        "Active coordinate mappings matching indexed text elements",
-        take_last,
-    ] = None
-    indexed_elements: Annotated[
-        list[dict] | None,
-        "Active interactable elements matching indexed text elements,"
-        " containing text, bounds, class, and center coordinates.",
-        take_last,
-    ] = None
 
-    async def asanitize_update(
-        self,
-        ctx: ArtemisContext,
-        update: dict,
-        agent: AgentNode | None = None,
-    ):
-        """Sanitizes the state update to ensure it is valid and apply side effect logic where required."""
-        return update
+    @classmethod
+    def initial(cls, goal: str) -> "State":
+        """Single source for the graph's initial state.
+
+        Every entrypoint (SDK, engine runners, tests) must construct the
+        first State through here so required fields stay in one place.
+        """
+        return cls(initial_goal=goal)

artemis/graph/visibility.py

@@ -0,0 +1,227 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Per-node State visibility manifests with test-time enforcement.
+
+``extra="forbid"`` on :class:`~artemis.graph.state.State` catches ghost
+*writes*; ghost *reads* (``getattr(state, key, default)`` silently returning
+the default) can only be caught by policing reads against a declared manifest.
+
+The manifest below is the declared visible domain of every graph node — the
+authoritative answer to "which State fields does this agent see / produce".
+Enforcement is zero-cost in production: :func:`strict_state` returns the state
+unchanged unless ``ARTEMIS_STRICT_STATE=1`` (set in tests), in which case field
+reads outside the node's manifest raise :class:`VisibilityError`.
+"""
+
+from __future__ import annotations
+
+import os
+from dataclasses import dataclass
+from typing import Any
+
+from artemis.graph.state import State
+
+
+class VisibilityError(Exception):
+    """A node touched a State field outside its declared visible domain."""
+
+
+@dataclass(frozen=True)
+class NodeVisibility:
+    reads: frozenset[str]
+    writes: frozenset[str]
+
+    @property
+    def visible(self) -> frozenset[str]:
+        return self.reads | self.writes
+
+
+def _vis(reads: set[str], writes: set[str]) -> NodeVisibility:
+    return NodeVisibility(reads=frozenset(reads), writes=frozenset(writes))
+
+
+#: Declared visible domain per node. Reads include fields touched by the tools
+#: a node invokes with ``InjectedState`` (they run inside the node's turn).
+#: Start generous, tighten over time — an entry here is a documented decision.
+NODE_VISIBILITY: dict[str, NodeVisibility] = {
+    "perception": _vis(
+        reads={"structured_decisions"},
+        writes={
+            "latest_screenshot",
+            "latest_ui_hierarchy",
+            "operator_raw_data",
+            "injected_instruction",
+            "user_stop_requested",
+        },
+    ),
+    "planner": _vis(
+        reads={"initial_goal", "latest_screenshot"},
+        # Mutates latest_screenshot in place when it captures its own frame
+        # (planner runs before perception on the first turn).
+        writes={"latest_screenshot"},
+    ),
+    "operator": _vis(
+        reads={
+            "initial_goal",
+            "injected_instruction",
+            "user_stop_requested",
+            "operator_feedback",
+            "operator_raw_data",
+            "latest_ui_hierarchy",
+            "latest_screenshot",
+            "indexed_points",
+            "indexed_elements",
+            "short_term_memory",
+            "subagent_calls",
+            "current_step_id",
+            "operator_tool_limit_exceeded",
+            "structured_decisions",
+            "operator_raw_thinking",
+            "operator_native_thinking",
+        },
+        writes={
+            "structured_decisions",
+            "operator_raw_thinking",
+            "operator_native_thinking",
+            "short_term_memory",
+            "indexed_points",
+            "indexed_elements",
+            "current_step_id",
+            "subagent_calls",
+            "operator_tool_limit_exceeded",
+        },
+    ),
+    "execution_check": _vis(
+        reads={
+            "operator_raw_data",
+            "structured_decisions",
+            "operator_raw_thinking",
+            "operator_native_thinking",
+            "user_stop_requested",
+            "initial_goal",
+        },
+        writes={
+            "checker_success",
+            "structured_decisions",
+            "current_step_id",
+            "operator_feedback",
+            "subagent_calls",
+        },
+    ),
+    "validator": _vis(
+        reads={
+            "initial_goal",
+            "structured_decisions",
+            "latest_screenshot",
+            "latest_ui_hierarchy",
+            "current_step_id",
+            "operator_raw_data",
+            "operator_raw_thinking",
+            "operator_native_thinking",
+            "indexed_points",
+            "indexed_elements",
+        },
+        writes={
+            "last_execution_result",
+            # In-place turn-scoped mutations (formalized as TurnWorkspace in
+            # Phase 2): post-action screenshot and re-derived index mappings.
+            "latest_screenshot",
+            "indexed_points",
+            "indexed_elements",
+        },
+    ),
+    "summarizer": _vis(
+        reads={
+            "current_step_id",
+            "structured_decisions",
+            "operator_raw_thinking",
+            "operator_native_thinking",
+            "last_execution_result",
+        },
+        writes=set(),
+    ),
+    "convergence": _vis(
+        reads={"checker_success", "user_stop_requested"},
+        writes=set(),
+    ),
+    "exit_settlement": _vis(
+        reads={"user_stop_requested", "initial_goal"},
+        writes={"exit_settlement_route", "operator_feedback", "run_outcome"},
+    ),
+}
+
+_STATE_FIELDS = frozenset(State.model_fields.keys())
+
+
+def strict_enabled() -> bool:
+    return os.environ.get("ARTEMIS_STRICT_STATE", "") == "1"
+
+
+class StateView:
+    """Read-policing proxy handed to nodes under ``ARTEMIS_STRICT_STATE=1``.
+
+    Only declared State *fields* are policed; methods and pydantic internals
+    pass through. Attribute writes delegate to the underlying State (in-place
+    turn mutations are Phase 2 scope) but are still checked against the
+    node's write set.
+    """
+
+    __slots__ = ("_state", "_node", "_visibility")
+
+    def __init__(self, state: State, node: str):
+        object.__setattr__(self, "_state", state)
+        object.__setattr__(self, "_node", node)
+        object.__setattr__(self, "_visibility", NODE_VISIBILITY[node])
+
+    def __getattr__(self, name: str) -> Any:
+        if name in _STATE_FIELDS and name not in self._visibility.visible:
+            raise VisibilityError(
+                f"Node '{self._node}' read undeclared State field '{name}'."
+                " Declare it in NODE_VISIBILITY or stop reading it."
+            )
+        return getattr(self._state, name)
+
+    def __setattr__(self, name: str, value: Any) -> None:
+        if name in _STATE_FIELDS and name not in self._visibility.writes:
+            raise VisibilityError(
+                f"Node '{self._node}' mutated undeclared State field '{name}'."
+                " Declare it in NODE_VISIBILITY or stop writing it."
+            )
+        setattr(self._state, name, value)
+
+
+def strict_state(state: State, node: str) -> State | StateView:
+    """Wraps ``state`` in a policing view when strict mode is on (tests)."""
+    if strict_enabled() and isinstance(state, State):
+        return StateView(state, node)
+    return state
+
+
+def check_update(node: str, update: dict | None) -> None:
+    """Validates a node's returned update keys against its write manifest."""
+    if not update:
+        return
+    illegal = set(update) & _STATE_FIELDS - NODE_VISIBILITY[node].writes
+    if illegal:
+        raise VisibilityError(f"Node '{node}' returned undeclared update keys: {sorted(illegal)}.")
+
+
+def validate_manifest() -> list[str]:
+    """Returns problems in the manifest itself (ghost fields). Empty = clean."""
+    problems = []
+    for node, vis in NODE_VISIBILITY.items():
+        for field in vis.visible - _STATE_FIELDS:
+            problems.append(f"{node}: '{field}' is not a declared State field")
+    return problems

artemis/llm/reliability.py

@@ -0,0 +1,252 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Failure classification, retry policies, and circuit breaking for LLM calls.
+
+This is the single source of truth for how ARTEMIS reacts to a failed model
+call.  Every provider/transport exception is converted into a ``Failure``
+carrying explicit recovery decisions, and every retry loop in the codebase is
+expected to consult ``retry_policy_for`` instead of inventing its own rules.
+
+The taxonomy was promoted from ``artemis.agents.video_analyzer.reliability``
+(which now extends it with media-specific categories) so that the same
+classification and circuit-breaker quality applies to every agent call, not
+just the video pipeline.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+from enum import StrEnum
+import random
+import threading
+import time
+from typing import Any
+
+
+class FailureCategory(StrEnum):
+    RATE_LIMIT = "rate_limit"
+    PROVIDER_UNAVAILABLE = "provider_unavailable"
+    TIMEOUT = "timeout"
+    CONNECTION = "connection"
+    AUTHENTICATION = "authentication"
+    BAD_REQUEST = "bad_request"
+    CANCELLED = "cancelled"
+    UNKNOWN = "unknown"
+
+
+#: Categories that count toward circuit-breaker state: transient provider-side
+#: trouble where hammering the endpoint makes things worse.
+TRANSIENT_CATEGORIES: frozenset[str] = frozenset(
+    {
+        FailureCategory.RATE_LIMIT.value,
+        FailureCategory.PROVIDER_UNAVAILABLE.value,
+        FailureCategory.TIMEOUT.value,
+        FailureCategory.CONNECTION.value,
+    }
+)
+
+
+@dataclass(frozen=True)
+class Failure:
+    category: FailureCategory
+    retryable: bool
+    should_fallback: bool
+
+
+def extract_status_code(error: BaseException) -> int | None:
+    """Best-effort extraction of an HTTP-ish status code from an exception."""
+    code = getattr(error, "code", None) or getattr(error, "status_code", None)
+    try:
+        return int(code) if code is not None else None
+    except (TypeError, ValueError):
+        return None
+
+
+_RATE_LIMIT_MARKERS = ("rate limit", "quota", "resource_exhausted")
+_UNAVAILABLE_MARKERS = ("unavailable", "overloaded", "high demand", "service unavailable")
+_AUTH_MARKERS = ("unauthorized", "forbidden", "api key", "credential")
+
+
+def classify_failure(error: BaseException) -> Failure:
+    """Convert a provider/transport exception into explicit recovery decisions."""
+    if isinstance(error, KeyboardInterrupt) or type(error).__name__ == "CancelledError":
+        return Failure(FailureCategory.CANCELLED, False, False)
+
+    code = extract_status_code(error)
+    message = str(error).lower()
+
+    if code == 429 or any(marker in message for marker in _RATE_LIMIT_MARKERS):
+        return Failure(FailureCategory.RATE_LIMIT, True, True)
+    if code in {500, 502, 503, 504} or any(marker in message for marker in _UNAVAILABLE_MARKERS):
+        return Failure(FailureCategory.PROVIDER_UNAVAILABLE, True, True)
+    if isinstance(error, TimeoutError) or "timed out" in message or "timeout" in message:
+        return Failure(FailureCategory.TIMEOUT, True, True)
+    if isinstance(error, (ConnectionError, OSError)):
+        return Failure(FailureCategory.CONNECTION, True, True)
+    if code in {401, 403} or any(marker in message for marker in _AUTH_MARKERS):
+        # Not retryable against the same endpoint, but a fallback endpoint may
+        # use a different (working) credential.
+        return Failure(FailureCategory.AUTHENTICATION, False, True)
+    if code in {400, 404, 413, 415, 422}:
+        # Programmer/request errors: retrying is useless and falling back only
+        # hides the bug inside a different model's output.
+        return Failure(FailureCategory.BAD_REQUEST, False, False)
+    return Failure(FailureCategory.UNKNOWN, True, True)
+
+
+# ---------------------------------------------------------------------------
+# Typed terminal errors
+# ---------------------------------------------------------------------------
+
+
+class LLMCallError(Exception):
+    """Base class for terminal LLM call failures raised by the gateway."""
+
+    def __init__(self, message: str, *, failure: Failure, cause: BaseException | None = None):
+        super().__init__(message)
+        self.failure = failure
+        if cause is not None:
+            self.__cause__ = cause
+
+
+class LLMPermanentError(LLMCallError):
+    """A non-retryable failure (auth, bad request). Retrying cannot help."""
+
+
+class LLMExhaustedError(LLMCallError):
+    """Retryable failures exhausted every attempt and the pause deadline."""
+
+
+# ---------------------------------------------------------------------------
+# Retry policies
+# ---------------------------------------------------------------------------
+
+
+@dataclass(frozen=True)
+class RetryPolicy:
+    """How many attempts a failure category deserves and how to space them."""
+
+    max_attempts: int
+    base_delay: float
+    max_delay: float
+
+    def delay_for(self, attempt: int) -> float:
+        """Backoff delay before retry number ``attempt`` (1-based)."""
+        delay = min(self.max_delay, self.base_delay * (2 ** max(0, attempt - 1)))
+        return delay + random.uniform(0.1, 0.5)
+
+
+_DEFAULT_RETRY_POLICIES: dict[FailureCategory, RetryPolicy] = {
+    FailureCategory.RATE_LIMIT: RetryPolicy(max_attempts=4, base_delay=10.0, max_delay=60.0),
+    FailureCategory.PROVIDER_UNAVAILABLE: RetryPolicy(
+        max_attempts=4, base_delay=5.0, max_delay=30.0
+    ),
+    FailureCategory.TIMEOUT: RetryPolicy(max_attempts=3, base_delay=2.0, max_delay=15.0),
+    FailureCategory.CONNECTION: RetryPolicy(max_attempts=3, base_delay=2.0, max_delay=15.0),
+    FailureCategory.UNKNOWN: RetryPolicy(max_attempts=2, base_delay=1.0, max_delay=5.0),
+    # Non-retryable categories get a single attempt by definition.
+    FailureCategory.AUTHENTICATION: RetryPolicy(max_attempts=1, base_delay=0.0, max_delay=0.0),
+    FailureCategory.BAD_REQUEST: RetryPolicy(max_attempts=1, base_delay=0.0, max_delay=0.0),
+    FailureCategory.CANCELLED: RetryPolicy(max_attempts=1, base_delay=0.0, max_delay=0.0),
+}
+
+
+def retry_policy_for(category: FailureCategory) -> RetryPolicy:
+    return _DEFAULT_RETRY_POLICIES.get(category, _DEFAULT_RETRY_POLICIES[FailureCategory.UNKNOWN])
+
+
+# ---------------------------------------------------------------------------
+# Circuit breaker
+# ---------------------------------------------------------------------------
+
+
+@dataclass
+class _CircuitState:
+    consecutive_failures: int = 0
+    open_until: float = 0.0
+    half_open_in_flight: bool = False
+
+
+class CircuitBreaker:
+    """Small circuit breaker keyed by provider/model identity.
+
+    Only transient categories (see ``TRANSIENT_CATEGORIES``) trip the breaker;
+    programmer errors and auth failures never open it.  ``failure`` may be any
+    object with a ``category`` attribute whose value matches this module's
+    category strings, so domain extensions (e.g. the video pipeline's failure
+    type) plug in without conversion.
+    """
+
+    def __init__(self, threshold: int = 3, cooldown_seconds: float = 60.0) -> None:
+        self.threshold = max(1, int(threshold))
+        self.cooldown_seconds = max(0.1, float(cooldown_seconds))
+        self._states: dict[str, _CircuitState] = {}
+        self._lock = threading.RLock()
+
+    def allow(self, key: str, *, now: float | None = None) -> bool:
+        current = time.time() if now is None else now
+        with self._lock:
+            state = self._states.get(key)
+            if state is None:
+                return True
+            if state.open_until:
+                if current < state.open_until or state.half_open_in_flight:
+                    return False
+                # Half-open: permit exactly one trial. Other concurrent calls
+                # continue to bypass the primary until it succeeds or fails.
+                state.half_open_in_flight = True
+                state.consecutive_failures = max(0, self.threshold - 1)
+                return True
+            return not state.half_open_in_flight
+
+    def open_remaining(self, key: str, *, now: float | None = None) -> float:
+        """Seconds until the circuit for ``key`` half-opens (0 if not open)."""
+        current = time.time() if now is None else now
+        with self._lock:
+            state = self._states.get(key)
+            if state is None or not state.open_until:
+                return 0.0
+            return max(0.0, state.open_until - current)
+
+    def record_success(self, key: str) -> None:
+        with self._lock:
+            self._states.pop(key, None)
+
+    def record_failure(self, key: str, failure: Any, *, now: float | None = None) -> None:
+        category = str(getattr(failure, "category", ""))
+        if category not in TRANSIENT_CATEGORIES:
+            with self._lock:
+                state = self._states.get(key)
+                if state is not None and state.half_open_in_flight:
+                    self._states.pop(key, None)
+            return
+        current = time.time() if now is None else now
+        with self._lock:
+            state = self._states.setdefault(key, _CircuitState())
+            state.half_open_in_flight = False
+            state.consecutive_failures += 1
+            if state.consecutive_failures >= self.threshold:
+                state.open_until = current + self.cooldown_seconds
+
+    def snapshot(self) -> dict[str, dict[str, Any]]:
+        with self._lock:
+            return {
+                key: {
+                    "consecutive_failures": state.consecutive_failures,
+                    "open_until": state.open_until,
+                    "half_open_in_flight": state.half_open_in_flight,
+                }
+                for key, state in self._states.items()
+            }

artemis/llm/structured.py

@@ -0,0 +1,194 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Unified structured-output parsing for LLM responses.
+
+Single pipeline replacing the ad-hoc JSON extraction scattered across agents:
+strip thinking tags → extract the JSON payload (fenced or balanced-span) →
+parse → tolerant repair pass → optional schema validation.  A failed parse
+returns a typed ``ParseFailure`` carrying the raw text and the reason — never
+a raw string masquerading as parsed data.
+"""
+
+from __future__ import annotations
+
+from dataclasses import dataclass
+import json
+import re
+from typing import Any
+
+from pydantic import BaseModel, ValidationError
+
+from artemis.llm.parser import extract_thinking_tags
+
+
+@dataclass(frozen=True)
+class ParseFailure:
+    """A parse miss: keeps the raw text and the reason, explicitly typed."""
+
+    raw: str
+    error: str
+
+
+class StructuredOutputError(ValueError):
+    """Raised when a required structured response could not be parsed."""
+
+    def __init__(self, failure: ParseFailure):
+        super().__init__(failure.error)
+        self.failure = failure
+
+
+def content_to_text(content: Any) -> str:
+    """Flatten a LangChain message content (str or block list) to plain text."""
+    if isinstance(content, str):
+        return content
+    if isinstance(content, list):
+        parts: list[str] = []
+        for item in content:
+            if isinstance(item, str):
+                parts.append(item)
+            elif isinstance(item, dict) and item.get("type") != "thinking":
+                text = item.get("text")
+                if isinstance(text, str):
+                    parts.append(text)
+        return "".join(parts)
+    return "" if content is None else str(content)
+
+
+_FENCE_RE = re.compile(r"```(?:json)?\s*([\s\S]*?)```", re.IGNORECASE)
+
+
+def _balanced_span(text: str) -> str | None:
+    """Return the first balanced {...} or [...] span, string/escape aware."""
+    start = None
+    for i, ch in enumerate(text):
+        if ch in "{[":
+            start = i
+            break
+    if start is None:
+        return None
+    stack: list[str] = []
+    in_string = False
+    escaped = False
+    for j in range(start, len(text)):
+        ch = text[j]
+        if in_string:
+            if escaped:
+                escaped = False
+            elif ch == "\\":
+                escaped = True
+            elif ch == '"':
+                in_string = False
+            continue
+        if ch == '"':
+            in_string = True
+        elif ch in "{[":
+            stack.append(ch)
+        elif ch in "}]":
+            if not stack or (stack[-1], ch) not in {("{", "}"), ("[", "]")}:
+                return None
+            stack.pop()
+            if not stack:
+                return text[start : j + 1]
+    return None
+
+
+def extract_json_candidate(text: str) -> str | None:
+    """Locate the JSON payload inside prose: fenced blocks first, then bare spans."""
+    text = text.strip()
+    for match in _FENCE_RE.finditer(text):
+        span = _balanced_span(match.group(1).strip())
+        if span:
+            return span
+    return _balanced_span(text)
+
+
+def repair_json(text: str) -> str:
+    """Remove comment and trailing-comma noise outside of strings."""
+    out: list[str] = []
+    i = 0
+    n = len(text)
+    in_string = False
+    escaped = False
+    while i < n:
+        ch = text[i]
+        if in_string:
+            out.append(ch)
+            if escaped:
+                escaped = False
+            elif ch == "\\":
+                escaped = True
+            elif ch == '"':
+                in_string = False
+            i += 1
+            continue
+        if ch == '"':
+            in_string = True
+            out.append(ch)
+            i += 1
+            continue
+        if ch == "/" and i + 1 < n and text[i + 1] == "/":
+            while i < n and text[i] != "\n":
+                i += 1
+            continue
+        if ch == "/" and i + 1 < n and text[i + 1] == "*":
+            i += 2
+            while i + 1 < n and not (text[i] == "*" and text[i + 1] == "/"):
+                i += 1
+            i += 2
+            continue
+        if ch == ",":
+            k = i + 1
+            while k < n and text[k] in " \t\r\n":
+                k += 1
+            if k < n and text[k] in "]}":
+                i += 1
+                continue
+        out.append(ch)
+        i += 1
+    return "".join(out)
+
+
+def parse_structured(text: Any, schema: type[BaseModel] | None = None) -> Any | ParseFailure:
+    """Parse an LLM response into JSON data (optionally schema-validated).
+
+    Returns the parsed object (or a validated ``schema`` instance), or a
+    ``ParseFailure`` describing why the text could not be parsed. Callers must
+    check ``isinstance(result, ParseFailure)`` — the raw text is never
+    silently passed through as a result.
+    """
+    if not isinstance(text, str):
+        text = content_to_text(text)
+    if not text or not text.strip():
+        return ParseFailure(raw=text or "", error="empty response")
+
+    cleaned, _ = extract_thinking_tags(text)
+    candidate = extract_json_candidate(cleaned)
+    if candidate is None:
+        return ParseFailure(raw=text, error="no JSON object or array found in response")
+
+    last_error = ""
+    for variant in (candidate, repair_json(candidate)):
+        try:
+            data = json.loads(variant)
+        except json.JSONDecodeError as decode_error:
+            last_error = str(decode_error)
+            continue
+        if schema is not None:
+            try:
+                return schema.model_validate(data)
+            except ValidationError as validation_error:
+                return ParseFailure(raw=text, error=f"schema validation failed: {validation_error}")
+        return data
+    return ParseFailure(raw=text, error=f"invalid JSON: {last_error}")

artemis/mcp/action_executor.py

@@ -42,6 +42,7 @@ from artemis.context import ArtemisContext
 from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.data_engine.trace import TraceSpan
 from artemis.mcp.action_manifest import OPTIONAL_ACTIONS, REQUIRED_ACTIONS
+from artemis.mcp.action_specs import exception_prefix
 from artemis.mcp.action_session import ActionSession, get_action_session
 from artemis.mcp.action_types import ActionCode, ActionResult
 from artemis.mcp.actuators.adb import AdbActuator
@@ -66,19 +67,6 @@ logger = get_logger(__name__)
 
 __all__ = ["McpActionExecutor"]
 
-# Wording for wrapped exceptions; matches the historical executor `except` arms.
-_EXCEPTION_PREFIX = {
-    "click": "Error during click",
-    "click_sequence": "Error executing click sequence",
-    "long_press": "Error during long press",
-    "input_text": "Error during input text",
-    "swipe": "Error during swipe",
-    "press_key": "Error during press_key",
-    "manage_app": "Error during manage_app",
-    "wait_for_delay": "Error during wait_for_delay",
-    "wait_for_text": "Error during wait_for_text",
-}
-
 
 class _ArgError(ValueError):
     """Argument-translation failure whose message is already fully formatted."""
@@ -94,19 +82,14 @@ class McpActionExecutor:
         actuator: AdbActuator | None = None,
     ):
         self.ctx = ctx
-        self.actuator = (
-            actuator or getattr(ctx, "actuator", None) or AdbActuator(ctx, controller)
-        )
+        self.actuator = actuator or getattr(ctx, "actuator", None) or AdbActuator(ctx, controller)
         self.controller = self.actuator.controller
         self._session: ActionSession | None = None
 
     @property
     def action_tool_names(self) -> frozenset[str]:
         """Device actions plus backend extension names -- the dynamic dispatch set."""
-        return (
-            (REQUIRED_ACTIONS | OPTIONAL_ACTIONS)
-            | {e.name for e in self.actuator.extensions()}
-        )
+        return (REQUIRED_ACTIONS | OPTIONAL_ACTIONS) | {e.name for e in self.actuator.extensions()}
 
     async def _session_or_start(self) -> ActionSession:
         if self._session is None or not self._session.started:
@@ -129,9 +112,7 @@ class McpActionExecutor:
             return await self._execute_agent_tool(raw_name, name, args, tool_call_id, state)
 
         if raw_name in self.action_tool_names:
-            return await self._execute_device_action(
-                raw_name, name, args, tool_call_id, state
-            )
+            return await self._execute_device_action(raw_name, name, args, tool_call_id, state)
 
         return ToolExecutionResult(
             tool_call_id=tool_call_id,
@@ -188,15 +169,12 @@ class McpActionExecutor:
                             try:
                                 img_bytes = Path(shot_path).read_bytes()
                             except Exception as read_err:
-                                logger.warning(
-                                    f"Failed to read observed screenshot: {read_err}"
-                                )
+                                logger.warning(f"Failed to read observed screenshot: {read_err}")
             except _ArgError as e:
                 res = ActionResult.failure(raw_name, str(e), code=ActionCode.INVALID_ARGS)
                 message = str(e)
             except Exception as e:
-                prefix = _EXCEPTION_PREFIX.get(raw_name, f"Error during {raw_name}")
-                message = f"{prefix}: {e}"
+                message = f"{exception_prefix(raw_name)}: {e}"
                 res = ActionResult.failure(raw_name, message, detail=repr(e))
 
             if shot_path:
@@ -365,9 +343,7 @@ class McpActionExecutor:
                     pass
 
         if not isinstance(sequence, (list, tuple)):
-            raise _ArgError(
-                f"Error during click sequence: Invalid sequence format: {sequence}"
-            )
+            raise _ArgError(f"Error during click sequence: Invalid sequence format: {sequence}")
 
         width = getattr(self.ctx.device, "device_width", 1080) if self.ctx.device else 1080
         height = getattr(self.ctx.device, "device_height", 2400) if self.ctx.device else 2400
@@ -389,15 +365,11 @@ class McpActionExecutor:
             elif isinstance(target, (list, tuple)) and len(target) == 2:
                 nx, ny = int(target[0]), int(target[1])
             else:
-                raise _ArgError(
-                    f"Error during click sequence: Invalid target format: {raw_target}"
-                )
+                raise _ArgError(f"Error during click sequence: Invalid target format: {raw_target}")
             resolved.append([nx, ny])
         return resolved
 
-    def _translate_swipe(
-        self, args: dict[str, Any], state: Any
-    ) -> tuple[str, dict[str, Any], Any]:
+    def _translate_swipe(self, args: dict[str, Any], state: Any) -> tuple[str, dict[str, Any], Any]:
         width = getattr(self.ctx.device, "device_width", 1080) if self.ctx.device else 1080
         height = getattr(self.ctx.device, "device_height", 2400) if self.ctx.device else 2400
         default_duration = args.get("duration", 400)
@@ -411,7 +383,7 @@ class McpActionExecutor:
                 direction=target,
                 target=args.get("target"),
                 indexed_elements=getattr(state, "indexed_elements", None) if state else None,
-                ui_hierarchy=getattr(state, "ui_tree", None) if state else None,
+                ui_hierarchy=getattr(state, "latest_ui_hierarchy", None) if state else None,
                 width=width,
                 height=height,
                 duration=final_duration,
@@ -441,10 +413,7 @@ class McpActionExecutor:
             def finalize(res: ActionResult) -> str:
                 if not res.ok:
                     return f"Error dragging: {res.detail}"
-                return (
-                    "Swipe completed successfully. Swiped from"
-                    f" [{x1}, {y1}] to [{x2}, {y2}]."
-                )
+                return f"Swipe completed successfully. Swiped from [{x1}, {y1}] to [{x2}, {y2}]."
 
             return (
                 "swipe",
@@ -494,9 +463,7 @@ class McpActionExecutor:
             text_summary=text,
         )
 
-    async def _read_note(
-        self, key: str, start_line: int | None, end_line: int | None
-    ) -> str:
+    async def _read_note(self, key: str, start_line: int | None, end_line: int | None) -> str:
         try:
             base_dir = self.ctx.data_engine.base_dir if self.ctx.data_engine else None
             if not base_dir:
@@ -514,9 +481,7 @@ class McpActionExecutor:
         except Exception as e:
             return format_list_notes_failure(str(e))
 
-    async def _ask_explorer(
-        self, query: str, context_feedback: str | None, state: Any
-    ) -> str:
+    async def _ask_explorer(self, query: str, context_feedback: str | None, state: Any) -> str:
         # Imported lazily: the explorer stack is heavy and recursive (it spawns an LLM
         # sub-agent), which is also why it must never live behind the action server.
         from artemis.tools.explorer_tool import _run_explorer_logic

artemis/mcp/action_manifest.py

@@ -122,7 +122,6 @@ BACKEND_INDEPENDENT_TOOLS: frozenset[str] = frozenset(
         "run_adb_command",
         "manage_task",
         "analyze_task_output",
-        "reply_to_checker",
         "report_task_status",
         "report_failure_analysis",
     }
@@ -289,8 +288,7 @@ def filter_declarations(
         for decl in declarations
         # Only device actions are gated. Backend-independent tools and anything the
         # manifest does not classify (e.g. an agent's own bespoke tool) pass through.
-        if _declaration_name(decl) not in DEVICE_ACTIONS
-        or _declaration_name(decl) in available
+        if _declaration_name(decl) not in DEVICE_ACTIONS or _declaration_name(decl) in available
     ]
 
     for ext in actuator.extensions():

artemis/mcp/action_server.py

@@ -37,6 +37,11 @@ from mcp.server.fastmcp import FastMCP
 from mcp.types import CallToolResult, ImageContent, TextContent
 
 from artemis.mcp.action_manifest import validate_actuator
+from artemis.mcp.action_specs import (
+    exception_prefix,
+    make_wire_handler,
+    wire_dialects,
+)
 from artemis.mcp.action_types import ActionCode, ActionResult, ObserveResult
 from artemis.mcp.actuators.base import Actuator
 from artemis.mcp.observation import observe as observe_impl
@@ -46,19 +51,6 @@ logger = get_logger(__name__)
 
 __all__ = ["build_action_server"]
 
-# Exception wrapping per tool; wording matches the historical executor `except` arms.
-_EXCEPTION_PREFIX = {
-    "click": "Error during click",
-    "click_sequence": "Error executing click sequence",
-    "long_press": "Error during long press",
-    "input_text": "Error during input text",
-    "swipe": "Error during swipe",
-    "press_key": "Error during press_key",
-    "manage_app": "Error during manage_app",
-    "wait_for_delay": "Error during wait_for_delay",
-    "wait_for_text": "Error during wait_for_text",
-}
-
 
 def _wrap(res: ActionResult) -> CallToolResult:
     return CallToolResult(
@@ -69,9 +61,8 @@ def _wrap(res: ActionResult) -> CallToolResult:
 
 
 def _wrap_exception(action: str, exc: Exception) -> CallToolResult:
-    prefix = _EXCEPTION_PREFIX.get(action, f"Error during {action}")
     return _wrap(
-        ActionResult.failure(action, f"{prefix}: {exc}", detail=repr(exc))
+        ActionResult.failure(action, f"{exception_prefix(action)}: {exc}", detail=repr(exc))
     )
 
 
@@ -86,166 +77,17 @@ def build_action_server(actuator: Actuator, name: str = "artemis_actions") -> Fa
     mcp = FastMCP(name)
 
     # --- Device actions (registered only when implemented) ---------------------------
+    # Declarations and argument conversion both come from the canonical manifest
+    # (artemis/mcp/action_specs.py); this server contributes only the transport
+    # wrapping, so the served schema cannot drift from the manifest.
 
-    if "click" in caps:
-
-        @mcp.tool()
-        async def click(
-            target: list[int], times: int = 1, delay_ms: int = 100
-        ) -> Annotated[CallToolResult, ActionResult]:
-            """Tap at a 0-1000 normalized [x, y] coordinate."""
-            try:
-                return _wrap(
-                    await actuator.click(
-                        int(target[0]), int(target[1]), times=times, delay_ms=delay_ms
-                    )
-                )
-            except Exception as e:
-                return _wrap_exception("click", e)
-
-    if "click_sequence" in caps:
-
-        @mcp.tool()
-        async def click_sequence(
-            sequence: list[list[int]], delay_ms: int = 50
-        ) -> Annotated[CallToolResult, ActionResult]:
-            """Tap a series of 0-1000 normalized [x, y] points in one atomic burst."""
-            try:
-                points = [(int(p[0]), int(p[1])) for p in sequence]
-                return _wrap(await actuator.click_sequence(points, delay_ms=delay_ms))
-            except Exception as e:
-                return _wrap_exception("click_sequence", e)
-
-    if "long_press" in caps:
-
-        @mcp.tool()
-        async def long_press(
-            target: list[int], duration_ms: int = 1000
-        ) -> Annotated[CallToolResult, ActionResult]:
-            """Long-press at a 0-1000 normalized [x, y] coordinate."""
-            try:
-                return _wrap(
-                    await actuator.long_press(
-                        int(target[0]), int(target[1]), duration_ms=duration_ms
-                    )
-                )
-            except Exception as e:
-                return _wrap_exception("long_press", e)
-
-    if "input_text" in caps:
-
-        @mcp.tool()
-        async def input_text(
-            text: str,
-            target: list[int] | None = None,
-            clear_exist: bool = True,
-        ) -> Annotated[CallToolResult, ActionResult]:
-            """Type text, optionally focusing a 0-1000 normalized [x, y] target first."""
-            try:
-                norm = (int(target[0]), int(target[1])) if target else None
-                return _wrap(
-                    await actuator.input_text(text, norm, clear_exist=clear_exist)
-                )
-            except Exception as e:
-                return _wrap_exception("input_text", e)
-
-    if "swipe" in caps:
-
-        @mcp.tool()
-        async def swipe(
-            start: list[int], end: list[int], duration_ms: int = 800
-        ) -> Annotated[CallToolResult, ActionResult]:
-            """Swipe between two 0-1000 normalized [x, y] points."""
-            try:
-                return _wrap(
-                    await actuator.swipe(
-                        (int(start[0]), int(start[1])),
-                        (int(end[0]), int(end[1])),
-                        duration_ms,
-                    )
-                )
-            except Exception as e:
-                return _wrap_exception("swipe", e)
-
-    if "press_key" in caps:
-
-        @mcp.tool()
-        async def press_key(key: str) -> Annotated[CallToolResult, ActionResult]:
-            """Press a device key (home, back, enter, delete, tab, search, menu, app_switch)."""
-            try:
-                return _wrap(await actuator.press_key(key))
-            except Exception as e:
-                return _wrap_exception("press_key", e)
-
-    if "manage_app" in caps:
-
-        @mcp.tool()
-        async def manage_app(
-            action: str, app_name: str
-        ) -> Annotated[CallToolResult, ActionResult]:
-            """Launch or stop an app by human-readable name or package."""
-            try:
-                return _wrap(await actuator.manage_app(action, app_name))
-            except Exception as e:
-                return _wrap_exception("manage_app", e)
-
-    if "wait_for_delay" in caps:
-
-        @mcp.tool()
-        async def wait_for_delay(
-            time_in_ms: int,
-        ) -> Annotated[CallToolResult, ActionResult]:
-            """Wait for a fixed number of milliseconds."""
-            try:
-                return _wrap(await actuator.wait_for_delay(time_in_ms))
-            except Exception as e:
-                return _wrap_exception("wait_for_delay", e)
-
-    if "wait_for_text" in caps:
-
-        @mcp.tool()
-        async def wait_for_text(
-            text: str, wait_state: str = "appear", timeout_ms: int = 5000
-        ) -> Annotated[CallToolResult, ActionResult]:
-            """Wait for text to appear on or disappear from the screen."""
-            try:
-                return _wrap(await actuator.wait_for_text(text, wait_state, timeout_ms))
-            except Exception as e:
-                return _wrap_exception("wait_for_text", e)
-
-    if "open_link" in caps:
-
-        @mcp.tool()
-        async def open_link(url: str) -> Annotated[CallToolResult, ActionResult]:
-            """Open a URL on the device."""
-            try:
-                return _wrap(await actuator.open_link(url))
-            except Exception as e:
-                return _wrap_exception("open_link", e)
-
-    if "erase_one_char" in caps:
-
-        @mcp.tool()
-        async def erase_one_char() -> Annotated[CallToolResult, ActionResult]:
-            """Erase a single character in the focused field."""
-            try:
-                return _wrap(await actuator.erase_one_char())
-            except Exception as e:
-                return _wrap_exception("erase_one_char", e)
-
-    if "focus_and_clear_text" in caps:
-
-        @mcp.tool()
-        async def focus_and_clear_text(
-            target: list[int],
-        ) -> Annotated[CallToolResult, ActionResult]:
-            """Focus the field at a 0-1000 normalized [x, y] and clear its text."""
-            try:
-                return _wrap(
-                    await actuator.focus_and_clear_text(int(target[0]), int(target[1]))
-                )
-            except Exception as e:
-                return _wrap_exception("focus_and_clear_text", e)
+    for spec in wire_dialects():
+        if spec.name in caps:
+            mcp.add_tool(
+                make_wire_handler(spec, actuator, _wrap, _wrap_exception),
+                name=spec.name,
+                description=spec.wire.description,
+            )
 
     # --- Internal observation tools (never declared to an LLM) -----------------------
 
@@ -315,7 +157,9 @@ def build_action_server(actuator: Actuator, name: str = "artemis_actions") -> Fa
             except Exception as e:
                 payload = {"ok": False, "error": str(e)}
             return CallToolResult(
-                content=[TextContent(type="text", text="screenshot" if payload["ok"] else str(payload))],
+                content=[
+                    TextContent(type="text", text="screenshot" if payload["ok"] else str(payload))
+                ],
                 structuredContent=payload,
                 isError=False,
             )
@@ -332,11 +176,16 @@ def build_action_server(actuator: Actuator, name: str = "artemis_actions") -> Fa
                 elements = await coro
                 payload: dict[str, Any] = {"ok": True, "elements": elements}
             except TimeoutError:
-                payload = {"ok": False, "error": f"ui hierarchy fetch timed out after {timeout_ms}ms"}
+                payload = {
+                    "ok": False,
+                    "error": f"ui hierarchy fetch timed out after {timeout_ms}ms",
+                }
             except Exception as e:
                 payload = {"ok": False, "error": str(e)}
             return CallToolResult(
-                content=[TextContent(type="text", text="ui_hierarchy" if payload["ok"] else str(payload))],
+                content=[
+                    TextContent(type="text", text="ui_hierarchy" if payload["ok"] else str(payload))
+                ],
                 structuredContent=payload,
                 isError=False,
             )

artemis/mcp/action_specs.py

@@ -0,0 +1,961 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""The canonical device-action manifest: every dialect of every action, side by side.
+
+Historically each device action was declared independently at three model-facing
+sites -- the Operator's inline LangChain shells, the Validator/Flash/FailureAnalyzer
+``ToolDeclaration`` constants, and the action MCP server's function signatures -- and
+the declarations drifted apart (``click.target`` took an element index in one place
+and only coordinates in another). This module is now the single place all three are
+*defined*; the historical sites import their surface from here, generated.
+
+An action has up to three dialects, and the differences between them are **deliberate
+and declared**, not accidental:
+
+* ``operator`` -- the LangChain shell tools the Operator binds. The Operator receives
+  an indexed "Visible UI Elements" list and its prompt teaches index-first targeting,
+  so target parameters accept ``int`` element indices as well as normalized
+  coordinates. The shells are declaration-only ("Action Recorded"): the Operator's
+  translate step lowers them into structured decisions.
+* ``declaration`` -- the JSON ``ToolDeclaration`` bound by the Validator's
+  FailureAnalyzer and the FlashRunner. These agents act on coordinates directly (the
+  element list they see carries coordinates), so target parameters are coordinate
+  pairs and the wording teaches the act-then-observe loop ("The screen after X will
+  be returned automatically").
+* ``wire`` -- what the action MCP server serves, mirroring the actuator protocol
+  one-to-one: normalized coordinates, millisecond durations, no addressing sugar.
+  Client-side executors (``McpActionExecutor``, ``MobileActionExecutor``) lower the
+  agent dialects onto it.
+
+Each ``ActionSpec.param_bridge`` maps operator parameter names onto declaration
+parameter names, so every rename (``duration`` -> ``duration_ms``, ``gesture`` ->
+``action``) is written down exactly once. ``tests/unit/mcp/test_action_specs.py``
+re-derives the bridge from the generated schemas, so an undeclared divergence --
+today's definition of drift -- fails CI instead of shipping.
+
+The classification of actions (required/optional/internal) stays in
+``action_manifest``; this module holds their schemas and teaching text.
+"""
+
+from collections.abc import Awaitable, Callable
+from dataclasses import dataclass, field
+from functools import cache
+import inspect
+from typing import Annotated, Any, Literal
+
+from langchain_core.tools import StructuredTool
+from mcp.types import CallToolResult
+from pydantic import Field, create_model
+
+from artemis.core.tool_declaration import ToolDeclaration
+from artemis.mcp.action_types import ActionResult
+
+__all__ = [
+    "ACTION_SPECS",
+    "ActionSpec",
+    "DeclarationDialect",
+    "EXCEPTION_PREFIXES",
+    "OPERATOR_SHELL_ORDER",
+    "OperatorDialect",
+    "ParamSpec",
+    "WireDialect",
+    "exception_prefix",
+    "make_wire_handler",
+    "operator_shell_tool",
+    "tool_declaration",
+    "wire_dialects",
+]
+
+
+# --- Spec structure ------------------------------------------------------------------
+
+
+@dataclass(frozen=True)
+class ParamSpec:
+    """One parameter of a Python-typed dialect (operator shell or wire)."""
+
+    name: str
+    annotation: Any
+    description: str | None = None
+    required: bool = True
+    default: Any = None
+
+
+@dataclass(frozen=True)
+class OperatorDialect:
+    """The Operator's LangChain shell: index-capable targets, declaration-only body."""
+
+    description: str
+    params: tuple[ParamSpec, ...]
+
+
+@dataclass(frozen=True)
+class DeclarationDialect:
+    """The Validator/Flash/FailureAnalyzer JSON declaration, wording preserved."""
+
+    description: str
+    parameters: dict[str, Any]
+
+
+@dataclass(frozen=True)
+class WireDialect:
+    """The action-server tool mirroring one actuator method.
+
+    ``bind`` receives the actuator and the validated, default-filled argument dict
+    and performs the (tiny) call-site conversion the historical hand-written wrapper
+    performed -- typically ``int()`` coercion of coordinate pairs.
+    """
+
+    description: str
+    params: tuple[ParamSpec, ...]
+    bind: Callable[[Any, dict[str, Any]], Awaitable[ActionResult]]
+
+
+@dataclass(frozen=True)
+class ActionSpec:
+    """All dialects of one canonical device action, plus their declared differences.
+
+    Attributes:
+        name: The canonical tool name every dialect shares.
+        operator: Operator shell dialect, or ``None`` when the Operator does not bind
+            this action.
+        declaration: Validator/Flash dialect, or ``None`` when no JSON declaration
+            exists (an action can be wire-only, reachable through executors but never
+            declared to a model directly).
+        wire: Action-server dialect; ``None`` only for purely virtual actions.
+        param_bridge: operator param name -> declaration param name, for every
+            operator param, when both dialects exist. Identical names map to
+            themselves; renames are the deliberate dialect differences.
+        differences: Human-readable record of every deliberate cross-dialect
+            divergence (semantics, not just spelling). Empty means the dialects
+            align modulo the documented dialect philosophy above.
+    """
+
+    name: str
+    operator: OperatorDialect | None = None
+    declaration: DeclarationDialect | None = None
+    wire: WireDialect | None = None
+    param_bridge: dict[str, str] = field(default_factory=dict)
+    differences: str = ""
+
+
+# --- Shared type aliases -------------------------------------------------------------
+
+#: Operator-dialect target: element index into the indexed UI list, or a normalized
+#: [x, y] pair. Only the Operator sees this union; see the module docstring.
+IndexOrCoords = int | list[int]
+
+Direction = Literal["up", "down", "left", "right"]
+
+_COORD_PAIR_JSON = {
+    "type": "array",
+    "items": {"type": "integer"},
+}
+
+
+# --- Wire bindings -------------------------------------------------------------------
+# Each binding reproduces the exact conversion of the historical hand-written FastMCP
+# wrapper it replaces. Arguments arrive validated and default-filled.
+
+
+async def _wire_click(actuator: Any, a: dict[str, Any]) -> ActionResult:
+    return await actuator.click(
+        int(a["target"][0]), int(a["target"][1]), times=a["times"], delay_ms=a["delay_ms"]
+    )
+
+
+async def _wire_click_sequence(actuator: Any, a: dict[str, Any]) -> ActionResult:
+    points = [(int(p[0]), int(p[1])) for p in a["sequence"]]
+    return await actuator.click_sequence(points, delay_ms=a["delay_ms"])
+
+
+async def _wire_long_press(actuator: Any, a: dict[str, Any]) -> ActionResult:
+    return await actuator.long_press(
+        int(a["target"][0]), int(a["target"][1]), duration_ms=a["duration_ms"]
+    )
+
+
+async def _wire_input_text(actuator: Any, a: dict[str, Any]) -> ActionResult:
+    target = a["target"]
+    norm = (int(target[0]), int(target[1])) if target else None
+    return await actuator.input_text(a["text"], norm, clear_exist=a["clear_exist"])
+
+
+async def _wire_swipe(actuator: Any, a: dict[str, Any]) -> ActionResult:
+    start, end = a["start"], a["end"]
+    return await actuator.swipe(
+        (int(start[0]), int(start[1])), (int(end[0]), int(end[1])), a["duration_ms"]
+    )
+
+
+async def _wire_press_key(actuator: Any, a: dict[str, Any]) -> ActionResult:
+    return await actuator.press_key(a["key"])
+
+
+async def _wire_manage_app(actuator: Any, a: dict[str, Any]) -> ActionResult:
+    return await actuator.manage_app(a["action"], a["app_name"])
+
+
+async def _wire_wait_for_delay(actuator: Any, a: dict[str, Any]) -> ActionResult:
+    return await actuator.wait_for_delay(a["time_in_ms"])
+
+
+async def _wire_wait_for_text(actuator: Any, a: dict[str, Any]) -> ActionResult:
+    return await actuator.wait_for_text(a["text"], a["wait_state"], a["timeout_ms"])
+
+
+async def _wire_open_link(actuator: Any, a: dict[str, Any]) -> ActionResult:
+    return await actuator.open_link(a["url"])
+
+
+async def _wire_erase_one_char(actuator: Any, a: dict[str, Any]) -> ActionResult:
+    return await actuator.erase_one_char()
+
+
+async def _wire_focus_and_clear_text(actuator: Any, a: dict[str, Any]) -> ActionResult:
+    return await actuator.focus_and_clear_text(int(a["target"][0]), int(a["target"][1]))
+
+
+# --- The manifest --------------------------------------------------------------------
+
+_SPECS: tuple[ActionSpec, ...] = (
+    ActionSpec(
+        name="click",
+        operator=OperatorDialect(
+            description=(
+                "[ACTION] Click on the target location on the screen (supports element"
+                " index or absolute normalized coordinates)."
+            ),
+            params=(
+                ParamSpec(
+                    "target",
+                    IndexOrCoords,
+                    "Click target. Can be an element index number (int, e.g. 3) OR"
+                    " normalized coordinates (list of 2 integers, e.g. [500, 600]).",
+                ),
+                ParamSpec(
+                    "times",
+                    int,
+                    "Number of consecutive clicks on this target. Use this for"
+                    " double-clicks or multi-clicks (e.g. 7 to enter developer"
+                    " mode). Default is 1.",
+                    required=False,
+                    default=1,
+                ),
+                ParamSpec(
+                    "delay_ms",
+                    int,
+                    "Delay in milliseconds between consecutive clicks. Default is 100.",
+                    required=False,
+                    default=100,
+                ),
+            ),
+        ),
+        declaration=DeclarationDialect(
+            description=(
+                "[ACTION] Tap/click on the target coordinate on the screen. The screen"
+                " after click will be returned automatically. For buttons, checkboxes,"
+                " tabs, icons, items in a list, tap ON the element. For text fields /"
+                " search bars / input boxes, tap INSIDE the input box to focus it."
+            ),
+            parameters={
+                "type": "object",
+                "properties": {
+                    "target": {
+                        "type": "array",
+                        "items": {"type": "integer"},
+                        "description": "Normalized coordinates [x, y] in 0-1000 scale.",
+                    },
+                    "times": {
+                        "type": "integer",
+                        "description": (
+                            "Number of taps to perform (default 1). Set to 2 for double click."
+                        ),
+                    },
+                    "delay_ms": {
+                        "type": "integer",
+                        "description": "Delay between taps in milliseconds (default 100).",
+                    },
+                },
+                "required": ["target"],
+            },
+        ),
+        wire=WireDialect(
+            description="Tap at a 0-1000 normalized [x, y] coordinate.",
+            params=(
+                ParamSpec("target", list[int]),
+                ParamSpec("times", int, required=False, default=1),
+                ParamSpec("delay_ms", int, required=False, default=100),
+            ),
+            bind=_wire_click,
+        ),
+        param_bridge={"target": "target", "times": "times", "delay_ms": "delay_ms"},
+        differences=(
+            "target: operator accepts an element index or a coordinate pair; the"
+            " declaration and wire dialects take coordinates only."
+        ),
+    ),
+    ActionSpec(
+        name="click_sequence",
+        declaration=DeclarationDialect(
+            description=(
+                "[ACTION] Executes a sequence of taps one by one in order on the"
+                " specified targets (e.g. [[500, 280], [885, 362]]). The screen will be returned ONLY after all clicks"
+                " in the sequence have completed."
+            ),
+            parameters={
+                "type": "object",
+                "properties": {
+                    "sequence": {
+                        "type": "array",
+                        "items": {
+                            "type": "array",
+                            "items": {"type": "integer"},
+                            "description": (
+                                "Normalized coordinates [x, y] in 0-1000 scale (e.g., [500, 280]),"
+                                " or a single integer element index."
+                            ),
+                        },
+                        "description": (
+                            "List of targets to tap in sequence, e.g. [[500, 280], [885, 362]]."
+                        ),
+                    },
+                    "delay_ms": {
+                        "type": "integer",
+                        "description": (
+                            "Delay between consecutive taps in milliseconds (default 50ms)."
+                        ),
+                    },
+                },
+                "required": ["sequence"],
+            },
+        ),
+        wire=WireDialect(
+            description=("Tap a series of 0-1000 normalized [x, y] points in one atomic burst."),
+            params=(
+                ParamSpec("sequence", list[list[int]]),
+                ParamSpec("delay_ms", int, required=False, default=50),
+            ),
+            bind=_wire_click_sequence,
+        ),
+        differences=(
+            "Declared to Flash and the disappeared-target FailureAnalyzer strategies"
+            " only; sequence entries may be element indices at the declaration layer"
+            " (resolved client-side), while the wire takes coordinate pairs only."
+        ),
+    ),
+    ActionSpec(
+        name="long_press",
+        operator=OperatorDialect(
+            description=(
+                "[ACTION] Long press on the target location on the screen (supports"
+                " element index or absolute normalized coordinates)."
+            ),
+            params=(
+                ParamSpec(
+                    "target",
+                    IndexOrCoords,
+                    "Long press target. Can be an element index number (int, e.g."
+                    " 3) OR normalized coordinates (list of 2 integers, e.g. [500,"
+                    " 600]).",
+                ),
+                ParamSpec(
+                    "duration",
+                    int,
+                    "Long press duration in milliseconds (default 1000).",
+                    required=False,
+                    default=1000,
+                ),
+            ),
+        ),
+        declaration=DeclarationDialect(
+            description=(
+                "[ACTION] Long press on a target coordinate on the screen. The screen"
+                " after long pressing will be returned automatically."
+            ),
+            parameters={
+                "type": "object",
+                "properties": {
+                    "target": {
+                        "type": "array",
+                        "items": {"type": "integer"},
+                        "description": "Normalized coordinates [x, y] in 0-1000 scale.",
+                    },
+                    "duration_ms": {
+                        "type": "integer",
+                        "description": "Duration of press in milliseconds (default 1000ms).",
+                    },
+                },
+                "required": ["target"],
+            },
+        ),
+        wire=WireDialect(
+            description="Long-press at a 0-1000 normalized [x, y] coordinate.",
+            params=(
+                ParamSpec("target", list[int]),
+                ParamSpec("duration_ms", int, required=False, default=1000),
+            ),
+            bind=_wire_long_press,
+        ),
+        param_bridge={"target": "target", "duration": "duration_ms"},
+        differences=(
+            "target: operator accepts an element index or a coordinate pair."
+            " duration: the operator spells the duration parameter `duration` (its"
+            " structured decisions and recorded traces carry that key); the"
+            " declaration and wire dialects spell it `duration_ms`. Executors accept"
+            " both spellings."
+        ),
+    ),
+    ActionSpec(
+        name="input_text",
+        operator=OperatorDialect(
+            description=(
+                "[ACTION] Type text into the target input field (supports replacing"
+                " whole text or appending to the end, and multi-line strings with"
+                " '\\n')."
+            ),
+            params=(
+                ParamSpec(
+                    "text",
+                    str,
+                    "The text content to input. Supports multi-line content with '\\n'.",
+                ),
+                ParamSpec(
+                    "target",
+                    IndexOrCoords,
+                    "Input target field. Can be an input box element index number"
+                    " (int, e.g. 3) OR normalized coordinates (list of 2 integers,"
+                    " e.g. [500, 600]).",
+                ),
+                ParamSpec(
+                    "clear_exist",
+                    bool,
+                    "Whether to clear existing text before typing. True (default):"
+                    " clear/replace entire text. False: append at the end of"
+                    " existing content.",
+                    required=False,
+                    default=True,
+                ),
+            ),
+        ),
+        declaration=DeclarationDialect(
+            description=(
+                "[ACTION] Type text into an input field on the screen. The screen after"
+                " typing will be returned automatically. Automatically taps inside the"
+                " input box at target [x, y] to focus before typing."
+            ),
+            parameters={
+                "type": "object",
+                "properties": {
+                    "text": {
+                        "type": "string",
+                        "description": "Text to type into the focused input field.",
+                    },
+                    "target": {
+                        "type": "array",
+                        "items": {"type": "integer"},
+                        "description": "Coordinates [x, y] of the input box in 0-1000 scale.",
+                    },
+                    "clear_exist": {
+                        "type": "boolean",
+                        "description": (
+                            "Whether to clear existing text in the input box before typing"
+                            " (default True)."
+                        ),
+                    },
+                },
+                "required": ["text", "target"],
+            },
+        ),
+        wire=WireDialect(
+            description=("Type text, optionally focusing a 0-1000 normalized [x, y] target first."),
+            params=(
+                ParamSpec("text", str),
+                ParamSpec("target", list[int] | None, required=False, default=None),
+                ParamSpec("clear_exist", bool, required=False, default=True),
+            ),
+            bind=_wire_input_text,
+        ),
+        param_bridge={"target": "target", "text": "text", "clear_exist": "clear_exist"},
+        differences=(
+            "target: operator accepts an element index or a coordinate pair, and"
+            " requires a target; the wire dialect allows omitting the target to type"
+            " into the already-focused field."
+        ),
+    ),
+    ActionSpec(
+        name="swipe",
+        operator=OperatorDialect(
+            description=(
+                "[ACTION] Perform a swipe, drag, or slider-adjustment gesture on the screen.\n"
+                "\n"
+                "• Directional Scrolling ('direction'): Recommended for general browsing and standard page scrolling in most scenarios. Automatically computes safe swipe vectors and adaptive duration, retains a ~40% visual overlap anchor for zero-omission traversal, and prevents inertial flings. Supports scoping to a sub-container via 'target'. If it fails on certain custom layouts, fall back to specifying exact coordinates ('start' and 'end') directly.\n"
+                "• Precise Coordinate Gestures ('start', 'end'): Best for local, fine-grained interactions such as adjusting sliders/SeekBars (e.g., volume, brightness, progress bars), drag-and-drop / list reordering, or as a reliable fallback when directional scrolling fails on specific containers. Always drag slightly PAST the target position to overcome touch slop and reliably trigger the update. When setting a slider to Maximum (100%) or Minimum (0%), swipe fully to the extreme boundary.\n"
+                "\n"
+                "Args:\n"
+                "    direction: Smart directional scrolling ('up', 'down', 'left', 'right'). Automatically computes safe swipe vectors, retaining 40% visual overlap: 'up' (reveal content below), 'down' (reveal content above), 'left', 'right'.\n"
+                "    start: Start normalized coordinates [start_x, start_y] in 0-1000 scale.\n"
+                "    end: End normalized coordinates [end_x, end_y] in 0-1000 scale.\n"
+                "    target: Optional target element index (e.g. 2) or container bounds [left, top, right, bottom] to scope the directional swipe within.\n"
+                "    gesture: Backward-compatible parameter: direction string OR custom coordinates list [start_x, start_y, end_x, end_y] in 0-1000 scale.\n"
+                "    duration: Optional gesture duration in milliseconds (default 800)."
+            ),
+            params=(
+                ParamSpec(
+                    "direction",
+                    Direction | None,
+                    "Direction for scrolling and swiping: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
+                    " 'down' (drags top-to-bottom, scrolling up to reveal content above),"
+                    " 'left' (drags right-to-left, scrolling right),"
+                    " 'right' (drags left-to-right, scrolling left).",
+                    required=False,
+                    default=None,
+                ),
+                ParamSpec(
+                    "start",
+                    list[int] | None,
+                    "Start normalized coordinates [start_x, start_y] in 0-1000 scale for precise,"
+                    " local interactions (e.g. adjusting sliders, SeekBars, fine range selection, or drag-and-drop).",
+                    required=False,
+                    default=None,
+                ),
+                ParamSpec(
+                    "end",
+                    list[int] | None,
+                    "End normalized coordinates [end_x, end_y] in 0-1000 scale for precise,"
+                    " local interactions (e.g. adjusting sliders, SeekBars, fine range selection, or drag-and-drop).",
+                    required=False,
+                    default=None,
+                ),
+                ParamSpec(
+                    "target",
+                    int | list[int] | str | None,
+                    "Optional target element index (e.g. 2) or container bounds [left, top, right, bottom] to scope the directional swipe within.",
+                    required=False,
+                    default=None,
+                ),
+                ParamSpec(
+                    "gesture",
+                    Direction | list[int] | None,
+                    "Backward-compatible swipe gesture: smart direction string ('up', 'down', 'left', 'right')"
+                    " OR precise custom coordinates [start_x, start_y, end_x, end_y] in 0-1000 scale.",
+                    required=False,
+                    default=None,
+                ),
+                ParamSpec(
+                    "duration",
+                    int | None,
+                    "Optional swipe/drag duration in milliseconds (default 800). For drag-and-drop,"
+                    " list reordering, or sliding/adjusting sliders (e.g., volume, brightness, SeekBars),"
+                    " set duration >= 1000 (e.g. 1500). If omitted for directional swipe, duration is computed automatically.",
+                    required=False,
+                    default=None,
+                ),
+            ),
+        ),
+        declaration=DeclarationDialect(
+            description=(
+                "[ACTION] Perform a swipe, drag, or slider-adjustment gesture on the screen. The screen and UI hierarchy after swipe will be returned automatically.\n\n"
+                "• Directional Scrolling ('direction' or 'action'): Recommended for general browsing and standard page scrolling in most scenarios. Automatically computes safe swipe vectors and adaptive duration, retains a ~40% visual overlap anchor for zero-omission traversal, and prevents inertial flings. Supports scoping to a sub-container via 'target'. If it fails on certain custom layouts, fall back to specifying exact coordinates ('start' and 'end') directly.\n"
+                "• Precise Coordinate Gestures ('start', 'end' or coordinates list): Best for local, fine-grained interactions such as adjusting sliders/SeekBars (e.g., volume, brightness, progress bars), drag-and-drop / list reordering, or as a reliable fallback when directional scrolling fails on specific containers. Always drag slightly PAST the target position to overcome touch slop and reliably trigger the update. When setting a slider to Maximum (100%) or Minimum (0%), swipe fully to the extreme boundary."
+            ),
+            parameters={
+                "type": "object",
+                "properties": {
+                    "direction": {
+                        "type": "string",
+                        "enum": ["up", "down", "left", "right"],
+                        "description": (
+                            "Direction for scrolling and swiping: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
+                            " 'down' (drags top-to-bottom, scrolling up to reveal content above),"
+                            " 'left' (drags right-to-left, scrolling right),"
+                            " 'right' (drags left-to-right, scrolling left)."
+                        ),
+                    },
+                    "start": {
+                        "type": "array",
+                        "items": {"type": "integer"},
+                        "description": (
+                            "Start normalized coordinates [start_x, start_y] in 0-1000 scale."
+                        ),
+                    },
+                    "end": {
+                        "type": "array",
+                        "items": {"type": "integer"},
+                        "description": (
+                            "End normalized coordinates [end_x, end_y] in 0-1000 scale."
+                        ),
+                    },
+                    "target": {
+                        "description": (
+                            "Optional target element index (e.g. 2) or container bounds"
+                            " [left, top, right, bottom] to scope the directional swipe within."
+                        ),
+                    },
+                    "action": {
+                        "description": (
+                            "Backward-compatible swipe gesture: smart direction string ('up', 'down', 'left', 'right')"
+                            " OR precise custom coordinates [start_x, start_y, end_x, end_y] in 0-1000 scale."
+                        ),
+                    },
+                    "duration": {
+                        "type": "integer",
+                        "description": (
+                            "Optional swipe/drag duration in milliseconds (default 800). For drag-and-drop,"
+                            " list reordering, or sliding/adjusting sliders (e.g., volume, brightness, SeekBars),"
+                            " set duration >= 1000 (e.g. 1500). If omitted for directional swipe, duration is computed automatically."
+                        ),
+                    },
+                },
+            },
+        ),
+        wire=WireDialect(
+            description="Swipe between two 0-1000 normalized [x, y] points.",
+            params=(
+                ParamSpec("start", list[int]),
+                ParamSpec("end", list[int]),
+                ParamSpec("duration_ms", int, required=False, default=800),
+            ),
+            bind=_wire_swipe,
+        ),
+        param_bridge={
+            "direction": "direction",
+            "start": "start",
+            "end": "end",
+            "target": "target",
+            "gesture": "action",
+            "duration": "duration",
+        },
+        differences=(
+            "gesture/action: the legacy combined direction-or-coordinates parameter is"
+            " spelled `gesture` in the operator dialect and `action` in the"
+            " declaration dialect (`action` collides with the operator's structured"
+            " decision verb field). Smart directional swipes exist only in the agent"
+            " dialects; the wire takes a resolved start/end pair, and direction"
+            " resolution happens client-side against the live UI tree."
+        ),
+    ),
+    ActionSpec(
+        name="press_key",
+        operator=OperatorDialect(
+            description=(
+                "[ACTION] Press a physical or virtual system button (e.g.\n"
+                "\n"
+                "ENTER, BACK, HOME, APP_SWITCH)."
+            ),
+            params=(
+                ParamSpec(
+                    "key",
+                    Literal["ENTER", "BACK", "HOME", "APP_SWITCH"],
+                    "Standard Android system button name (ENTER, BACK, HOME, APP_SWITCH).",
+                ),
+            ),
+        ),
+        declaration=DeclarationDialect(
+            description=(
+                "[ACTION] Press a physical or virtual system button. The screen after"
+                " pressing will be returned automatically."
+            ),
+            parameters={
+                "type": "object",
+                "properties": {
+                    "key": {
+                        "type": "string",
+                        "description": (
+                            "Standard Android system button name (ENTER, BACK, HOME, APP_SWITCH)."
+                        ),
+                    }
+                },
+                "required": ["key"],
+            },
+        ),
+        wire=WireDialect(
+            description=(
+                "Press a device key (home, back, enter, delete, tab, search, menu, app_switch)."
+            ),
+            params=(ParamSpec("key", str),),
+            bind=_wire_press_key,
+        ),
+        param_bridge={"key": "key"},
+        differences=(
+            "key vocabulary: the operator dialect is a closed uppercase enum (its"
+            " translate step emits Android KEYCODE_* names); the declaration dialect"
+            " is an open string taught the same uppercase names; the wire takes the"
+            " actuator's bare lowercase key words. `to_canonical_call` and the"
+            " executors normalize between them."
+        ),
+    ),
+    ActionSpec(
+        name="manage_app",
+        operator=OperatorDialect(
+            description="[ACTION] Launch or force stop a specified application.",
+            params=(
+                ParamSpec("action", Literal["launch", "stop"], "The action type."),
+                ParamSpec(
+                    "app_name",
+                    str,
+                    "Display name or package name of the application.",
+                ),
+            ),
+        ),
+        declaration=DeclarationDialect(
+            description=(
+                "[ACTION] Launch or force stop a specified application. The screen"
+                " after launch/stop will be returned automatically."
+            ),
+            parameters={
+                "type": "object",
+                "properties": {
+                    "action": {
+                        "type": "string",
+                        "description": "The action type ('launch' or 'stop').",
+                    },
+                    "app_name": {
+                        "type": "string",
+                        "description": "Display name or package name of the application.",
+                    },
+                },
+                "required": ["action", "app_name"],
+            },
+        ),
+        wire=WireDialect(
+            description="Launch or stop an app by human-readable name or package.",
+            params=(
+                ParamSpec("action", str),
+                ParamSpec("app_name", str),
+            ),
+            bind=_wire_manage_app,
+        ),
+        param_bridge={"action": "action", "app_name": "app_name"},
+    ),
+    ActionSpec(
+        name="wait_for_delay",
+        operator=OperatorDialect(
+            description=(
+                "[ACTION] Pause execution and wait for a specified duration in milliseconds.\n"
+                "\n"
+                "Use this whenever you need time to elapse—whether for UI loading, animations,\n"
+                "screen transitions, or longer scheduled delays and intervals specified in the task."
+            ),
+            params=(
+                ParamSpec(
+                    "time_in_ms",
+                    int,
+                    "The exact duration to wait in milliseconds. Accurately convert the"
+                    " required time duration into milliseconds based on your objective"
+                    " or plan (e.g., 2000 for 2s, 5000 for 5s, 60000 for 1 minute,"
+                    " 180000 for 3 minutes, 300000 for 5 minutes).",
+                ),
+            ),
+        ),
+        declaration=DeclarationDialect(
+            description=(
+                "[ACTION] Pause execution and wait for a specified duration in milliseconds."
+                " The screen after pause will be returned automatically."
+            ),
+            parameters={
+                "type": "object",
+                "properties": {
+                    "time_in_ms": {
+                        "type": "integer",
+                        "description": (
+                            "The exact duration to pause in milliseconds (e.g., 2000 for 2s,"
+                            " 60000 for 1 min, 180000 for 3 mins, 300000 for 5 mins)."
+                            " Convert any required waiting duration into milliseconds."
+                        ),
+                    }
+                },
+                "required": ["time_in_ms"],
+            },
+        ),
+        wire=WireDialect(
+            description="Wait for a fixed number of milliseconds.",
+            params=(ParamSpec("time_in_ms", int),),
+            bind=_wire_wait_for_delay,
+        ),
+        param_bridge={"time_in_ms": "time_in_ms"},
+    ),
+    ActionSpec(
+        name="wait_for_text",
+        declaration=None,  # Listed in the legacy tool-name sets but never declared to
+        # any LLM; kept wire-reachable for executors and external MCP clients.
+        wire=WireDialect(
+            description="Wait for text to appear on or disappear from the screen.",
+            params=(
+                ParamSpec("text", str),
+                ParamSpec("wait_state", str, required=False, default="appear"),
+                ParamSpec("timeout_ms", int, required=False, default=5000),
+            ),
+            bind=_wire_wait_for_text,
+        ),
+    ),
+    ActionSpec(
+        name="open_link",
+        wire=WireDialect(
+            description="Open a URL on the device.",
+            params=(ParamSpec("url", str),),
+            bind=_wire_open_link,
+        ),
+    ),
+    ActionSpec(
+        name="erase_one_char",
+        wire=WireDialect(
+            description="Erase a single character in the focused field.",
+            params=(),
+            bind=_wire_erase_one_char,
+        ),
+    ),
+    ActionSpec(
+        name="focus_and_clear_text",
+        wire=WireDialect(
+            description=("Focus the field at a 0-1000 normalized [x, y] and clear its text."),
+            params=(ParamSpec("target", list[int]),),
+            bind=_wire_focus_and_clear_text,
+        ),
+    ),
+)
+
+#: Canonical manifest, keyed by action name, in declaration order.
+ACTION_SPECS: dict[str, ActionSpec] = {spec.name: spec for spec in _SPECS}
+
+#: The order the Operator binds its shells (historically the factory-dict order,
+#: identical to the prompt's "Physical device actions" enumeration order).
+OPERATOR_SHELL_ORDER: tuple[str, ...] = (
+    "click",
+    "input_text",
+    "swipe",
+    "press_key",
+    "manage_app",
+    "wait_for_delay",
+    "long_press",
+)
+
+#: Exception wording per action; matches the historical executor `except` arms.
+EXCEPTION_PREFIXES: dict[str, str] = {
+    "click": "Error during click",
+    "click_sequence": "Error executing click sequence",
+    "long_press": "Error during long press",
+    "input_text": "Error during input text",
+    "swipe": "Error during swipe",
+    "press_key": "Error during press_key",
+    "manage_app": "Error during manage_app",
+    "wait_for_delay": "Error during wait_for_delay",
+    "wait_for_text": "Error during wait_for_text",
+}
+
+
+def exception_prefix(action: str) -> str:
+    """Returns the historical human-readable exception prefix for an action."""
+    return EXCEPTION_PREFIXES.get(action, f"Error during {action}")
+
+
+# --- Projections ---------------------------------------------------------------------
+
+
+@cache
+def _operator_args_model(name: str):
+    """Builds (once) the pydantic argument model for an operator shell."""
+    dialect = ACTION_SPECS[name].operator
+    if dialect is None:
+        raise ValueError(f"Action '{name}' has no operator dialect.")
+    fields: dict[str, Any] = {}
+    for p in dialect.params:
+        if p.required:
+            fields[p.name] = (p.annotation, Field(description=p.description))
+        else:
+            fields[p.name] = (
+                p.annotation,
+                Field(default=p.default, description=p.description),
+            )
+    return create_model(name, **fields)
+
+
+def operator_shell_tool(name: str) -> StructuredTool:
+    """Builds the Operator's declaration-only shell for one action.
+
+    The body is never executed by the Operator loop -- action calls are translated
+    into structured decisions instead -- but returns the historical marker string for
+    any caller that does invoke it.
+    """
+    dialect = ACTION_SPECS[name].operator
+    if dialect is None:
+        raise ValueError(f"Action '{name}' has no operator dialect.")
+    return StructuredTool(
+        name=name,
+        description=dialect.description,
+        args_schema=_operator_args_model(name),
+        func=lambda **kwargs: "Action Recorded",
+    )
+
+
+def tool_declaration(name: str) -> ToolDeclaration:
+    """Builds the Validator/Flash ``ToolDeclaration`` for one action."""
+    dialect = ACTION_SPECS[name].declaration
+    if dialect is None:
+        raise ValueError(f"Action '{name}' has no declaration dialect.")
+    return ToolDeclaration(
+        name=name,
+        description=dialect.description,
+        parameters=dialect.parameters,
+    )
+
+
+def wire_dialects() -> tuple[ActionSpec, ...]:
+    """Returns every spec that has a wire dialect, in manifest order."""
+    return tuple(spec for spec in _SPECS if spec.wire is not None)
+
+
+def make_wire_handler(
+    spec: ActionSpec,
+    actuator: Any,
+    wrap: Callable[[ActionResult], CallToolResult],
+    wrap_exception: Callable[[str, Exception], CallToolResult],
+) -> Callable[..., Awaitable[CallToolResult]]:
+    """Builds the FastMCP tool function for one wire dialect.
+
+    The returned coroutine carries an explicit ``__signature__`` so FastMCP derives
+    the same argument model a literal ``async def`` produced historically, and the
+    same ``Annotated[CallToolResult, ActionResult]`` return annotation so the
+    structured-output schema is unchanged.
+    """
+    wire = spec.wire
+    if wire is None:
+        raise ValueError(f"Action '{spec.name}' has no wire dialect.")
+
+    async def handler(**kwargs: Any) -> Any:
+        try:
+            return wrap(await wire.bind(actuator, kwargs))
+        except Exception as e:  # pylint: disable=broad-exception-caught
+            return wrap_exception(spec.name, e)
+
+    return_annotation = Annotated[CallToolResult, ActionResult]
+    parameters = [
+        inspect.Parameter(
+            p.name,
+            inspect.Parameter.POSITIONAL_OR_KEYWORD,
+            annotation=p.annotation,
+            default=inspect.Parameter.empty if p.required else p.default,
+        )
+        for p in wire.params
+    ]
+    handler.__name__ = spec.name
+    handler.__qualname__ = spec.name
+    handler.__doc__ = wire.description
+    handler.__signature__ = inspect.Signature(  # type: ignore[attr-defined]
+        parameters, return_annotation=return_annotation
+    )
+    handler.__annotations__ = {p.name: p.annotation for p in wire.params}
+    handler.__annotations__["return"] = return_annotation
+    return handler

artemis/sdk/agent.py

@@ -35,7 +35,6 @@ import uuid
 from adbutils import AdbClient
 from dotenv import load_dotenv
 from google import genai
-from langchain_core.messages import AIMessage
 from langchain_google_genai import ChatGoogleGenerativeAI
 from PIL import Image
 from pydantic import BaseModel
@@ -117,14 +116,13 @@ class Agent:
         session_id: str | None = None,
     ):
         raw_sid = (
-            session_id
-            or os.getenv("ARTEMIS_SESSION_ID")
-            or os.getenv("ARTEMIS_CLOUD_SESSION_ID")
+            session_id or os.getenv("ARTEMIS_SESSION_ID") or os.getenv("ARTEMIS_CLOUD_SESSION_ID")
         )
         self._session_id: str | None = str(raw_sid).strip() if raw_sid else None
         target_dev = device_serial or device_id
         if config is None:
             from artemis.sdk.builders import Builders
+
             builder = Builders.AgentConfig
             if target_dev:
                 builder.for_device(DevicePlatform.ANDROID, target_dev)
@@ -203,7 +201,9 @@ class Agent:
             raise DeviceNotFoundError(error_msg)
 
         # Initialize clients
-        publish_startup_progress("device_check", "Checking the Android device", session_id=self._session_id)
+        publish_startup_progress(
+            "device_check", "Checking the Android device", session_id=self._session_id
+        )
         if os.environ.get("ARTEMIS_CLOUD_MODE") != "1":
             self._init_clients(
                 device_id=device_id,
@@ -222,7 +222,9 @@ class Agent:
             device_id=device_id, platform=platform
         )
         logger.info(self._device_context.to_str())
-        publish_startup_progress("device_ready", "Android device connected", session_id=self._session_id)
+        publish_startup_progress(
+            "device_ready", "Android device connected", session_id=self._session_id
+        )
 
         # Asynchronously pre-warm LLM connection pools in the background
         asyncio.create_task(self._prewarm_llm_connections(api_key))
@@ -236,9 +238,13 @@ class Agent:
         """Pre-warms the HTTP2/gRPC connection pools for both Native GenAI and LangChain clients in the background."""
         if os.environ.get("ARTEMIS_FAKE_LLM") == "1":
             logger.info("ARTEMIS_FAKE_LLM=1 — skipping real LLM connection pre-warming.")
-            publish_startup_progress("model_ready", "Model connection is ready (fake LLM)", session_id=self._session_id)
+            publish_startup_progress(
+                "model_ready", "Model connection is ready (fake LLM)", session_id=self._session_id
+            )
             return
-        publish_startup_progress("model_warmup", "Warming the model connection", session_id=self._session_id)
+        publish_startup_progress(
+            "model_warmup", "Warming the model connection", session_id=self._session_id
+        )
         logger.info("Starting background pre-warming of Gemini API connection pools...")
         try:
             key = api_key
@@ -248,7 +254,9 @@ class Agent:
             if not key:
                 logger.warning("Skipping LLM pre-warming: No API key available.")
                 publish_startup_progress(
-                    "model_ready", "Model connection will initialize on first use", session_id=self._session_id
+                    "model_ready",
+                    "Model connection will initialize on first use",
+                    session_id=self._session_id,
                 )
                 return
 
@@ -265,11 +273,15 @@ class Agent:
                 return_exceptions=True,
             )
             logger.success("Gemini API connection pools successfully pre-warmed.")
-            publish_startup_progress("model_ready", "Model connection is ready", session_id=self._session_id)
+            publish_startup_progress(
+                "model_ready", "Model connection is ready", session_id=self._session_id
+            )
         except Exception as e:
             logger.warning(f"Failed to pre-warm LLM connections: {e}")
             publish_startup_progress(
-                "model_ready", "Model connection will initialize on first use", session_id=self._session_id
+                "model_ready",
+                "Model connection will initialize on first use",
+                session_id=self._session_id,
             )
 
     async def install_apk(self, apk_path: str | Path) -> None:
@@ -549,13 +561,17 @@ class Agent:
                     or active_owner.token == self._device_context.device_id
                 )
             )
-            device_lock = None if already_held else DeviceExecutionLock(
-                self._device_context.device_id,
-                description=f"{request.goal[:120]}",
-                concurrency_mode=effective_mode,
-                max_concurrency=effective_max,
-                session_id=str(sess_id) if sess_id else None,
-                ingress=os.getenv("ARTEMIS_TASK_INGRESS") or "agent",
+            device_lock = (
+                None
+                if already_held
+                else DeviceExecutionLock(
+                    self._device_context.device_id,
+                    description=f"{request.goal[:120]}",
+                    concurrency_mode=effective_mode,
+                    max_concurrency=effective_max,
+                    session_id=str(sess_id) if sess_id else None,
+                    ingress=os.getenv("ARTEMIS_TASK_INGRESS") or "agent",
+                )
             )
             try:
                 if device_lock is not None and os.environ.get("ARTEMIS_CLOUD_MODE") != "1":
@@ -707,7 +723,6 @@ class Agent:
                                     context.data_engine.end_session("failed")
                                 return None
 
-                            print_ai_response_to_stderr(graph_result=last_state)
                             output = await self._extract_output(
                                 task_name=task_name,
                                 ctx=context,
@@ -715,7 +730,43 @@ class Agent:
                                 output_config=output_config,
                                 state=last_state,
                             )
-                            logger.info(f"✅ Automation '{task_name}' is success ✅")
+                            # Run outcome (goal axis x test axis) drives the
+                            # wrap-up instead of an unconditional success path.
+                            run_outcome = getattr(last_state, "run_outcome", None)
+                            context.run_outcome = run_outcome
+                            output = attach_test_summary(output, run_outcome)
+                            task_status_axis = (
+                                run_outcome.get("task_status") if run_outcome else None
+                            )
+                            tests_failed = (
+                                int((run_outcome.get("tests") or {}).get("failed", 0))
+                                if run_outcome
+                                else 0
+                            )
+                            if task_status_axis == "blocked":
+                                err = (
+                                    f"[{task_name}] Task ended blocked: verify"
+                                    " criteria unmet after exhausting the final"
+                                    " check budget."
+                                )
+                                logger.warning(err)
+                                await task.finalize(
+                                    content=output,
+                                    state=last_state_snapshot,
+                                    error=err,
+                                )
+                                if context.data_engine:
+                                    context.data_engine.end_session("failed")
+                                return output
+
+                            if tests_failed > 0:
+                                logger.warning(
+                                    f"[{task_name}] Task completed, but"
+                                    f" {tests_failed} assertion(s) failed —"
+                                    " see the test summary."
+                                )
+                            else:
+                                logger.info(f"✅ Automation '{task_name}' is success ✅")
                             await task.finalize(content=output, state=last_state_snapshot)
                             if context.data_engine:
                                 context.data_engine.end_session("completed")
@@ -723,13 +774,18 @@ class Agent:
                             return output
                     finally:
                         if recording_started:
+
                             async def _safe_stop_recording():
                                 try:
-                                    logger.info(f"[{task_name}] Stopping automated screen recording...")
+                                    logger.info(
+                                        f"[{task_name}] Stopping automated screen recording..."
+                                    )
                                     controller = get_controller(context)
                                     return await controller.stop_video_recording()
                                 except Exception as e:
-                                    logger.error(f"[{task_name}] Error stopping screen recording: {e}")
+                                    logger.error(
+                                        f"[{task_name}] Error stopping screen recording: {e}"
+                                    )
                                     return None
 
                             stop_task = asyncio.create_task(_safe_stop_recording())
@@ -982,8 +1038,16 @@ class Agent:
             else False,
             video_recording_tools_enabled=self._config.video_recording_tools_enabled,
             disable_checker=self._config.disable_checker,
+            disable_midway_checks=self._config.disable_midway_checks,
+            disable_final_check=self._config.disable_final_check,
             checker_max_iterations=self._config.checker_max_iterations,
-            checker_max_chat_rounds=self._config.checker_max_chat_rounds,
+            final_check_max_attempts=self._config.final_check_max_attempts,
+            checkpoint_max_repairs=self._config.checkpoint_max_repairs,
+            max_concurrent_checkpoints=self._config.max_concurrent_checkpoints,
+            checkpoint_timeout=self._config.checkpoint_timeout,
+            settlement_timeout=self._config.settlement_timeout,
+            assert_failure_policy=self._config.assert_failure_policy,
+            disable_device_probes=self._config.disable_device_probes,
             disable_planner_validation=self._config.disable_planner_validation,
             planner_validation_threshold=self._config.planner_validation_threshold,
             enable_committee=self._config.enable_committee,
@@ -1058,7 +1122,7 @@ class Agent:
                 logger.error(f"[{task_name}] Failed to clean up temp trace folder: {e}")
             return
 
-        status = "_PASS" if task.status == "completed" else "_FAIL"
+        status = resolve_trace_suffix(task.status, getattr(context, "run_outcome", None))
         ts = task.created_at.strftime("%Y-%m-%dT%H-%M-%S")
         new_name = f"{exec_setup_ctx.trace_name}{status}_{ts}"
 
@@ -1133,28 +1197,10 @@ class Agent:
             except Exception as e:
                 logger.error(f"[{task_name}] Failed to generate structured output: {e}")
                 return None
-        if state and state.messages:
-            for msg in reversed(state.messages):
-                if msg.content:
-                    logger.info(f"Fallback output from message: {msg.content}")
-                    record_events(output_path=request.llm_output_path, events=msg.content)
-                    return msg.content
         return None
 
     def _get_graph_state(self, task: Task):
-        return State(
-            messages=[],
-            initial_goal=task.request.goal,
-            subgoal_plan=[],
-            latest_ui_hierarchy=None,
-            latest_screenshot=None,
-            focused_app_info=None,
-            device_date=None,
-            structured_decisions=None,
-            complete_subgoals_by_ids=[],
-            remaining_steps=task.request.max_steps,
-            validator_messages=[],
-        )
+        return State.initial(task.request.goal)
 
     def _init_clients(
         self,
@@ -1229,8 +1275,44 @@ def _validate_and_prepare_file(file_path: Path):
         raise AgentTaskRequestError(f"Error creating file '{file_path}': {e}")
 
 
-def print_ai_response_to_stderr(graph_result: State):
-    for msg in reversed(graph_result.messages):
-        if isinstance(msg, AIMessage):
-            print(msg.content, file=sys.stderr)
-            return
+def attach_test_summary(output, run_outcome: dict | None):
+    """Surfaces the machine-readable test summary in the task's return value.
+
+    Only applies when the run actually had check items (the summary carries at
+    least one counted item) — otherwise the historical return shape is kept
+    untouched. String outputs are wrapped into a dict so that callers such as
+    ``mobile_run_task`` receive the summary without parsing report prose.
+    """
+    if not run_outcome:
+        return output
+    tests = run_outcome.get("tests") or {}
+    total = (
+        int(tests.get("passed", 0))
+        + int(tests.get("failed", 0))
+        + int(tests.get("inconclusive", 0))
+        + int(tests.get("unchecked", 0))
+    )
+    if total <= 0:
+        return output
+    summary = {"task_status": run_outcome.get("task_status"), **tests}
+    if isinstance(output, dict):
+        merged = dict(output)
+        merged.setdefault("test_summary", summary)
+        return merged
+    if output is None:
+        return {"test_summary": summary}
+    if isinstance(output, str):
+        return {"result": output, "test_summary": summary}
+    # Typed/structured outputs keep their shape; the summary stays available in
+    # run_outcome.json and the session state.
+    return output
+
+
+def resolve_trace_suffix(task_status: str, run_outcome: dict | None) -> str:
+    """Trace naming: assertion failures must be distinguishable from _PASS."""
+    if task_status != "completed":
+        return "_FAIL"
+    tests = (run_outcome or {}).get("tests") or {}
+    if int(tests.get("failed", 0)) > 0:
+        return "_TESTFAIL"
+    return "_PASS"

artemis/sdk/builders/agent_config_builder.py

@@ -88,7 +88,15 @@ class AgentConfigBuilder:
         self._committee_debate_rounds = agent_cfg.committee.debate_rounds
         self._disable_checker = not agent_cfg.checker.enabled
         self._checker_max_iterations = agent_cfg.checker.max_iterations
-        self._checker_max_chat_rounds = agent_cfg.checker.max_chat_rounds
+        self._disable_midway_checks = not agent_cfg.checker.midway_checks
+        self._disable_final_check = not agent_cfg.checker.final_check
+        self._final_check_max_attempts = agent_cfg.checker.final_check_max_attempts
+        self._checkpoint_max_repairs = agent_cfg.checker.checkpoint_max_repairs
+        self._max_concurrent_checkpoints = agent_cfg.checker.max_concurrent_checkpoints
+        self._checkpoint_timeout = agent_cfg.checker.checkpoint_timeout
+        self._settlement_timeout = agent_cfg.checker.settlement_timeout
+        self._assert_failure_policy = agent_cfg.checker.assert_failure_policy
+        self._disable_device_probes = not agent_cfg.checker.device_probes
         self._outputter = agent_cfg.outputter
         self._disable_outputter = not agent_cfg.outputter.enabled
         self._flash = agent_cfg.flash
@@ -162,9 +170,7 @@ class AgentConfigBuilder:
         """Target a specific Android device by its ADB serial number."""
         return self.for_device(DevicePlatform.ANDROID, device_serial)
 
-    def with_concurrency_mode(
-        self, mode: str
-    ) -> "AgentConfigBuilder":
+    def with_concurrency_mode(self, mode: str) -> "AgentConfigBuilder":
         """Configure concurrency mode: 'global' (1 task globally) or 'per_device' (1 task per device)."""
         self._concurrency_mode = str(mode).strip().lower()
         return self
@@ -244,20 +250,62 @@ class AgentConfigBuilder:
         self,
         enabled: bool = True,
         max_iterations: int | None = None,
-        max_chat_rounds: int | None = None,
+        midway_checks: bool | None = None,
+        final_check: bool | None = None,
+        final_check_max_attempts: int | None = None,
+        checkpoint_max_repairs: int | None = None,
+        max_concurrent_checkpoints: int | None = None,
+        checkpoint_timeout: float | None = None,
+        settlement_timeout: float | None = None,
+        assert_failure_policy: str | None = None,
+        device_probes: bool | None = None,
     ) -> "AgentConfigBuilder":
-        """Configure Checker subgoal verification and rollback settings.
+        """Configure plan-driven checkpoint verification and the final review.
 
         Args:
-            enabled: Whether checker visual verification is enabled
+            enabled: Master switch (compat alias); False disables both gates
             max_iterations: Optional maximum iterations for Checker reasoning (1-50)
-            max_chat_rounds: Optional maximum debate rounds with Operator (1-10)
+            midway_checks: Whether plan-declared midway checkpoints run
+            final_check: Whether the exit final review runs
+            final_check_max_attempts: Max final-review attempts before a blocked exit
+            checkpoint_max_repairs: Per-checkpoint verify-fail repair quota
+            max_concurrent_checkpoints: Concurrency cap for checkpoint attempts
+            checkpoint_timeout: Per-attempt timeout in seconds
+            settlement_timeout: Aggregate exit-settlement timeout in seconds
+            assert_failure_policy: 'continue' or 'halt' on assert failures
+            device_probes: Whether Checker read-only device probes are registered
         """
         self._disable_checker = not enabled
         if max_iterations is not None:
             self._checker_max_iterations = max_iterations
-        if max_chat_rounds is not None:
-            self._checker_max_chat_rounds = max_chat_rounds
+        if midway_checks is not None:
+            self._disable_midway_checks = not midway_checks
+        if final_check is not None:
+            self._disable_final_check = not final_check
+        if final_check_max_attempts is not None:
+            self._final_check_max_attempts = final_check_max_attempts
+        if checkpoint_max_repairs is not None:
+            self._checkpoint_max_repairs = checkpoint_max_repairs
+        if max_concurrent_checkpoints is not None:
+            self._max_concurrent_checkpoints = max_concurrent_checkpoints
+        if checkpoint_timeout is not None:
+            self._checkpoint_timeout = checkpoint_timeout
+        if settlement_timeout is not None:
+            self._settlement_timeout = settlement_timeout
+        if assert_failure_policy is not None:
+            self._assert_failure_policy = assert_failure_policy
+        if device_probes is not None:
+            self._disable_device_probes = not device_probes
+        return self
+
+    def with_midway_checks(self, enabled: bool = True) -> "AgentConfigBuilder":
+        """Enable or disable plan-declared midway checkpoints."""
+        self._disable_midway_checks = not enabled
+        return self
+
+    def with_final_check(self, enabled: bool = True) -> "AgentConfigBuilder":
+        """Enable or disable the exit final review."""
+        self._disable_final_check = not enabled
         return self
 
     def with_disable_planner_validation(self, disable: bool = True) -> "AgentConfigBuilder":
@@ -521,8 +569,16 @@ class AgentConfigBuilder:
             video_recording_tools_enabled=self._video_recording_tools_enabled,
             force_web_accessibility=self._force_web_accessibility,
             disable_checker=self._disable_checker,
+            disable_midway_checks=self._disable_midway_checks,
+            disable_final_check=self._disable_final_check,
             checker_max_iterations=self._checker_max_iterations,
-            checker_max_chat_rounds=self._checker_max_chat_rounds,
+            final_check_max_attempts=self._final_check_max_attempts,
+            checkpoint_max_repairs=self._checkpoint_max_repairs,
+            max_concurrent_checkpoints=self._max_concurrent_checkpoints,
+            checkpoint_timeout=self._checkpoint_timeout,
+            settlement_timeout=self._settlement_timeout,
+            assert_failure_policy=self._assert_failure_policy,
+            disable_device_probes=self._disable_device_probes,
             disable_planner_validation=self._disable_planner_validation,
             planner_validation_threshold=self._planner_validation_threshold,
             enable_committee=self._enable_committee,

artemis/sdk/types/agent.py

@@ -112,8 +112,16 @@ class AgentConfig(BaseModel):
     video_recording_tools_enabled: bool = Field(default_factory=detect_video_tools_enabled)
     force_web_accessibility: bool = False
     disable_checker: bool = True
+    disable_midway_checks: bool = False
+    disable_final_check: bool = False
     checker_max_iterations: int = 20
-    checker_max_chat_rounds: int = 4
+    final_check_max_attempts: int = 3
+    checkpoint_max_repairs: int = 2
+    max_concurrent_checkpoints: int = 3
+    checkpoint_timeout: float = 180.0
+    settlement_timeout: float = 120.0
+    assert_failure_policy: Literal["continue", "halt"] = "continue"
+    disable_device_probes: bool = False
     disable_planner_validation: bool = True
     planner_validation_threshold: float = 0.85
     enable_committee: bool = False

artemis/services/llm.py

@@ -25,7 +25,6 @@ from dataclasses import replace
 import functools
 import logging
 from pathlib import Path
-import random
 import re
 import sys
 import time
@@ -33,6 +32,7 @@ from typing import Any, Literal, TypeVar, overload
 from uuid import uuid4
 
 from langchain_core.language_models.chat_models import BaseChatModel
+from langchain_core.messages import AIMessage, BaseMessage, HumanMessage
 
 from artemis.config import (
     PAUSE_FILE,
@@ -46,7 +46,22 @@ from artemis.config import (
 )
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import CURRENT_TRACE_ID, DataEngineCallbackHandler
+from artemis.llm.reliability import (
+    CircuitBreaker,
+    FailureCategory,
+    LLMCallError,
+    LLMExhaustedError,
+    LLMPermanentError,
+    classify_failure,
+    retry_policy_for,
+)
 from artemis.llm.router import ModelEndpoint, ModelFactory, ModelProvider
+from artemis.llm.structured import (
+    ParseFailure,
+    StructuredOutputError,
+    content_to_text,
+    parse_structured,
+)
 from artemis.utils.logger import get_logger
 
 # Logger for internal messages
@@ -69,26 +84,32 @@ _ACTIVE_LLM_REQUEST: ContextVar[dict[str, Any] | None] = ContextVar(
 )
 
 
-def _provider_name_from_call(args: tuple[Any, ...]) -> str | None:
-    wrapper = args[0] if args else None
-    endpoint = getattr(wrapper, "endpoint", None)
-    provider = getattr(endpoint, "provider", None)
-    if provider is None:
-        return None
-    return str(getattr(provider, "value", provider))
-
-
-def _begin_llm_request(args: tuple[Any, ...]) -> Token:
+def _begin_llm_request(provider: str | None) -> Token:
     return _ACTIVE_LLM_REQUEST.set(
         {
             "request_id": str(uuid4()),
-            "provider": _provider_name_from_call(args),
+            "provider": provider,
             "started_at": time.time(),
             "retries": [],
         }
     )
 
 
+# Whether the current async context has a configured fallback model waiting
+# behind this call (set by with_fallback).  When a fallback exists, exhausting
+# retries hands over to it immediately instead of pausing the whole task.
+_FALLBACK_AVAILABLE: ContextVar[bool] = ContextVar("llm_fallback_available", default=False)
+
+# Shared circuit breaker keyed by "provider:model".  Transient failures open
+# it; while open, concurrent calls wait out the cooldown instead of hammering
+# a provider that is already melting down.
+_ENDPOINT_BREAKER = CircuitBreaker(threshold=3, cooldown_seconds=30.0)
+
+# Endpoints observed to not support streaming.  Detected once, loudly, then
+# remembered so subsequent calls go straight to non-streaming invocation.
+_NON_STREAMING_ENDPOINTS: set[str] = set()
+
+
 def _get_current_data_engine():
     if _CURRENT_DATA_ENGINE is not None:
         return _CURRENT_DATA_ENGINE
@@ -170,6 +191,52 @@ def _record_llm_retry(
         llm_logger.warning("Failed to publish LLM retry event: %s", publish_error)
 
 
+def _record_llm_event(name: str, payload: dict[str, Any], *, status: str = "retrying") -> None:
+    """Persist and publish one degradation/lifecycle event on the LLM IO path.
+
+    Every degradation (fallback, stream downgrade, stream reset, giving up)
+    must be observable: WARNING-level logging is handled by callers, this
+    records the structured trace + live event, best-effort.
+    """
+    request = _ACTIVE_LLM_REQUEST.get()
+    engine = _get_current_data_engine()
+    if not engine or not getattr(engine, "current_session_id", None):
+        return
+
+    full_payload = dict(payload)
+    if request is not None:
+        full_payload.setdefault("request_id", request.get("request_id"))
+        if request.get("provider"):
+            full_payload.setdefault("provider", request.get("provider"))
+    full_payload = {key: value for key, value in full_payload.items() if value is not None}
+
+    step_id = getattr(engine, "current_step_id", None)
+    trace_id = None
+    try:
+        trace_id = engine.record_trace(
+            type="llm_call",
+            name=name,
+            payload=full_payload,
+            step_id=step_id,
+            parent_trace_id=CURRENT_TRACE_ID.get(),
+            status=status,
+        )
+    except Exception as trace_error:
+        llm_logger.warning("Failed to persist LLM event trace %s: %s", name, trace_error)
+    try:
+        engine._publish(
+            name,
+            {
+                **full_payload,
+                "step_id": str(step_id) if step_id else None,
+                "trace_id": str(trace_id) if trace_id else None,
+                "timestamp": time.time(),
+            },
+        )
+    except Exception as publish_error:
+        llm_logger.warning("Failed to publish LLM event %s: %s", name, publish_error)
+
+
 class _ProviderRetryTelemetryHandler(logging.Handler):
     """Turn provider SDK retry log records into structured Artemis telemetry."""
 
@@ -217,8 +284,7 @@ class _ProviderRetryTelemetryHandler(logging.Handler):
 def _install_provider_retry_telemetry() -> None:
     provider_logger = logging.getLogger("google_genai._api_client")
     if any(
-        isinstance(handler, _ProviderRetryTelemetryHandler)
-        for handler in provider_logger.handlers
+        isinstance(handler, _ProviderRetryTelemetryHandler) for handler in provider_logger.handlers
     ):
         return
     provider_logger.addHandler(_ProviderRetryTelemetryHandler(level=logging.INFO))
@@ -285,126 +351,149 @@ def _handle_llm_pause_and_resume(last_error: Exception) -> Path:
     return pause_file
 
 
-def robust_retry_async(func: Callable) -> Callable:
-    """Decorator to add robust retry and pause/resume logic to an async LLM call."""
+async def _wait_for_resume(pause_file: Path) -> bool:
+    """Wait for the pause file to be cleared. Returns False on deadline."""
+    deadline = float(getattr(settings, "LLM_PAUSE_TIMEOUT_SECONDS", 0.0) or 0.0)
+    waited = 0.0
+    while pause_file.exists():
+        if deadline > 0 and waited >= deadline:
+            return False
+        await asyncio.sleep(1)
+        waited += 1
+    return True
 
-    @functools.wraps(func)
-    async def wrapper(*args, **kwargs):
-        while True:
-            request_token = _begin_llm_request(args)
-            max_retries = 3
-            retry_delay = 0.0
-            last_error = None
-            for attempt in range(max_retries):
+
+async def _wait_for_breaker(key: str) -> None:
+    """Wait out an open circuit instead of hammering a melting provider."""
+    waited = 0.0
+    warned = False
+    while not _ENDPOINT_BREAKER.allow(key):
+        if not warned:
+            warned = True
+            llm_logger.warning(f"LLM circuit open for {key}; waiting for cooldown...")
+        step = min(max(_ENDPOINT_BREAKER.open_remaining(key), 0.5), 2.0)
+        await asyncio.sleep(step)
+        waited += step
+        if waited >= 120.0:
+            # Never deadlock behind a stuck half-open trial; proceed anyway.
+            break
+
+
+async def _run_with_recovery[T](
+    call_once: Callable[[], Awaitable[T]],
+    *,
+    provider: str | None = None,
+    endpoint_key: str | None = None,
+) -> T:
+    """Execute one LLM call under the classified retry / pause / breaker policy.
+
+    Failure handling is decided per category (see artemis.llm.reliability):
+
+    - Retryable categories back off per policy; each retry is recorded for UI
+      transparency via _record_llm_retry.
+    - Non-retryable categories (auth, bad request) raise LLMPermanentError
+      immediately: retrying cannot help and pausing would hang the task.
+    - When retryable attempts are exhausted: if a fallback model is waiting
+      (with_fallback), raise LLMExhaustedError so it takes over immediately;
+      otherwise pause the task (bounded by settings.LLM_PAUSE_TIMEOUT_SECONDS)
+      and retry from scratch on resume.
+    """
+    while True:
+        request_token = _begin_llm_request(provider)
+        last_error: Exception | None = None
+        last_failure = None
+        attempts: dict[FailureCategory, int] = {}
+        pause_file: Path | None = None
+        try:
+            while True:
+                if endpoint_key:
+                    await _wait_for_breaker(endpoint_key)
                 try:
-                    result = await func(*args, **kwargs)
-                    _ACTIVE_LLM_REQUEST.reset(request_token)
-                    return result
-                except Exception as e:
-                    last_error = e
-                    err_str = str(e).lower()
-                    if attempt == max_retries - 1:
-                        break
-                    if attempt > 0:
-                        retry_delay = 0.5 * (2 ** (attempt - 1)) + random.uniform(0.1, 0.3)
-                    if any(
-                        err in err_str
-                        for err in [
-                            "503",
-                            "throttled",
-                            "overloaded",
-                            "429",
-                            "quota",
-                            "resource_exhausted",
-                        ]
-                    ):
-                        retry_delay = max(retry_delay, 10.0)
-                    if retry_delay > 0:
-                        llm_logger.warning(f"LLM call failed, retrying in {retry_delay:.2f}s...")
-
-                        await asyncio.sleep(retry_delay)
-                except BaseException:
-                    _ACTIVE_LLM_REQUEST.reset(request_token)
+                    result = await call_once()
+                except (KeyboardInterrupt, asyncio.CancelledError):
+                    raise
+                except LLMCallError:
+                    # Already classified terminal by a nested recovery layer.
                     raise
-
-            try:
-                pause_file = _handle_llm_pause_and_resume(last_error)
-            finally:
-                _ACTIVE_LLM_REQUEST.reset(request_token)
-            while pause_file.exists():
-                await asyncio.sleep(1)
-
-            llm_logger.info("Resume signal received, retrying LLM call...")
-
-            current_engine = _get_current_data_engine()
-            if current_engine:
-                current_engine._publish("task_resumed", {})
-
-    return wrapper
-
-
-def robust_retry_astream(func: Callable) -> Callable:
-    """Decorator to add robust retry and pause/resume logic to an async stream LLM call."""
-
-    @functools.wraps(func)
-    async def wrapper(*args, **kwargs):
-        while True:
-            request_token = _begin_llm_request(args)
-            max_retries = 3
-            retry_delay = 0.0
-            last_error = None
-            for attempt in range(max_retries):
-                try:
-                    gen = func(*args, **kwargs)
-                    first = await gen.__anext__()
-                    yield first
-                    async for chunk in gen:
-                        yield chunk
-                    _ACTIVE_LLM_REQUEST.reset(request_token)
-                    return
                 except Exception as e:
-                    last_error = e
-                    err_str = str(e).lower()
-                    if attempt == max_retries - 1:
-                        break
-                    if attempt > 0:
-                        retry_delay = 0.5 * (2 ** (attempt - 1)) + random.uniform(0.1, 0.3)
-                    if any(
-                        err in err_str
-                        for err in [
-                            "503",
-                            "throttled",
-                            "overloaded",
-                            "429",
-                            "quota",
-                            "resource_exhausted",
-                        ]
-                    ):
-                        retry_delay = max(retry_delay, 10.0)
-                    if retry_delay > 0:
+                    failure = classify_failure(e)
+                    if failure.category is FailureCategory.CANCELLED:
+                        raise
+                    if endpoint_key:
+                        _ENDPOINT_BREAKER.record_failure(endpoint_key, failure)
+                    if not failure.retryable:
                         llm_logger.warning(
-                            f"LLM stream handshake throttled, retrying in {retry_delay:.2f}s..."
+                            f"LLM call failed permanently ({failure.category.value}): {e}"
                         )
+                        _record_llm_event(
+                            "llm_gave_up",
+                            {
+                                "error": str(e)[:1000],
+                                "category": failure.category.value,
+                                "retryable": False,
+                            },
+                            status="failed",
+                        )
+                        raise LLMPermanentError(str(e), failure=failure, cause=e) from e
+                    last_error, last_failure = e, failure
+                    attempt = attempts.get(failure.category, 0) + 1
+                    attempts[failure.category] = attempt
+                    policy = retry_policy_for(failure.category)
+                    if attempt >= policy.max_attempts:
+                        break
+                    delay = policy.delay_for(attempt)
+                    _record_llm_retry(
+                        str(e),
+                        delay,
+                        attempt=attempt,
+                        max_retries=policy.max_attempts,
+                        provider=provider,
+                    )
+                    llm_logger.warning(
+                        f"LLM call failed ({failure.category.value}), retrying in {delay:.2f}s..."
+                    )
+                    await asyncio.sleep(delay)
+                else:
+                    if endpoint_key:
+                        _ENDPOINT_BREAKER.record_success(endpoint_key)
+                    return result
 
-                        await asyncio.sleep(retry_delay)
-                except BaseException:
-                    _ACTIVE_LLM_REQUEST.reset(request_token)
-                    raise
+            # Retryable attempts exhausted.
+            if _FALLBACK_AVAILABLE.get():
+                _record_llm_event(
+                    "llm_gave_up",
+                    {
+                        "error": str(last_error)[:1000],
+                        "category": last_failure.category.value,
+                        "retryable": True,
+                        "handover": "fallback",
+                    },
+                    status="failed",
+                )
+                raise LLMExhaustedError(
+                    str(last_error), failure=last_failure, cause=last_error
+                ) from last_error
+            pause_file = _handle_llm_pause_and_resume(last_error)
+        finally:
+            _ACTIVE_LLM_REQUEST.reset(request_token)
 
+        if not await _wait_for_resume(pause_file):
+            llm_logger.error(
+                "LLM pause deadline"
+                f" ({settings.LLM_PAUSE_TIMEOUT_SECONDS:.0f}s) exceeded; giving up."
+            )
             try:
-                pause_file = _handle_llm_pause_and_resume(last_error)
-            finally:
-                _ACTIVE_LLM_REQUEST.reset(request_token)
-            while pause_file.exists():
-                await asyncio.sleep(1)
+                pause_file.unlink(missing_ok=True)
+            except OSError:
+                pass
+            raise LLMExhaustedError(
+                str(last_error), failure=last_failure, cause=last_error
+            ) from last_error
 
-            llm_logger.info("Resume signal received, retrying LLM stream...")
-
-            current_engine = _get_current_data_engine()
-            if current_engine:
-                current_engine._publish("task_resumed", {})
-
-    return wrapper
+        llm_logger.info("Resume signal received, retrying LLM call...")
+        current_engine = _get_current_data_engine()
+        if current_engine:
+            current_engine._publish("task_resumed", {})
 
 
 def _inject_parent_trace_id(trace_id, *args, **kwargs):
@@ -524,49 +613,229 @@ class RobustChatModelWrapper:
             self.base_model.__ror__(other), self.ctx, endpoint=self.endpoint
         )
 
-    @robust_retry_async
+    def _provider_value(self) -> str | None:
+        provider = getattr(self.endpoint, "provider", None)
+        if provider is None:
+            return None
+        return str(getattr(provider, "value", provider))
+
+    def _endpoint_key(self) -> str:
+        if self.endpoint is not None:
+            return f"{self._provider_value()}:{self.endpoint.model_name}"
+        return type(self.base_model).__name__
+
+    def _traced_call(self, args: tuple, kwargs: dict) -> tuple[tuple, dict, Any]:
+        trace_id = None
+        if self.ctx and self.ctx.data_engine:
+            trace_id = CURRENT_TRACE_ID.get()
+        if trace_id:
+            args, kwargs = _inject_parent_trace_id(trace_id, *args, **kwargs)
+        return args, kwargs, trace_id
+
     async def ainvoke(self, *args, **kwargs):
-        trace_id = None
-        if self.ctx and self.ctx.data_engine:
-            trace_id = CURRENT_TRACE_ID.get()
-        if trace_id:
-            args, kwargs = _inject_parent_trace_id(trace_id, *args, **kwargs)
-        return await self.base_model.ainvoke(*args, **kwargs)
+        args, kwargs, _ = self._traced_call(args, kwargs)
+        return await _run_with_recovery(
+            functools.partial(self.base_model.ainvoke, *args, **kwargs),
+            provider=self._provider_value(),
+            endpoint_key=self._endpoint_key(),
+        )
 
-    @robust_retry_astream
-    async def astream(self, *args, **kwargs):
-        trace_id = None
-        if self.ctx and self.ctx.data_engine:
-            trace_id = CURRENT_TRACE_ID.get()
+    async def complete(self, *args, **kwargs):
+        """Single completion entry point: returns the full final message.
 
-        if trace_id:
-            args, kwargs = _inject_parent_trace_id(trace_id, *args, **kwargs)
+        Streaming is a transport detail handled internally — provider chunks
+        are forwarded to the live UI as deltas and accumulated here, so
+        downstream consumers only ever see one complete message (or a typed
+        LLMCallError).  A mid-stream failure discards the partial output,
+        signals the UI to drop it, and retries the whole call per policy;
+        partial or duplicated chunks can never reach message history.
+        """
+        args, kwargs, _ = self._traced_call(args, kwargs)
+        emit_deltas = bool(self.ctx and self.ctx.data_engine)
+        return await _run_with_recovery(
+            functools.partial(self._complete_attempt, emit_deltas, *args, **kwargs),
+            provider=self._provider_value(),
+            endpoint_key=self._endpoint_key(),
+        )
+
+    def _emit_stream_delta(self, stream_exec_id, chunk) -> None:
+        content = getattr(chunk, "content", None)
+        if not content:
+            return
+        text_to_stream = ""
+        thinking_to_stream = ""
+        if isinstance(content, str):
+            text_to_stream = content
+        elif isinstance(content, list):
+            for item in content:
+                if isinstance(item, dict):
+                    if item.get("type") == "text":
+                        text_to_stream += item.get("text", "")
+                    elif item.get("type") == "thinking":
+                        thinking_to_stream += item.get("thinking", "")
+        if text_to_stream:
+            self.ctx.data_engine.stream_output(stream_exec_id, text_to_stream, is_thinking=False)
+        if thinking_to_stream:
+            self.ctx.data_engine.stream_output(stream_exec_id, thinking_to_stream, is_thinking=True)
+
+    async def _complete_attempt(self, emit_deltas: bool, *args, **kwargs):
+        endpoint_key = self._endpoint_key()
+        if endpoint_key in _NON_STREAMING_ENDPOINTS:
+            return await self.base_model.ainvoke(*args, **kwargs)
 
         stream_exec_id = uuid4()
+        full_response = None
+        try:
+            async for chunk in self.base_model.astream(*args, **kwargs):
+                if full_response is None:
+                    full_response = chunk
+                else:
+                    full_response = full_response + chunk
+                if emit_deltas:
+                    self._emit_stream_delta(stream_exec_id, chunk)
+        except (KeyboardInterrupt, asyncio.CancelledError):
+            raise
+        except Exception as stream_error:
+            if full_response is not None:
+                # Mid-stream failure: the partial output must never reach
+                # message history. Tell the UI to discard it, then let the
+                # recovery policy retry the whole call with a fresh stream id.
+                llm_logger.warning(
+                    f"LLM stream broke mid-response ({stream_error}); discarding partial output."
+                )
+                _record_llm_event(
+                    "llm_stream_reset",
+                    {
+                        "stream_exec_id": str(stream_exec_id),
+                        "error": str(stream_error)[:500],
+                        "category": classify_failure(stream_error).category.value,
+                    },
+                )
+                raise
+            if _is_stream_unsupported_error(stream_error):
+                _NON_STREAMING_ENDPOINTS.add(endpoint_key)
+                llm_logger.warning(
+                    f"Endpoint {endpoint_key} does not support streaming"
+                    f" ({stream_error}); switching to non-streaming calls."
+                )
+                _record_llm_event(
+                    "llm_stream_downgrade",
+                    {"endpoint": endpoint_key, "error": str(stream_error)[:500]},
+                )
+                return await self.base_model.ainvoke(*args, **kwargs)
+            raise
+        if full_response is None:
+            llm_logger.warning("LLM stream yielded no chunks; using non-streaming call.")
+            return await self.base_model.ainvoke(*args, **kwargs)
+        return full_response
 
-        async for chunk in self.base_model.astream(*args, **kwargs):
-            if trace_id and self.ctx and self.ctx.data_engine and chunk.content:
-                text_to_stream = ""
-                thinking_to_stream = ""
-                if isinstance(chunk.content, str):
-                    text_to_stream = chunk.content
-                elif isinstance(chunk.content, list):
-                    for item in chunk.content:
-                        if isinstance(item, dict):
-                            if item.get("type") == "text":
-                                text_to_stream += item.get("text", "")
-                            elif item.get("type") == "thinking":
-                                thinking_to_stream += item.get("thinking", "")
+    async def astream(self, *args, **kwargs):
+        """Deprecated compatibility shim: yields exactly one final message.
 
-                if text_to_stream:
-                    self.ctx.data_engine.stream_output(
-                        stream_exec_id, text_to_stream, is_thinking=False
-                    )
-                if thinking_to_stream:
-                    self.ctx.data_engine.stream_output(
-                        stream_exec_id, thinking_to_stream, is_thinking=True
-                    )
-            yield chunk
+        The old chunk-level astream retry could re-deliver already-yielded
+        chunks after a mid-stream failure, corrupting accumulated message
+        history. Streaming now happens inside complete(); live token deltas
+        still reach the UI through the data engine.
+        """
+        yield await self.complete(*args, **kwargs)
+
+
+def _is_stream_unsupported_error(error: BaseException) -> bool:
+    """Detect 'this endpoint/model cannot stream' as opposed to a transient failure."""
+    if isinstance(error, (NotImplementedError, AttributeError, TypeError)):
+        return True
+    message = str(error).lower()
+    return "stream" in message and any(
+        marker in message
+        for marker in ("not support", "unsupported", "not implemented", "not available")
+    )
+
+
+async def acomplete(llm, *args, **kwargs):
+    """Get one complete response from any chat model object.
+
+    This is the single call-shape agents should use. For gateway-managed
+    models (RobustChatModelWrapper) it delegates to complete(), which owns
+    streaming, classified retries, and telemetry. For raw models and test
+    doubles it preserves the legacy accumulate-stream-else-ainvoke shape.
+    """
+    if isinstance(llm, RobustChatModelWrapper):
+        return await llm.complete(*args, **kwargs)
+
+    try:
+        full_response = None
+        async for chunk in llm.astream(*args, **kwargs):
+            if full_response is None:
+                full_response = chunk
+            else:
+                full_response = full_response + chunk
+        if full_response is not None:
+            return full_response
+    except (KeyboardInterrupt, asyncio.CancelledError):
+        raise
+    except Exception as stream_error:
+        llm_logger.warning(
+            f"astream unavailable on {type(llm).__name__} ({stream_error}); using ainvoke."
+        )
+    return await llm.ainvoke(*args, **kwargs)
+
+
+async def acomplete_structured(
+    llm,
+    messages: list,
+    *,
+    schema=None,
+    correction_attempts: int = 1,
+):
+    """Complete and parse a JSON response, with one corrective re-ask.
+
+    On a parse failure the model is shown its own output's parse error and
+    asked to re-emit valid JSON (up to ``correction_attempts`` times) — giving
+    the model a chance to see its mistake instead of feeding garbage
+    downstream. If it still fails, raises StructuredOutputError (never
+    returns raw text masquerading as parsed data). Each repair round emits an
+    llm_parse_repair telemetry event.
+    """
+    response = await acomplete(llm, messages)
+    text = content_to_text(getattr(response, "content", ""))
+    parsed = parse_structured(text, schema=schema)
+    attempt = 0
+    while isinstance(parsed, ParseFailure) and attempt < correction_attempts:
+        attempt += 1
+        llm_logger.warning(
+            f"Structured output parse failed ({parsed.error});"
+            f" asking the model to correct itself (attempt {attempt})..."
+        )
+        _record_llm_event(
+            "llm_parse_repair",
+            {"error": parsed.error[:500], "attempt": attempt},
+        )
+        correction_messages = [
+            *messages,
+            response if isinstance(response, BaseMessage) else AIMessage(content=text),
+            HumanMessage(
+                content=(
+                    "Your previous reply could not be parsed as the required"
+                    f" JSON ({parsed.error}). Re-emit ONLY the corrected JSON"
+                    " payload, with no surrounding prose or code fences."
+                )
+            ),
+        ]
+        response = await acomplete(llm, correction_messages)
+        text = content_to_text(getattr(response, "content", ""))
+        parsed = parse_structured(text, schema=schema)
+    if isinstance(parsed, ParseFailure):
+        _record_llm_event(
+            "llm_gave_up",
+            {
+                "error": f"structured output unparseable: {parsed.error[:400]}",
+                "category": "bad_request",
+                "retryable": False,
+            },
+            status="failed",
+        )
+        raise StructuredOutputError(parsed)
+    return parsed
 
 
 async def invoke_llm_with_timeout_message[T](
@@ -578,9 +847,7 @@ async def invoke_llm_with_timeout_message[T](
     llm_task = asyncio.create_task(llm_call)
     waiter_task = asyncio.create_task(asyncio.sleep(timeout_seconds))
     try:
-        done, _ = await asyncio.wait(
-            {llm_task, waiter_task}, return_when=asyncio.FIRST_COMPLETED
-        )
+        done, _ = await asyncio.wait({llm_task, waiter_task}, return_when=asyncio.FIRST_COMPLETED)
 
         if llm_task in done:
             return llm_task.result()
@@ -846,12 +1113,55 @@ async def with_fallback[T](
     fallback_call: Callable[[], Awaitable[T]],
     none_should_fallback: bool = True,
 ) -> T:
+    """Run main_call, switching to fallback_call only when it can actually help.
+
+    Falling back is an explicit, observed decision: the failure is classified
+    and only categories where a different endpoint might succeed trigger the
+    fallback (a bad request would just hide the bug inside a weaker model's
+    output). Every switch is logged at WARNING and recorded as an
+    llm_fallback telemetry event. While main_call runs, the recovery layer
+    knows a fallback exists and hands over immediately on retry exhaustion
+    instead of pausing the task.
+    """
+
+    def _switch(reason: str, category: str | None, error: str | None) -> None:
+        llm_logger.warning(
+            f"❗ Main LLM inference failed ({reason}"
+            f"{f': {error}' if error else ''}). Falling back..."
+        )
+        _record_llm_event(
+            "llm_fallback",
+            {
+                "reason": reason,
+                "category": category,
+                "error": error[:500] if error else None,
+            },
+        )
+
+    # The contextvar must be reset BEFORE fallback_call runs: the fallback has
+    # no further fallback behind it, so its own exhaustion should pause.
+    fallback_token = _FALLBACK_AVAILABLE.set(True)
     try:
         result = await main_call()
-        if result is None and none_should_fallback:
-            llm_logger.warning("Main LLM inference returned None. Falling back...")
-            return await fallback_call()
-        return result
-    except Exception as e:
-        llm_logger.warning(f"❗ Main LLM inference failed: {e}. Falling back...")
+    except (KeyboardInterrupt, asyncio.CancelledError):
+        raise
+    except LLMCallError as e:
+        _FALLBACK_AVAILABLE.reset(fallback_token)
+        if not e.failure.should_fallback:
+            raise
+        _switch("terminal_error", e.failure.category.value, str(e))
         return await fallback_call()
+    except Exception as e:
+        _FALLBACK_AVAILABLE.reset(fallback_token)
+        failure = classify_failure(e)
+        if failure.category is FailureCategory.CANCELLED or not failure.should_fallback:
+            raise
+        _switch("error", failure.category.value, str(e))
+        return await fallback_call()
+    else:
+        _FALLBACK_AVAILABLE.reset(fallback_token)
+
+    if result is None and none_should_fallback:
+        _switch("empty_result", None, None)
+        return await fallback_call()
+    return result

artemis/tools/actions/__init__.py

@@ -1,45 +0,0 @@
-# Copyright 2026 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Universal device action tools."""
-
-from artemis.tools.actions.device_actions import (
-    ClickArgs,
-    InputTextArgs,
-    LongPressArgs,
-    PressKeyArgs,
-    SwipeArgs,
-    WaitForDelayArgs,
-    click,
-    input_text,
-    long_press,
-    press_key,
-    swipe,
-    wait_for_delay,
-)
-
-__all__ = [
-    "click",
-    "long_press",
-    "input_text",
-    "swipe",
-    "press_key",
-    "wait_for_delay",
-    "ClickArgs",
-    "LongPressArgs",
-    "InputTextArgs",
-    "SwipeArgs",
-    "PressKeyArgs",
-    "WaitForDelayArgs",
-]

artemis/tools/actions/device_actions.py

@@ -1,250 +0,0 @@
-# Copyright 2026 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Universal device action tools implemented on top of BaseDeviceDriver."""
-
-import asyncio
-from typing import Any, Literal
-from artemis.drivers.base import BaseDeviceDriver
-from artemis.tools.base import artemis_tool
-from artemis.utils.coordinates import (
-    compute_smart_swipe_coordinates,
-    parse_swipe_parameters,
-)
-from artemis.utils.logger import get_logger
-from pydantic import BaseModel, Field
-
-logger = get_logger(__name__)
-
-
-# Schemas
-class ClickArgs(BaseModel):
-    target: list[int] = Field(
-        ..., description="Click target normalized coordinates [x, y] in 0-1000 scale."
-    )
-
-
-class LongPressArgs(BaseModel):
-    target: list[int] = Field(
-        ..., description="Long press target normalized coordinates [x, y] in 0-1000 scale."
-    )
-    duration: int = Field(1000, description="Long press duration in milliseconds (default 1000).")
-
-
-class InputTextArgs(BaseModel):
-    text: str = Field(..., description="The text content to input.")
-    target: list[int] = Field(
-        ..., description="Input target field normalized coordinates [x, y] in 0-1000 scale."
-    )
-    clear_exist: bool = Field(
-        True, description="Whether to clear existing text in the input box before typing."
-    )
-
-
-class SwipeArgs(BaseModel):
-    direction: Literal["up", "down", "left", "right"] | None = Field(
-        None,
-        description=(
-            "Direction for scrolling and swiping: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
-            " 'down' (drags top-to-bottom, scrolling up to reveal content above),"
-            " 'left' (drags right-to-left, scrolling right),"
-            " 'right' (drags left-to-right, scrolling left)."
-        ),
-    )
-    start: list[int] | None = Field(
-        None,
-        description=(
-            "Start normalized coordinates [start_x, start_y] in 0-1000 scale for precise,"
-            " local interactions (e.g. adjusting sliders, SeekBars, fine range selection, or drag-and-drop)."
-        ),
-    )
-    end: list[int] | None = Field(
-        None,
-        description=(
-            "End normalized coordinates [end_x, end_y] in 0-1000 scale for precise,"
-            " local interactions (e.g. adjusting sliders, SeekBars, fine range selection, or drag-and-drop)."
-        ),
-    )
-    action: Literal["up", "down", "left", "right"] | list[int] | None = Field(
-        None,
-        description=(
-            "Backward-compatible swipe gesture: smart direction string ('up', 'down', 'left', 'right')"
-            " OR precise custom coordinates [start_x, start_y, end_x, end_y] in 0-1000 scale."
-        ),
-    )
-    duration: int = Field(
-        800,
-        description=(
-            "Swipe/drag duration in milliseconds (default 800). For drag-and-drop, "
-            "list reordering, or sliding/adjusting sliders (e.g. volume, brightness, SeekBars), "
-            "set duration >= 1000 (e.g. 1500)."
-        ),
-    )
-
-
-class PressKeyArgs(BaseModel):
-    key: str = Field(
-        ...,
-        description="Key name to press (e.g. 'home', 'back', 'enter', 'delete', 'power').",
-    )
-
-
-class WaitForDelayArgs(BaseModel):
-    seconds: float = Field(
-        ...,
-        description=(
-            "Duration in seconds to wait (e.g., 2 for 2s, 60 for 1 min, 180 for 3 mins)."
-            " Use whenever time needs to elapse for animations, loading, or scheduled delays."
-        ),
-    )
-
-
-# Universal Tool Definitions
-@artemis_tool(
-    name="click",
-    description="[ACTION] Clicks on the specified screen coordinate [x, y] in normalized 0-1000 scale.",
-    args_schema=ClickArgs,
-    category="action",
-)
-async def click(target: list[int], driver: BaseDeviceDriver) -> str:
-    width, height = driver.screen_size
-    abs_x = int(target[0] * width / 1000.0)
-    abs_y = int(target[1] * height / 1000.0)
-    success = await driver.tap(abs_x, abs_y)
-    return (
-        f"Click executed at normalized {target} (pixels: {abs_x}, {abs_y})"
-        if success
-        else f"Click failed at {target}"
-    )
-
-
-@artemis_tool(
-    name="long_press",
-    description="[ACTION] Long presses on the specified screen coordinate [x, y] in normalized 0-1000 scale.",
-    args_schema=LongPressArgs,
-    category="action",
-)
-async def long_press(
-    target: list[int], duration: int = 1000, driver: BaseDeviceDriver = None
-) -> str:
-    width, height = driver.screen_size
-    abs_x = int(target[0] * width / 1000.0)
-    abs_y = int(target[1] * height / 1000.0)
-    success = await driver.long_press(abs_x, abs_y, duration_ms=duration)
-    return (
-        f"Long press executed at normalized {target} for {duration}ms"
-        if success
-        else f"Long press failed at {target}"
-    )
-
-
-@artemis_tool(
-    name="input_text",
-    description="[ACTION] Clicks an input field target [x, y] and types text into it.",
-    args_schema=InputTextArgs,
-    category="action",
-)
-async def input_text(
-    text: str, target: list[int], clear_exist: bool = True, driver: BaseDeviceDriver = None
-) -> str:
-    # First tap to focus
-    width, height = driver.screen_size
-    abs_x = int(target[0] * width / 1000.0)
-    abs_y = int(target[1] * height / 1000.0)
-    await driver.tap(abs_x, abs_y)
-    await asyncio.sleep(0.2)
-
-    # Then input text
-    success = await driver.input_text(text, clear_existing=clear_exist)
-    return (
-        f"Input text '{text}' into target {target}"
-        if success
-        else f"Failed to input text into {target}"
-    )
-
-
-@artemis_tool(
-    name="swipe",
-    description=(
-        "[ACTION] Performs smart directional swipe for browsing, or precise coordinate drag/slider gestures."
-    ),
-    args_schema=SwipeArgs,
-    category="action",
-)
-async def swipe(
-    action: Any = None,
-    direction: str | None = None,
-    start: list[int] | None = None,
-    end: list[int] | None = None,
-    duration: int = 800,
-    driver: BaseDeviceDriver = None,
-    **kwargs: Any,
-) -> str:
-    params = dict(kwargs)
-    if action is not None:
-        params["action"] = action
-    if direction is not None:
-        params["direction"] = direction
-    if start is not None:
-        params["start"] = start
-    if end is not None:
-        params["end"] = end
-    params["duration"] = duration
-
-    kind, target, dur = parse_swipe_parameters(params, default_duration=duration)
-
-    if kind == "direction" and isinstance(target, str):
-        width, height = driver.screen_size
-        sx, sy, ex, ey, smart_dur = compute_smart_swipe_coordinates(
-            direction=target,
-            target=params.get("target"),
-            width=width,
-            height=height,
-            duration=dur,
-        )
-        success = await driver.swipe(sx, sy, ex, ey, duration_ms=smart_dur)
-        return f"Swiped {target}" if success else f"Swipe {target} failed"
-    elif kind == "coords" and isinstance(target, list) and len(target) == 4:
-        width, height = driver.screen_size
-        start_x = int(target[0] * width / 1000.0)
-        start_y = int(target[1] * height / 1000.0)
-        end_x = int(target[2] * width / 1000.0)
-        end_y = int(target[3] * height / 1000.0)
-        success = await driver.swipe(start_x, start_y, end_x, end_y, duration_ms=dur)
-        return (
-            f"Swiped from {target[:2]} to {target[2:]}" if success else "Swipe coordinates failed"
-        )
-    return f"Invalid swipe action parameter: {action or direction or (start, end)}"
-
-
-@artemis_tool(
-    name="press_key",
-    description="[ACTION] Presses a hardware or system key on the device ('home', 'back', 'enter', 'delete', 'power').",
-    args_schema=PressKeyArgs,
-    category="action",
-)
-async def press_key(key: str, driver: BaseDeviceDriver) -> str:
-    success = await driver.press_key(key)
-    return f"Key '{key}' pressed" if success else f"Failed to press key '{key}'"
-
-
-@artemis_tool(
-    name="wait_for_delay",
-    description="[ACTION] Pauses execution for a specified number of seconds to allow UI loading or transition.",
-    args_schema=WaitForDelayArgs,
-    category="action",
-)
-async def wait_for_delay(seconds: float, driver: BaseDeviceDriver = None) -> str:
-    await asyncio.sleep(seconds)
-    return f"Waited for {seconds} seconds."

artemis/tools/committee_tool.py

@@ -198,13 +198,6 @@ async def _execute_committee(
                     logger.error(f"Failed to build plan and history in committee: {e}")
                     history_summary += f"Error building plan and history: {e}"
 
-                # Append immediate failure reason if present in state
-                replan_reason = (
-                    getattr(state, "operator_replan_reason", None) if state is not None else None
-                )
-                if replan_reason:
-                    history_summary += f"\n\n### Current Failure Reason\n{replan_reason}\n"
-
                 # Read failed plans history to ensure the committee avoids
                 # repeating failed strategies
                 failed_plans_history = ""

artemis/tools/mobile/exec_tools.py

@@ -17,10 +17,8 @@ from typing import Any, Literal
 
 from langchain_core.messages import ToolMessage
 from langchain_core.tools import BaseTool
-from langgraph.types import Command
 from pydantic import BaseModel, Field
 
-from artemis.constants import VALIDATOR_MESSAGES_KEY
 from artemis.context import ArtemisContext
 from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.data_engine.trace import trace_langchain_tool
@@ -206,19 +204,12 @@ class ClickTool(ArtemisTool):
             success = False
             outcome = f"Error during click: {e}"
 
-        if st and callable(getattr(st, "asanitize_update", None)):
-            tool_message = ToolMessage(
+        if st is not None:
+            return ToolMessage(
                 tool_call_id=tcid or "",
                 content=outcome,
                 status="success" if success else "error",
             )
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={VALIDATOR_MESSAGES_KEY: [tool_message]},
-                    agent="validator",
-                )
-            )
 
         return outcome
 
@@ -306,19 +297,12 @@ class LongPressTool(ArtemisTool):
             success = False
             outcome = f"Error during long press: {e}"
 
-        if st and callable(getattr(st, "asanitize_update", None)):
-            tool_message = ToolMessage(
+        if st is not None:
+            return ToolMessage(
                 tool_call_id=tcid or "",
                 content=outcome,
                 status="success" if success else "error",
             )
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={VALIDATOR_MESSAGES_KEY: [tool_message]},
-                    agent="validator",
-                )
-            )
 
         return outcome
 
@@ -414,19 +398,12 @@ class InputTextTool(ArtemisTool):
             success = False
             outcome = f"Error during input text: {e}"
 
-        if st and callable(getattr(st, "asanitize_update", None)):
-            tool_message = ToolMessage(
+        if st is not None:
+            return ToolMessage(
                 tool_call_id=tcid or "",
                 content=outcome,
                 status="success" if success else "error",
             )
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={VALIDATOR_MESSAGES_KEY: [tool_message]},
-                    agent="validator",
-                )
-            )
 
         return outcome
 
@@ -545,7 +522,7 @@ class SwipeTool(ArtemisTool):
 
                 if kind == "direction" and isinstance(target, str):
                     indexed_elems = getattr(state, "indexed_elements", None) if state else None
-                    ui_hier = getattr(state, "ui_tree", None) if state else None
+                    ui_hier = getattr(state, "latest_ui_hierarchy", None) if state else None
                     start_x, start_y, end_x, end_y, smart_dur = compute_smart_swipe_coordinates(
                         direction=target,
                         target=params.get("target"),
@@ -575,19 +552,12 @@ class SwipeTool(ArtemisTool):
             success = False
             outcome = f"Error during swipe: {e}"
 
-        if st and callable(getattr(st, "asanitize_update", None)):
-            tool_message = ToolMessage(
+        if st is not None:
+            return ToolMessage(
                 tool_call_id=tcid or "",
                 content=outcome,
                 status="success" if success else "error",
             )
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={VALIDATOR_MESSAGES_KEY: [tool_message]},
-                    agent="validator",
-                )
-            )
 
         return outcome
 
@@ -664,19 +634,12 @@ class PressKeyTool(ArtemisTool):
             success = False
             outcome = f"Error during press key: {e}"
 
-        if st and callable(getattr(st, "asanitize_update", None)):
-            tool_message = ToolMessage(
+        if st is not None:
+            return ToolMessage(
                 tool_call_id=tcid or "",
                 content=outcome,
                 status="success" if success else "error",
             )
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={VALIDATOR_MESSAGES_KEY: [tool_message]},
-                    agent="validator",
-                )
-            )
 
         return outcome
 

artemis/tools/mobile/launch_app.py

@@ -16,11 +16,9 @@ from typing import Any
 
 from langchain_core.messages import ToolMessage
 from langchain_core.tools import BaseTool
-from langgraph.types import Command
 from pydantic import BaseModel, Field
 
 from artemis.agents.hopper.hopper import HopperOutput, hopper
-from artemis.constants import VALIDATOR_MESSAGES_KEY
 from artemis.context import ArtemisContext
 from artemis.controllers.platform_specific_commands_controller import (
     list_packages_async,
@@ -176,23 +174,14 @@ class LaunchAppTool(ArtemisTool):
             outcome = f"Error during launch app: {e}"
             error_msg = str(e)
 
-        if st and callable(getattr(st, "asanitize_update", None)):
+        if st is not None:
             additional_kwargs = {} if success else ({"error": error_msg} if error_msg else {})
-            tool_message = ToolMessage(
+            return ToolMessage(
                 tool_call_id=tcid or "",
                 content=outcome,
                 additional_kwargs=additional_kwargs,
                 status="success" if success else "error",
             )
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={
-                        VALIDATOR_MESSAGES_KEY: [tool_message],
-                    },
-                    agent="validator",
-                )
-            )
 
         return outcome
 

artemis/tools/mobile/ocr.py

@@ -19,10 +19,8 @@ from typing import Any
 
 from langchain_core.messages import ToolMessage
 from langchain_core.tools import BaseTool
-from langgraph.types import Command
 from pydantic import BaseModel
 
-from artemis.constants import VALIDATOR_MESSAGES_KEY
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import trace_langchain_tool
 from artemis.drivers.base import BaseDeviceDriver
@@ -169,8 +167,8 @@ class OcrRecognitionTool(ArtemisTool):
             has_failed = True
             error_message = f"Failed to perform OCR: {str(e)}"
 
-        if st and callable(getattr(st, "asanitize_update", None)):
-            tool_message = ToolMessage(
+        if st is not None:
+            return ToolMessage(
                 tool_call_id=tcid or "",
                 content=ocr_recognition_wrapper.on_failure_fn(error_message)
                 if has_failed
@@ -178,15 +176,6 @@ class OcrRecognitionTool(ArtemisTool):
                 additional_kwargs={"error": error_message} if has_failed else {},
                 status="error" if has_failed else "success",
             )
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={
-                        VALIDATOR_MESSAGES_KEY: [tool_message],
-                    },
-                    agent="validator",
-                )
-            )
 
         if has_failed:
             return ocr_recognition_wrapper.on_failure_fn(error_message)

artemis/tools/object_detection_tool.py

@@ -95,7 +95,7 @@ async def _run_object_detection_logic(
 
     if not target_image:
         error_msg = "No image path provided and no screenshot available in state."
-        if state and callable(getattr(state, "asanitize_update", None)):
+        if state is not None:
             return await _create_error_command(ctx, state, tool_call_id, error_msg, active_wrapper)
         return f"Object Detection failed: {error_msg}"
 
@@ -107,12 +107,12 @@ async def _run_object_detection_logic(
             span.result = result
             output = json.dumps(result) if not isinstance(result, str) else result
 
-        if state and callable(getattr(state, "asanitize_update", None)):
+        if state is not None:
             return await _create_success_command(ctx, state, tool_call_id, output, active_wrapper)
         return output
     except Exception as e:
         logger.error(f"Object detection execution failed: {e}")
-        if state and callable(getattr(state, "asanitize_update", None)):
+        if state is not None:
             return await _create_error_command(ctx, state, tool_call_id, str(e), active_wrapper)
         return f"Object Detection failed: {e}"
 

artemis/tools/scratchpad.py

@@ -16,10 +16,8 @@ from typing import Any
 
 from langchain_core.messages import ToolMessage
 from langchain_core.tools.base import BaseTool
-from langgraph.types import Command
 from pydantic import BaseModel, Field
 
-from artemis.constants import VALIDATOR_MESSAGES_KEY
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import trace_langchain_tool
 from artemis.drivers.base import BaseDeviceDriver
@@ -155,27 +153,13 @@ class SaveNoteTool(ArtemisTool):
             agent_outcome = f"Failed to save note {k}.md: {e}"
             status = "error"
 
-        if st and callable(getattr(st, "asanitize_update", None)):
-            tool_message = ToolMessage(
+        if st is not None:
+            return ToolMessage(
                 tool_call_id=tcid,
                 content=agent_outcome,
                 status=status,
             )
 
-            current_agent = "operator"
-            if hasattr(st, "current_agent") and st.current_agent:
-                current_agent = st.current_agent
-
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={
-                        VALIDATOR_MESSAGES_KEY: [tool_message],
-                    },
-                    agent=current_agent,
-                ),
-            )
-
         return agent_outcome
 
 
@@ -249,27 +233,13 @@ class AppendNoteTool(ArtemisTool):
             agent_outcome = f"Failed to append note {k}.md: {e}"
             status = "error"
 
-        if st and callable(getattr(st, "asanitize_update", None)):
-            tool_message = ToolMessage(
+        if st is not None:
+            return ToolMessage(
                 tool_call_id=tcid,
                 content=agent_outcome,
                 status=status,
             )
 
-            current_agent = "operator"
-            if hasattr(st, "current_agent") and st.current_agent:
-                current_agent = st.current_agent
-
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={
-                        VALIDATOR_MESSAGES_KEY: [tool_message],
-                    },
-                    agent=current_agent,
-                ),
-            )
-
         return agent_outcome
 
 
@@ -348,27 +318,13 @@ class UpdateNoteTool(ArtemisTool):
             agent_outcome = f"Failed to update note '{k}': {e}"
             status = "error"
 
-        if st and callable(getattr(st, "asanitize_update", None)):
-            tool_message = ToolMessage(
+        if st is not None:
+            return ToolMessage(
                 tool_call_id=tcid,
                 content=agent_outcome,
                 status=status,
             )
 
-            current_agent = "operator"
-            if hasattr(st, "current_agent") and st.current_agent:
-                current_agent = st.current_agent
-
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={
-                        VALIDATOR_MESSAGES_KEY: [tool_message],
-                    },
-                    agent=current_agent,
-                ),
-            )
-
         return agent_outcome
 
 
@@ -447,27 +403,13 @@ class ReadNoteTool(ArtemisTool):
                 agent_outcome = f"Failed to read note '{k}': {e}"
             status = "error"
 
-        if st and callable(getattr(st, "asanitize_update", None)):
-            tool_message = ToolMessage(
+        if st is not None:
+            return ToolMessage(
                 tool_call_id=tcid,
                 content=agent_outcome,
                 status=status,
             )
 
-            current_agent = "operator"
-            if hasattr(st, "current_agent") and st.current_agent:
-                current_agent = st.current_agent
-
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={
-                        VALIDATOR_MESSAGES_KEY: [tool_message],
-                    },
-                    agent=current_agent,
-                ),
-            )
-
         return agent_outcome
 
 
@@ -523,27 +465,13 @@ class ListNotesTool(ArtemisTool):
             agent_outcome = f"Failed to list notes: {e}"
             status = "error"
 
-        if st and callable(getattr(st, "asanitize_update", None)):
-            tool_message = ToolMessage(
+        if st is not None:
+            return ToolMessage(
                 tool_call_id=tcid,
                 content=agent_outcome,
                 status=status,
             )
 
-            current_agent = "operator"
-            if hasattr(st, "current_agent") and st.current_agent:
-                current_agent = st.current_agent
-
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={
-                        VALIDATOR_MESSAGES_KEY: [tool_message],
-                    },
-                    agent=current_agent,
-                ),
-            )
-
         return agent_outcome
 
 

artemis/tools/tool_wrapper.py

@@ -19,13 +19,12 @@ from typing import Annotated, Any, get_args, get_origin
 from unittest.mock import MagicMock, Mock
 import uuid
 
+from langchain_core.messages import ToolMessage
 from langchain_core.tools import BaseTool
 from langchain_core.tools.base import InjectedToolCallId
 from langgraph.prebuilt import InjectedState
-from langgraph.types import Command
 from pydantic import BaseModel
 
-from artemis.constants import VALIDATOR_MESSAGES_KEY
 from artemis.context import ArtemisContext
 from artemis.data_engine import engine as engine_mod
 from artemis.data_engine.trace import CURRENT_TRACE_ID, smart_serialize
@@ -169,12 +168,7 @@ async def invoke_tool_with_injection(
 
 
 def get_tool_result_content(result: Any) -> Any:
-    """Extracts the content from a tool result, handling Command objects."""
-    if isinstance(result, Command):
-        updates = result.update
-        if VALIDATOR_MESSAGES_KEY in updates:
-            msgs = updates[VALIDATOR_MESSAGES_KEY]
-            if msgs and hasattr(msgs[0], "content"):
-                return msgs[0].content
-        return result
+    """Extracts the content from a tool result, handling ToolMessage returns."""
+    if isinstance(result, ToolMessage):
+        return result.content
     return result

artemis/tools/video_tool.py

@@ -23,11 +23,9 @@ from typing import Any, Literal
 
 from langchain_core.messages import ToolMessage
 from langchain_core.tools import BaseTool
-from langgraph.types import Command
 from pydantic import BaseModel, Field
 
 from artemis.agents.video_analyzer.video_analyzer import VideoAnalyzer
-from artemis.constants import VALIDATOR_MESSAGES_KEY
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import trace_langchain_tool
 from artemis.drivers.base import BaseDeviceDriver
@@ -148,21 +146,12 @@ class VideoAnalyzerTool(ArtemisTool):
             agent_outcome = f"Error running video analyzer: {e}"
             status = "error"
 
-        if st and callable(getattr(st, "asanitize_update", None)):
-            tool_message = ToolMessage(
+        if st is not None:
+            return ToolMessage(
                 tool_call_id=tcid or "",
                 content=agent_outcome,
                 status=status,
             )
-            return Command(
-                update=await st.asanitize_update(
-                    ctx=ctx,
-                    update={
-                        VALIDATOR_MESSAGES_KEY: [tool_message],
-                    },
-                    agent="validator",
-                ),
-            )
 
         if status == "failed":
             return f"Video analysis failed: {agent_outcome}"

artemis/utils/file.py

@@ -13,11 +13,8 @@
 # limitations under the License.
 
 import json
-from pathlib import Path
 import re
-import shutil
 from typing import IO
-from unittest.mock import MagicMock
 
 
 def strip_json_comments(text: str) -> str:
@@ -28,28 +25,3 @@ def strip_json_comments(text: str) -> str:
 
 def load_jsonc(file: IO) -> dict:
     return json.loads(strip_json_comments(file.read()))
-
-
-def create_snapshot(src_dir: Path, snapshot_dir: Path):
-    """Creates a snapshot of a directory by copying it."""
-    if (
-        isinstance(src_dir, MagicMock)
-        or isinstance(snapshot_dir, MagicMock)
-        or "MagicMock" in str(src_dir)
-        or "MagicMock" in str(snapshot_dir)
-    ):
-        return
-    if not src_dir.exists():
-        return
-    if snapshot_dir.exists():
-        shutil.rmtree(snapshot_dir)
-    shutil.copytree(src_dir, snapshot_dir)
-
-
-def restore_snapshot(snapshot_dir: Path, dst_dir: Path):
-    """Restores a directory from a snapshot."""
-    if not snapshot_dir.exists():
-        raise FileNotFoundError(f"Snapshot directory {snapshot_dir} does not exist.")
-    if dst_dir.exists():
-        shutil.rmtree(dst_dir)
-    shutil.copytree(snapshot_dir, dst_dir)

artemis/utils/plan_grammar.py

@@ -0,0 +1,362 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Single source of truth for the task-plan grammar (the "machine channel").
+
+``task_plan.md`` is a dual-channel document:
+
+- **Machine channel**: the checkbox micro-grammar parsed here (status chars,
+  indentation, ``[Loop]`` / ``[Loop:continuous]`` tags). Harness code may only
+  ever read this channel.
+- **Semantic channel**: all remaining free text. Only LLM agents interpret it;
+  harness code must never branch on wording.
+
+Every deterministic decision the harness makes about the plan (termination,
+loop protection, validation/checker triggers, hand-slip detection) derives
+from the :class:`PlanSnapshot` produced by :func:`parse_plan`. The same
+constants render :data:`PLAN_GRAMMAR_SPEC`, the grammar description injected
+into agent prompts via the ``{{ plan_grammar }}`` template variable — so the
+contract the agents are taught and the contract the code enforces cannot
+drift apart.
+"""
+
+from __future__ import annotations
+
+import hashlib
+import re
+from collections import Counter
+from dataclasses import dataclass, field
+from typing import Literal
+
+STATUS_PENDING = " "
+STATUS_ACTIVE = "/"
+STATUS_DONE = "x"
+STATUS_BLOCKED = "!"
+STATUS_CHARS = STATUS_PENDING + STATUS_ACTIVE + STATUS_DONE + STATUS_BLOCKED
+
+LOOP_TAG = "[Loop]"
+CONTINUOUS_LOOP_TAG = "[Loop:continuous]"
+
+CHECKBOX_LINE_RE = re.compile(
+    r"^(?P<indent>\s*)-\s*\[(?P<status>[" + re.escape(STATUS_CHARS) + r"])\]\s*(?P<text>.*)$"
+)
+_LOOP_TAG_RE = re.compile(r"\[Loop(?::continuous)?\]")
+_CONTINUOUS_TAG_RE = re.compile(re.escape(CONTINUOUS_LOOP_TAG))
+
+#: Check lines are indented attachments under a top-level subgoal
+#: (``- verify: ...`` / ``- assert: ...``) or task-level free items
+#: (``- assert@end: ...`` at zero indent). The optional ``@end`` suffix moves
+#: the judgment moment from the anchored subgoal's completion to task exit.
+CHECK_LINE_RE = re.compile(
+    r"^(?P<indent>\s*)-\s*(?P<kind>verify|assert)(?P<at_end>@end)?\s*:\s*(?P<text>\S.*?)\s*$"
+)
+
+
+def subgoal_hash(text: str) -> str:
+    """Stable identity hash for a subgoal text (used for per-subgoal artifacts)."""
+    return hashlib.md5(text.strip().encode("utf-8")).hexdigest()
+
+
+@dataclass(frozen=True)
+class PlanItem:
+    """One checkbox line of the plan's machine channel."""
+
+    indent: int
+    status: str
+    text: str
+    line_no: int
+
+    @property
+    def is_top_level(self) -> bool:
+        return self.indent == 0
+
+    @property
+    def is_pending(self) -> bool:
+        return self.status == STATUS_PENDING
+
+    @property
+    def is_active(self) -> bool:
+        return self.status == STATUS_ACTIVE
+
+    @property
+    def is_done(self) -> bool:
+        return self.status == STATUS_DONE
+
+    @property
+    def is_loop(self) -> bool:
+        return bool(_LOOP_TAG_RE.search(self.text))
+
+    @property
+    def is_continuous(self) -> bool:
+        return bool(_CONTINUOUS_TAG_RE.search(self.text))
+
+    @property
+    def key(self) -> str:
+        return subgoal_hash(self.text)
+
+
+@dataclass(frozen=True)
+class CheckItem:
+    """One declared check line of the plan's machine channel.
+
+    ``verify`` marks a completion acceptance criterion (a failure triggers
+    repair); ``assert`` marks a test assertion (a failure is recorded as a
+    legitimate test result, never repaired).
+    """
+
+    kind: Literal["verify", "assert"]
+    when: Literal["on_complete", "at_end"]
+    text: str
+    parent_key: str | None
+
+    @property
+    def key(self) -> str:
+        """Stable identity of the check line (kind + when + text + anchor)."""
+        return subgoal_hash(f"{self.kind}|{self.when}|{self.parent_key}|{self.text}")
+
+
+@dataclass(frozen=True)
+class PlanSnapshot:
+    """Read-only derived view of a plan. Never persisted, never written back."""
+
+    items: tuple[PlanItem, ...]
+    check_items: tuple[CheckItem, ...] = field(default=())
+
+    @property
+    def top_level(self) -> tuple[PlanItem, ...]:
+        return tuple(i for i in self.items if i.is_top_level)
+
+    @property
+    def has_top_level(self) -> bool:
+        return bool(self.top_level)
+
+    @property
+    def all_top_level_done(self) -> bool:
+        top = self.top_level
+        return bool(top) and all(i.is_done for i in top)
+
+    @property
+    def continuous_top_level(self) -> tuple[PlanItem, ...]:
+        return tuple(i for i in self.top_level if i.is_continuous)
+
+    @property
+    def milestone_texts(self) -> tuple[str, ...]:
+        return tuple(i.text.strip() for i in self.top_level)
+
+    def first_active(self) -> PlanItem | None:
+        for item in self.items:
+            if item.is_active:
+                return item
+        return None
+
+    def last_active(self) -> PlanItem | None:
+        for item in reversed(self.items):
+            if item.is_active:
+                return item
+        return None
+
+    def parent_of(self, item: PlanItem) -> PlanItem | None:
+        """Nearest top-level item above an indented item."""
+        parent = None
+        for candidate in self.items:
+            if candidate.line_no >= item.line_no:
+                break
+            if candidate.is_top_level:
+                parent = candidate
+        return parent
+
+    def check_items_of(self, item: PlanItem) -> tuple[CheckItem, ...]:
+        """Check lines anchored to the given top-level subgoal."""
+        return tuple(ci for ci in self.check_items if ci.parent_key == item.key)
+
+    @property
+    def task_level_check_items(self) -> tuple[CheckItem, ...]:
+        return tuple(ci for ci in self.check_items if ci.parent_key is None)
+
+    @property
+    def all_check_items(self) -> tuple[CheckItem, ...]:
+        return self.check_items
+
+
+def parse_plan(content: str | None) -> PlanSnapshot:
+    """Parses plan markdown into its machine channel. Non-checkbox lines are ignored.
+
+    Check lines (``- verify:`` / ``- assert:``) are parsed into
+    :class:`CheckItem` entries; they never contribute to the checkbox item list
+    and therefore never influence milestone hashing, drift detection, or
+    completion triggers.
+    """
+    items: list[PlanItem] = []
+    checks: list[CheckItem] = []
+    current_top: PlanItem | None = None
+    if content:
+        for line_no, line in enumerate(content.splitlines()):
+            match = CHECKBOX_LINE_RE.match(line)
+            if match:
+                item = PlanItem(
+                    indent=len(match.group("indent")),
+                    status=match.group("status"),
+                    text=match.group("text").strip(),
+                    line_no=line_no,
+                )
+                items.append(item)
+                if item.is_top_level:
+                    current_top = item
+                continue
+            check_match = CHECK_LINE_RE.match(line)
+            if check_match:
+                indented = bool(check_match.group("indent"))
+                at_end = bool(check_match.group("at_end"))
+                parent_key = current_top.key if (indented and current_top) else None
+                checks.append(
+                    CheckItem(
+                        kind=check_match.group("kind"),
+                        when="at_end" if at_end else "on_complete",
+                        text=check_match.group("text").strip(),
+                        parent_key=parent_key,
+                    )
+                )
+    return PlanSnapshot(items=tuple(items), check_items=tuple(checks))
+
+
+def milestones_changed(before: PlanSnapshot, after: PlanSnapshot) -> bool:
+    """True if the top-level milestone texts differ (status flips don't count)."""
+    return before.milestone_texts != after.milestone_texts
+
+
+def _check_multiset(snapshot: PlanSnapshot) -> Counter:
+    return Counter((ci.kind, ci.when, ci.text, ci.parent_key) for ci in snapshot.check_items)
+
+
+def check_items_changed(before: PlanSnapshot, after: PlanSnapshot) -> bool:
+    """True if the multiset of check lines differs between the two snapshots."""
+    return _check_multiset(before) != _check_multiset(after)
+
+
+def missing_check_items(before: PlanSnapshot, after: PlanSnapshot) -> list[CheckItem]:
+    """Check items present in ``before`` but absent from ``after`` (multiset diff).
+
+    Additions in ``after`` are ignored — adding check lines is allowed; only
+    deletions/rewrites of existing ones are guarded.
+    """
+    deficit = _check_multiset(before) - _check_multiset(after)
+    missing: list[CheckItem] = []
+    for ci in before.check_items:
+        sig = (ci.kind, ci.when, ci.text, ci.parent_key)
+        if deficit.get(sig, 0) > 0:
+            deficit[sig] -= 1
+            missing.append(ci)
+    return missing
+
+
+def render_check_line(item: CheckItem, indent: int = 2) -> str:
+    suffix = "@end" if item.when == "at_end" else ""
+    prefix = " " * indent if item.parent_key is not None else ""
+    return f"{prefix}- {item.kind}{suffix}: {item.text}"
+
+
+def restore_missing_check_items(content_before: str, content_after: str) -> str | None:
+    """Deterministic guard: merge check lines deleted/rewritten by a plan write
+    back into the new content.
+
+    Returns the merged plan text, or ``None`` when nothing is missing. Lines
+    whose parent subgoal still exists are re-anchored directly under it; lines
+    whose parent subgoal was removed are converted to task-level ``@end`` items
+    appended at the end of the plan.
+    """
+    before = parse_plan(content_before)
+    after = parse_plan(content_after)
+    missing = missing_check_items(before, after)
+    if not missing:
+        return None
+
+    lines = (content_after or "").splitlines()
+    top_line_by_key: dict[str, int] = {}
+    for item in after.items:
+        if item.is_top_level:
+            top_line_by_key.setdefault(item.key, item.line_no)
+
+    inserts: dict[int, list[str]] = {}
+    tail: list[str] = []
+    for ci in missing:
+        if ci.parent_key is not None and ci.parent_key in top_line_by_key:
+            inserts.setdefault(top_line_by_key[ci.parent_key], []).append(render_check_line(ci))
+        else:
+            orphan = CheckItem(kind=ci.kind, when="at_end", text=ci.text, parent_key=None)
+            tail.append(render_check_line(orphan))
+
+    merged: list[str] = []
+    for line_no, line in enumerate(lines):
+        merged.append(line)
+        if line_no in inserts:
+            merged.extend(inserts[line_no])
+    merged.extend(tail)
+    return "\n".join(merged) + ("\n" if (content_after or "").endswith("\n") else "")
+
+
+def new_top_level_completions(before: PlanSnapshot, after: PlanSnapshot) -> list[str]:
+    """Top-level texts newly marked done, in plan order."""
+    before_done = {i.text for i in before.top_level if i.is_done}
+    return [i.text for i in after.top_level if i.is_done and i.text not in before_done]
+
+
+def unintended_milestone_edits(before: PlanSnapshot, after: PlanSnapshot) -> list[tuple[str, str]]:
+    """Hand-slip signature of a full-file rewrite: positional top-level pairs
+    whose status char is unchanged but whose text was reworded.
+
+    Only defined when the milestone count is unchanged; structural changes
+    (add/remove) are a declared replan and are judged by planner validation
+    instead.
+    """
+    top_before = before.top_level
+    top_after = after.top_level
+    if len(top_before) != len(top_after):
+        return []
+    return [
+        (b.text, a.text)
+        for b, a in zip(top_before, top_after)
+        if b.status == a.status and b.text != a.text
+    ]
+
+
+_PLAN_GRAMMAR_BASE = f"""### Task Plan Grammar (machine-enforced contract)
+Every checklist line MUST match `<indent>- [<status>] <text>`:
+- Status characters: `[{STATUS_PENDING}]` pending, `[{STATUS_ACTIVE}]` in progress, `[{STATUS_DONE}]` completed, `[{STATUS_BLOCKED}]` blocked.
+- Top-level lines (no indentation) are strategic milestones; their sub-tasks are indented by 2 spaces. All other text in the note is free-form context.
+- `{LOOP_TAG}` tags a BOUNDED iterative milestone: declare its exit boundary (e.g., `(Exit: <condition>; Interval: <cadence>)`) and mark it `[{STATUS_DONE}]` only once that exit condition is verifiably met.
+- `{CONTINUOUS_LOOP_TAG}` tags an UNBOUNDED continuous-monitoring milestone: it must stay `[{STATUS_ACTIVE}]` and can never be marked `[{STATUS_DONE}]`, deleted, or untagged by you — the system mechanically rejects such edits. Only an explicit external stop signal injected by the user unlocks its completion.
+- The task terminates only when every top-level milestone is `[{STATUS_DONE}]`."""
+
+_CHECK_GRAMMAR_EXTENSION = """### Check Line Grammar (declared verification standards)
+A top-level milestone may carry indented check lines, and the plan may end with task-level check lines at zero indentation:
+- `  - verify: <expected state>` — an acceptance criterion for its parent milestone. It is judged by an independent Checker at the moment the milestone is marked completed; a failed verify triggers a repair loop.
+- `  - assert: <expected observation>` — a test assertion. It is judged by the independent Checker and a failure is recorded verbatim as a legitimate test result; assertions are NEVER repaired, worked around, or satisfied by constructing state.
+- `- verify@end: ...` / `- assert@end: ...` — the `@end` suffix defers judgment to task exit, using the final device state. Items without `@end` are judged with the evidence available when their anchored milestone completes.
+- Capability boundary (declare honestly, never promise more): all checks are POST-HOC audits over recorded evidence. They detect and record violations after the fact; they CANNOT block an action from happening, so "B must not run before A is confirmed" ordering rules are verified retroactively, not enforced. Transient prompts that were never captured in the execution history cannot be recovered — prefer expressing expectations as persistently probeable state."""
+
+
+def render_plan_grammar_spec(include_checks: bool) -> str:
+    """Renders the grammar spec taught to agents.
+
+    ``include_checks=False`` (both check gates disabled) returns exactly the
+    base grammar — zero context pollution from the checking feature.
+    """
+    if include_checks:
+        return _PLAN_GRAMMAR_BASE + "\n\n" + _CHECK_GRAMMAR_EXTENSION
+    return _PLAN_GRAMMAR_BASE
+
+
+#: Backward-compatible module constant: the base grammar without the check
+#: extension. Prompt injection sites use :func:`render_plan_grammar_spec`.
+PLAN_GRAMMAR_SPEC = _PLAN_GRAMMAR_BASE

artemis/utils/task_tree.py

@@ -13,12 +13,13 @@
 # limitations under the License.
 
 import ast
-import hashlib
 import json
 from pathlib import Path
 import re
 from typing import Any
 
+from artemis.utils.plan_grammar import parse_plan
+
 
 def format_action_clean(action_obj) -> str:
     if not action_obj:
@@ -952,53 +953,26 @@ def get_active_subgoal_hashes(task_plan: str) -> tuple[str, str | None]:
         return "default", None
 
     try:
-        lines = task_plan.split("\n")
-        active_line_idx = -1
-        # Search from bottom to top to prioritize the most specific deeply nested active subgoal
-        for i in range(len(lines) - 1, -1, -1):
-            if lines[i].strip().startswith("- [/]"):
-                active_line_idx = i
-                break
+        snapshot = parse_plan(task_plan)
 
-        if active_line_idx != -1:
-            active_line = lines[active_line_idx]
-            active_text = active_line.strip()[5:].strip()
-
-            # Check if it is indented (sub-subgoal)
-            if active_line.startswith(" "):
-                # Find parent (first non-indented line above it)
-                parent_text = ""
-                for i in range(active_line_idx - 1, -1, -1):
-                    if not lines[i].startswith(" ") and lines[i].strip().startswith("- ["):
-                        parent_text = lines[i].strip()[5:].strip()
-                        break
-
-                # Consolidate under the parent Level 1 subgoal
-                if parent_text:
-                    parent_hash = hashlib.md5(parent_text.encode("utf-8")).hexdigest()
-                    return parent_hash, None
-
-            # If not indented, it is a top-level Level 1 subgoal
-            active_hash = hashlib.md5(active_text.encode("utf-8")).hexdigest()
-            return active_hash, None
+        # Bottom-most active item wins (most specific deeply nested subgoal)
+        active_item = snapshot.last_active()
+        if active_item is not None:
+            if not active_item.is_top_level:
+                parent = snapshot.parent_of(active_item)
+                if parent is not None:
+                    return parent.key, None
+            return active_item.key, None
 
         # No active subgoal found. Check if all are completed.
-        top_level_lines = [line for line in lines if line.strip().startswith("- [")]
-        all_done = all(line.strip().startswith("- [x]") for line in top_level_lines)
-        if all_done:
+        if all(item.is_done for item in snapshot.top_level):
             return "default", None
 
-        # SAFE FALLBACK: Check if all subgoals are not started (all are "[ ]")
-        all_pending = all(line.strip().startswith("- [ ]") for line in top_level_lines)
-        if all_pending:
-            # If all are pending, we are at the start of the task.
-            # The active subgoal is guaranteed to be the first one.
-            for line in lines:
-                stripped = line.strip()
-                if stripped.startswith("- [ ]"):
-                    first_pending_text = stripped[5:].strip()
-                    first_hash = hashlib.md5(first_pending_text.encode("utf-8")).hexdigest()
-                    return first_hash, None
+        # SAFE FALLBACK: if every item is untouched, the first pending one is active.
+        if all(item.is_pending for item in snapshot.top_level):
+            for item in snapshot.items:
+                if item.is_pending:
+                    return item.key, None
 
     except Exception:
         pass
@@ -1011,13 +985,7 @@ def get_completed_subgoal_hashes(task_plan: str) -> set[str]:
     if not task_plan:
         return set()
 
-    completed_hashes = set()
-
-    pattern = re.compile(r"^-\s*\[x\]\s*(.*)$", re.MULTILINE)
-    for match in pattern.findall(task_plan):
-        completed_hashes.add(hashlib.md5(match.strip().encode("utf-8")).hexdigest())
-
-    return completed_hashes
+    return {item.key for item in parse_plan(task_plan).top_level if item.is_done}
 
 
 def get_all_subgoal_aliases(target_hash: str, base_dir: str | Path | None = None) -> set[str]:

artemis/utils/verification.py

@@ -1,62 +0,0 @@
-# Copyright 2026 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-import json
-from pathlib import Path
-from artemis.utils.logger import get_logger
-from artemis.utils.notes import get_notes_dir
-
-logger = get_logger(__name__)
-
-
-def get_verification_chat_path(base_dir: str | Path, subgoal_hash: str) -> Path:
-    """Gets the path to the verification chat file for a given subgoal."""
-    notes_dir = get_notes_dir(base_dir)
-    return notes_dir / f"verification_chat_{subgoal_hash}.json"
-
-
-def read_verification_chat(chat_path: Path) -> list[dict]:
-    """Reads the verification chat history from the given path."""
-    if not chat_path.exists():
-        return []
-    try:
-        return json.loads(chat_path.read_text(encoding="utf-8"))
-    except Exception as e:
-        logger.error(f"Failed to load verification chat from {chat_path}: {e}")
-        return []
-
-
-def get_verification_chat_rounds(turns: list[dict]) -> tuple[int, int]:
-    """Calculates the maximum rounds for operator and checker from the turns."""
-    max_op = max(
-        [t.get("round", 0) for t in turns if t.get("role") == "operator"],
-        default=0,
-    )
-    max_chk = max(
-        [t.get("round", 0) for t in turns if t.get("role") == "checker"],
-        default=0,
-    )
-    return max_op, max_chk
-
-
-def append_verification_chat(chat_path: Path, role: str, content: str, round_num: int) -> bool:
-    """Appends a new turn to the verification chat file."""
-    turns = read_verification_chat(chat_path)
-    turns.append({"role": role, "round": round_num, "content": content})
-    try:
-        chat_path.write_text(json.dumps(turns, indent=2, ensure_ascii=False), encoding="utf-8")
-        return True
-    except Exception as e:
-        logger.error(f"Failed to save verification chat to {chat_path}: {e}")
-        return False