feat: harden agent verification and runtime contracts

Introduce two-gate checker settlement, explicit graph visibility and checkpoints, structured LLM reliability, centralized action specifications, and corresponding tests and design documentation.

artemis/agents/checker/checker.json

@@ -1,4 +1,9 @@
 {
-  "checker_system": "You are a precise and reliable verification expert (Checker). Your task is to determine whether the Operator has successfully completed the active subgoal by combining visual evidence from the current screenshot with data stored in persistent notes.",
-    "checker_prompt": "# Inputs\n- Active Subgoal: {{ subgoal }}\n- Execution History:\n{{ plan_and_history }}\n- Dialogue History: {{ dialogue_history }}\n- Visual State: The provided screenshot.\n\n# Guidelines\n- **Visual + Data Verification**: You must combine the screenshot with note contents. If the subgoal involves extracting, saving, or recording information (e.g., \"save contacts\", \"record address\"), you **MUST** verify that the content actually exists in the note. Do not assume success just because the information is visible on the screen. If the signs of success are not obvious, you can check the history, especially the most recent steps, to see if the operator performed their own check and whether that check was reasonable, to determine if the subgoal was truly successful.\n- **Visual Priority (Anti-History-Bias)**: Your primary ground truth is the visual evidence on the **Current Screenshot**. The `Execution History` lists previous successes or failures, which may be outdated due to dynamic background recovery or self-healing. If the screenshot demonstrably shows the active subgoal's target state is achieved, you **MUST** mark `success: true`, regardless of previous failed steps. Trust what you see, not past FAILED texts.\n- **Constraint Verification**: For subgoals containing specific parameters (such as names, brands, topics, events, numbers, or dates), confirm that the visible text precisely reflects these specified details. A successful state is defined by a clear, direct match with the subgoal's constraints.\n- **Iteration & Monitoring Subgoals Validation**: For concrete iteration sub-tasks (e.g., inspecting Candidate #1) or periodic polling subgoals (e.g., Polling Round #1 check), verify whether the Operator correctly executed the step and logged the progress/findings into notes or the task plan. Do NOT mark a monitoring or iteration step as failed simply because the overall monitoring session is ongoing or no new incoming events were detected in that specific polling cycle. As long as the discrete check was performed and recorded, mark `success: true`.\n- **Realistic Expectations**: You cannot scroll the screen. If a subgoal requires saving all items but only some are visible on the screen, verify that at least the **visible items** are recorded in the note. As long as the visible part is recorded and there are no obvious errors, consider it a success. Do not fail the task simply because off-screen content cannot be verified.\n- **Success Criteria**: Set `success` to `true` if the core objective of the subgoal is met and it does not block subsequent steps. Set `success` to `false` if the state is incorrect, required data was not saved, or the operator is stuck/looping.\n\n# Output Protocol\nInvoke the `submit_verification` tool with `success` (boolean) and `reason` (string) to submit your final decision. If you need to inspect note data first, use `read_note`."
+  "system": "You are a precise, evidence-driven verification expert (Checker). You audit declared check items against recorded execution evidence and issue verdicts. You never operate the device, never modify any state, and never negotiate with the executing agent — you only observe, probe read-only state where allowed, and judge.",
+  "base_rules": "# Verdict Rules (always apply)\n- Quote every check item verbatim in `item_text` — never paraphrase.\n- Every verdict MUST carry concrete evidence: a specific probe output, a specific note excerpt, or a specific on-screen/screenshot observation. A `failed` verdict without concrete evidence is invalid and will be downgraded.\n- `inconclusive` means the evidence needed for a judgment was never captured or is genuinely ambiguous. When evidence was never collected, output `inconclusive` and state explicitly that the evidence is missing — do NOT output `failed` for lack of evidence.\n- Judge only what the check item declares. Do not invent additional criteria and do not relax the declared ones.",
+  "verify_semantics": "# `verify` items (completion acceptance criteria)\n- A `verify` item asks: was the anchored subgoal actually accomplished to this standard?\n- `failed` verify verdicts trigger a repair loop, so a failed verdict MUST include a `suggestion`: a concrete, actionable hint telling the executor what is missing or wrong.\n- The release decision (whether execution proceeds) is computed outside of you and is fail-open on errors; your job is only the truthful verdict value.",
+  "assert_semantics": "# `assert` items (test assertions)\n- An `assert` item is a test expectation, not an instruction. A failed assertion is a legitimate, valuable test result: record it truthfully with evidence. Nothing will (or should) try to repair it.\n- Never mark an assert `failed` merely because evidence is absent — absence of captured evidence is `inconclusive` with an explicit note that the evidence is missing.\n- `suggestion` is not needed for assert verdicts.",
+  "anchor_guide": "# Evidence anchoring (checkpoint audit)\n- You are auditing a subgoal at its completion moment. The authoritative evidence is the execution history around the anchor step (the anchor step's pre-action screenshot and the preceding step's post-action state) plus persistently probeable device state.\n- For transient expectations (toasts, dialogs, momentary states) and ordering expectations, you MUST inspect the recorded history near the anchor step — the live device has already moved on and is NOT valid evidence for them.\n- Prefer persistent state probes and recorded notes over inference from summaries.",
+  "final_guide": "# Final review (task exit audit)\n- Audit against the USER'S ORIGINAL GOAL, not merely the plan's checkboxes.\n- `@end` items: judge them using the FINAL device state provided to you.\n- `on_complete` items: cite the verdict ledger — an already-recorded PASS/FAIL from the item's checkpoint attempt is authoritative (an assert that ever failed stays failed). For items never checked, judge from the historical evidence near their subgoal's completion; if that evidence was never captured, output `inconclusive` stating the evidence is missing. NEVER re-judge an `on_complete` item from the current screen: the current screen only proves the present, not the past moment the item was declared for.\n- `unmet_subgoals`: list, verbatim, any plan subgoal marked complete whose verify criteria are demonstrably not met. Leave it empty otherwise; failed asserts alone do NOT make a subgoal unmet.",
+  "probe_guide": "# Device probes (read-only)\n- `probe_device` runs a fixed, enumerated set of read-only state probes. Prefer probing persistent state over inferring it: alarms (`alarms`), battery status (`battery`), foreground activities (`foreground`), notifications (`notifications`), a settings value (`setting` with `namespace` + `key`), a content provider query (`content` with `uri`), installed packages (`packages`), a system property (`prop` with `key`).\n- Probes outside this table do not exist; you cannot run arbitrary commands."
 }

artemis/agents/diagnoser/diagnoser.py

@@ -30,7 +30,7 @@ from artemis.data_engine.trace import (
     trace_langchain_tool,
 )
 from artemis.graph.state import State
-from artemis.services.llm import get_llm, invoke_llm_with_timeout_message
+from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
 from artemis.tools.command_tool import get_run_short_adb_command_tool
 from artemis.tools.diagnoser_submit_answer_tool import get_submit_answer_tool
 from artemis.tools.index import get_tool_by_name
@@ -173,21 +173,11 @@ class Diagnoser:
                 logger.error(f"Failed to build plan and history in Diagnoser: {e}")
                 plan_and_history = f"Error building plan and history: {e}"
 
-        # Get Checker Feedback
+        # Get verification findings injected by the checkpoint harvest, if any
         checker_feedback = ""
-        if self.ctx.data_engine:
-            notes_dir = Path(self.ctx.data_engine.base_dir) / "notes"
-            verification_chat_path = notes_dir / f"verification_chat_{subgoal_hash}.json"
-            if verification_chat_path.exists():
-                try:
-                    turns = json.loads(verification_chat_path.read_text(encoding="utf-8"))
-                    dialogue_lines = []
-                    for t in turns:
-                        role = "Operator" if t["role"] == "operator" else "Checker"
-                        dialogue_lines.append(f"**{role} (Round {t['round']})**:\n{t['content']}")
-                    checker_feedback = "\n\n".join(dialogue_lines)
-                except Exception as e:
-                    logger.error(f"Error reading verification chat in Diagnoser: {e}")
+        findings = getattr(state, "operator_feedback", None)
+        if findings:
+            checker_feedback = "\n".join(f"- {f}" for f in findings)
 
         # Get Current Screenshot and UI Hierarchy
         latest_screenshot_b64 = None
@@ -309,18 +299,6 @@ class Diagnoser:
                         )
                     llm = base_llm.bind_tools(tools=traced_tools)
 
-                async def run_stream():
-                    full_response = None
-                    trace_id = CURRENT_TRACE_ID.get()
-                    async for chunk in llm.astream(current_messages):
-                        if full_response is None:
-                            full_response = chunk
-                        else:
-                            full_response += chunk
-
-                        pass
-                    return full_response
-
                 with TraceSpan(name="gemini_diagnoser_call", ctx=self.ctx) as span:
                     span.payload = {
                         "context_components": {
@@ -331,7 +309,9 @@ class Diagnoser:
                             "diagnostic_query": prompt,
                         }
                     }
-                    response = await invoke_llm_with_timeout_message(run_stream())
+                    response = await invoke_llm_with_timeout_message(
+                        acomplete(llm, current_messages)
+                    )
                     span.result = (
                         response.content if hasattr(response, "content") else str(response)
                     )

artemis/agents/flash/runner.py

@@ -49,9 +49,11 @@ from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.data_engine.trace import trace
 from artemis.graph.perception import _check_injected_instruction_file
 from artemis.graph.state import State
+from artemis.llm.structured import ParseFailure, parse_structured
 from artemis.mcp.action_executor import McpActionExecutor
 from artemis.services.llm import (
     RobustChatModelWrapper,
+    acomplete,
     get_google_llm,
     get_llm,
     invoke_llm_with_timeout_message,
@@ -182,20 +184,23 @@ class FlashRunner:
             # Check for real-time injected instructions
             if self.ctx.data_engine and self.ctx.data_engine.base_dir:
                 try:
-                    injected_instruction = await asyncio.to_thread(
+                    injected_payload = await asyncio.to_thread(
                         _check_injected_instruction_file,
                         str(self.ctx.data_engine.base_dir),
                     )
-                    if injected_instruction:
-                        messages.append(
-                            HumanMessage(
-                                content=(
-                                    "[REAL-TIME INJECTED INSTRUCTION from user]:"
-                                    f" {injected_instruction}\nYou MUST immediately follow"
-                                    " this instruction and adjust your plan/actions."
-                                )
-                            )
+                    if injected_payload and injected_payload.get("instruction"):
+                        injected_text = (
+                            "[REAL-TIME INJECTED INSTRUCTION from user]:"
+                            f" {injected_payload['instruction']}\nYou MUST immediately"
+                            " follow this instruction and adjust your plan/actions."
                         )
+                        if injected_payload.get("release_loop"):
+                            injected_text += (
+                                "\nThe user has explicitly authorized stopping any"
+                                " ongoing monitoring loop; you may now wrap up and"
+                                " complete the task."
+                            )
+                        messages.append(HumanMessage(content=injected_text))
                 except Exception as e:
                     logger.warning(f"Failed to check injected instruction in FlashRunner: {e}")
 
@@ -224,61 +229,12 @@ class FlashRunner:
             # Bind active tools
             bound_llm = llm.bind_tools(current_tools)
 
-            # Invoke Model with Streaming & Retry
-            response = None
-            max_retries = 3
-
-            for attempt in range(max_retries):
-                try:
-                    stream_exec_id = uuid.uuid4()
-
-                    async def run_stream():
-                        full_response = None
-                        async for chunk in bound_llm.astream(messages):
-                            if full_response is None:
-                                full_response = chunk
-                            else:
-                                full_response += chunk
-
-                            # Real-time token streaming to DataEngine for live UI display
-                            if (
-                                getattr(self, "ctx", None)
-                                and getattr(self.ctx, "data_engine", None)
-                                and getattr(chunk, "content", None)
-                            ):
-                                text_to_stream = ""
-                                thinking_to_stream = ""
-                                if isinstance(chunk.content, str):
-                                    text_to_stream = chunk.content
-                                elif isinstance(chunk.content, list):
-                                    for item in chunk.content:
-                                        if isinstance(item, str):
-                                            text_to_stream += item
-                                        elif isinstance(item, dict):
-                                            if item.get("type") == "text":
-                                                text_to_stream += item.get("text", "")
-                                            elif item.get("type") == "thinking":
-                                                thinking_to_stream += item.get("thinking", "")
-                                if text_to_stream:
-                                    self.ctx.data_engine.stream_output(
-                                        stream_exec_id, text_to_stream, is_thinking=False
-                                    )
-                                if thinking_to_stream:
-                                    self.ctx.data_engine.stream_output(
-                                        stream_exec_id, thinking_to_stream, is_thinking=True
-                                    )
-                        return full_response
-
-                    response = await invoke_llm_with_timeout_message(
-                        run_stream(), timeout_seconds=10, hard_timeout=180
-                    )
-                    if response:
-                        break
-                except Exception as e:
-                    logger.warning(f"FlashRunner LLM turn failed on attempt {attempt + 1}: {e}")
-                    if attempt == max_retries - 1:
-                        raise e
-                    await asyncio.sleep(2.0**attempt)
+            # Invoke Model. Streaming, live-token UI deltas, classified
+            # retries, and pause/resume are all owned by the LLM gateway
+            # (acomplete); a typed LLMCallError propagates if it gives up.
+            response = await invoke_llm_with_timeout_message(
+                acomplete(bound_llm, messages), timeout_seconds=10, hard_timeout=180
+            )
 
             if response is None:
                 break
@@ -372,19 +328,20 @@ class FlashRunner:
 
             # Fallback text parsing if tool_calls not parsed natively
             if not tool_calls and "```json" in raw_text:
-                try:
-                    snippet = raw_text.split("```json")[1].split("```")[0].strip()
-                    parsed = json.loads(snippet)
-                    if isinstance(parsed, dict) and "name" in parsed:
-                        tool_calls = [
-                            {
-                                "name": parsed["name"],
-                                "args": parsed.get("args", {}),
-                                "id": str(uuid.uuid4()),
-                            }
-                        ]
-                except Exception:
-                    pass
+                parsed = parse_structured(raw_text)
+                if isinstance(parsed, ParseFailure):
+                    logger.warning(
+                        "FlashRunner response contained a JSON block that could"
+                        f" not be parsed: {parsed.error}"
+                    )
+                elif isinstance(parsed, dict) and "name" in parsed:
+                    tool_calls = [
+                        {
+                            "name": parsed["name"],
+                            "args": parsed.get("args", {}),
+                            "id": str(uuid.uuid4()),
+                        }
+                    ]
 
             if not tool_calls:
                 logger.info(

artemis/agents/history_analyzer/history_analyzer.py

@@ -25,7 +25,7 @@ from langchain_core.messages import (
 from langchain_core.tools import BaseTool, tool
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan, trace
-from artemis.services.llm import get_llm, invoke_llm_with_timeout_message
+from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
 from artemis.tools.scratchpad import (
     get_list_notes_tool_pure,
     get_read_note_tool_pure,
@@ -164,19 +164,7 @@ class HistoryAnalyzer:
         for turn in range(max_turns):
             logger.info(f"HistoryAnalyzer ReAct turn {turn + 1}")
 
-            async def run_stream():
-                full_response = None
-                trace_id = CURRENT_TRACE_ID.get()
-                async for chunk in llm.astream(messages):
-                    if full_response is None:
-                        full_response = chunk
-                    else:
-                        full_response += chunk
-
-                    pass
-                return full_response
-
-            response = await invoke_llm_with_timeout_message(run_stream())
+            response = await invoke_llm_with_timeout_message(acomplete(llm, messages))
             messages.append(response)
 
             if not response.tool_calls:

artemis/agents/log_analyzer/log_analyzer.py

@@ -58,7 +58,7 @@ from artemis.data_engine.trace import (
     trace,
 )
 from artemis.graph.state import State
-from artemis.services.llm import get_llm, invoke_llm_with_timeout_message
+from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
 from artemis.tools.index import get_tool_by_name
 from artemis.tools.mobile.log_utils import fetch_and_filter_logs
 from artemis.tools.mobile.read_logs import get_read_logs_tool
@@ -220,19 +220,7 @@ class LogAnalyzerNode:
                     )
                 llm = base_llm.bind_tools(tools=tools_for_binding)
 
-            async def run_stream():
-                full_response = None
-                trace_id = CURRENT_TRACE_ID.get()
-                async for chunk in llm.astream(current_messages):
-                    if full_response is None:
-                        full_response = chunk
-                    else:
-                        full_response += chunk
-
-                    pass
-                return full_response
-
-            response = await invoke_llm_with_timeout_message(run_stream())
+            response = await invoke_llm_with_timeout_message(acomplete(llm, current_messages))
 
             if response.content:
                 outcome = response.content

artemis/agents/object_detector/object_detector.py

@@ -22,9 +22,8 @@ from pathlib import Path
 import re
 
 from langchain_core.messages import HumanMessage, ToolMessage
-from langgraph.types import Command
 
-from artemis.constants import VALIDATOR_MESSAGES_KEY
+from artemis.llm.structured import ParseFailure, parse_structured
 from artemis.services.llm import get_llm
 from artemis.utils.logger import get_logger
 
@@ -78,22 +77,24 @@ async def _detect_single_label(
                 )
             logger.info(f"Received response for '{label}' (Attempt {i + 1})")
 
-            cleaned_initial = output.strip()
-            code_block_match = re.search(r"```(?:json)?\s*([\s\S]*?)\s*```", cleaned_initial)
-            if code_block_match:
-                cleaned_initial = code_block_match.group(1).strip()
-
-            try:
-                res_json = json.loads(cleaned_initial)
-                if isinstance(res_json, list):
-                    valid_results = []
-                    for item in res_json:
-                        if isinstance(item, dict):
-                            item["label"] = label
-                            valid_results.append(item)
-                    return valid_results
-            except json.JSONDecodeError:
-                pass
+            res_json = parse_structured(output)
+            if isinstance(res_json, list):
+                valid_results = []
+                for item in res_json:
+                    if isinstance(item, dict):
+                        item["label"] = label
+                        valid_results.append(item)
+                return valid_results
+            if isinstance(res_json, ParseFailure):
+                logger.warning(
+                    f"Object detector response for '{label}' was not valid"
+                    f" JSON (attempt {i + 1}): {res_json.error}"
+                )
+            else:
+                logger.warning(
+                    f"Object detector response for '{label}' parsed to"
+                    f" {type(res_json).__name__} instead of a list (attempt {i + 1})."
+                )
 
         except Exception as e:
             logger.warning(f"Detection attempt {i + 1} failed for '{label}': {e}")
@@ -194,35 +195,17 @@ async def _run_object_detection(
 
 
 async def _create_error_command(ctx, state, tool_call_id, error_message, wrapper):
-    tool_message = ToolMessage(
+    return ToolMessage(
         tool_call_id=tool_call_id or "default_tool_call",
         content=wrapper.on_failure_fn(error_message),
         additional_kwargs={"error": error_message},
         status="error",
     )
-    return Command(
-        update=await state.asanitize_update(
-            ctx=ctx,
-            update={
-                VALIDATOR_MESSAGES_KEY: [tool_message],
-            },
-            agent="validator",
-        ),
-    )
 
 
 async def _create_success_command(ctx, state, tool_call_id, output, wrapper):
-    tool_message = ToolMessage(
+    return ToolMessage(
         tool_call_id=tool_call_id or "default_tool_call",
         content=wrapper.on_success_fn(output),
         status="success",
     )
-    return Command(
-        update=await state.asanitize_update(
-            ctx=ctx,
-            update={
-                VALIDATOR_MESSAGES_KEY: [tool_message],
-            },
-            agent="validator",
-        ),
-    )

artemis/agents/operator/operator.json

@@ -1,4 +1,3 @@
 {
-  "main_template": "# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen. For any task requiring continuous range interactions, sliding, or boundary selection, prefer using targeted swipe/drag gestures spanning from the element's start bounds [xmin, ymin] to end bounds [xmax, ymax] to ensure the action is strictly confined to the target element. For drag-and-drop, list reordering, or sliding sliders/SeekBars, set duration >= 1000 (e.g. 1500) and always drag slightly PAST the target position to overcome touch slop and reliably trigger the drop/snap/value update. When setting a slider to Maximum (100%) or Minimum (0%), always swipe fully to the extreme boundary to guarantee reaching the absolute limits.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n- **Tool Calling Iron Rule**: Do NOT submit a Turn-Ending Action at the same time (in the same tool-call list) as a Pre-Decision Exploratory Tool. You must first gather information (e.g., finding coordinates or diagnosing a state), manage ADB background tasks, or update memory notes. You can use pre-decision exploratory tools to complete this task. You will be prompted again with the tool's result, at which point you can output your final physical/turn-ending action.\n- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n- **Memory Preservation & Paginated Exploration**: Using note tools is entirely zero-cost. Actively use this tool to maintain goals and logs. The note tool is the only reliable way to persist memory. Therefore, when you find information on the screen that might be relevant to the task, even if you are not certain, you should first save the information currently visible on the screen before exploring further. If you have already recorded some information, you can use `append_note` to append the new information to the existing content. The note tool can also help you organize lists or information across multiple screens; you can consolidate all information into notes for unified processing.\n- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor. Calibrate your scroll distance so that this anchor remains visible near the top of the subsequent screen, creating a seamless visual overlap that guarantees zero missed items. Terminate exploration when a definitive boundary is reached (e.g., list bottom reached or no new content appears after scrolling).\n  2. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.\n  3. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for \"double-checking\". Only perform incremental re-verification if the list has been refreshed or contentLine truncated
-  "troubleshooter_template": "# ROLE\nYou are the Expert Android Device Troubleshooter. A previous attempt to complete the task was rejected by the verification agent (Checker). Your priority is diagnosis and execution accuracy to resolve the rejection.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen. For any task requiring continuous range interactions, sliding, or boundary selection, prefer using targeted swipe/drag gestures spanning from the element's start bounds [xmin, ymin] to end bounds [xmax, ymax] to ensure the action is strictly confined to the target element. For drag-and-drop, list reordering, or sliding sliders/SeekBars, set duration >= 1000 (e.g. 1500) and always drag slightly PAST the target position to overcome touch slop and reliably trigger the drop/snap/value update. When setting a slider to Maximum (100%) or Minimum (0%), always swipe fully to the extreme boundary to guarantee reaching the absolute limits.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n- **Tool Calling Iron Rule**: Do NOT submit a Turn-Ending Action at the same time (in the same tool-call list) as a Pre-Decision Exploratory Tool. You must first gather information (e.g., finding coordinates or diagnosing a state), manage ADB background tasks, or update memory notes. You can use pre-decision exploratory tools to complete this task. You will be prompted again with the tool's result, at which point you can output your final physical/turn-ending action.\n- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n- **Memory Preservation & Paginated Exploration**: Using note tools is entirely zero-cost. Actively use this tool to maintain goals and logs. The note tool is the only reliable way to persist memory. Therefore, when you find information on the screen that might be relevant to the task, even if you are not certain, you should first save the information currently visible on the screen before exploring further. If you have already recorded some information, you can use `append_note` to append the new information to the existing content. The note tool can also help you organize lists or information across multiple screens; you can consolidate all information into notes for unified processing.\n- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor. Calibrate your scroll distance so that this anchor remains visible near the top of the subsequent screen, creating a seamless visual overlap that guarantees zero missed items. Terminate exploration when a definitive boundary is reached (e.g., list bottom reached or no new content appears after scrolling).\n  2. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.\n  3. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for \"double-checking\". Only perform incremental rLine truncated
+  "main_template": "# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen.[% if \"swipe\" in available_tools %] For any task requiring continuous range interactions, sliding, or boundary selection, prefer using targeted swipe/drag gestures spanning from the element's start bounds [xmin, ymin] to end bounds [xmax, ymax] to ensure the action is strictly confined to the target element. For drag-and-drop, list reordering, or sliding sliders/SeekBars, set duration >= 1000 (e.g. 1500) and always drag slightly PAST the target position to overcome touch slop and reliably trigger the drop/snap/value update. When setting a slider to Maximum (100%) or Minimum (0%), always swipe fully to the extreme boundary to guarantee reaching the absolute limits.[% endif %]\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n- **Tool Calling Contract**: Tools that return information needed for the decision ([[ tool_enum(pre_decision_tools, \"and\") ]]) are result-dependent Pre-Decision Tools. Do not submit them in the same tool-call list as a Turn-Ending Action; inspect their result first. The write-through tools `update_note` and `append_note` may accompany at most one Turn-Ending Action when their content is based entirely on evidence already available before that action. Never write an action's expected outcome as if it had already been observed.\n[% if \"wait_for_delay\" in available_tools %]- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n[% endif %]- **Memory Preservation & Paginated Exploration**: Using note tools is entirely zero-cost. Actively use this tool to maintain goals and logs. The note tool is the only reliable way to persist memory. Therefore, when you find information on the screen that might be relevant to the task, even if you are not certain, you should first save the information currently visible on the screen before exploring further. If you have already recorded some information, you can use `append_note` to append the new information to the existing content. The note tool can also help you organize lists or information across multiple screens; you can consolidate all information into notes for unified processing.\n- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.\n  2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading state. This boundary probe must remain unidLine truncated
 }

artemis/agents/operator/operator.py

@@ -12,21 +12,14 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-import asyncio
 import json
-from pathlib import Path
 import re
-from typing import Annotated, Any, Literal
+from typing import Any
 from uuid import uuid4
 
 from langchain_core.messages import SystemMessage, ToolMessage
-from langchain_core.tools import BaseTool, tool
-from langgraph.types import Command
+from langchain_core.tools import BaseTool
 
-from artemis.constants import (
-    OPERATOR_MAX_CHAT_ROUNDS,
-    VALIDATOR_MESSAGES_KEY,
-)
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import (
     TraceSpan,
@@ -34,7 +27,9 @@ from artemis.data_engine.trace import (
     trace_langchain_tool,
 )
 from artemis.graph.state import State
-from artemis.services.llm import get_llm, invoke_llm_with_timeout_message
+from artemis.graph.visibility import strict_state
+from artemis.mcp.action_specs import OPERATOR_SHELL_ORDER, operator_shell_tool
+from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
 from artemis.tools.command_tool import (
     _BACKGROUND_TASKS,
     _FINISHED_TASKS_LOGS,
@@ -51,19 +46,12 @@ from artemis.utils.coordinates import (
     parse_swipe_parameters,
 )
 from artemis.utils.decorators import wrap_with_callbacks
-from artemis.utils.file import create_snapshot
 from artemis.utils.logger import get_logger
-from artemis.utils.notes import get_note_file_path, get_notes_dir
+from artemis.utils.notes import get_note_file_path
 from artemis.utils.task_tree import (
     build_plan_and_history,
     get_active_subgoal_hashes,
 )
-from artemis.utils.verification import (
-    append_verification_chat,
-    get_verification_chat_path,
-    get_verification_chat_rounds,
-    read_verification_chat,
-)
 from artemis.utils.visualization import format_minimal_list_with_elements
 
 logger = get_logger(__name__)
@@ -86,7 +74,8 @@ from artemis.agents.operator.prompts import (
     TemplatePromptComponent,
     ObservationPromptComponent,
     ScreenshotSimilarityPromptComponent,
-    CheckerFeedbackPromptComponent,
+    FeedbackPromptComponent,
+    CheckItemsExplainerPromptComponent,
     BackgroundTasksPromptComponent,
     ShortTermMemoryPromptComponent,
     TaskPlanWarningPromptComponent,
@@ -169,56 +158,6 @@ class OperatorNode:
 
         return steps, task_plan
 
-    def _get_verification_chat_rounds(self) -> tuple[int, int]:
-        if not self.ctx.data_engine:
-            return 0, 0
-
-        subgoal_hash = self._get_active_subgoal_hash()
-        chat_path = get_verification_chat_path(self.ctx.data_engine.base_dir, subgoal_hash)
-        turns = read_verification_chat(chat_path)
-        return get_verification_chat_rounds(turns)
-
-    def _has_checker_feedback(self) -> bool:
-        if not self.ctx.data_engine:
-            return False
-
-        subgoal_hash = self._get_active_subgoal_hash()
-        chat_path = get_verification_chat_path(self.ctx.data_engine.base_dir, subgoal_hash)
-        return chat_path.exists()
-
-    def _get_reply_to_checker_tool(self) -> BaseTool:
-        @tool
-        def reply_to_checker(
-            reasoning: Annotated[
-                str,
-                "Your reasoning or observations regarding the verification feedback.",
-            ],
-        ):
-            """[TERMINAL] Use this tool to reply to the verification feedback with your reasoning or observations.
-
-            This ends your turn.
-            """
-
-            if self.ctx.data_engine:
-                subgoal_hash = self._get_active_subgoal_hash()
-                chat_path = get_verification_chat_path(self.ctx.data_engine.base_dir, subgoal_hash)
-
-                turns = read_verification_chat(chat_path)
-                max_op, max_chk = get_verification_chat_rounds(turns)
-
-                round_num = max(max_op + 1, max_chk)
-                if round_num == 0:
-                    round_num = 1
-
-                if append_verification_chat(chat_path, "operator", reasoning, round_num):
-                    return "Successfully recorded your reply to the Checker."
-                else:
-                    return "Failed to save reply."
-            else:
-                return "Error: DataEngine not available."
-
-        return reply_to_checker
-
     async def _build_prompt(
         self,
         state: State,
@@ -235,39 +174,26 @@ class OperatorNode:
 
         builder = PromptBuilder()
 
-        has_feedback = self._has_checker_feedback()
-
+        # One constant component list: the prompt template is never switched by
+        # verification results. Check-related components are append-only and
+        # render nothing when there is nothing to say.
         components = self.prompt_components
         if not components:
-            if has_feedback:
-                components = [
-                    (
-                        TemplatePromptComponent(),
-                        {"template_name": "troubleshooter_template"},
-                    ),
-                    (ObservationPromptComponent(), {}),
-                    (ScreenshotSimilarityPromptComponent(), {}),
-                    (InjectedInstructionPromptComponent(), {}),
-                    (ShortTermMemoryPromptComponent(), {}),
-                    (BackgroundTasksPromptComponent(), {}),
-                    (CheckerFeedbackPromptComponent(), {}),
-                    (TaskPlanWarningPromptComponent(), {}),
-                    (ToolLimitWarningPromptComponent(), {}),
-                ]
-            else:
-                components = [
-                    (
-                        TemplatePromptComponent(),
-                        {"template_name": "main_template"},
-                    ),
-                    (ObservationPromptComponent(), {}),
-                    (ScreenshotSimilarityPromptComponent(), {}),
-                    (InjectedInstructionPromptComponent(), {}),
-                    (ShortTermMemoryPromptComponent(), {}),
-                    (BackgroundTasksPromptComponent(), {}),
-                    (TaskPlanWarningPromptComponent(), {}),
-                    (ToolLimitWarningPromptComponent(), {}),
-                ]
+            components = [
+                (
+                    TemplatePromptComponent(),
+                    {"template_name": "main_template"},
+                ),
+                (CheckItemsExplainerPromptComponent(), {}),
+                (ObservationPromptComponent(), {}),
+                (ScreenshotSimilarityPromptComponent(), {}),
+                (InjectedInstructionPromptComponent(), {}),
+                (ShortTermMemoryPromptComponent(), {}),
+                (BackgroundTasksPromptComponent(), {}),
+                (FeedbackPromptComponent(), {}),
+                (TaskPlanWarningPromptComponent(), {}),
+                (ToolLimitWarningPromptComponent(), {}),
+            ]
         else:
             components = [(c, {}) for c in components] + [(ToolLimitWarningPromptComponent(), {})]
 
@@ -298,31 +224,13 @@ class OperatorNode:
         traced_tools: list,
         new_subagent_calls: list,
         state: State,
-    ) -> tuple[list[dict] | None, bool, str | None, str | None, bool]:
+    ) -> tuple[list[dict] | None, str | None, str | None, bool]:
         max_iterations = 20
         action_result = None
-        requested_argue = False
         raw_thoughts = []
         native_thoughts = []
         tool_limit_exceeded = False
 
-        async def read_stream_and_accumulate(target_llm, msgs):
-            try:
-                full_response = None
-                has_chunks = False
-                async for chunk in target_llm.astream(msgs):
-                    has_chunks = True
-                    if full_response is None:
-                        full_response = chunk
-                    else:
-                        full_response += chunk
-                if has_chunks and full_response is not None:
-                    return full_response
-            except Exception as e:
-                logger.debug(f"astream failed or not supported: {e}. Falling back to ainvoke.")
-
-            return await target_llm.ainvoke(msgs)
-
         for iteration in range(max_iterations):
             if iteration == max_iterations - 1:
                 logger.warning(
@@ -340,9 +248,7 @@ class OperatorNode:
                 )
 
             bound_llm = base_llm.bind_tools(tools=traced_tools)
-            response = await invoke_llm_with_timeout_message(
-                read_stream_and_accumulate(bound_llm, current_messages)
-            )
+            response = await invoke_llm_with_timeout_message(acomplete(bound_llm, current_messages))
 
             if hasattr(response, "response_metadata") and response.response_metadata:
                 usage = (
@@ -420,23 +326,19 @@ class OperatorNode:
             action_calls = [
                 tc for tc in response.tool_calls if normalize_name(tc["name"]) in action_tool_names
             ]
-            reply_to_checker_calls = [
-                tc for tc in response.tool_calls if normalize_name(tc["name"]) == "reply_to_checker"
-            ]
             other_calls = [
                 tc
                 for tc in response.tool_calls
-                if normalize_name(tc["name"]) not in action_tool_names + ["reply_to_checker"]
+                if normalize_name(tc["name"]) not in action_tool_names
             ]
 
             tool_outputs = []
             validation_errors = False
-            requested_argue = False
 
             other_tool_failed = False
             other_tool_failure_msg = ""
 
-            has_terminal_calls = bool(action_calls or reply_to_checker_calls)
+            has_terminal_calls = bool(action_calls)
 
             if other_calls:
                 results = []
@@ -478,12 +380,8 @@ class OperatorNode:
                                 content = get_tool_result_content(result_obj)
                                 status = "success"
 
-                                if isinstance(result_obj, Command):
-                                    updates = result_obj.update
-                                    if VALIDATOR_MESSAGES_KEY in updates:
-                                        msgs = updates[VALIDATOR_MESSAGES_KEY]
-                                        if msgs and hasattr(msgs[0], "status"):
-                                            status = msgs[0].status
+                                if isinstance(result_obj, ToolMessage) and result_obj.status:
+                                    status = result_obj.status
 
                                 is_err = status == "error"
                                 if not is_err:
@@ -567,29 +465,6 @@ class OperatorNode:
                             )
                         action_calls = []
 
-                    if reply_to_checker_calls:
-                        logger.warning(
-                            "Scratchpad/helper tool call failed. Deferring"
-                            f" reply to checker: {other_tool_failure_msg}"
-                        )
-                        for tc in reply_to_checker_calls:
-                            tool_outputs.append(
-                                ToolMessage(
-                                    tool_call_id=tc["id"],
-                                    content=(
-                                        "Your reply to the checker was"
-                                        " rejected because an accompanying"
-                                        " tool call failed. Error:"
-                                        f" {other_tool_failure_msg}. Currently,"
-                                        " no response has been sent to the"
-                                        " checker. Please correct the error"
-                                        " and try again."
-                                    ),
-                                    status="error",
-                                )
-                            )
-                        reply_to_checker_calls = []
-
                 elif has_terminal_calls:
                     has_deferring_calls = any(
                         normalize_name(tc["name"]) in DEFERRING_TOOLS for tc in other_calls
@@ -623,28 +498,7 @@ class OperatorNode:
                                 )
                             )
 
-                        for tc in reply_to_checker_calls:
-                            tool_outputs.append(
-                                ToolMessage(
-                                    tool_call_id=tc["id"],
-                                    content=(
-                                        "Your pre-decision tools have been"
-                                        " successfully processed. However, your"
-                                        " response to the checker was rejected"
-                                        " because you are not allowed to"
-                                        " simultaneously use result-dependent"
-                                        " pre-decision tools and reply to the"
-                                        " checker. Currently, no response has been"
-                                        " sent to the checker. Please review"
-                                        " your updated context and re-output"
-                                        " your reply."
-                                    ),
-                                    status="success",
-                                )
-                            )
-
                         action_calls = []
-                        reply_to_checker_calls = []
 
             if action_calls:
                 logger.info(
@@ -676,32 +530,10 @@ class OperatorNode:
                 if not validation_errors:
                     action_result = actions_list
 
-            if reply_to_checker_calls:
-                logger.info("Operator replied to checker.")
-                tc = reply_to_checker_calls[0]
-                requested_argue = True
-
-                # Execute it immediately
-                result = await self._run_other_tool(
-                    tc,
-                    traced_tools,
-                    new_subagent_calls,
-                    state,
-                    raw_thoughts,
-                    native_thoughts,
-                )
-                tool_outputs.append(
-                    ToolMessage(
-                        tool_call_id=tc["id"],
-                        content=result,
-                        status="success" if not result.startswith("Error") else "error",
-                    )
-                )
-
             for tm in tool_outputs:
                 current_messages.append(tm)
 
-            if (action_calls and not validation_errors) or requested_argue:
+            if action_calls and not validation_errors:
                 break
 
             if validation_errors:
@@ -714,7 +546,6 @@ class OperatorNode:
         native_thinking = "\n".join(native_thoughts) if native_thoughts else None
         return (
             action_result,
-            requested_argue,
             raw_thinking,
             native_thinking,
             tool_limit_exceeded,
@@ -765,18 +596,10 @@ class OperatorNode:
     )
     @trace(type="agent", name="operator")
     async def __call__(self, state: State):
+        state = strict_state(state, "operator")
         # 1. Check for infinite loops
         self._check_infinite_loop(state)
 
-        # Create snapshot for optimistic execution
-        if self.ctx.data_engine:
-            notes_dir = get_notes_dir(self.ctx.data_engine.base_dir)
-            if notes_dir.exists():
-                snapshot_dir = notes_dir.with_name("notes_snapshot")
-                await asyncio.to_thread(create_snapshot, notes_dir, snapshot_dir)
-                self.ctx.task_plan_snapshot = snapshot_dir
-                logger.info(f"Created snapshot of notes at {snapshot_dir}")
-
         # 2. Get perception data from state (populated by Perception node)
         operator_raw_data = getattr(state, "operator_raw_data", {}) or {}
         latest_screenshot_b64 = operator_raw_data.get("screenshot_b64")
@@ -822,45 +645,16 @@ class OperatorNode:
 
         current_step_num = len(steps) + 1
 
-        # 5. Prepare Tools. Device-action declarations are assembled against the
-        # installed actuator backend's capabilities: an action the backend does not
-        # implement is simply never declared (and the prompt assembly drops its
-        # teaching segments in lockstep).
+        # 5. Prepare Tools. Device-action shells come from the canonical manifest
+        # (artemis/mcp/action_specs.py) and are assembled against the installed
+        # actuator backend's capabilities: an action the backend does not implement
+        # is simply never declared (and the prompt assembly drops its teaching
+        # segments in lockstep).
         available_actions = self._available_device_actions()
-        action_tool_factories = {
-            "click": self._get_click_tool,
-            "input_text": self._get_input_text_tool,
-            "swipe": self._get_swipe_tool,
-            "press_key": self._get_press_key_tool,
-            "manage_app": self._get_manage_app_tool,
-            "wait_for_delay": self._get_wait_for_delay_tool,
-            "long_press": self._get_long_press_tool,
-        }
         all_tools = [
-            factory()
-            for name, factory in action_tool_factories.items()
-            if name in available_actions
+            operator_shell_tool(name) for name in OPERATOR_SHELL_ORDER if name in available_actions
         ] + self.tools
 
-        if self._has_checker_feedback():
-            max_op, max_chk = self._get_verification_chat_rounds()
-            max_rounds = (
-                getattr(
-                    getattr(self.ctx, "execution_setup", None),
-                    "checker_max_chat_rounds",
-                    OPERATOR_MAX_CHAT_ROUNDS,
-                )
-                if self.ctx and getattr(self.ctx, "execution_setup", None)
-                else OPERATOR_MAX_CHAT_ROUNDS
-            )
-            if max_op < max_rounds:
-                reply_to_checker_tool = self._get_reply_to_checker_tool()
-                all_tools.append(reply_to_checker_tool)
-            else:
-                logger.info(
-                    f"Maximum chat rounds ({max_rounds}) reached. Removing reply_to_checker tool."
-                )
-
         # 5. Evaluate dynamic tools and prepare background tasks
         has_long_output = False
         for tinfo in _FINISHED_TASKS_LOGS.values():
@@ -924,7 +718,6 @@ class OperatorNode:
         new_subagent_calls = []
         (
             action_result,
-            requested_argue,
             raw_thinking,
             native_thinking,
             tool_limit_exceeded,
@@ -955,22 +748,17 @@ class OperatorNode:
             if stm_match:
                 short_term_memory = stm_match.group(1).strip()
 
-        return await state.asanitize_update(
-            ctx=self.ctx,
-            update={
-                "structured_decisions": structured_decisions,
-                "operator_raw_thinking": raw_thinking,
-                "operator_native_thinking": native_thinking,
-                "short_term_memory": short_term_memory,
-                "indexed_points": state.indexed_points,
-                "complete_subgoals_by_ids": [],
-                "current_step_id": state.current_step_id,
-                "subagent_calls": ((state.subagent_calls or []) + new_subagent_calls),
-                "operator_replied": requested_argue,
-                "operator_tool_limit_exceeded": tool_limit_exceeded,
-            },
-            agent="operator",
-        )
+        return {
+            "structured_decisions": structured_decisions,
+            "operator_raw_thinking": raw_thinking,
+            "operator_native_thinking": native_thinking,
+            "short_term_memory": short_term_memory,
+            "indexed_points": state.indexed_points,
+            "indexed_elements": state.indexed_elements,
+            "current_step_id": state.current_step_id,
+            "subagent_calls": ((state.subagent_calls or []) + new_subagent_calls),
+            "operator_tool_limit_exceeded": tool_limit_exceeded,
+        }
 
     def _get_active_subgoal_hash(self) -> str:
         """Parses task_plan.md to find the active top-level subgoal hash."""
@@ -1007,198 +795,6 @@ class OperatorNode:
             chronological_last_step=True,
         )
 
-    def _get_long_press_tool(self) -> BaseTool:
-        @tool
-        def long_press(
-            target: Annotated[
-                int | list[int],
-                "Long press target. Can be an element index number (int, e.g."
-                " 3) OR normalized coordinates (list of 2 integers, e.g. [500,"
-                " 600]).",
-            ],
-            duration: Annotated[int, "Long press duration in milliseconds (default 1000)."] = 1000,
-        ):
-            """[ACTION] Long press on the target location on the screen (supports element index or absolute normalized coordinates)."""
-            return "Action Recorded"
-
-        return long_press
-
-    def _get_click_tool(self) -> BaseTool:
-        @tool
-        def click(
-            target: Annotated[
-                int | list[int],
-                "Click target. Can be an element index number (int, e.g. 3) OR"
-                " normalized coordinates (list of 2 integers, e.g. [500,"
-                " 600]).",
-            ],
-            times: Annotated[
-                int,
-                "Number of consecutive clicks on this target. Use this for"
-                " double-clicks or multi-clicks (e.g. 7 to enter developer"
-                " mode). Default is 1.",
-            ] = 1,
-            delay_ms: Annotated[
-                int,
-                "Delay in milliseconds between consecutive clicks. Default is 100.",
-            ] = 100,
-        ):
-            """[ACTION] Click on the target location on the screen (supports element index or absolute normalized coordinates)."""
-            return "Action Recorded"
-
-        return click
-
-    def _get_input_text_tool(self) -> BaseTool:
-        @tool
-        def input_text(
-            text: Annotated[
-                str,
-                "The text content to input. Supports multi-line content with '\\n'.",
-            ],
-            target: Annotated[
-                int | list[int],
-                "Input target field. Can be an input box element index number"
-                " (int, e.g. 3) OR normalized coordinates (list of 2 integers,"
-                " e.g. [500, 600]).",
-            ],
-            clear_exist: Annotated[
-                bool,
-                "Whether to clear existing text before typing. True (default):"
-                " clear/replace entire text. False: append at the end of"
-                " existing content.",
-            ] = True,
-        ):
-            """[ACTION] Type text into the target input field (supports replacing whole text or appending to the end, and multi-line strings with '\\n')."""
-            return "Action Recorded"
-
-        return input_text
-
-    def _get_swipe_tool(self) -> BaseTool:
-        @tool
-        def swipe(
-            direction: Annotated[
-                Literal["up", "down", "left", "right"] | None,
-                "Direction for scrolling and swiping: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
-                " 'down' (drags top-to-bottom, scrolling up to reveal content above),"
-                " 'left' (drags right-to-left, scrolling right),"
-                " 'right' (drags left-to-right, scrolling left).",
-            ] = None,
-            start: Annotated[
-                list[int] | None,
-                "Start normalized coordinates [start_x, start_y] in 0-1000 scale for precise,"
-                " local interactions (e.g. adjusting sliders, SeekBars, fine range selection, or drag-and-drop).",
-            ] = None,
-            end: Annotated[
-                list[int] | None,
-                "End normalized coordinates [end_x, end_y] in 0-1000 scale for precise,"
-                " local interactions (e.g. adjusting sliders, SeekBars, fine range selection, or drag-and-drop).",
-            ] = None,
-            target: Annotated[
-                int | list[int] | str | None,
-                "Optional target element index (e.g. 2) or container bounds [left, top, right, bottom] to scope the directional swipe within.",
-            ] = None,
-            gesture: Annotated[
-                Literal["up", "down", "left", "right"] | list[int] | None,
-                "Backward-compatible swipe gesture: smart direction string ('up', 'down', 'left', 'right')"
-                " OR precise custom coordinates [start_x, start_y, end_x, end_y] in 0-1000 scale.",
-            ] = None,
-            duration: Annotated[
-                int | None,
-                "Optional swipe/drag duration in milliseconds (default 800). For drag-and-drop,"
-                " list reordering, or sliding/adjusting sliders (e.g., volume, brightness, SeekBars),"
-                " set duration >= 1000 (e.g. 1500). If omitted for directional swipe, duration is computed automatically.",
-            ] = None,
-        ):
-            """[ACTION] Perform a swipe, drag, or slider-adjustment gesture on the screen.
-
-            • Directional Scrolling ('direction'): Recommended for general browsing and standard page scrolling in most scenarios. Automatically computes safe swipe vectors and adaptive duration, retains a ~40% visual overlap anchor for zero-omission traversal, and prevents inertial flings. Supports scoping to a sub-container via 'target'. If it fails on certain custom layouts, fall back to specifying exact coordinates ('start' and 'end') directly.
-            • Precise Coordinate Gestures ('start', 'end'): Best for local, fine-grained interactions such as adjusting sliders/SeekBars (e.g., volume, brightness, progress bars), drag-and-drop / list reordering, or as a reliable fallback when directional scrolling fails on specific containers. Always drag slightly PAST the target position to overcome touch slop and reliably trigger the update. When setting a slider to Maximum (100%) or Minimum (0%), swipe fully to the extreme boundary.
-
-            Args:
-                direction: Smart directional scrolling ('up', 'down', 'left', 'right'). Automatically computes safe swipe vectors, retaining 40% visual overlap: 'up' (reveal content below), 'down' (reveal content above), 'left', 'right'.
-                start: Start normalized coordinates [start_x, start_y] in 0-1000 scale.
-                end: End normalized coordinates [end_x, end_y] in 0-1000 scale.
-                target: Optional target element index (e.g. 2) or container bounds [left, top, right, bottom] to scope the directional swipe within.
-                gesture: Backward-compatible parameter: direction string OR custom coordinates list [start_x, start_y, end_x, end_y] in 0-1000 scale.
-                duration: Optional gesture duration in milliseconds (default 800).
-            """
-            return "Action Recorded"
-
-        return swipe
-
-    def _get_press_key_tool(self) -> BaseTool:
-        @tool
-        def press_key(
-            key: Annotated[
-                Literal["ENTER", "BACK", "HOME", "APP_SWITCH"],
-                "Standard Android system button name (ENTER, BACK, HOME, APP_SWITCH).",
-            ],
-        ):
-            """[ACTION] Press a physical or virtual system button (e.g.
-
-            ENTER, BACK, HOME, APP_SWITCH).
-            """
-            return "Action Recorded"
-
-        return press_key
-
-    def _get_manage_app_tool(self) -> BaseTool:
-        @tool
-        def manage_app(
-            action: Annotated[Literal["launch", "stop"], "The action type."],
-            app_name: Annotated[str, "Display name or package name of the application."],
-        ):
-            """[ACTION] Launch or force stop a specified application."""
-            return "Action Recorded"
-
-        return manage_app
-
-    def _get_wait_for_delay_tool(self) -> BaseTool:
-        @tool
-        def wait_for_delay(
-            time_in_ms: Annotated[
-                int,
-                "The exact duration to wait in milliseconds. Accurately convert the"
-                " required time duration into milliseconds based on your objective"
-                " or plan (e.g., 2000 for 2s, 5000 for 5s, 60000 for 1 minute,"
-                " 180000 for 3 minutes, 300000 for 5 minutes).",
-            ],
-        ):
-            """[ACTION] Pause execution and wait for a specified duration in milliseconds.
-
-            Use this whenever you need time to elapse—whether for UI loading, animations,
-            screen transitions, or longer scheduled delays and intervals specified in the task.
-            """
-            return "Action Recorded"
-
-        return wait_for_delay
-
-    def _get_wait_for_text_tool(self) -> BaseTool:
-        @tool
-        def wait_for_text(
-            text: Annotated[
-                str,
-                "A simple, distinct keyword (e.g., 'Success', 'Done',"
-                " 'Loading') to watch on the screen. Try to pass a clean word"
-                " without trailing punctuation (e.g. 'Loading' instead of"
-                " 'Loading...') for optimal matching reliability.",
-            ],
-            state: Annotated[
-                Literal["visible", "hidden"],
-                "Wait for the keyword to appear ('visible') or disappear"
-                " ('hidden') from the screen.",
-            ] = "visible",
-            timeout_ms: Annotated[int, "Maximum timeout in milliseconds (default 5000)."] = 5000,
-        ):
-            """[ACTION] Pause execution and wait intelligently for a specific, distinct text or keyword to appear or disappear on the screen.
-
-            Use this when waiting for a specific loading screen to finish or a
-            specific success/confirmation state to appear.
-            """
-            return "Action Recorded"
-
-        return wait_for_text
-
     def _translate_and_validate_tool(self, tc: dict, state: State) -> tuple[list[dict], str | None]:
         actions, err = self._translate_and_validate_tool_inner(tc, state)
         if err:
@@ -1384,7 +980,7 @@ class OperatorNode:
                     direction=target,
                     target=args.get("target"),
                     indexed_elements=getattr(state, "indexed_elements", None),
-                    ui_hierarchy=getattr(state, "ui_tree", None),
+                    ui_hierarchy=getattr(state, "latest_ui_hierarchy", None),
                     width=width,
                     height=height,
                     duration=duration,

artemis/agents/operator/prompts.py

@@ -14,10 +14,9 @@
 
 import base64
 import io
-import json
 from pathlib import Path
 
-from jinja2 import Template
+from jinja2 import Environment, StrictUndefined, Template
 from langchain_core.messages import HumanMessage, SystemMessage
 from PIL import Image
 
@@ -28,27 +27,27 @@ from artemis.tools.command_tool import (
     _is_output_long,
 )
 from artemis.utils.logger import get_logger
-from artemis.utils.task_tree import get_active_subgoal_hashes
+from artemis.utils.notes import get_note_file_path
+from artemis.utils.plan_grammar import parse_plan, render_plan_grammar_spec
 
 logger = get_logger(__name__)
 
 
-from artemis.agents.prompt_assembly import (
-    gate_segment,
-    render_tool_enum,
-    resolve_available,
-)
-
-_LEGACY_TOOL_CALLING_RULE = """- **Tool Calling Iron Rule**: Do NOT submit a Turn-Ending Action at the same time (in the same tool-call list) as a Pre-Decision Exploratory Tool. You must first gather information (e.g., finding coordinates or diagnosing a state), manage ADB background tasks, or update memory notes. You can use pre-decision exploratory tools to complete this task. You will be prompted again with the tool's result, at which point you can output your final physical/turn-ending action."""
-
-_LEGACY_TOOL_PROTOCOL = """1. **Pre-Decision Exploratory Tools**:
-   - *What they are*: Helper/Subagent tools (such as `ask_explorer`, `ask_diagnoser`, `video_analyzer`), ADB command tools (`run_adb_command`, `manage_task`), and memory note tools (`read_note`, `list_notes`, `save_note`, `update_note`, `append_note`).
-   - *How they work*: These tools gather details or update/read memory. When you invoke these tools, the framework will immediately execute them and return the results, allowing you to continue thinking and make your final decision.
-2. **Turn-Ending Actions**:"""
+from artemis.agents.prompt_assembly import render_tool_enum, resolve_available
+from artemis.mcp.action_specs import OPERATOR_SHELL_ORDER
 
 # --- Assembled tool references -------------------------------------------------------
+# The operator.json templates carry availability slots rendered by
+# ``apply_operator_prompt_contract``: ``[[ tool_enum(...) ]]`` expands an ordered,
+# backticked enumeration limited to the available tool set, and
+# ``[% if "x" in available_tools %]...[% endif %]`` gates a self-contained teaching
+# segment. An unavailable tool therefore leaves no trace in the prompt at all. The
+# slot delimiters are square-bracketed so the standard ``{{ ... }}`` context
+# variables (initial_goal, plan_and_history, plan_grammar, ...) pass through
+# untouched for the later context render.
+#
 # The orderings below reproduce the historical prompt wording exactly; with the full
-# tool set the rendered output is byte-identical to the previous literals. See
+# tool set the rendered output is byte-identical to the pre-assembly literals. See
 # artemis/agents/prompt_assembly.py for the assembly rationale.
 
 _PRE_DECISION_HELPER_TOOLS = ("ask_explorer", "ask_diagnoser", "video_analyzer")
@@ -59,16 +58,9 @@ _PRE_DECISION_ALL_TOOLS = (
 )
 
 # The two device-action enumeration slots in operator.json (identical in both
-# templates), in their historical orders.
-_PHYSICAL_ACTIONS_ORDER = (
-    "click",
-    "input_text",
-    "swipe",
-    "press_key",
-    "manage_app",
-    "wait_for_delay",
-    "long_press",
-)
+# templates), in their historical orders. The physical order is the canonical
+# manifest's shell-binding order; the turn-ending order is a prompt property.
+_PHYSICAL_ACTIONS_ORDER = OPERATOR_SHELL_ORDER
 _TURN_ENDING_ORDER = (
     "click",
     "swipe",
@@ -79,120 +71,31 @@ _TURN_ENDING_ORDER = (
     "wait_for_delay",
 )
 
-_PHYSICAL_ACTIONS_SLOT = (
-    "Physical device actions (`click`, `input_text`, `swipe`, `press_key`,"
-    " `manage_app`, `wait_for_delay`, `long_press`)"
-)
-_TURN_ENDING_SLOT = (
-    "Turn-Ending Action (`click`, `swipe`, `input_text`, `long_press`,"
-    " `press_key`, `manage_app`, or `wait_for_delay`)"
-)
-
-# Self-contained instruction segments that teach a single tool. Removed wholesale when
-# the tool is unavailable, so nothing in the prompt dangles.
-_WAIT_FOR_DELAY_TRANSITIONS_SEGMENT = (
-    "- **Loading & Transitions**: When entering page transitions (e.g., app launching,"
-    " web page loading, or progress indicators), use `wait_for_delay` to let the screen"
-    " settle before executing target actions. Some transition states are difficult to"
-    " discern, such as a thin progress line at the top of the browser screen during"
-    " webpage loads. Before questioning whether an error has occurred, you can first"
-    " adopt a waiting strategy to see how the device state evolves. If the state"
-    " remains ambiguous after waiting, certain UI components might be malfunctioning or"
-    " failing to load; in this case, consider retrying the action or pivoting to an"
-    " alternative path.\n"
-)
-_WAIT_FOR_DELAY_INTERVAL_PARENTHETICAL = (
-    " (convert minutes to milliseconds accurately for `wait_for_delay`)"
-)
-_VIDEO_ANALYZER_TRANSIENT_SEGMENT = (
-    "   - *Missed Transient States*: If the current state jumped ahead and you suspect"
-    " you missed an important initial or intermediate screen (e.g., a number or flash"
-    " message that was overwritten by your last action), consider using the"
-    " `video_analyzer` tool to look back at the recent history.\n"
-)
-_VIDEO_ANALYZER_SNAPSHOT_SENTENCE = (
-    " If you have already missed certain critical information and cannot re-view it by"
-    " going back or other operations, you can use the `video_analyzer` to look back at"
-    " historical information."
-)
-# Tail of the "Action Scope & Interaction Precision" bullet: everything past the two
-# generic gesture-scope sentences is swipe/drag teaching.
-_SWIPE_PRECISION_SEGMENT = (
-    " For any task requiring continuous range interactions, sliding, or boundary"
-    " selection, prefer using targeted swipe/drag gestures spanning from the element's"
-    " start bounds [xmin, ymin] to end bounds [xmax, ymax] to ensure the action is"
-    " strictly confined to the target element. For drag-and-drop, list reordering, or"
-    " sliding sliders/SeekBars, set duration >= 1000 (e.g. 1500) and always drag"
-    " slightly PAST the target position to overcome touch slop and reliably trigger the"
-    " drop/snap/value update. When setting a slider to Maximum (100%) or Minimum (0%),"
-    " always swipe fully to the extreme boundary to guarantee reaching the absolute"
-    " limits."
-)
-
 #: Every tool name the operator prompt slots can reference. ``available_tools=None``
 #: resolves to this set, preserving the historical output.
 OPERATOR_PROMPT_TOOLSET: frozenset[str] = frozenset(
     _PRE_DECISION_ALL_TOOLS + _PHYSICAL_ACTIONS_ORDER
 )
 
-
-def _tool_calling_rule(available: frozenset[str]) -> str:
-    tools = render_tool_enum(_PRE_DECISION_ALL_TOOLS, available, final_sep="and")
-    return (
-        "- **Tool Calling Contract**: Tools that return information needed for the"
-        f" decision ({tools}) are result-dependent Pre-Decision Tools. Do not submit"
-        " them in the same tool-call list as a Turn-Ending Action; inspect their result"
-        " first. The write-through tools `update_note` and `append_note` may accompany"
-        " at most one Turn-Ending Action when their content is based entirely on"
-        " evidence already available before that action. Never write an action's"
-        " expected outcome as if it had already been observed."
-    )
-
-
-def _tool_protocol(available: frozenset[str]) -> str:
-    helper = render_tool_enum(_PRE_DECISION_HELPER_TOOLS, available)
-    adb = render_tool_enum(_PRE_DECISION_ADB_TOOLS, available)
-    memory = render_tool_enum(_PRE_DECISION_MEMORY_TOOLS, available)
-    return (
-        "1. **Result-Dependent Pre-Decision Tools**:\n"
-        f"   - *What they are*: Helper/Subagent tools ({helper}), ADB/task tools"
-        f" ({adb}), and memory tools whose result must be inspected ({memory}).\n"
-        "   - *How they work*: Invoke these without a Turn-Ending Action, inspect the"
-        " returned result, and then decide.\n"
-        "2. **Write-Through Memory Tools**:\n"
-        "   - `update_note` and `append_note` may run alongside at most one Turn-Ending"
-        " Action only when recording facts already observed in the current context. If"
-        " the note content depends on the action's result, wait for the next"
-        " observation before writing it.\n"
-        "3. **Turn-Ending Actions**:"
-    )
-
-_LEGACY_CHECKER_REJECTION_TRIGGER = """   - **Validation/Checker Rejection**: The verification agent (Checker) rejected your subgoal completion (i.e., you are in troubleshooter mode and this is a retry)."""
-
-_AMBIGUOUS_CHECKER_REJECTION_TRIGGER = """   - **Ambiguous Validation/Checker Rejection**: Use `ask_diagnoser` only when the rejection cause is unclear, the evidence conflicts, or no safe local correction is evident. A Checker rejection alone is not a mandatory diagnosis trigger."""
-
-_LEGACY_TROUBLESHOOTER_CHALLENGE = """   - *Challenge Checker*: If you visually confirm the subgoal's target state is *already achieved*, you **MUST** invoke the `reply_to_checker` tool to state your observation."""
-
-_TROUBLESHOOTER_CHALLENGE = """   - *Challenge Checker*: If current evidence clearly proves the subgoal is already achieved, invoke `reply_to_checker` directly. This branch takes precedence over diagnosis; do not call `ask_diagnoser` first.
-   - *Clear Local Correction*: If the rejection cause is explicit and a safe correction is evident from the current screen, perform that correction directly. This branch also takes precedence over generic diagnosis triggers. Use `ask_diagnoser` only for ambiguity, conflicting evidence, or repeated failure."""
-
-_LEGACY_LARGE_LIST_STRATEGY = """- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:
-  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor. Calibrate your scroll distance so that this anchor remains visible near the top of the subsequent screen, creating a seamless visual overlap that guarantees zero missed items. Terminate exploration when a definitive boundary is reached (e.g., list bottom reached or no new content appears after scrolling).
-  2. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.
-  3. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for "double-checking". Only perform incremental re-verification if the list has been refreshed or content has mutated, while maintaining strict single-pass handling of new events."""
-
-_LARGE_LIST_STRATEGY = """- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:
-  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.
-  2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading state. This boundary probe must remain unidirectional; never reverse direction merely to prove completion. If the task declares an earlier exit condition (for example, find one verified match or process N items), that explicit condition may end the traversal without reaching the list boundary.
-  3. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.
-  4. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed with confirmed anchor continuity and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for "double-checking", even if older execution history has been pruned. Resume from the last recorded anchor and direction. Only the single bounded recovery above, or a refreshed/mutated list, permits limited re-verification; otherwise continue the single pass over new content."""
+#: Renders the availability slots in operator.json. Square-bracket delimiters keep
+#: the standard ``{{ ... }}`` context placeholders inert during this phase.
+_CONTRACT_ENV = Environment(
+    block_start_string="[%",
+    block_end_string="%]",
+    variable_start_string="[[",
+    variable_end_string="]]",
+    comment_start_string="[#",
+    comment_end_string="#]",
+    keep_trailing_newline=True,
+    undefined=StrictUndefined,
+)
 
 
 def apply_operator_prompt_contract(
     prompt_template: str,
     available_tools: frozenset[str] | None = None,
 ) -> str:
-    """Align prompt-level tool and recovery rules with Operator runtime semantics.
+    """Renders a template's availability slots against the actually-available tools.
 
     Args:
         prompt_template: One of the raw operator.json templates.
@@ -203,65 +106,19 @@ def apply_operator_prompt_contract(
     """
     available = frozenset(resolve_available(available_tools, OPERATOR_PROMPT_TOOLSET))
 
-    prompt_template = prompt_template.replace(
-        _LEGACY_TOOL_CALLING_RULE,
-        _tool_calling_rule(available),
-    )
-    prompt_template = prompt_template.replace(
-        _LEGACY_TOOL_PROTOCOL,
-        _tool_protocol(available),
-    )
-    prompt_template = prompt_template.replace(
-        _LEGACY_CHECKER_REJECTION_TRIGGER,
-        _AMBIGUOUS_CHECKER_REJECTION_TRIGGER,
-    )
-    prompt_template = prompt_template.replace(
-        _LEGACY_TROUBLESHOOTER_CHALLENGE,
-        _TROUBLESHOOTER_CHALLENGE,
-    )
-    prompt_template = prompt_template.replace(
-        _LEGACY_LARGE_LIST_STRATEGY,
-        _LARGE_LIST_STRATEGY,
-    )
+    def tool_enum(names, final_sep: str | None = None) -> str:
+        return render_tool_enum(tuple(names), available, final_sep=final_sep)
 
-    # Device-action enumeration slots (identical in both templates).
-    prompt_template = prompt_template.replace(
-        _PHYSICAL_ACTIONS_SLOT,
-        "Physical device actions ("
-        + render_tool_enum(_PHYSICAL_ACTIONS_ORDER, available)
-        + ")",
+    return _CONTRACT_ENV.from_string(prompt_template).render(
+        available_tools=available,
+        tool_enum=tool_enum,
+        pre_decision_tools=_PRE_DECISION_ALL_TOOLS,
+        helper_tools=_PRE_DECISION_HELPER_TOOLS,
+        adb_tools=_PRE_DECISION_ADB_TOOLS,
+        memory_tools=_PRE_DECISION_MEMORY_TOOLS,
+        physical_actions=_PHYSICAL_ACTIONS_ORDER,
+        turn_ending_actions=_TURN_ENDING_ORDER,
     )
-    prompt_template = prompt_template.replace(
-        _TURN_ENDING_SLOT,
-        "Turn-Ending Action ("
-        + render_tool_enum(_TURN_ENDING_ORDER, available, final_sep="or")
-        + ")",
-    )
-
-    # Instruction segments that teach a single tool.
-    prompt_template = prompt_template.replace(
-        _WAIT_FOR_DELAY_TRANSITIONS_SEGMENT,
-        gate_segment(_WAIT_FOR_DELAY_TRANSITIONS_SEGMENT, available, "wait_for_delay"),
-    )
-    prompt_template = prompt_template.replace(
-        _WAIT_FOR_DELAY_INTERVAL_PARENTHETICAL,
-        gate_segment(
-            _WAIT_FOR_DELAY_INTERVAL_PARENTHETICAL, available, "wait_for_delay"
-        ),
-    )
-    prompt_template = prompt_template.replace(
-        _VIDEO_ANALYZER_TRANSIENT_SEGMENT,
-        gate_segment(_VIDEO_ANALYZER_TRANSIENT_SEGMENT, available, "video_analyzer"),
-    )
-    prompt_template = prompt_template.replace(
-        _VIDEO_ANALYZER_SNAPSHOT_SENTENCE,
-        gate_segment(_VIDEO_ANALYZER_SNAPSHOT_SENTENCE, available, "video_analyzer"),
-    )
-    prompt_template = prompt_template.replace(
-        _SWIPE_PRECISION_SEGMENT,
-        gate_segment(_SWIPE_PRECISION_SEGMENT, available, "swipe"),
-    )
-    return prompt_template
 
 
 class PromptBuilder:
@@ -340,11 +197,22 @@ class TemplatePromptComponent(PromptComponent):
 
         plan_and_history = kwargs.get("plan_and_history", "No plan or history yet.")
 
+        # Grammar spec assembly is a function of configuration: with both check
+        # gates disabled, the check-line grammar never enters any prompt.
+        include_checks = bool(setup and getattr(setup, "checks_enabled", False))
+        # The rejection/finding diagnosis trigger only exists while a mechanism
+        # that can produce rejections or findings is active.
+        verification_active = include_checks or bool(
+            setup and not getattr(setup, "disable_planner_validation", True)
+        )
+
         full_prompt = Template(prompt_template).render(
             initial_goal=state.initial_goal,
             subgoals_status="",
             plan_and_history=plan_and_history,
             unified_history="",
+            plan_grammar=render_plan_grammar_spec(include_checks),
+            verification_active=verification_active,
         )
 
         parts = full_prompt.split("# CURRENT OBSERVATION")
@@ -369,43 +237,64 @@ class ObservationPromptComponent(PromptComponent):
         builder.add_human_content(f"--- Visible UI Elements ---\n{minimal_list}")
 
 
-class CheckerFeedbackPromptComponent(PromptComponent):
+class FeedbackPromptComponent(PromptComponent):
+    """Append-only injection of source-tagged findings.
+
+    Reads ``state.operator_feedback`` (written by ``execution_check_node`` at
+    harvest / planner rejection and by exit settlement on a bounce-back).
+    Renders nothing when there are no findings — the prompt template itself
+    is never switched.
+    """
+
     async def __call__(self, builder: PromptBuilder, state: State, ctx: ArtemisContext, **kwargs):
-        if not ctx.data_engine:
+        findings = getattr(state, "operator_feedback", None)
+        if not findings:
             return
+        lines = "\n".join(f"- {f}" for f in findings)
+        builder.add_human_content(
+            "--- Verification Findings ---\n"
+            f"{lines}\n"
+            "Each finding is tagged with its source. Checker verdicts"
+            " ([verify failed], [final check]) are independent judgments —"
+            " address any reverted subgoal accordingly; do not re-litigate"
+            " them. [planner] findings explain why a plan change was rejected"
+            " and rolled back — factor the reason into your next strategy."
+        )
 
-        notes_dir = Path(ctx.data_engine.base_dir) / "notes"
 
-        subgoal_hash = "default"
-        task_plan_path = notes_dir / "task_plan.md"
-        if task_plan_path.exists():
-            try:
-                content = task_plan_path.read_text(encoding="utf-8")
+class CheckItemsExplainerPromptComponent(PromptComponent):
+    """Behavioral guidance for check lines, rendered iff the CURRENT plan
+    actually contains check lines (content-driven, not switch-driven: a resumed
+    plan carrying check lines still gets the explanation)."""
 
-                parent_hash, _ = get_active_subgoal_hashes(content)
-                subgoal_hash = parent_hash
-            except Exception as e:
-                logger.error(f"Failed to parse active subgoal in component: {e}")
-
-        verification_chat_path = notes_dir / f"verification_chat_{subgoal_hash}.json"
-        turns = []
-        if verification_chat_path.exists():
-            try:
-                turns = json.loads(verification_chat_path.read_text(encoding="utf-8"))
-                logger.info(f"Read verification chat for {subgoal_hash}: {len(turns)} turns")
-            except Exception as e:
-                logger.error(f"Error reading verification chat: {e}")
-
-        if turns:
-            dialogue_lines = []
-            for t in turns:
-                role = "Operator" if t["role"] == "operator" else "Checker"
-                dialogue_lines.append(f"**{role} (Round {t['round']})**:\n{t['content']}")
-            checker_feedback = "\n\n".join(dialogue_lines)
-
-            feedback_prompt = f"--- Checker Feedback ---\n{checker_feedback}"
-
-            builder.add_human_content(feedback_prompt)
+    async def __call__(self, builder: PromptBuilder, state: State, ctx: ArtemisContext, **kwargs):
+        if not ctx or not getattr(ctx, "data_engine", None):
+            return
+        try:
+            task_plan_path = get_note_file_path(ctx.data_engine.base_dir, "task_plan")
+            if not task_plan_path.exists():
+                return
+            snapshot = parse_plan(task_plan_path.read_text(encoding="utf-8"))
+        except Exception as e:
+            logger.error(f"Failed to parse plan for check-items explainer: {e}")
+            return
+        if not snapshot.all_check_items:
+            return
+        builder.add_human_content(
+            "--- About the plan's check lines ---\n"
+            "The task plan declares `- verify:` / `- assert:` check lines."
+            " `verify:` lines are the acceptance criteria for their subgoal — use"
+            " them to confirm your work is complete, but the judgment is made by"
+            " an independent Checker: never declare a check passed yourself or"
+            " record conclusions on its behalf. `assert:` lines are test"
+            " assertions — take NO extra actions for them and never construct or"
+            " fake state to satisfy one; a failing assertion is a legitimate test"
+            " result. Check lines must not be deleted or reworded (deletions are"
+            " automatically restored by the system); keep them verbatim when"
+            " rewriting the plan — adding new ones is allowed. Simply mark"
+            " completions per the plan grammar as usual; checking runs"
+            " asynchronously in the background and does not block you."
+        )
 
 
 class BackgroundTasksPromptComponent(PromptComponent):

artemis/agents/outputter/outputter.py

@@ -26,6 +26,7 @@ from artemis.config import OutputConfig
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import trace
 from artemis.graph.state import State
+from artemis.llm.structured import ParseFailure, parse_structured
 from artemis.services.llm import get_llm, invoke_llm_with_timeout_message, with_fallback
 from artemis.tools.scratchpad import (
     get_append_note_tool_pure,
@@ -141,6 +142,47 @@ async def outputter(
     )
 
     human_message_content = [{"type": "text", "text": human_message}]
+
+    # Verification material (only present when the run declared check items):
+    # the full append-only verdict sequence plus the four-state test summary.
+    if ctx.data_engine:
+        try:
+            from artemis.graph.checkpoints import read_ledger, read_run_outcome
+
+            ledger_records = read_ledger(ctx.data_engine.base_dir)
+            run_outcome = read_run_outcome(ctx.data_engine.base_dir) or getattr(
+                graph_output, "run_outcome", None
+            )
+            if ledger_records or run_outcome:
+                lines = ["--- Verification Verdict Ledger (append-only) ---"]
+                for r in ledger_records:
+                    lines.append(
+                        f"- attempt {r.get('attempt_id')} [{r.get('kind')}]"
+                        f" '{r.get('item_text')}' -> {r.get('status')}:"
+                        f" {r.get('evidence', '')}"
+                    )
+                if run_outcome:
+                    tests = run_outcome.get("tests") or {}
+                    lines.append(
+                        "--- Test Summary ---\n"
+                        f"task_status={run_outcome.get('task_status')},"
+                        f" passed={tests.get('passed', 0)},"
+                        f" failed={tests.get('failed', 0)},"
+                        f" inconclusive={tests.get('inconclusive', 0)},"
+                        f" unchecked={tests.get('unchecked', 0)}"
+                    )
+                lines.append(
+                    "Report every declared check item with its final state"
+                    " (passed / failed / inconclusive / unchecked) and include"
+                    " the complete verdict sequence above. A failed assertion"
+                    " is a test result to report verbatim, not something to"
+                    " explain away. Items reported unchecked were never"
+                    " judged — never present them as passed."
+                )
+                human_message_content.append({"type": "text", "text": "\n".join(lines)})
+        except Exception as e:
+            logger.error(f"Failed to attach verdict ledger to outputter: {e}")
+
     if screenshot_b64:
         human_message_content.append(
             {
@@ -306,16 +348,15 @@ async def outputter(
     # Fallback to old behavior if it didn't match the template
     if isinstance(raw_answer, str):
         raw_answer_stripped = raw_answer.strip()
-        if raw_answer_stripped.startswith("{") and raw_answer_stripped.endswith("}"):
-            try:
-                return json.loads(raw_answer_stripped)
-            except Exception:
-                pass
-        elif raw_answer_stripped.startswith("```json") and raw_answer_stripped.endswith("```"):
-            try:
-                content = raw_answer_stripped[7:-3].strip()
-                return json.loads(content)
-            except Exception:
-                pass
+        if raw_answer_stripped.startswith(("{", "[", "```")):
+            parsed = parse_structured(raw_answer_stripped)
+            if not isinstance(parsed, ParseFailure):
+                return parsed
+            # The raw text is a legitimate final answer; the miss is only
+            # noteworthy because the answer *looked* like JSON.
+            logger.warning(
+                "Outputter answer looked like JSON but could not be parsed"
+                f" ({parsed.error}); returning it as plain text."
+            )
 
     return raw_answer

artemis/agents/planner/planner.json

@@ -1,12 +1,14 @@
 {
   "blocks": {
     "role": "## Role\nYou are a Senior Test Engineer leading a team of mobile execution agents. Your objective is to deeply analyze and understand user goals, anticipate execution pitfalls, and formulate robust, observable, and feasibility-bounded milestone blueprints.",
-    "thinking_process": "## Systematic Thinking Process & Operating Principles\nBefore calling any tools or formulating your final milestones, you MUST systematically analyze the task following these 5 sequential thinking steps in your reasoning:\n\n### 1. Deep Intent Deconstruction & Task Composition\n**Core Principle**: Natural language requests describe the *substantive intent and desired real-world outcome*, not the *rigid system keywords or database filters* of mobile apps. Never mechanically map user phrasing into strict literal searches or surface filters.\n\n**Thinking Checklist (Ask yourself)**:\n- *Task Composition*: What phases make up this task? Is it a sequence of one-off linear steps (e.g., open app, configure settings), an iterative loop over multiple items (e.g., inspecting a list of search results), or a periodic polling loop (e.g., refreshing to monitor new messages, waiting for battery level)?\n- *Substantive Intent & Semantic Expansion (Connecting User Words to App Reality)*: What information or outcome is the user truly looking for, and how might that actually appear inside the target app? For example, details for an event taking place in November might have been sent months earlier (in August or September), and the email subject might use terms like \"Summit\", \"Invitation\", or \"Confirmation\" rather than the literal words \"November conference\". Proactively brainstorm the synonyms, related terms, project codes, and indirect records the app might use.\n\n**Abstract Behavioral Contrast**:\n- ❌ *Anti-Pattern (Mechanical Overfitting)*: Treating descriptive modifiers as strict literal filters -> Pre-judging items solely by surface list titles -> Prematurely missing valid candidates.\n- ✅ *Pro-Pattern (Intent Expansion & Deep Inspection)*: Decoupling core intent from surface wording -> Considering all plausible forms and timeframes -> Opening items to verify their real contents.\n\n### 2. Visual Grounding & Starting Context\n**Core Principle**: Every plan must be rooted strictly in the current physical reality of the device.\n\n**Thinking Checklist (Ask yourself)**:\n- *Current State Analysis*: What active app, screen layout, and interactive elements are visible in the provided screenshot to serve as the immediate starting point?\n\n### 3. Tiered Strategy, Funnel Approach & Feasibility Safeguards\n**Core Principle**: Formulate an efficient hierarchy of execution strategies, use a broad-to-narrow funnel for finding information, and enforce clear boundaries on loops and exploration.\n\n**Thinking Checklist (Ask yourself)**:\n- *Tiered Paths*: What is the fastest, highest-signal optimal path (e.g., targeted in-app search, direct shortcuts)? If that fails or yields no results, what is the pragmatic fallback path (e.g., browsing categories, chronological scanning)?\n- *The Funnel Approach (Broad Discovery -> Deep Verification)*:\n  - **Top of Funnel (Broad Gathering)**: First collect all potentially relevant candidates (via multi-keyword search, category browsing, or scanning). Do not filter out candidates prematurely based only on incomplete snippet text in list views.\n  - **Bottom of Funnel (Deep Inspection)**:\n  - Systematically open each candidate to thoroughly inspect its details and confirm whether it meets the criteria.\n- *Feasibility Safeguards & Loop Boundaries*:\n  - *For Item Iteration & Exploration*: Set explicit upper bounds (e.g., \"cap inspection at top 15 candidates\", \"scan recent 3 months\") to prevent endless scrolling.\n  - *For Periodic Monitoring / Polling*: Specify the check action (e.g., pull-to-refresh, reading system telemetry via ADB), the wait interval between checks (e.g., check every 5 minutes to conserve resources), and the exit criteria (e.g., stop when condition is met, or continue until manually stopped).\n\n### 4. Risk Pre-mortem & Defensive Design\n**Core Principle**: Proactively assume that execution will encounter obstacles, and build self-healing and verification checkpoints directly into the milestones.\n\n**Thinking Checklist (Ask yourself)**:\n- *Anticipate Roadblocks*: What common mobile friction points could occur (e.g., login/auth walls, permission dialogs, promotional popups/ads, search ambiguities, empty states, network latency)?\n- *Embedded Defense*: What observable criteria can be embedded into the milestone to help the executor immediately verify success or detect derailment without mistaking normal wait intervals for system stalls?\n\n### 5. Milestone Granularity, Outcome-Orientation & Dynamic Live-Iteration Tracking\n**Core Principle**: Maintain a strict two-tier hierarchy: top-level milestones define strategic macro-phases, while indented sub-tasks encapsulate every individual iteration instance within a loop.\n\n**Thinking Checklist (Ask yourself)**:\n- *Outcome-Oriented Granularity (2-5 Strategic Phases)*: Top-level milestones describe *what macro result to achieve* (e.g., \"Search for candidate emails and extract confirmed itiLine truncated
+    "thinking_process": "## Systematic Thinking Process & Operating Principles\nBefore calling any tools or formulating your final milestones, you MUST systematically analyze the task following these 5 sequential thinking steps in your reasoning:\n\n### 1. Deep Intent Deconstruction & Task Composition\n**Core Principle**: Natural language requests describe the *substantive intent and desired real-world outcome*, not the *rigid system keywords or database filters* of mobile apps. Never mechanically map user phrasing into strict literal searches or surface filters.\n\n**Thinking Checklist (Ask yourself)**:\n- *Task Composition*: What phases make up this task? Is it a sequence of one-off linear steps (e.g., open app, configure settings), an iterative loop over multiple items (e.g., inspecting a list of search results), or a periodic polling loop (e.g., refreshing to monitor new messages, waiting for battery level)?\n- *Substantive Intent & Semantic Expansion (Connecting User Words to App Reality)*: What information or outcome is the user truly looking for, and how might that actually appear inside the target app? For example, details for an event taking place in November might have been sent months earlier (in August or September), and the email subject might use terms like \"Summit\", \"Invitation\", or \"Confirmation\" rather than the literal words \"November conference\". Proactively brainstorm the synonyms, related terms, project codes, and indirect records the app might use.\n\n**Abstract Behavioral Contrast**:\n- ❌ *Anti-Pattern (Mechanical Overfitting)*: Treating descriptive modifiers as strict literal filters -> Pre-judging items solely by surface list titles -> Prematurely missing valid candidates.\n- ✅ *Pro-Pattern (Intent Expansion & Deep Inspection)*: Decoupling core intent from surface wording -> Considering all plausible forms and timeframes -> Opening items to verify their real contents.\n\n### 2. Visual Grounding & Starting Context\n**Core Principle**: Every plan must be rooted strictly in the current physical reality of the device.\n\n**Thinking Checklist (Ask yourself)**:\n- *Current State Analysis*: What active app, screen layout, and interactive elements are visible in the provided screenshot to serve as the immediate starting point?\n\n### 3. Tiered Strategy, Funnel Approach & Feasibility Safeguards\n**Core Principle**: Formulate an efficient hierarchy of execution strategies, use a broad-to-narrow funnel for finding information, and enforce clear boundaries on loops and exploration.\n\n**Thinking Checklist (Ask yourself)**:\n- *Tiered Paths*: What is the fastest, highest-signal optimal path (e.g., targeted in-app search, direct shortcuts)? If that fails or yields no results, what is the pragmatic fallback path (e.g., browsing categories, chronological scanning)?\n- *The Funnel Approach (Broad Discovery -> Deep Verification)*:\n  - **Top of Funnel (Broad Gathering)**: First collect all potentially relevant candidates (via multi-keyword search, category browsing, or scanning). Do not filter out candidates prematurely based only on incomplete snippet text in list views.\n  - **Bottom of Funnel (Deep Inspection)**:\n  - Systematically open each candidate to thoroughly inspect its details and confirm whether it meets the criteria.\n- *Feasibility Safeguards & Loop Boundaries*:\n  - *For Item Iteration & Exploration*: Set explicit upper bounds (e.g., \"cap inspection at top 15 candidates\", \"scan recent 3 months\") to prevent endless scrolling.\n  - *For Periodic Monitoring / Polling*: Specify the check action (e.g., pull-to-refresh, reading system telemetry via ADB), the wait interval between checks (e.g., check every 5 minutes to conserve resources), and the exit criteria (e.g., stop when condition is met, or continue until manually stopped — tag such unbounded milestones `[Loop:continuous]`).\n\n### 4. Risk Pre-mortem & Defensive Design\n**Core Principle**: Proactively assume that execution will encounter obstacles, and build self-healing and verification checkpoints directly into the milestones.\n\n**Thinking Checklist (Ask yourself)**:\n- *Anticipate Roadblocks*: What common mobile friction points could occur (e.g., login/auth walls, permission dialogs, promotional popups/ads, search ambiguities, empty states, network latency)?\n- *Embedded Defense*: What observable criteria can be embedded into the milestone to help the executor immediately verify success or detect derailment without mistaking normal wait intervals for system stalls?\n\n### 5. Milestone Granularity, Outcome-Orientation & Dynamic Live-Iteration Tracking\n**Core Principle**: Maintain a strict two-tier hierarchy: top-level milestones define strategic macro-phases, while indented sub-tasks encapsulate every individual iteration instance within a loop.\n\n**Thinking Checklist (Ask yourself)**:\n- *Outcome-Oriented Granularity (2-5 Strategic Phases)*: Top-level milestones describe *what macro result to achieve* (e.g., \Line truncated
     "capabilities_operator": "## Execution Capabilities\n\nThe execution agent primarily interacts with the mobile device and analyzes screen recordings and logs. It can perform the following types of actions:\n- **UI Interaction**: Tapping, swiping, and entering text on the screen.\n- **App Management**: Launching and stopping specific applications.\n- **Information Retrieval & Analysis**: Reading on-screen text and UI hierarchy, searching the web for external information, querying system logs, and analyzing screen recordings.\n- **Terminal & Background Operations**: Executing arbitrary ADB shell commands on the device for exploration. It is capable of running long-running ADB commands asynchronously in the background as tasks.\n\n### Capability Boundaries & Planning Rules\n- **Cross-App Operations**: When a task requires interacting with multiple applications, split the operations into distinct milestones for each app. Avoid mixing actions for different apps in a single milestone.",
-    "output_protocol_initial": "## Task Plan & Scratchpad Protocol\n\n1. **Articulate Thinking**: Begin by reasoning through the 5-step **Systematic Thinking Process** to establish your strategy, funnel approach, bounds, and pre-mortem checks.\n2. **Designate Notes in Milestones & Pre-generate Answer Cards**: If the task involves collecting findings, logging periodic monitoring events, recording candidates, or analytical summaries, designate clear, descriptive note names directly in the milestone descriptions (e.g., `candidates_list`, `monitoring_log`, `schedule_summary`). You can and should invoke the `save_note` tool to pre-generate these structured template files (answer card / scratchpad skeletons with Markdown headers, sections, or tables) before submitting `task_plan`. This provides downstream executors with a clear, ready-to-fill schema.\n3. **Submit Plan**: Submit your initial plan by invoking the `save_note` tool with `key='task_plan'`. Avoid writing the plan only in your text response, as it will only be recognized by the system if submitted via the tool.\n\n### Plan Formatting Rules\n- Format the plan strictly as a Markdown task list starting with `- [ ]` for uncompleted items.\n- For linear macro-milestones: `- [ ] <Phase Description>`\n- For repetitive, iterative, or periodic polling milestones, tag the item with `[Loop]` and declare its Exit Boundary and Interval (e.g., `- [ ] [Loop] <Overall Loop / Monitoring Goal> (Exit: <Condition or Continuous until manually stopped>; Interval: <e.g., every 5 minutes>)`).\n- **Strict Two-Tier Hierarchy & Loop Subtask Contract**:\n  1. **Subtask Nesting Rule**: ALL individual iteration instances, polling checks, candidate inspections, or baseline setups MUST be formatted as 2-space indented sub-tasks (`  - [ ]`) directly nested under their parent `[Loop]` milestone.\n  2. **Mandatory First-Instance Initialization**: Every `[Loop]` milestone MUST be initialized with its first concrete child sub-task (e.g., `  - [ ] Inspect Candidate #1: ...` or `  - [ ] Polling Check #1 (Baseline & Initial Sync): ...`).\n  3. **No Flattened Cycles**: NEVER elevate individual checks, recurring polling cycles, or candidate inspections to top-level milestones outside the `[Loop]`. Top-level items are reserved exclusively for strategic macro-phases.\n- Ensure the content of the `task_plan` note contains only the task list, excluding introductions, explanations, conversational text, or headers.",
+    "output_protocol_initial": "## Task Plan & Scratchpad Protocol\n\n1. **Articulate Thinking**: Begin by reasoning through the 5-step **Systematic Thinking Process** to establish your strategy, funnel approach, bounds, and pre-mortem checks.\n2. **Designate Notes in Milestones & Pre-generate Answer Cards**: If the task involves collecting findings, logging periodic monitoring events, recording candidates, or analytical summaries, designate clear, descriptive note names directly in the milestone descriptions (e.g., `candidates_list`, `monitoring_log`, `schedule_summary`). You can and should invoke the `save_note` tool to pre-generate these structured template files (answer card / scratchpad skeletons with Markdown headers, sections, or tables) before submitting `task_plan`. This provides downstream executors with a clear, ready-to-fill schema.\n3. **Submit Plan**: Submit your initial plan by invoking the `save_note` tool with `key='task_plan'`. Avoid writing the plan only in your text response, as it will only be recognized by the system if submitted via the tool.\n\n### Plan Formatting Rules\n- Format the plan strictly as a Markdown task list starting with `- [ ]` for uncompleted items.\n{{ plan_grammar }}\n- For linear macro-milestones: `- [ ] <Phase Description>`\n- For bounded repetitive, iterative, or periodic polling milestones, tag the item with `[Loop]` and declare its Exit Boundary and Interval (e.g., `- [ ] [Loop] <Overall Loop / Monitoring Goal> (Exit: <Condition>; Interval: <e.g., every 5 minutes>)`).\n- For unbounded continuous monitoring that only the user can stop, tag the item with `[Loop:continuous]` instead (e.g., `- [ ] [Loop:continuous] <Monitoring Goal> (Interval: <e.g., every 5 minutes>)`); never declare a self-serve exit condition for it.\n- **Strict Two-Tier Hierarchy & Loop Subtask Contract**:\n  1. **Subtask Nesting Rule**: ALL individual iteration instances, polling checks, candidate inspections, or baseline setups MUST be formatted as 2-space indented sub-tasks (`  - [ ]`) directly nested under their parent `[Loop]` milestone.\n  2. **Mandatory First-Instance Initialization**: Every `[Loop]` milestone MUST be initialized with its first concrete child sub-task (e.g., `  - [ ] Inspect Candidate #1: ...` or `  - [ ] Polling Check #1 (Baseline & Initial Sync): ...`).\n  3. **No Flattened Cycles**: NEVER elevate individual checks, recurring polling cycles, or candidate inspections to top-level milestones outside the `[Loop]`. Top-level items are reserved exclusively for strategic macro-phases.\n- Ensure the content of the `task_plan` note contains only the task list, excluding introductions, explanations, conversational text, or headers.",
     "examples_initial": "## Master Example: Demonstrating the Complete Cognitive Paradigm\n\n**Goal**: \"Find and summarize all schedule arrangements and venue details for the upcoming November tech conference from my emails.\"\n\n### Naive / Incorrect Approaches:\n1. *Flawed Semantics & Too Granular*:\n- [ ] Tap search icon in Gmail\n- [ ] Type \"November\"\n- [ ] Open first email and tap back\n*(Fails: Micromanages atomic UI taps; assumes emails for November arrangements must have \"November\" in the title or be received in November)*\n2. *Vague & Unbounded Trap*:\n- [ ] Open email and check all emails for conference details\n*(Fails: Unbounded scan trap; causes infinite scrolling and step budget exhaustion on mobile)*\n\n---\n\n### Correct Approach (Full Cognitive Thinking & Robust Plan):\n\n#### Step-by-Step Thinking:\n1. **Intent & Semantic Expansion**:\n   - *Core Objective*: Extract confirmed schedule dates, sessions, and venue info for the November tech conference.\n   - *Semantic Insight*: Conference communications are often sent months in advance (e.g., ticket confirmations or registration emails in August/September). The subject line may say \"Global Developer Summit\", \"Tech Expo Agenda\", or \"Registration Confirmed\" rather than literally \"November conference\".\n   - *Task Composition*: Linear setup (open email app and search) -> Iterative candidate inspection loop (examine each promising email) -> Final consolidation.\n2. **Visual Grounding**:\n   - The device screenshot shows the Home screen with the Gmail app icon visible. The plan must start by opening the email client.\n3. **Tiered Strategy, Funnel Approach & Safeguards**:\n   - *Tier 1 (Optimal)*: Use in-app search with rich semantic keywords (\"conference\", \"summit\", \"schedule\", \"agenda\", \"venue\", \"November\").\n   - *Tier 2 (Fallback)*: If search is unavailable or yields zero results, fallback to chronological scanning of recent 3 months.\n   - *Funnel (Broad Gathering -> Deep Inspection)*: Gather candidate emails from search results; do not discard candidates based solely on list previews. Open each candidate thread to inspect full body details.\n   - *Feasibility Boundary*: Cap the candidate email inspection pool at top 15 emails to prevent infinite scrolling.\n4. **Risk Pre-mortem**:\n   - *Risks*: Promotional marketing spam; conflicting drafts with preliminary dates; cancelled sessions.\n   - *Defensive Check*: Instruct downstream executor to filter out promotional newsletters, verify session timestamps, and confirm final venue address.\n5. **Milestone Granularity & Scratchpad Blueprint**:\n   - Focus milestones on business outcomes (search, inspect candidates, consolidate), avoiding micromanaged click-by-click steps.\n   - Use indented sub-tasks to guide the loop of inspecting each candidate email, initializing with Item #1 and adding subsequent items as they are identified. This prevents the agent from losing state or re-inspecting the same items.\n   - Designate note `november_conference_summary` as the destination for recording findings and pre-generate the structured template note.\n\n#### Tool Invocation:\n1. Pre-generate designated answer card note template:\n`save_note(key='november_conference_summary', content='# November Tech Conference Summary\\n\\n## Key Details\\n- Conference Name: \\n- Dates: \\n- Venue / Address: \\n\\n## Daily Schedule & Sessions\\n(To be populated during email inspection)\\n')`\n\n2. Submit milestone plan:\n`save_note(key='task_plan', content='- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\\n  - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\\n- [ ] Review and consolidate all extracted information in `november_conference_summary` into a clear, finalized conference itinerary')`\n\n#### Content of `task_plan` Note:\n- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\n  - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\n- [ ] ReviLine truncated
+    "check_generation": "## Declaring Verification Standards (check lines)\nYou may attach check lines to the plan; an independent Checker judges them asynchronously.\n\n- **User-declared expectations become `- assert:` lines.** Whenever the user's goal states an explicit expectation (\"should show X\", \"must not crash\", \"the toast Y should appear\"), declare it as an assert line preserving the user's wording verbatim. Expectations about the whole run or the final state use the `@end` suffix (task-level lines at the end of the plan).\n- **`- verify:` lines only for milestones that genuinely need acceptance checks** — completion points of irreversible operations, or milestones that directly contribute to the final goal. Do NOT attach a verify line to every subgoal: restraint is an explicit requirement (checking cost scales with the number of check lines).\n- **Do not fabricate assertions** for subgoals that have nothing verifiable. Rewrite transient expectations as persistently probeable state wherever possible (e.g., \"alarm exists in the alarm list\" instead of \"a toast flashed\") — evidence for transient moments only exists if the execution history happened to capture it.\n- **Ordering requirements are post-hoc.** If the user demands \"A must be confirmed before B runs\", still generate the assertion preserving their semantics, but annotate in the plan that ordering is verified retroactively from history — the system performs post-hoc audits and CANNOT block B from executing. Never silently promise enforcement that does not exist.",
+    "check_audit": "- **Check Standard Protection**: Check lines (`- verify:` / `- assert:`) declared in the plan are the task's verification standards. REJECT any proposed plan that deletes, weakens, or rewords existing check lines — softening a declared standard is never an acceptable adaptation. Adding new check lines is allowed.",
     "role_validator": "## Role\nYou are the Task Plan Validator. The Operator has attempted to modify the top-level task plan. Your job is to ensure the proposed new plan still aligns with the user's original intent and is viable.",
-    "validator_guidelines": "## Validation Principles\n- **Goal Alignment**: Ensure the new plan ultimately accomplishes the `Initial Goal` requested by the user. If the Operator has abandoned the core objective or fundamentally altered it, you must reject the plan.\n- **Logical Coherence**: Verify that the proposed milestones form a logical, achievable sequence to reach the goal. Support valid ongoing iteration/polling subgoals and trigger branches without misconstruing periodic checks as infinite loops. Reject the plan if it introduces contradictory steps, unhandled deadlocks, or obvious logical fallacies.\n- **Loop & Monitoring Protection**: If the existing plan contains an active `[Loop]` milestone (especially continuous polling / monitoring with exit condition `until manually stopped`), you MUST REJECT any proposed plan that deletes the `[Loop]` milestone, removes the loop tag, or marks a continuous monitoring milestone as completed (`[x]`) without explicit external user instruction.\n- **Adaptive Acceptance**: The Operator operates in a dynamic environment and may encounter unexpected UI states or dynamic event triggers. You should approve alternative strategies, extra intermediate steps, loop status tracking, or necessary workarounds as long as they serve the primary goal effectively.\n- **Constructive Feedback**: If you must reject the plan, provide clear, objective, and actionable feedback explaining the logical flaw or goal deviation, and suggest how the Operator should correct the plan."
+    "validator_guidelines": "## Validation Principles\n- **Goal Alignment**: Ensure the new plan ultimately accomplishes the `Initial Goal` requested by the user. If the Operator has abandoned the core objective or fundamentally altered it, you must reject the plan.\n- **Logical Coherence**: Verify that the proposed milestones form a logical, achievable sequence to reach the goal. Support valid ongoing iteration/polling subgoals and trigger branches without misconstruing periodic checks as infinite loops. Reject the plan if it introduces contradictory steps, unhandled deadlocks, or obvious logical fallacies.\n{{ plan_grammar }}\n- **Loop & Monitoring Protection**: If the existing plan contains an active `[Loop]` or `[Loop:continuous]` milestone, you MUST REJECT any proposed plan that deletes the milestone or removes its loop tag, as well as any plan that marks a `[Loop:continuous]` continuous monitoring milestone as completed (`[x]`) without an explicit external user stop signal.\n- **Adaptive Acceptance**: The Operator operates in a dynamic environment and may encounter unexpected UI states or dynamic event triggers. You should approve alternative strategies, extra intermediate steps, loop status tracking, or necessary workarounds as long as they serve the primary goal effectively.\n- **Constructive Feedback**: If you must reject the plan, provide clear, objective, and actionable feedback explaining the logical flaw or goal deviation, and suggest how the Operator should correct the plan."
   },
   "modes": {
     "initial_plan": {

artemis/agents/planner/planner.py

@@ -25,7 +25,9 @@ from artemis.context import ArtemisContext
 from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan, trace
 from artemis.graph.state import State
+from artemis.graph.visibility import strict_state
 from artemis.services.llm import (
+    acomplete,
     get_llm,
     invoke_llm_with_timeout_message,
     with_fallback,
@@ -44,11 +46,30 @@ from artemis.tools.tool_wrapper import (
 from artemis.utils.decorators import wrap_with_callbacks
 from artemis.utils.logger import get_logger
 from artemis.utils.notes import get_note_file_path
+from artemis.utils.plan_grammar import render_plan_grammar_spec
 from artemis.utils.task_tree import build_plan_and_history
 
 logger = get_logger(__name__)
 
 
+def _checks_enabled(ctx: ArtemisContext) -> bool:
+    """Config-driven prompt assembly gate: with both check gates disabled, no
+    check-related instruction reaches any planner prompt."""
+    setup = getattr(ctx, "execution_setup", None)
+    return bool(setup and getattr(setup, "checks_enabled", False))
+
+
+def build_planner_system_blocks(prompts_data: dict, mode: str, include_checks: bool) -> list[str]:
+    """Assembly point for the planner prompt's component list (§1.3): the
+    check-related block mounts only while the checking feature is active."""
+    blocks = list(prompts_data["modes"][mode]["system"])
+    if include_checks:
+        extra = "check_generation" if mode == "initial_plan" else "check_audit"
+        if extra in prompts_data.get("blocks", {}):
+            blocks.append(extra)
+    return blocks
+
+
 class _CyFunctionDetectorMeta(type):
     def __instancecheck__(self, instance):
         name = type(instance).__name__
@@ -97,10 +118,13 @@ async def run_async_planner_validation(
         with open(prompts_path, encoding="utf-8") as f:
             prompts_data = json.load(f)
 
+        include_checks = _checks_enabled(ctx)
         mode_config = prompts_data["modes"]["validator"]
-        system_blocks = mode_config["system"]
+        system_blocks = build_planner_system_blocks(prompts_data, "validator", include_checks)
         system_content = "\n\n".join(prompts_data["blocks"][block] for block in system_blocks)
-        system_message = Template(system_content).render()
+        system_message = Template(system_content).render(
+            plan_grammar=render_plan_grammar_spec(include_checks)
+        )
 
         history_str = "No history available."
         if ctx.data_engine:
@@ -163,6 +187,7 @@ class PlannerNode:
     )
     @trace(type="agent", name="planner")
     async def __call__(self, state: State):
+        state = strict_state(state, "planner")
         prompts_path = Path(__file__).parent / "planner.json"
         with open(prompts_path, encoding="utf-8") as f:
             prompts_data = json.load(f)
@@ -170,10 +195,16 @@ class PlannerNode:
         mode = "initial_plan"
         mode_config = prompts_data["modes"][mode]
 
-        system_blocks = mode_config["system"]
+        include_checks = _checks_enabled(self.ctx)
+        # Assembly point (not in-template conditionals): check-generation
+        # instructions only mount while the checking feature is active, so a
+        # fully-disabled configuration yields a byte-identical planner prompt.
+        system_blocks = build_planner_system_blocks(prompts_data, mode, include_checks)
         system_content = "\n\n".join(prompts_data["blocks"][block] for block in system_blocks)
 
-        system_message = Template(system_content).render()
+        system_message = Template(system_content).render(
+            plan_grammar=render_plan_grammar_spec(include_checks)
+        )
 
         # Get screenshot
         screenshot_b64 = None
@@ -287,22 +318,14 @@ class PlannerNode:
         for validation_attempt in range(max_validation_attempts):
             tool_called = False
 
-            async def run_stream(llm_model):
-                full_response = None
-                trace_id = CURRENT_TRACE_ID.get()
-                async for chunk in llm_model.astream(current_messages):
-                    if full_response is None:
-                        full_response = chunk
-                    else:
-                        full_response += chunk
-
-                    pass
-                return full_response
-
             for _ in range(max_iterations):
                 response = await with_fallback(
-                    main_call=lambda: invoke_llm_with_timeout_message(run_stream(llm)),
-                    fallback_call=lambda: invoke_llm_with_timeout_message(run_stream(llm_fallback)),
+                    main_call=lambda: invoke_llm_with_timeout_message(
+                        acomplete(llm, current_messages)
+                    ),
+                    fallback_call=lambda: invoke_llm_with_timeout_message(
+                        acomplete(llm_fallback, current_messages)
+                    ),
                 )
 
                 if response is None or not response.tool_calls:
@@ -415,8 +438,4 @@ class PlannerNode:
                 f"Failed to generate a valid plan after {max_validation_attempts} attempts."
             )
 
-        return await state.asanitize_update(
-            ctx=self.ctx,
-            update={},
-            agent="planner",
-        )
+        return {}

artemis/agents/summarizer/summarizer.py

@@ -25,7 +25,8 @@ from langchain_core.messages import (
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import CURRENT_TRACE_ID, trace
 from artemis.graph.state import State
-from artemis.services.llm import get_llm
+from artemis.graph.visibility import strict_state
+from artemis.services.llm import acomplete, get_llm
 from artemis.utils.coordinates import normalize_any_structure
 from artemis.utils.logger import get_logger
 from artemis.utils.task_tree import (
@@ -42,6 +43,7 @@ class SummarizerNode:
 
     @trace(type="agent", name="summarizer")
     async def __call__(self, state: State):
+        state = strict_state(state, "summarizer")
         # 1. Trigger background task for summary and data engine update
         if self.ctx.data_engine and state.current_step_id:
             step_id = UUID(state.current_step_id)
@@ -179,26 +181,16 @@ class SummarizerNode:
                     HumanMessage(content="\n\n".join(content)),
                 ]
 
+                aggregated_chunk = await acomplete(llm, messages)
+
                 generated_text = ""
-                aggregated_chunk = None
-                async for chunk in llm.astream(messages):
-                    if aggregated_chunk is None:
-                        aggregated_chunk = chunk
-                    else:
-                        aggregated_chunk += chunk
-
-                    if chunk.content:
-                        text_to_stream = ""
-                        if isinstance(chunk.content, str):
-                            text_to_stream = chunk.content
-                        elif isinstance(chunk.content, list):
-                            for item in chunk.content:
-                                if isinstance(item, dict) and item.get("type") == "text":
-                                    text_to_stream += item.get("text", "")
-
-                        if text_to_stream:
-                            generated_text += text_to_stream
-                    pass
+                content = getattr(aggregated_chunk, "content", "") if aggregated_chunk else ""
+                if isinstance(content, str):
+                    generated_text = content
+                elif isinstance(content, list):
+                    for item in content:
+                        if isinstance(item, dict) and item.get("type") == "text":
+                            generated_text += item.get("text", "")
 
                 summary = generated_text.strip()
                 logger.info(f"Summary generated: {summary}")

artemis/agents/validator/tool_declarations.py

@@ -30,6 +30,7 @@ from artemis.context import ArtemisContext
 from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.core.tool_declaration import ToolDeclaration
 from artemis.data_engine.trace import trace
+from artemis.mcp.action_specs import tool_declaration
 from artemis.mcp.action_types import ActionCode
 from artemis.mcp.actuators.adb import AdbActuator
 from artemis.mcp.observation import observe
@@ -187,9 +188,7 @@ async def capture_screenshot_and_parse_ui(
     Thin adapter over :func:`artemis.mcp.observation.observe` that adds the LangGraph
     ``State`` write-back (indexed points/elements) the agents rely on.
     """
-    obs, screenshot_bytes = await observe(
-        ctx, controller, settle_ms=0 if skip_settling else 400
-    )
+    obs, screenshot_bytes = await observe(ctx, controller, settle_ms=0 if skip_settling else 400)
     if not obs.ok:
         return None, None, None
 
@@ -473,7 +472,7 @@ class MobileActionExecutor:
                     direction=target,
                     target=swipe_input.get("target"),
                     indexed_elements=getattr(state, "indexed_elements", None) if state else None,
-                    ui_hierarchy=getattr(state, "ui_tree", None) if state else None,
+                    ui_hierarchy=getattr(state, "latest_ui_hierarchy", None) if state else None,
                     width=width,
                     height=height,
                     duration=final_duration,
@@ -753,189 +752,19 @@ class MobileActionExecutor:
         )
 
 
-CLICK_TOOL = ToolDeclaration(
-    name="click",
-    description=(
-        "[ACTION] Tap/click on the target coordinate on the screen. The screen"
-        " after click will be returned automatically. For buttons, checkboxes,"
-        " tabs, icons, items in a list, tap ON the element. For text fields /"
-        " search bars / input boxes, tap INSIDE the input box to focus it."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "target": {
-                "type": "array",
-                "items": {"type": "integer"},
-                "description": "Normalized coordinates [x, y] in 0-1000 scale.",
-            },
-            "times": {
-                "type": "integer",
-                "description": "Number of taps to perform (default 1). Set to 2 for double click.",
-            },
-            "delay_ms": {
-                "type": "integer",
-                "description": "Delay between taps in milliseconds (default 100).",
-            },
-        },
-        "required": ["target"],
-    },
-)
+# Device-action declarations are generated from the canonical manifest
+# (artemis/mcp/action_specs.py); the historical constant names remain the public API.
+CLICK_TOOL = tool_declaration("click")
 
-CLICK_SEQUENCE_TOOL = ToolDeclaration(
-    name="click_sequence",
-    description=(
-        "[ACTION] Executes a sequence of taps one by one in order on the"
-        " specified targets (e.g. [[500, 280], [885, 362]]). The screen will be returned ONLY after all clicks"
-        " in the sequence have completed."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "sequence": {
-                "type": "array",
-                "items": {
-                    "type": "array",
-                    "items": {"type": "integer"},
-                    "description": (
-                        "Normalized coordinates [x, y] in 0-1000 scale (e.g., [500, 280]),"
-                        " or a single integer element index."
-                    ),
-                },
-                "description": "List of targets to tap in sequence, e.g. [[500, 280], [885, 362]].",
-            },
-            "delay_ms": {
-                "type": "integer",
-                "description": "Delay between consecutive taps in milliseconds (default 50ms).",
-            },
-        },
-        "required": ["sequence"],
-    },
-)
+CLICK_SEQUENCE_TOOL = tool_declaration("click_sequence")
 
-LONG_PRESS_TOOL = ToolDeclaration(
-    name="long_press",
-    description=(
-        "[ACTION] Long press on a target coordinate on the screen. The screen"
-        " after long pressing will be returned automatically."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "target": {
-                "type": "array",
-                "items": {"type": "integer"},
-                "description": "Normalized coordinates [x, y] in 0-1000 scale.",
-            },
-            "duration_ms": {
-                "type": "integer",
-                "description": "Duration of press in milliseconds (default 1000ms).",
-            },
-        },
-        "required": ["target"],
-    },
-)
+LONG_PRESS_TOOL = tool_declaration("long_press")
 
-INPUT_TEXT_TOOL = ToolDeclaration(
-    name="input_text",
-    description=(
-        "[ACTION] Type text into an input field on the screen. The screen after"
-        " typing will be returned automatically. Automatically taps inside the"
-        " input box at target [x, y] to focus before typing."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "text": {
-                "type": "string",
-                "description": "Text to type into the focused input field.",
-            },
-            "target": {
-                "type": "array",
-                "items": {"type": "integer"},
-                "description": "Coordinates [x, y] of the input box in 0-1000 scale.",
-            },
-            "clear_exist": {
-                "type": "boolean",
-                "description": (
-                    "Whether to clear existing text in the input box before typing (default True)."
-                ),
-            },
-        },
-        "required": ["text", "target"],
-    },
-)
+INPUT_TEXT_TOOL = tool_declaration("input_text")
 
-SWIPE_TOOL = ToolDeclaration(
-    name="swipe",
-    description=(
-        "[ACTION] Perform a swipe, drag, or slider-adjustment gesture on the screen. The screen and UI hierarchy after swipe will be returned automatically.\n\n"
-        "• Directional Scrolling ('direction' or 'action'): Recommended for general browsing and standard page scrolling in most scenarios. Automatically computes safe swipe vectors and adaptive duration, retains a ~40% visual overlap anchor for zero-omission traversal, and prevents inertial flings. Supports scoping to a sub-container via 'target'. If it fails on certain custom layouts, fall back to specifying exact coordinates ('start' and 'end') directly.\n"
-        "• Precise Coordinate Gestures ('start', 'end' or coordinates list): Best for local, fine-grained interactions such as adjusting sliders/SeekBars (e.g., volume, brightness, progress bars), drag-and-drop / list reordering, or as a reliable fallback when directional scrolling fails on specific containers. Always drag slightly PAST the target position to overcome touch slop and reliably trigger the update. When setting a slider to Maximum (100%) or Minimum (0%), swipe fully to the extreme boundary."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "direction": {
-                "type": "string",
-                "enum": ["up", "down", "left", "right"],
-                "description": (
-                    "Direction for scrolling and swiping: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
-                    " 'down' (drags top-to-bottom, scrolling up to reveal content above),"
-                    " 'left' (drags right-to-left, scrolling right),"
-                    " 'right' (drags left-to-right, scrolling left)."
-                ),
-            },
-            "start": {
-                "type": "array",
-                "items": {"type": "integer"},
-                "description": "Start normalized coordinates [start_x, start_y] in 0-1000 scale.",
-            },
-            "end": {
-                "type": "array",
-                "items": {"type": "integer"},
-                "description": "End normalized coordinates [end_x, end_y] in 0-1000 scale.",
-            },
-            "target": {
-                "description": "Optional target element index (e.g. 2) or container bounds [left, top, right, bottom] to scope the directional swipe within.",
-            },
-            "action": {
-                "description": (
-                    "Backward-compatible swipe gesture: smart direction string ('up', 'down', 'left', 'right')"
-                    " OR precise custom coordinates [start_x, start_y, end_x, end_y] in 0-1000 scale."
-                ),
-            },
-            "duration": {
-                "type": "integer",
-                "description": (
-                    "Optional swipe/drag duration in milliseconds (default 800). For drag-and-drop,"
-                    " list reordering, or sliding/adjusting sliders (e.g., volume, brightness, SeekBars),"
-                    " set duration >= 1000 (e.g. 1500). If omitted for directional swipe, duration is computed automatically."
-                ),
-            },
-        },
-    },
-)
+SWIPE_TOOL = tool_declaration("swipe")
 
-PRESS_KEY_TOOL = ToolDeclaration(
-    name="press_key",
-    description=(
-        "[ACTION] Press a physical or virtual system button. The screen after"
-        " pressing will be returned automatically."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "key": {
-                "type": "string",
-                "description": (
-                    "Standard Android system button name (ENTER, BACK, HOME, APP_SWITCH)."
-                ),
-            }
-        },
-        "required": ["key"],
-    },
-)
+PRESS_KEY_TOOL = tool_declaration("press_key")
 
 READ_NOTE_TOOL = ToolDeclaration(
     name="read_note",
@@ -966,49 +795,9 @@ LIST_NOTES_TOOL = ToolDeclaration(
     parameters={"type": "object", "properties": {}},
 )
 
-MANAGE_APP_TOOL = ToolDeclaration(
-    name="manage_app",
-    description=(
-        "[ACTION] Launch or force stop a specified application. The screen"
-        " after launch/stop will be returned automatically."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "action": {
-                "type": "string",
-                "description": "The action type ('launch' or 'stop').",
-            },
-            "app_name": {
-                "type": "string",
-                "description": "Display name or package name of the application.",
-            },
-        },
-        "required": ["action", "app_name"],
-    },
-)
+MANAGE_APP_TOOL = tool_declaration("manage_app")
 
-WAIT_FOR_DELAY_TOOL = ToolDeclaration(
-    name="wait_for_delay",
-    description=(
-        "[ACTION] Pause execution and wait for a specified duration in milliseconds."
-        " The screen after pause will be returned automatically."
-    ),
-    parameters={
-        "type": "object",
-        "properties": {
-            "time_in_ms": {
-                "type": "integer",
-                "description": (
-                    "The exact duration to pause in milliseconds (e.g., 2000 for 2s,"
-                    " 60000 for 1 min, 180000 for 3 mins, 300000 for 5 mins)."
-                    " Convert any required waiting duration into milliseconds."
-                ),
-            }
-        },
-        "required": ["time_in_ms"],
-    },
-)
+WAIT_FOR_DELAY_TOOL = tool_declaration("wait_for_delay")
 
 REPORT_FAILURE_ANALYSIS_TOOL = ToolDeclaration(
     name="report_failure_analysis",

artemis/agents/validator/validator.py

@@ -38,9 +38,10 @@ from artemis.context import ArtemisContext
 from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.data_engine.trace import CURRENT_TRACE_ID, trace
 from artemis.graph.state import State
+from artemis.graph.visibility import strict_state
 from artemis.mcp.action_names import to_canonical_call
 from artemis.mcp.action_session import ActionSession, get_action_session
-from artemis.services.llm import get_llm
+from artemis.services.llm import acomplete_structured, get_llm
 from artemis.utils import image_diff, visualization
 from artemis.utils.decorators import wrap_with_callbacks
 from artemis.utils.logger import get_logger
@@ -154,11 +155,7 @@ class ValidatorNode:
     async def _execute_validation_loop(self, state: State):
         actions, error_msg = self._parse_decisions(state.structured_decisions)
         if error_msg:
-            return await state.asanitize_update(
-                ctx=self.ctx,
-                update={},
-                agent="validator",
-            )
+            return {}
 
         execution = []
         failed_action = None
@@ -176,11 +173,7 @@ class ValidatorNode:
             decision_screenshot_b64 = last_screenshot_b64
             decision_screenshot_name = last_screenshot_name
         except Exception:
-            return await state.asanitize_update(
-                ctx=self.ctx,
-                update={},
-                agent="validator",
-            )
+            return {}
 
         step_id = None
         if state.current_step_id:
@@ -355,7 +348,9 @@ class ValidatorNode:
                 if post_screenshot_b64:
                     decoded_bytes = base64.b64decode(post_screenshot_b64)
                     last_screenshot_b64 = post_screenshot_b64
-                    logger.info(f"Successfully decoded post_screenshot_b64 ({len(decoded_bytes)} bytes)")
+                    logger.info(
+                        f"Successfully decoded post_screenshot_b64 ({len(decoded_bytes)} bytes)"
+                    )
                     if self.ctx.data_engine:
                         post_image_name = self.ctx.data_engine.get_or_create_image(decoded_bytes)
                         last_screenshot_name = post_image_name
@@ -466,11 +461,7 @@ class ValidatorNode:
                 step_id, report, post_image_name=distinct_post_image_name
             )
 
-        return await state.asanitize_update(
-            ctx=self.ctx,
-            update={"last_execution_result": report},
-            agent="validator",
-        )
+        return {"last_execution_result": report}
 
     @wrap_with_callbacks(
         before=lambda: logger.info("Starting Validator Agent..."),
@@ -479,6 +470,7 @@ class ValidatorNode:
     )
     @trace(type="agent", name="validator")
     async def __call__(self, state: State):
+        state = strict_state(state, "validator")
 
         step_id = None
         if state.current_step_id:
@@ -1196,22 +1188,16 @@ class ValidatorNode:
                     HumanMessage(content=user_content),
                 ]
 
-                # D. Invoke Universal VLM
-                response = await llm.ainvoke(messages)
-                output = response.content if isinstance(response.content, str) else ""
-                if isinstance(response.content, list):
-                    output = "".join(
-                        b.get("text", "")
-                        for b in response.content
-                        if isinstance(b, dict) and "text" in b
+                # D. Invoke Universal VLM. Parsing (fences, repair) and one
+                # corrective re-ask on unparseable JSON are handled by the
+                # structured-output layer; a StructuredOutputError lands in
+                # this attempt loop's except like any other attempt failure.
+                res_json = await acomplete_structured(llm, messages)
+                if not isinstance(res_json, dict):
+                    raise ValueError(
+                        "Pixel validation response parsed to"
+                        f" {type(res_json).__name__}, expected a JSON object."
                     )
-
-                if output.startswith("```"):
-                    code_match = re.search(r"```(?:json)?\s*([\s\S]*?)\s*```", output)
-                    if code_match:
-                        output = code_match.group(1).strip()
-
-                res_json = json.loads(output)
                 reasoning = res_json.get("reasoning", "")
                 is_present = res_json.get("is_present", True)
                 confidence = res_json.get("confidence", 1.0)

artemis/agents/video_analyzer/reliability.py

@@ -12,15 +12,24 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-"""Failure classification and circuit breaking for video model calls."""
+"""Failure classification and circuit breaking for video model calls.
+
+Thin media-aware extension of :mod:`artemis.llm.reliability`, which owns the
+generic taxonomy and circuit breaker.  This module adds the media-processing
+category and the ``should_split`` recovery decision (bisecting long clips).
+"""
 
 from __future__ import annotations
 
 from dataclasses import dataclass
 from enum import StrEnum
-import threading
-import time
-from typing import Any
+
+from artemis.llm.reliability import (
+    CircuitBreaker,
+    FailureCategory,
+    classify_failure,
+    extract_status_code,
+)
 
 
 class VideoFailureCategory(StrEnum):
@@ -43,115 +52,37 @@ class VideoFailure:
     should_fallback: bool
 
 
+_MEDIA_MARKERS = ("ffmpeg", "codec", "corrupt", "invalid video", "invalid audio", "media")
+
+
 def classify_video_failure(error: BaseException) -> VideoFailure:
     """Convert provider/transport/media exceptions into recovery decisions."""
-
-    if isinstance(error, KeyboardInterrupt) or type(error).__name__ == "CancelledError":
-        return VideoFailure(VideoFailureCategory.CANCELLED, False, False, False)
-
-    code = getattr(error, "code", None) or getattr(error, "status_code", None)
-    try:
-        code = int(code) if code is not None else None
-    except (TypeError, ValueError):
-        code = None
-
+    generic = classify_failure(error)
     message = str(error).lower()
-    if code == 429 or any(marker in message for marker in ("rate limit", "quota", "resource_exhausted")):
-        return VideoFailure(VideoFailureCategory.RATE_LIMIT, True, False, True)
-    if code in {500, 502, 503, 504} or any(
-        marker in message
-        for marker in ("unavailable", "overloaded", "high demand", "service unavailable")
-    ):
-        return VideoFailure(VideoFailureCategory.PROVIDER_UNAVAILABLE, True, False, True)
-    if isinstance(error, TimeoutError) or "timed out" in message or "timeout" in message:
-        return VideoFailure(VideoFailureCategory.TIMEOUT, True, True, True)
-    if isinstance(error, (ConnectionError, OSError)) and not any(
-        marker in message for marker in ("ffmpeg", "video", "audio", "codec", "media")
-    ):
-        return VideoFailure(VideoFailureCategory.CONNECTION, True, False, True)
-    if code in {401, 403} or any(
-        marker in message for marker in ("unauthorized", "forbidden", "api key", "credential")
-    ):
-        return VideoFailure(VideoFailureCategory.AUTHENTICATION, False, False, True)
-    if any(
-        marker in message
-        for marker in ("ffmpeg", "codec", "corrupt", "invalid video", "invalid audio", "media")
-    ):
+    has_media_marker = any(marker in message for marker in _MEDIA_MARKERS)
+
+    # Media markers take precedence over generic connection / bad-request /
+    # unknown classification (rate-limit, unavailable, timeout, and auth
+    # signals still win, matching the original ordering of checks).
+    if has_media_marker and generic.category in {
+        FailureCategory.CONNECTION,
+        FailureCategory.BAD_REQUEST,
+        FailureCategory.UNKNOWN,
+    }:
         return VideoFailure(VideoFailureCategory.MEDIA_PROCESSING, False, True, True)
-    if code in {400, 404, 413, 415, 422}:
-        return VideoFailure(VideoFailureCategory.BAD_REQUEST, False, code == 413, True)
-    return VideoFailure(VideoFailureCategory.UNKNOWN, True, True, True)
+
+    category = VideoFailureCategory(generic.category.value)
+    should_split = category in {VideoFailureCategory.TIMEOUT, VideoFailureCategory.UNKNOWN} or (
+        category is VideoFailureCategory.BAD_REQUEST and extract_status_code(error) == 413
+    )
+    # The video pipeline keeps its historical behavior of falling back to the
+    # universal analyzer even for bad requests (a different provider may accept
+    # the same media payload).
+    should_fallback = generic.should_fallback or category is VideoFailureCategory.BAD_REQUEST
+    if category is VideoFailureCategory.CANCELLED:
+        should_fallback = False
+    return VideoFailure(category, generic.retryable, should_split, should_fallback)
 
 
-@dataclass
-class _CircuitState:
-    consecutive_failures: int = 0
-    open_until: float = 0.0
-    half_open_in_flight: bool = False
-
-
-class VideoCircuitBreaker:
-    """Small context-scoped circuit breaker keyed by provider/model identity."""
-
-    def __init__(self, threshold: int = 3, cooldown_seconds: float = 60.0) -> None:
-        self.threshold = max(1, int(threshold))
-        self.cooldown_seconds = max(0.1, float(cooldown_seconds))
-        self._states: dict[str, _CircuitState] = {}
-        self._lock = threading.RLock()
-
-    def allow(self, key: str, *, now: float | None = None) -> bool:
-        current = time.time() if now is None else now
-        with self._lock:
-            state = self._states.get(key)
-            if state is None:
-                return True
-            if state.open_until:
-                if current < state.open_until or state.half_open_in_flight:
-                    return False
-                # Half-open: permit exactly one trial. Other concurrent calls
-                # continue to bypass the primary until it succeeds or fails.
-                state.half_open_in_flight = True
-                state.consecutive_failures = max(0, self.threshold - 1)
-                return True
-            return not state.half_open_in_flight
-
-    def record_success(self, key: str) -> None:
-        with self._lock:
-            self._states.pop(key, None)
-
-    def record_failure(
-        self,
-        key: str,
-        failure: VideoFailure,
-        *,
-        now: float | None = None,
-    ) -> None:
-        if failure.category not in {
-            VideoFailureCategory.RATE_LIMIT,
-            VideoFailureCategory.PROVIDER_UNAVAILABLE,
-            VideoFailureCategory.TIMEOUT,
-            VideoFailureCategory.CONNECTION,
-        }:
-            with self._lock:
-                state = self._states.get(key)
-                if state is not None and state.half_open_in_flight:
-                    self._states.pop(key, None)
-            return
-        current = time.time() if now is None else now
-        with self._lock:
-            state = self._states.setdefault(key, _CircuitState())
-            state.half_open_in_flight = False
-            state.consecutive_failures += 1
-            if state.consecutive_failures >= self.threshold:
-                state.open_until = current + self.cooldown_seconds
-
-    def snapshot(self) -> dict[str, dict[str, Any]]:
-        with self._lock:
-            return {
-                key: {
-                    "consecutive_failures": state.consecutive_failures,
-                    "open_until": state.open_until,
-                    "half_open_in_flight": state.half_open_in_flight,
-                }
-                for key, state in self._states.items()
-            }
+class VideoCircuitBreaker(CircuitBreaker):
+    """Context-scoped circuit breaker keyed by provider/model identity."""