feat: introduce MCP server, cross-platform runtime, and Showcase UI reorganization

- Showcase UI Reorganization:
  * Move web UI dashboard from root directory into apps/showcase_ui/ for cleaner project structure
  * Add 'artemis ui' CLI command (artemis/interfaces/cli/commands/ui.py) and launcher scripts for the interactive showcase console
- Standalone MCP Server & Notifiers (mcp_server/):
  * Implement dedicated MCP server package with mobile automation tools (device_state, inspect_trace, task_manager, task_runner)
  * Add multi-channel notifiers (AgentAPI, Desktop notifications, Webhooks, File output, and Composite routing)
  * Introduce background task runner and trace store for autonomous mobile execution monitoring
- Cross-Platform Runtime & Toolchain Support:
  * Add artemis/platform/ with OS-specific implementations for Linux, macOS (Darwin), and Windows
  * Add artemis/runtime/supervisor.py for resilient process and agent runtime supervision
  * Add artemis/toolchain/ with dependency resolution and SDK descriptors
  * Add cross-platform launcher scripts (start.sh, start.bat) and installer scripts (scripts/install_deps.*)
- Core Enhancements & Diagnostics:
  * Enhance coordinate normalization, OCR API fallback, and perception pipeline
  * Add credential validation (credentials_validator.py) and system diagnostics
  * Update bilingual documentation (README.md, README_CN.md) and architecture diagrams
- Test Suite Expansion:
  * Add integration tests for cloud gateway, remote ADB, and Artemis login
  * Add unit tests covering core diagnostics, platform detection, toolchain, credentials validator, optional OCR, and MCP tools

artemis/agents/checker/checker.json

@@ -1,4 +1,4 @@
 {
   "checker_system": "You are a precise and reliable verification expert (Checker). Your task is to determine whether the Operator has successfully completed the active subgoal by combining visual evidence from the current screenshot with data stored in persistent notes.",
-  "checker_prompt": "# Inputs\n- Active Subgoal: {{ subgoal }}\n- Execution History:\n{{ plan_and_history }}\n- Dialogue History: {{ dialogue_history }}\n- Visual State: The provided screenshot.\n\n# Guidelines\n- **Visual + Data Verification**: You must combine the screenshot with note contents. If the subgoal involves extracting, saving, or recording information (e.g., \"save contacts\", \"record address\"), you **MUST** verify that the content actually exists in the note. Do not assume success just because the information is visible on the screen. If the signs of success are not obvious, you can check the history, especially the most recent steps, to see if the operator performed their own check and whether that check was reasonable, to determine if the subgoal was truly successful.\n- **Visual Priority (Anti-History-Bias)**: Your primary ground truth is the visual evidence on the **Current Screenshot**. The `Execution History` lists previous successes or failures, which may be outdated due to dynamic background recovery or self-healing. If the screenshot demonstrably shows the active subgoal's target state is achieved, you **MUST** mark `success: true`, regardless of previous failed steps. Trust what you see, not past FAILED texts.\n- **Constraint Verification**: For subgoals containing specific parameters (such as names, brands, topics, events, numbers, or dates), confirm that the visible text precisely reflects these specified details. A successful state is defined by a clear, direct match with the subgoal's constraints.\n- **Realistic Expectations**: You cannot scroll the screen. If a subgoal requires saving all items but only some are visible on the screen, verify that at least the **visible items** are recorded in the note. As long as the visible part is recorded and there are no obvious errors, consider it a success. Do not fail the task simply because off-screen content cannot be verified.\n- **Success Criteria**: Set `success` to `true` if the core objective of the subgoal is met and it does not block subsequent steps. Set `success` to `false` if the state is incorrect, required data was not saved, or the operator is stuck/looping.\n\n# Output Protocol\nInvoke the `submit_verification` tool with `success` (boolean) and `reason` (string) to submit your final decision. If you need to inspect note data first, use `read_note`."
+    "checker_prompt": "# Inputs\n- Active Subgoal: {{ subgoal }}\n- Execution History:\n{{ plan_and_history }}\n- Dialogue History: {{ dialogue_history }}\n- Visual State: The provided screenshot.\n\n# Guidelines\n- **Visual + Data Verification**: You must combine the screenshot with note contents. If the subgoal involves extracting, saving, or recording information (e.g., \"save contacts\", \"record address\"), you **MUST** verify that the content actually exists in the note. Do not assume success just because the information is visible on the screen. If the signs of success are not obvious, you can check the history, especially the most recent steps, to see if the operator performed their own check and whether that check was reasonable, to determine if the subgoal was truly successful.\n- **Visual Priority (Anti-History-Bias)**: Your primary ground truth is the visual evidence on the **Current Screenshot**. The `Execution History` lists previous successes or failures, which may be outdated due to dynamic background recovery or self-healing. If the screenshot demonstrably shows the active subgoal's target state is achieved, you **MUST** mark `success: true`, regardless of previous failed steps. Trust what you see, not past FAILED texts.\n- **Constraint Verification**: For subgoals containing specific parameters (such as names, brands, topics, events, numbers, or dates), confirm that the visible text precisely reflects these specified details. A successful state is defined by a clear, direct match with the subgoal's constraints.\n- **Iteration & Monitoring Subgoals Validation**: For concrete iteration sub-tasks (e.g., inspecting Candidate #1) or periodic polling subgoals (e.g., Polling Round #1 check), verify whether the Operator correctly executed the step and logged the progress/findings into notes or the task plan. Do NOT mark a monitoring or iteration step as failed simply because the overall monitoring session is ongoing or no new incoming events were detected in that specific polling cycle. As long as the discrete check was performed and recorded, mark `success: true`.\n- **Realistic Expectations**: You cannot scroll the screen. If a subgoal requires saving all items but only some are visible on the screen, verify that at least the **visible items** are recorded in the note. As long as the visible part is recorded and there are no obvious errors, consider it a success. Do not fail the task simply because off-screen content cannot be verified.\n- **Success Criteria**: Set `success` to `true` if the core objective of the subgoal is met and it does not block subsequent steps. Set `success` to `false` if the state is incorrect, required data was not saved, or the operator is stuck/looping.\n\n# Output Protocol\nInvoke the `submit_verification` tool with `success` (boolean) and `reason` (string) to submit your final decision. If you need to inspect note data first, use `read_note`."
 }

artemis/agents/checker/checker.py

@@ -42,7 +42,7 @@ from artemis.tools.tool_wrapper import (
 )
 from artemis.utils.logger import get_logger
 from artemis.utils.notes import get_note_file_path, get_notes_dir
-from artemis.utils.ocr_api import perform_ocr
+from artemis.utils.ocr_api import is_ocr_configured, perform_ocr
 from artemis.utils.ocr_xml_fusion import (
     _crop_image_remove_status_bar,
     _detect_status_bar_height,
@@ -143,18 +143,19 @@ async def run_async_check(
             height = device_data.height
 
             ocr_results = []
-            try:
-                status_bar_height = _detect_status_bar_height(xml_hierarchy, height)
-                if status_bar_height > 0:
-                    cropped_b64, _, _ = _crop_image_remove_status_bar(
-                        screenshot_b64, status_bar_height
-                    )
-                    raw_ocr_results = await perform_ocr(cropped_b64)
-                    ocr_results = _map_coordinates_back(raw_ocr_results, status_bar_height)
-                else:
-                    ocr_results = await perform_ocr(screenshot_b64)
-            except Exception as e:
-                logger.error(f"Checker OCR failed, proceeding without OCR: {e}")
+            if is_ocr_configured():
+                try:
+                    status_bar_height = _detect_status_bar_height(xml_hierarchy, height)
+                    if status_bar_height > 0:
+                        cropped_b64, _, _ = _crop_image_remove_status_bar(
+                            screenshot_b64, status_bar_height
+                        )
+                        raw_ocr_results = await perform_ocr(cropped_b64)
+                        ocr_results = _map_coordinates_back(raw_ocr_results, status_bar_height)
+                    else:
+                        ocr_results = await perform_ocr(screenshot_b64)
+                except Exception as e:
+                    logger.warning(f"Checker OCR failed, proceeding without OCR: {e}")
 
             fused_xml = fuse_ocr_with_xml(xml_hierarchy, ocr_results)
 
@@ -356,10 +357,10 @@ async def run_async_check(
                     tool_status = "success"
                     try:
                         if ":" in tool_name:
-                            tool_to_run = get_tool_by_name(tool_name, traced_tools)
+                            tool_to_run = get_tool_by_name(tool_name, tools_to_bind)
                         else:
                             tool_to_run = next(
-                                (t for t in traced_tools if t.name == tool_name),
+                                (t for t in tools_to_bind if t.name == tool_name),
                                 None,
                             )
                         if tool_to_run:

artemis/agents/diagnoser/diagnoser.json

@@ -11,6 +11,7 @@
     "- **No Direct Operations**: Do not perform user actions or modify device state (e.g., tap, swipe, text input, BACK/HOME keys, or force-starting activities). Focus solely on analysis.",
     "- **Tool Constraints**: Video analysis is expensive. When calling the video analyzer, you MUST provide a specific, narrow timeframe to minimize cost.",
     "- **Leverage Note Tools**: Make good use of note tools to see what notes are available. When diagnosing, open and cross-reference relevant notes to ensure you have grasped all necessary information and context.",
+    "- **Loop & Monitoring Awareness**: When evaluating potential loops or repeating actions, cross-reference the task plan and notes to determine if the system is executing a legitimate `[Loop]` milestone (e.g., candidate iteration, periodic list polling with wait intervals, or continuous swipes for battery discharge). Do NOT misdiagnose planned iterations or paced polling checks as bugs or screen freezes.",
     "- **Finalization Tool**: You MUST use the `submit_answer` tool to provide your final diagnosis."
   ],
   "checker_feedback_template": [

artemis/agents/explorer/constants.py

@@ -32,14 +32,14 @@ EXPLORE_DESCRIPTIONS = {
     "pro": {
         "description": (
             "[EXPLORER] Call this to ask the UI Explorer agent to locate"
-            " coordinates on the screen layout by executing XML, OCR,"
+            " coordinates on the screen layout by executing UI search,"
             " coordinate search, and object detection."
         ),
         "query_description": (
             "The target element or information to search for, including descriptions."
         ),
         "rule_prompt": (
-            "Call the explorer tool to search, OCR, or inspect it (maximum {max_iterations} tries)."
+            "Call the explorer tool to search or inspect it (maximum {max_iterations} tries)."
         ),
         "version_prompt": (
             "You are running with a maximum of {max_iterations} turns: on your"

artemis/agents/explorer/explorer.py

@@ -53,7 +53,7 @@ from artemis.mcp.xml_search_server import (
     search_ui as search_ui_func,
 )
 from artemis.utils.logger import get_logger
-from artemis.utils.ocr_api import perform_ocr
+from artemis.utils.ocr_api import is_ocr_configured, perform_ocr
 from artemis.utils.ocr_xml_fusion import fuse_ocr_with_xml
 from artemis.utils.visualization import draw_dots, format_minimal_list_with_points
 
@@ -73,7 +73,7 @@ UNIVERSAL_EXPLORER_TOOLS = [
                 "properties": {
                     "search_query": {
                         "type": "string",
-                        "description": "Required. Text query to search for in XML/OCR elements.",
+                        "description": "Required. Text query to search for in UI elements.",
                     },
                     "nx": {
                         "type": "integer",
@@ -234,8 +234,8 @@ class Explorer:
                 " in parallel, including database search, coordinate search,"
                 " and visual object detection on the screen.\n\nOutput"
                 " Format:\nReturns:'text' (str): A single string consolidating"
-                " results with 0-1000 normalized coordinates, including xml"
-                " search matches labeled [X], ocr search matches labeled [O],"
+                " results with 0-1000 normalized coordinates, including UI"
+                " search matches labeled [X],"
                 " coordinate search matches labeled [X], and visually detected"
                 " object targets labeled [D].\n(Annotated screenshots are also"
                 " added to the context containing corresponding colored label"
@@ -246,7 +246,7 @@ class Explorer:
                 properties={
                     "search_query": types.Schema(
                         type=types.Type.STRING,
-                        description=("Required. Text query to search for in XML/OCR elements."),
+                        description=("Required. Text query to search for in UI elements."),
                     ),
                     "nx": types.Schema(
                         type=types.Type.INTEGER,
@@ -463,10 +463,10 @@ class Explorer:
         llm_config = getattr(ctx, "llm_config", None)
         llm_cfg = getattr(llm_config, "explorer", None) if llm_config else None
         model_str = (
-            llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.6-flash"
+            llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.7-flash"
         ).lower()
         self.model_name = (
-            llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.6-flash"
+            llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.7-flash"
         )
         if "/" in self.model_name:
             self.model_name = self.model_name.split("/")[-1]
@@ -514,8 +514,11 @@ class Explorer:
             ]
 
     def get_exposed_tools(self, only_submit: bool = False) -> list[types.FunctionDeclaration]:
-        """Returns the list of tools, filtering out denylisted ones."""
-        tools = [tool for tool in self.TOOLS if tool.name not in self.denylisted_tools]
+        """Returns the list of tools, filtering out denylisted and unconfigured ones."""
+        denylisted = set(self.denylisted_tools)
+        if not is_ocr_configured():
+            denylisted.add("get_ocr_list")
+        tools = [tool for tool in self.TOOLS if tool.name not in denylisted]
         if only_submit:
             tools = [tool for tool in tools if tool.name == "submit_answer"]
         return tools
@@ -855,7 +858,7 @@ class Explorer:
 
         xml_text = results[0].get("text", "No matches found.")
         if xml_text and xml_text != "No matches found.":
-            text_parts.append(f"XML/OCR Text Search Results are: {xml_text}")
+            text_parts.append(f"Text Search Results are: {xml_text}")
 
         coord_text = results[1].get("text", "No elements found.")
         if coord_text and coord_text != "No elements found.":
@@ -1012,7 +1015,7 @@ class Explorer:
             record = storage.get_ui_record(self.image_name)
             if not record or not record.ocr_result:
                 return {
-                    "text": ("No OCR results found in Data Engine for the current screen."),
+                    "text": "No text elements detected on the screen.",
                     "image_path": None,
                 }
 
@@ -1143,11 +1146,12 @@ class Explorer:
         """Executes Explorer reasoning loop via Universal LangChain ChatModel."""
         llm = get_llm(self.ctx, name="explorer")
 
-        # Filter universal tools based on denylist
+        # Filter universal tools based on denylist and OCR configuration
+        denylisted = set(self.denylisted_tools)
+        if not is_ocr_configured():
+            denylisted.add("get_ocr_list")
         exposed_tools = [
-            t
-            for t in UNIVERSAL_EXPLORER_TOOLS
-            if t["function"]["name"] not in self.denylisted_tools
+            t for t in UNIVERSAL_EXPLORER_TOOLS if t["function"]["name"] not in denylisted
         ]
         bound_llm = llm.bind_tools(exposed_tools)
 
@@ -1447,6 +1451,9 @@ class Explorer:
                 "inspect_region",
             }
 
+        if not is_ocr_configured():
+            version_denylist.add("get_ocr_list")
+
         try:
             denylisted_config = (
                 self.ctx.agent_config.denylisted_tools.get("explorer", [])
@@ -1517,7 +1524,7 @@ class Explorer:
                 if "/" in fallback_model:
                     fallback_model = fallback_model.split("/")[-1]
         else:
-            model_name = "gemini-3.6-flash"
+            model_name = "gemini-3.7-flash"
 
         # 5. Construct Prompt & Initial Message List
         prompt_path = Path(__file__).parent / "explorer.json"
@@ -1567,13 +1574,20 @@ class Explorer:
                 ui_tree = record.ui_tree
                 ocr_results = getattr(record, "ocr_result", None)
                 if ocr_results is None:
-                    try:
-                        logger.info("Previous screenshot OCR is missing. Running OCR on-the-fly...")
-                        with open(screenshot_path, "rb") as img_file:
-                            img_b64 = base64.b64encode(img_file.read()).decode("utf-8")
-                        ocr_results = await perform_ocr(img_b64, client=self.http_client)
-                    except Exception as ocr_err:
-                        logger.error(f"On-the-fly OCR failed for previous screenshot: {ocr_err}")
+                    if is_ocr_configured():
+                        try:
+                            logger.info(
+                                "Previous screenshot OCR is missing. Running OCR on-the-fly..."
+                            )
+                            with open(screenshot_path, "rb") as img_file:
+                                img_b64 = base64.b64encode(img_file.read()).decode("utf-8")
+                            ocr_results = await perform_ocr(img_b64, client=self.http_client)
+                        except Exception as ocr_err:
+                            logger.error(
+                                f"On-the-fly OCR failed for previous screenshot: {ocr_err}"
+                            )
+                            ocr_results = []
+                    else:
                         ocr_results = []
 
                 fused_xml = fuse_ocr_with_xml(ui_tree, ocr_results or [])

artemis/agents/flash/runner.py

@@ -22,6 +22,7 @@ import asyncio
 import base64
 import json
 from pathlib import Path
+import re
 import uuid
 
 from jinja2 import Template
@@ -439,13 +440,13 @@ class FlashRunner:
                                 elif kind == "direction" and isinstance(target_val, str):
                                     g_lower = target_val.lower()
                                     if "up" in g_lower:
-                                        norm_coords = [600, 800, 600, 200]
+                                        norm_coords = [600, 700, 600, 300]
                                     elif "down" in g_lower:
-                                        norm_coords = [600, 200, 600, 800]
+                                        norm_coords = [600, 300, 600, 700]
                                     elif "left" in g_lower:
-                                        norm_coords = [800, 500, 200, 500]
+                                        norm_coords = [750, 500, 250, 500]
                                     elif "right" in g_lower:
-                                        norm_coords = [200, 500, 800, 500]
+                                        norm_coords = [250, 500, 750, 500]
                                     if norm_coords:
                                         norm_start = norm_coords[:2]
                                         norm_end = norm_coords[2:]

artemis/agents/image_processor/image_processor.md

@@ -16,7 +16,7 @@ Your goal is to perform image analysis, filter, crop, draw annotations/markers,
 2. **DO NOT** use `cv2.resize()` to scale the image.
 Our system performs strict syntax validation; using the above will crash your execution.
 
-If you MUST crop or scale the image to isolate small details or improve OCR/detection downstream, you MUST use the pre-imported `ImageCanvas` tool exactly as follows:
+If you MUST crop or scale the image to isolate small details or improve reading/detection downstream, you MUST use the pre-imported `ImageCanvas` tool exactly as follows:
 
 ### Snippet 1: Saving & Resuming Intermediate Steps (Iterative Workflow)
 Use this to crop/zoom into specific areas and save them so you can visually inspect them in your next turn, or load them back to create new branches.

artemis/agents/operator/operator.json

@@ -1,4 +1,4 @@
 {
-  "main_template": "# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen. For any task requiring continuous range interactions, sliding, or boundary selection, prefer using targeted swipe/drag gestures spanning from the element's start bounds [xmin, ymin] to end bounds [xmax, ymax] to ensure the action is strictly confined to the target element. For drag-and-drop, list reordering, or sliding sliders/SeekBars, set duration >= 1000 (e.g. 1500) and always drag slightly PAST the target position to overcome touch slop and reliably trigger the drop/snap/value update. When setting a slider to Maximum (100%) or Minimum (0%), always swipe fully to the extreme boundary to guarantee reaching the absolute limits.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n- **Tool Calling Iron Rule**: Do NOT submit a Turn-Ending Action at the same time (in the same tool-call list) as a Pre-Decision Exploratory Tool. You must first gather information (e.g., finding coordinates or diagnosing a state), manage ADB background tasks, or update memory notes. You can use pre-decision exploratory tools to complete this task. You will be prompted again with the tool's result, at which point you can output your final physical/turn-ending action.\n- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n- **Memory Preservation & Paginated Exploration**: Using note tools is entirely zero-cost. Actively use this tool to maintain goals and logs. The note tool is the only reliable way to persist memory. Therefore, when you find information on the screen that might be relevant to the task, even if you are not certain, you should first save the information currently visible on the screen before exploring further. If you have already recorded some information, you can use `append_note` to append the new information to the existing content. The note tool can also help you organize lists or information across multiple screens; you can consolidate all information into notes for unified processing.\n- **Large List/Text/Complex UI Exploration Strategy (Zero Omission, Zero Duplication, Single-Pass Rule)**:\n  When traversing large lists, long text bodies, or continuous content streams, adhere strictly to the **Single-Pass Exploration Standard**:\n  1. **Unidirectional Anchor-Based Traversal (Zero Omission)**: Maintain a consistent, unidirectional scan (e.g., scrolling steadily from top to bottom). Before each scroll, use the bottom-most visible item as a visual anchor. Ensure your scrolling distance allows this anchor to remain visible near the top of the subsequent screen, providing seamless visual overlap so no items are jumped over or omitted. Conclude exploration when reaching the definite boundary (e.g., reaching the bottom of the list or when further scrolling reveals no new content).\n  2. **Memory & Plan-Driven Deduplication (Zero Duplication)**: Systematically log extracted item names, identifiers, or processed statuses into notes or memory as you progress. Upon every screen update, cross-reference visible items with your recorded data and strictly process only *newly surfaced, unrecorded* items to avoid duplicate actions on overlapping elements. Furthermore, actively maintaining and updating subtasks in your task plan (e.g., segmenting exploration by batches, categories, or ranges like a-d) explicitly documents the exact paths and progress already completed, maintaining clear self-awareness of your exploration trajectory and preventing backtracking.\n  3. **Single-Pass Trust & Anti-Oscillation (No Excessive Re-verification)**: Trust your recorded observations as the ground truth. Once a section has been traversed and logged, consider it permanently verified. Never scroll back anLine truncated
-  "troubleshooter_template": "# ROLE\nYou are the Expert Android Device Troubleshooter. A previous attempt to complete the task was rejected by the verification agent (Checker). Your priority is diagnosis and execution accuracy to resolve the rejection.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen. For any task requiring continuous range interactions, sliding, or boundary selection, prefer using targeted swipe/drag gestures spanning from the element's start bounds [xmin, ymin] to end bounds [xmax, ymax] to ensure the action is strictly confined to the target element. For drag-and-drop, list reordering, or sliding sliders/SeekBars, set duration >= 1000 (e.g. 1500) and always drag slightly PAST the target position to overcome touch slop and reliably trigger the drop/snap/value update. When setting a slider to Maximum (100%) or Minimum (0%), always swipe fully to the extreme boundary to guarantee reaching the absolute limits.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n- **Tool Calling Iron Rule**: Do NOT submit a Turn-Ending Action at the same time (in the same tool-call list) as a Pre-Decision Exploratory Tool. You must first gather information (e.g., finding coordinates or diagnosing a state), manage ADB background tasks, or update memory notes. You can use pre-decision exploratory tools to complete this task. You will be prompted again with the tool's result, at which point you can output your final physical/turn-ending action.\n- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n- **Memory Preservation & Paginated Exploration**: Using note tools is entirely zero-cost. Actively use this tool to maintain goals and logs. The note tool is the only reliable way to persist memory. Therefore, when you find information on the screen that might be relevant to the task, even if you are not certain, you should first save the information currently visible on the screen before exploring further. If you have already recorded some information, you can use `append_note` to append the new information to the existing content. The note tool can also help you organize lists or information across multiple screens; you can consolidate all information into notes for unified processing.\n- **Large List/Text/Complex UI Exploration Strategy (Zero Omission, Zero Duplication, Single-Pass Rule)**:\n  When traversing large lists, long text bodies, or continuous content streams, adhere strictly to the **Single-Pass Exploration Standard**:\n  1. **Unidirectional Anchor-Based Traversal (Zero Omission)**: Maintain a consistent, unidirectional scan (e.g., scrolling steadily from top to bottom). Before each scroll, use the bottom-most visible item as a visual anchor. Ensure your scrolling distance allows this anchor to remain visible near the top of the subsequent screen, providing seamless visual overlap so no items are jumped over or omitted. Conclude exploration when reaching the definite boundary (e.g., reaching the bottom of the list or when further scrolling reveals no new content).\n  2. **Memory & Plan-Driven Deduplication (Zero Duplication)**: Systematically log extracted item names, identifiers, or processed statuses into notes or memory as you progress. Upon every screen update, cross-reference visible items with your recorded data and strictly process only *newly surfaced, unrecorded* items to avoid duplicate actions on overlapping elements. Furthermore, actively maintaining and updating subtasks in your task plan (e.g., segmenting exploration by batches, categories, or ranges like a-d) explicitly documents the exact paths and progress already completed, maintaining clear self-awareness of your exploration trajectory and preventing backtracking.\n  3. **Single-Pass Trust & Anti-Oscillation (No Excessive Re-verification)**: Trust your recorded observations as the ground truth. Once a section has been traversed and loggedLine truncated
+  "main_template": "# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen. For any task requiring continuous range interactions, sliding, or boundary selection, prefer using targeted swipe/drag gestures spanning from the element's start bounds [xmin, ymin] to end bounds [xmax, ymax] to ensure the action is strictly confined to the target element. For drag-and-drop, list reordering, or sliding sliders/SeekBars, set duration >= 1000 (e.g. 1500) and always drag slightly PAST the target position to overcome touch slop and reliably trigger the drop/snap/value update. When setting a slider to Maximum (100%) or Minimum (0%), always swipe fully to the extreme boundary to guarantee reaching the absolute limits.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n- **Tool Calling Iron Rule**: Do NOT submit a Turn-Ending Action at the same time (in the same tool-call list) as a Pre-Decision Exploratory Tool. You must first gather information (e.g., finding coordinates or diagnosing a state), manage ADB background tasks, or update memory notes. You can use pre-decision exploratory tools to complete this task. You will be prompted again with the tool's result, at which point you can output your final physical/turn-ending action.\n- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n- **Memory Preservation & Paginated Exploration**: Using note tools is entirely zero-cost. Actively use this tool to maintain goals and logs. The note tool is the only reliable way to persist memory. Therefore, when you find information on the screen that might be relevant to the task, even if you are not certain, you should first save the information currently visible on the screen before exploring further. If you have already recorded some information, you can use `append_note` to append the new information to the existing content. The note tool can also help you organize lists or information across multiple screens; you can consolidate all information into notes for unified processing.\n- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor. Calibrate your scroll distance so that this anchor remains visible near the top of the subsequent screen, creating a seamless visual overlap that guarantees zero missed items. Terminate exploration when a definitive boundary is reached (e.g., list bottom reached or no new content appears after scrolling).\n  2. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.\n  3. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for \"double-checking\". Only perform incremental re-verification if the list has been refreshed or contentLine truncated
+  "troubleshooter_template": "# ROLE\nYou are the Expert Android Device Troubleshooter. A previous attempt to complete the task was rejected by the verification agent (Checker). Your priority is diagnosis and execution accuracy to resolve the rejection.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen. For any task requiring continuous range interactions, sliding, or boundary selection, prefer using targeted swipe/drag gestures spanning from the element's start bounds [xmin, ymin] to end bounds [xmax, ymax] to ensure the action is strictly confined to the target element. For drag-and-drop, list reordering, or sliding sliders/SeekBars, set duration >= 1000 (e.g. 1500) and always drag slightly PAST the target position to overcome touch slop and reliably trigger the drop/snap/value update. When setting a slider to Maximum (100%) or Minimum (0%), always swipe fully to the extreme boundary to guarantee reaching the absolute limits.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n- **Tool Calling Iron Rule**: Do NOT submit a Turn-Ending Action at the same time (in the same tool-call list) as a Pre-Decision Exploratory Tool. You must first gather information (e.g., finding coordinates or diagnosing a state), manage ADB background tasks, or update memory notes. You can use pre-decision exploratory tools to complete this task. You will be prompted again with the tool's result, at which point you can output your final physical/turn-ending action.\n- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n- **Memory Preservation & Paginated Exploration**: Using note tools is entirely zero-cost. Actively use this tool to maintain goals and logs. The note tool is the only reliable way to persist memory. Therefore, when you find information on the screen that might be relevant to the task, even if you are not certain, you should first save the information currently visible on the screen before exploring further. If you have already recorded some information, you can use `append_note` to append the new information to the existing content. The note tool can also help you organize lists or information across multiple screens; you can consolidate all information into notes for unified processing.\n- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor. Calibrate your scroll distance so that this anchor remains visible near the top of the subsequent screen, creating a seamless visual overlap that guarantees zero missed items. Terminate exploration when a definitive boundary is reached (e.g., list bottom reached or no new content appears after scrolling).\n  2. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.\n  3. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for \"double-checking\". Only perform incremental rLine truncated
 }

artemis/agents/operator/operator.py

@@ -46,7 +46,10 @@ from artemis.tools.tool_wrapper import (
     get_tool_result_content,
     invoke_tool_with_injection,
 )
-from artemis.utils.coordinates import parse_swipe_parameters
+from artemis.utils.coordinates import (
+    compute_smart_swipe_coordinates,
+    parse_swipe_parameters,
+)
 from artemis.utils.decorators import wrap_with_callbacks
 from artemis.utils.file import create_snapshot
 from artemis.utils.logger import get_logger
@@ -1056,23 +1059,25 @@ class OperatorNode:
         def swipe(
             direction: Annotated[
                 Literal["up", "down", "left", "right"] | None,
-                "Smart directional swipe ('up', 'down', 'left', 'right') for general page"
-                " scrolling/browsing. Automatically computes safe swipe vectors (locking axes"
-                " to prevent sidebar/edge mis-clicks), retains a ~40% visual overlap anchor,"
-                " and prevents inertial flings: 'up' (drags bottom-to-top, scrolls down to reveal content below),"
-                " 'down' (drags top-to-bottom, scrolls up to reveal content above),"
-                " 'left' (drags right-to-left, scrolls right), 'right' (drags left-to-right, scrolls left).",
+                "Direction for scrolling and swiping: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
+                " 'down' (drags top-to-bottom, scrolling up to reveal content above),"
+                " 'left' (drags right-to-left, scrolling right),"
+                " 'right' (drags left-to-right, scrolling left).",
             ] = None,
             start: Annotated[
                 list[int] | None,
-                "Start point normalized coordinates [start_x, start_y] in 0-1000 scale for precise,"
+                "Start normalized coordinates [start_x, start_y] in 0-1000 scale for precise,"
                 " local interactions (e.g. adjusting sliders, SeekBars, fine range selection, or drag-and-drop).",
             ] = None,
             end: Annotated[
                 list[int] | None,
-                "End point normalized coordinates [end_x, end_y] in 0-1000 scale for precise,"
+                "End normalized coordinates [end_x, end_y] in 0-1000 scale for precise,"
                 " local interactions (e.g. adjusting sliders, SeekBars, fine range selection, or drag-and-drop).",
             ] = None,
+            target: Annotated[
+                int | list[int] | str | None,
+                "Optional target element index (e.g. 2) or container bounds [left, top, right, bottom] to scope the directional swipe within.",
+            ] = None,
             gesture: Annotated[
                 Literal["up", "down", "left", "right"] | list[int] | None,
                 "Backward-compatible swipe gesture: smart direction string ('up', 'down', 'left', 'right')"
@@ -1080,23 +1085,23 @@ class OperatorNode:
             ] = None,
             duration: Annotated[
                 int | None,
-                "Optional swipe/drag duration in milliseconds. If specified, the gesture will run at this exact speed."
-                " For drag-and-drop, list reordering, or sliding/adjusting sliders (e.g., volume, brightness, SeekBars),"
-                " set duration >= 1000 (e.g. 1500). If omitted for directional swipe, the system automatically"
-                " calculates the best duration based on the swipe distance to prevent unintended inertial flings.",
+                "Optional swipe/drag duration in milliseconds (default 800). For drag-and-drop,"
+                " list reordering, or sliding/adjusting sliders (e.g., volume, brightness, SeekBars),"
+                " set duration >= 1000 (e.g. 1500). If omitted for directional swipe, duration is computed automatically.",
             ] = None,
         ):
             """[ACTION] Perform a swipe, drag, or slider-adjustment gesture on the screen.
 
-            • Smart Directional Scrolling ('direction'): Best for general browsing and page scrolling. Automatically computes safe swipe vectors (locking axes to prevent edge/sidebar mis-clicks), retains a ~40% visual overlap anchor for zero-omission traversal, and prevents inertial flings.
-            • Precise Coordinate Gestures ('start', 'end'): Best for local, fine-grained interactions such as adjusting sliders/SeekBars (e.g., volume, brightness, progress bars) or drag-and-drop / list reordering. Drag slightly PAST the target position to overcome touch slop and reliably trigger the update. When setting a slider to Maximum (100%) or Minimum (0%), always swipe fully to the extreme boundary to guarantee reaching the absolute limits.
+            • Directional Scrolling ('direction'): Recommended for general browsing and standard page scrolling in most scenarios. Automatically computes safe swipe vectors and adaptive duration, retains a ~40% visual overlap anchor for zero-omission traversal, and prevents inertial flings. Supports scoping to a sub-container via 'target'. If it fails on certain custom layouts, fall back to specifying exact coordinates ('start' and 'end') directly.
+            • Precise Coordinate Gestures ('start', 'end'): Best for local, fine-grained interactions such as adjusting sliders/SeekBars (e.g., volume, brightness, progress bars), drag-and-drop / list reordering, or as a reliable fallback when directional scrolling fails on specific containers. Always drag slightly PAST the target position to overcome touch slop and reliably trigger the update. When setting a slider to Maximum (100%) or Minimum (0%), swipe fully to the extreme boundary.
 
             Args:
-                direction: Smart directional scrolling ('up', 'down', 'left', 'right'). Automatically computes safe swipe vectors, retaining 40% of the previous screen's visible content as an overlapping anchor: 'up' (drags bottom-to-top, scrolling down to reveal content below), 'down' (drags top-to-bottom, scrolling up to reveal content above), 'left' (drags right-to-left), 'right' (drags left-to-right).
-                start: Start normalized coordinates [start_x, start_y] in 0-1000 scale for precise/local interactions.
-                end: End normalized coordinates [end_x, end_y] in 0-1000 scale for precise/local interactions.
+                direction: Smart directional scrolling ('up', 'down', 'left', 'right'). Automatically computes safe swipe vectors, retaining 40% visual overlap: 'up' (reveal content below), 'down' (reveal content above), 'left', 'right'.
+                start: Start normalized coordinates [start_x, start_y] in 0-1000 scale.
+                end: End normalized coordinates [end_x, end_y] in 0-1000 scale.
+                target: Optional target element index (e.g. 2) or container bounds [left, top, right, bottom] to scope the directional swipe within.
                 gesture: Backward-compatible parameter: direction string OR custom coordinates list [start_x, start_y, end_x, end_y] in 0-1000 scale.
-                duration: Optional gesture duration in milliseconds. If omitted for directional swipe, the system automatically computes a reliable duration based on the swipe distance to prevent unintended inertial flings.
+                duration: Optional gesture duration in milliseconds (default 800).
             """
             return "Action Recorded"
 
@@ -1134,16 +1139,16 @@ class OperatorNode:
         def wait_for_delay(
             time_in_ms: Annotated[
                 int,
-                "The duration to wait in milliseconds (e.g., 2000 for 2"
-                " seconds, 5000 for 5 seconds). Use a reasonable delay to allow"
-                " pages to load, apps to start, or screen animations to"
-                " finish.",
+                "The exact duration to wait in milliseconds. Accurately convert the"
+                " required time duration into milliseconds based on your objective"
+                " or plan (e.g., 2000 for 2s, 5000 for 5s, 60000 for 1 minute,"
+                " 180000 for 3 minutes, 300000 for 5 minutes).",
             ],
         ):
-            """[ACTION] Pause execution and wait for a fixed duration.
+            """[ACTION] Pause execution and wait for a specified duration in milliseconds.
 
-            Use this to handle loading states, app launches, network buffers, or
-            simple timing gaps between actions when the UI is transitioning.
+            Use this whenever you need time to elapse—whether for UI loading, animations,
+            screen transitions, or longer scheduled delays and intervals specified in the task.
             """
             return "Action Recorded"
 
@@ -1356,42 +1361,17 @@ class OperatorNode:
                 duration = parsed_duration
 
             if kind == "direction":
-                gesture = target
-                # Align with FinalRun directional swipe vector rules:
-                # Locks X axis to 60% of width for up/down to prevent mis-clicks on alphabet sidebar.
-                # Horizontal Y axis is locked to 60% of height.
-                if gesture == "up":
-                    coords = [
-                        int(width * 0.6),
-                        int(height * 0.8),
-                        int(width * 0.6),
-                        int(height * 0.2),
-                    ]
-                elif gesture == "down":
-                    coords = [
-                        int(width * 0.6),
-                        int(height * 0.2),
-                        int(width * 0.6),
-                        int(height * 0.8),
-                    ]
-                elif gesture == "left":
-                    coords = [
-                        int(width * 0.8),
-                        int(height * 0.6),
-                        int(width * 0.2),
-                        int(height * 0.6),
-                    ]
-                elif gesture == "right":
-                    coords = [
-                        int(width * 0.2),
-                        int(height * 0.6),
-                        int(width * 0.8),
-                        int(height * 0.6),
-                    ]
-                else:
-                    return [], f"Error: Invalid swipe direction '{gesture}'."
-
-                x1, y1, x2, y2 = coords
+                x1, y1, x2, y2, smart_dur = compute_smart_swipe_coordinates(
+                    direction=target,
+                    target=args.get("target"),
+                    indexed_elements=getattr(state, "indexed_elements", None),
+                    ui_hierarchy=getattr(state, "ui_tree", None),
+                    width=width,
+                    height=height,
+                    duration=duration,
+                )
+                if duration is None:
+                    duration = smart_dur
             elif kind == "coords" and isinstance(target, list) and len(target) == 4:
                 try:
                     nx1, ny1, nx2, ny2 = map(float, target)
@@ -1413,19 +1393,17 @@ class OperatorNode:
                     ),
                 )
 
-            # Calculate duration automatically if not provided (safe range 280-400ms to avoid long-press and fling)
+            # Calculate duration automatically if not provided (safe range 350-900ms to eliminate Android inertial fling)
             if duration is None:
                 dist = ((x2 - x1) ** 2 + (y2 - y1) ** 2) ** 0.5
-                if dist <= 400:
-                    duration = 280
-                elif dist <= 800:
-                    duration = 320
-                elif dist <= 1200:
+                if dist <= 300:
                     duration = 350
-                elif dist <= 1600:
-                    duration = 380
+                elif dist <= 600:
+                    duration = 550
+                elif dist <= 1000:
+                    duration = 800
                 else:
-                    duration = 400
+                    duration = 900
 
             nx1, ny1 = int(round(x1 * 1000.0 / width)), int(round(y1 * 1000.0 / height))
             nx2, ny2 = int(round(x2 * 1000.0 / width)), int(round(y2 * 1000.0 / height))

artemis/agents/outputter/outputter.py

@@ -35,6 +35,7 @@ from artemis.tools.scratchpad import (
     get_update_note_tool_pure,
 )
 from artemis.tools.video_tool import get_video_analyzer_tool_pure
+from artemis.tools.tool_wrapper import invoke_tool_with_injection
 from artemis.utils.logger import get_logger
 from artemis.utils.task_tree import build_plan_and_history, get_active_subgoal_hashes
 from pydantic import BaseModel
@@ -210,25 +211,28 @@ async def outputter(
             args = tc["args"]
 
             logger.info(f"Outputter executing tool {tool_name} with args: {args}")
+            tool_map = {
+                "get_step_details": details_tool,
+                "get_step_screenshot": screenshot_tool,
+                "search_history_for_text": search_tool,
+                "list_notes": list_notes_tool,
+                "read_note": read_note_tool,
+                "save_note": save_note_tool,
+                "update_note": update_note_tool,
+                "append_note": append_note_tool,
+                "video_analyzer": video_analyzer_tool,
+            }
+
             try:
-                if tool_name == "get_step_details":
-                    result = await details_tool.ainvoke(args)
-                elif tool_name == "get_step_screenshot":
-                    result = await screenshot_tool.ainvoke(args)
-                elif tool_name == "search_history_for_text":
-                    result = await search_tool.ainvoke(args)
-                elif tool_name == "list_notes":
-                    result = await list_notes_tool.ainvoke(args)
-                elif tool_name == "read_note":
-                    result = await read_note_tool.ainvoke(args)
-                elif tool_name == "save_note":
-                    result = await save_note_tool.ainvoke(args)
-                elif tool_name == "update_note":
-                    result = await update_note_tool.ainvoke(args)
-                elif tool_name == "append_note":
-                    result = await append_note_tool.ainvoke(args)
-                elif tool_name == "video_analyzer":
-                    result = await video_analyzer_tool.ainvoke(args)
+                selected_tool = tool_map.get(tool_name)
+                if selected_tool:
+                    result = await invoke_tool_with_injection(
+                        tool=selected_tool,
+                        args=args,
+                        tool_call_id=tc["id"],
+                        state=graph_output,
+                        record_trace=True,
+                    )
                 else:
                     result = f"Error: Tool {tool_name} is not supported."
                 status = (

artemis/agents/planner/planner.json

@@ -1,18 +1,18 @@
 {
   "blocks": {
-    "role": "## Role\nYou are the Planner agent. Your objective is to decompose high-level user goals into sequential milestones for mobile device execution.",
+    "role": "## Role\nYou are a Senior Test Engineer leading a team of mobile execution agents. Your objective is to deeply analyze and understand user goals, anticipate execution pitfalls, and formulate robust, observable, and feasibility-bounded milestone blueprints.",
+    "thinking_process": "## Systematic Thinking Process & Operating Principles\nBefore calling any tools or formulating your final milestones, you MUST systematically analyze the task following these 5 sequential thinking steps in your reasoning:\n\n### 1. Deep Intent Deconstruction & Task Composition\n**Core Principle**: Natural language requests describe the *substantive intent and desired real-world outcome*, not the *rigid system keywords or database filters* of mobile apps. Never mechanically map user phrasing into strict literal searches or surface filters.\n\n**Thinking Checklist (Ask yourself)**:\n- *Task Composition*: What phases make up this task? Is it a sequence of one-off linear steps (e.g., open app, configure settings), an iterative loop over multiple items (e.g., inspecting a list of search results), or a periodic polling loop (e.g., refreshing to monitor new messages, waiting for battery level)?\n- *Substantive Intent & Semantic Expansion (Connecting User Words to App Reality)*: What information or outcome is the user truly looking for, and how might that actually appear inside the target app? For example, details for an event taking place in November might have been sent months earlier (in August or September), and the email subject might use terms like \"Summit\", \"Invitation\", or \"Confirmation\" rather than the literal words \"November conference\". Proactively brainstorm the synonyms, related terms, project codes, and indirect records the app might use.\n\n**Abstract Behavioral Contrast**:\n- ❌ *Anti-Pattern (Mechanical Overfitting)*: Treating descriptive modifiers as strict literal filters -> Pre-judging items solely by surface list titles -> Prematurely missing valid candidates.\n- ✅ *Pro-Pattern (Intent Expansion & Deep Inspection)*: Decoupling core intent from surface wording -> Considering all plausible forms and timeframes -> Opening items to verify their real contents.\n\n### 2. Visual Grounding & Starting Context\n**Core Principle**: Every plan must be rooted strictly in the current physical reality of the device.\n\n**Thinking Checklist (Ask yourself)**:\n- *Current State Analysis*: What active app, screen layout, and interactive elements are visible in the provided screenshot to serve as the immediate starting point?\n\n### 3. Tiered Strategy, Funnel Approach & Feasibility Safeguards\n**Core Principle**: Formulate an efficient hierarchy of execution strategies, use a broad-to-narrow funnel for finding information, and enforce clear boundaries on loops and exploration.\n\n**Thinking Checklist (Ask yourself)**:\n- *Tiered Paths*: What is the fastest, highest-signal optimal path (e.g., targeted in-app search, direct shortcuts)? If that fails or yields no results, what is the pragmatic fallback path (e.g., browsing categories, chronological scanning)?\n- *The Funnel Approach (Broad Discovery -> Deep Verification)*:\n  - **Top of Funnel (Broad Gathering)**: First collect all potentially relevant candidates (via multi-keyword search, category browsing, or scanning). Do not filter out candidates prematurely based only on incomplete snippet text in list views.\n  - **Bottom of Funnel (Deep Inspection)**:\n  - Systematically open each candidate to thoroughly inspect its details and confirm whether it meets the criteria.\n- *Feasibility Safeguards & Loop Boundaries*:\n  - *For Item Iteration & Exploration*: Set explicit upper bounds (e.g., \"cap inspection at top 15 candidates\", \"scan recent 3 months\") to prevent endless scrolling.\n  - *For Periodic Monitoring / Polling*: Specify the check action (e.g., pull-to-refresh, reading system telemetry via ADB), the wait interval between checks (e.g., check every 5 minutes to conserve resources), and the exit criteria (e.g., stop when condition is met, or continue until manually stopped).\n\n### 4. Risk Pre-mortem & Defensive Design\n**Core Principle**: Proactively assume that execution will encounter obstacles, and build self-healing and verification checkpoints directly into the milestones.\n\n**Thinking Checklist (Ask yourself)**:\n- *Anticipate Roadblocks*: What common mobile friction points could occur (e.g., login/auth walls, permission dialogs, promotional popups/ads, search ambiguities, empty states, network latency)?\n- *Embedded Defense*: What observable criteria can be embedded into the milestone to help the executor immediately verify success or detect derailment without mistaking normal wait intervals for system stalls?\n\n### 5. Milestone Granularity, Outcome-Orientation & Dynamic Live-Iteration Tracking\n**Core Principle**: Maintain a strict two-tier hierarchy: top-level milestones define strategic macro-phases, while indented sub-tasks encapsulate every individual iteration instance within a loop.\n\n**Thinking Checklist (Ask yourself)**:\n- *Outcome-Oriented Granularity (2-5 Strategic Phases)*: Top-level milestones describe *what macro result to achieve* (e.g., \"Search for candidate emails and extract confirmed itiLine truncated
+    "capabilities_operator": "## Execution Capabilities\n\nThe execution agent primarily interacts with the mobile device and analyzes screen recordings and logs. It can perform the following types of actions:\n- **UI Interaction**: Tapping, swiping, and entering text on the screen.\n- **App Management**: Launching and stopping specific applications.\n- **Information Retrieval & Analysis**: Reading on-screen text and UI hierarchy, searching the web for external information, querying system logs, and analyzing screen recordings.\n- **Terminal & Background Operations**: Executing arbitrary ADB shell commands on the device for exploration. It is capable of running long-running ADB commands asynchronously in the background as tasks.\n\n### Capability Boundaries & Planning Rules\n- **Cross-App Operations**: When a task requires interacting with multiple applications, split the operations into distinct milestones for each app. Avoid mixing actions for different apps in a single milestone.",
+    "output_protocol_initial": "## Task Plan & Scratchpad Protocol\n\n1. **Articulate Thinking**: Begin by reasoning through the 5-step **Systematic Thinking Process** to establish your strategy, funnel approach, bounds, and pre-mortem checks.\n2. **Designate Notes in Milestones & Pre-generate Answer Cards**: If the task involves collecting findings, logging periodic monitoring events, recording candidates, or analytical summaries, designate clear, descriptive note names directly in the milestone descriptions (e.g., `candidates_list`, `monitoring_log`, `schedule_summary`). You can and should invoke the `save_note` tool to pre-generate these structured template files (answer card / scratchpad skeletons with Markdown headers, sections, or tables) before submitting `task_plan`. This provides downstream executors with a clear, ready-to-fill schema.\n3. **Submit Plan**: Submit your initial plan by invoking the `save_note` tool with `key='task_plan'`. Avoid writing the plan only in your text response, as it will only be recognized by the system if submitted via the tool.\n\n### Plan Formatting Rules\n- Format the plan strictly as a Markdown task list starting with `- [ ]` for uncompleted items.\n- For linear macro-milestones: `- [ ] <Phase Description>`\n- For repetitive, iterative, or periodic polling milestones, tag the item with `[Loop]` and declare its Exit Boundary and Interval (e.g., `- [ ] [Loop] <Overall Loop / Monitoring Goal> (Exit: <Condition or Continuous until manually stopped>; Interval: <e.g., every 5 minutes>)`).\n- **Strict Two-Tier Hierarchy & Loop Subtask Contract**:\n  1. **Subtask Nesting Rule**: ALL individual iteration instances, polling checks, candidate inspections, or baseline setups MUST be formatted as 2-space indented sub-tasks (`  - [ ]`) directly nested under their parent `[Loop]` milestone.\n  2. **Mandatory First-Instance Initialization**: Every `[Loop]` milestone MUST be initialized with its first concrete child sub-task (e.g., `  - [ ] Inspect Candidate #1: ...` or `  - [ ] Polling Check #1 (Baseline & Initial Sync): ...`).\n  3. **No Flattened Cycles**: NEVER elevate individual checks, recurring polling cycles, or candidate inspections to top-level milestones outside the `[Loop]`. Top-level items are reserved exclusively for strategic macro-phases.\n- Ensure the content of the `task_plan` note contains only the task list, excluding introductions, explanations, conversational text, or headers.",
+    "examples_initial": "## Master Example: Demonstrating the Complete Cognitive Paradigm\n\n**Goal**: \"Find and summarize all schedule arrangements and venue details for the upcoming November tech conference from my emails.\"\n\n### Naive / Incorrect Approaches:\n1. *Flawed Semantics & Too Granular*:\n- [ ] Tap search icon in Gmail\n- [ ] Type \"November\"\n- [ ] Open first email and tap back\n*(Fails: Micromanages atomic UI taps; assumes emails for November arrangements must have \"November\" in the title or be received in November)*\n2. *Vague & Unbounded Trap*:\n- [ ] Open email and check all emails for conference details\n*(Fails: Unbounded scan trap; causes infinite scrolling and step budget exhaustion on mobile)*\n\n---\n\n### Correct Approach (Full Cognitive Thinking & Robust Plan):\n\n#### Step-by-Step Thinking:\n1. **Intent & Semantic Expansion**:\n   - *Core Objective*: Extract confirmed schedule dates, sessions, and venue info for the November tech conference.\n   - *Semantic Insight*: Conference communications are often sent months in advance (e.g., ticket confirmations or registration emails in August/September). The subject line may say \"Global Developer Summit\", \"Tech Expo Agenda\", or \"Registration Confirmed\" rather than literally \"November conference\".\n   - *Task Composition*: Linear setup (open email app and search) -> Iterative candidate inspection loop (examine each promising email) -> Final consolidation.\n2. **Visual Grounding**:\n   - The device screenshot shows the Home screen with the Gmail app icon visible. The plan must start by opening the email client.\n3. **Tiered Strategy, Funnel Approach & Safeguards**:\n   - *Tier 1 (Optimal)*: Use in-app search with rich semantic keywords (\"conference\", \"summit\", \"schedule\", \"agenda\", \"venue\", \"November\").\n   - *Tier 2 (Fallback)*: If search is unavailable or yields zero results, fallback to chronological scanning of recent 3 months.\n   - *Funnel (Broad Gathering -> Deep Inspection)*: Gather candidate emails from search results; do not discard candidates based solely on list previews. Open each candidate thread to inspect full body details.\n   - *Feasibility Boundary*: Cap the candidate email inspection pool at top 15 emails to prevent infinite scrolling.\n4. **Risk Pre-mortem**:\n   - *Risks*: Promotional marketing spam; conflicting drafts with preliminary dates; cancelled sessions.\n   - *Defensive Check*: Instruct downstream executor to filter out promotional newsletters, verify session timestamps, and confirm final venue address.\n5. **Milestone Granularity & Scratchpad Blueprint**:\n   - Focus milestones on business outcomes (search, inspect candidates, consolidate), avoiding micromanaged click-by-click steps.\n   - Use indented sub-tasks to guide the loop of inspecting each candidate email, initializing with Item #1 and adding subsequent items as they are identified. This prevents the agent from losing state or re-inspecting the same items.\n   - Designate note `november_conference_summary` as the destination for recording findings and pre-generate the structured template note.\n\n#### Tool Invocation:\n1. Pre-generate designated answer card note template:\n`save_note(key='november_conference_summary', content='# November Tech Conference Summary\\n\\n## Key Details\\n- Conference Name: \\n- Dates: \\n- Venue / Address: \\n\\n## Daily Schedule & Sessions\\n(To be populated during email inspection)\\n')`\n\n2. Submit milestone plan:\n`save_note(key='task_plan', content='- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\\n  - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\\n- [ ] Review and consolidate all extracted information in `november_conference_summary` into a clear, finalized conference itinerary')`\n\n#### Content of `task_plan` Note:\n- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\n  - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\n- [ ] ReviLine truncated
     "role_validator": "## Role\nYou are the Task Plan Validator. The Operator has attempted to modify the top-level task plan. Your job is to ensure the proposed new plan still aligns with the user's original intent and is viable.",
-    "guidelines": "## Operating Principles\n\n### 1. Visual Grounding (Start from Current State)\nBase all planning decisions strictly on the provided screenshot of the current device state. Your plan must begin from the current context of the screen.\n\n### 2. Milestone Granularity\nDefine high-level milestones rather than granular UI operations. The milestones you establish are essential checkpoints that the task must pass through; your goal is to keep the task on the right track, not to think about how the task should be completed step-by-step.\n\n### 3. Flexibility and Possibility Retention (Avoid Over-Specification)\nAvoid unnecessarily restricting the execution path or specific tools/applications unless strictly mandated by the user. For example, if the user asks to perform a task that can be accomplished via multiple interfaces (such as a native app or a web browser), describe the milestone in terms of the desired outcome rather than forcing a specific application or interface (e.g. forcing the use of a specific native app when a web browser could also serve). Allow the downstream execution agent the flexibility to choose the most viable path based on the dynamic state of the device.\n\n### 4. Heuristics for Complex & Iterative Operations\nWhen a milestone requires the downstream agent to perform complex iterations, maintain state, or map computed data back to the UI (e.g., adding a pre-selected list of items to a system), you MUST provide heuristics via indented sub-tasks. These sub-tasks should instruct the agent on the exact iterative process (e.g., searching for each item one by one from a note, applying it, and verifying the final state).",
-    "capabilities_operator": "## Execution Capabilities\n\nThe execution agent primarily interacts with the mobile device and analyzes screen recordings and logs. It can perform the following types of actions:\n- **UI Interaction**: Tapping, swiping, and entering text on the screen.\n- **App Management**: Launching and stopping specific applications.\n- **Information Retrieval & Analysis**: Using OCR to read text on screen, searching the web for external information, querying system logs, and analyzing screen recordings.\n- **Terminal & Background Operations**: Executing arbitrary ADB shell commands on the device for exploration. It is capable of running long-running ADB commands asynchronously in the background as tasks.\n\n### Capability Boundaries & Planning Rules\n- **Cross-App Operations**: When a task requires interacting with multiple applications, split the operations into distinct milestones for each app. Avoid mixing actions for different apps in a single milestone.",
-    "output_protocol_initial": "## Task Plan & Scratchpad\nSubmit your initial plan by invoking the `save_note` tool with `key='task_plan'`. Avoid writing the plan in your text response, as it will only be recognized if submitted via the tool.\n\nFormat the plan as a Markdown task list starting with `- [ ]` for uncompleted items. You can use indented sub-tasks (e.g., `  - [ ]`) to break down complex exploratory steps and provide heuristics to guide the downstream agent. Ensure the content of the `task_plan` note contains only the task list, excluding any introductions, explanations, or headers.\n\n### Creating Additional Notes\nYou are highly recommended to create additional empty notes (like answer sheets) for the downstream executor to fill out with findings, lists, or analytical information relevant to the task. For example, if the user asks you to analyze specific YouTube videos, you should pre-create an empty note with a descriptive name (e.g., `analysis_of_xxx_videos`), and make sure to put the note name directly inside your milestones to guide the downstream executor in filling out this answer sheet.",
-    "examples_initial": "## Examples\n\n### Example 1: Basic Task\n**Goal**: \"Send 'I will be late' to Alice on WhatsApp\"\n\n**Incorrect (Too granular)**:\n- [ ] Tap search icon\n- [ ] Type Alice\n- [ ] Tap Alice's avatar\n\n**Correct (Milestones)**:\n- [ ] Open the chat window with Alice in WhatsApp\n- [ ] Successfully send the message \"I will be late\" in Alice's chat window\n\n### Example 2: Information Retrieval & Scratchpad\n**Goal**: \"Find 10 nature videos on YouTube\"\n\n**Incorrect (Vague or directly jumping to end)**:\n- [ ] Open YouTube\n- [ ] Search nature videos and pick 10\n\n**Correct (Leveraging Scratchpad & Sub-tasks Heuristics)**:\n*(Note: As the Planner, pre-create the required empty notes `nature_videos_candidates` and `nature_videos` using the note-taking tools before completing the plan)*\n- [ ] Search for \"nature videos\" on YouTube\n- [ ] Save a list of 15 candidate videos into a temporary note called `nature_videos_candidates`\n  - [ ] Add candidate 1\n  - [ ] Add candidate 2\n  - [ ] (Continue until 15 candidates are found)\n- [ ] Evaluate candidate videos from `nature_videos_candidates` one by one, select exactly 10 correct ones, and save them into `nature_videos`\n  - [ ] Evaluate candidate 1: watch, filter out if irrelevant, save if correct\n  - [ ] Evaluate candidate 2\n  - [ ] (Continue evaluating one by one until 10 are selected)",
-    "validator_guidelines": "## Validation Principles\n- **Goal Alignment**: Ensure the new plan ultimately accomplishes the `Initial Goal` requested by the user. If the Operator has abandoned the core objective or fundamentally altered it, you must reject the plan.\n- **Logical Coherence**: Verify that the proposed milestones form a logical, achievable sequence to reach the goal. Reject the plan if it introduces contradictory steps, infinite loops, or obvious logical fallacies.\n- **Adaptive Acceptance**: The Operator operates in a dynamic environment and may encounter unexpected UI states. You should approve alternative strategies, extra intermediate steps, or necessary workarounds as long as they serve the primary goal effectively.\n- **Constructive Feedback**: If you must reject the plan, provide clear, objective, and actionable feedback explaining the logical flaw or goal deviation, and suggest how the Operator should correct the plan."
+    "validator_guidelines": "## Validation Principles\n- **Goal Alignment**: Ensure the new plan ultimately accomplishes the `Initial Goal` requested by the user. If the Operator has abandoned the core objective or fundamentally altered it, you must reject the plan.\n- **Logical Coherence**: Verify that the proposed milestones form a logical, achievable sequence to reach the goal. Support valid ongoing iteration/polling subgoals and trigger branches without misconstruing periodic checks as infinite loops. Reject the plan if it introduces contradictory steps, unhandled deadlocks, or obvious logical fallacies.\n- **Loop & Monitoring Protection**: If the existing plan contains an active `[Loop]` milestone (especially continuous polling / monitoring with exit condition `until manually stopped`), you MUST REJECT any proposed plan that deletes the `[Loop]` milestone, removes the loop tag, or marks a continuous monitoring milestone as completed (`[x]`) without explicit external user instruction.\n- **Adaptive Acceptance**: The Operator operates in a dynamic environment and may encounter unexpected UI states or dynamic event triggers. You should approve alternative strategies, extra intermediate steps, loop status tracking, or necessary workarounds as long as they serve the primary goal effectively.\n- **Constructive Feedback**: If you must reject the plan, provide clear, objective, and actionable feedback explaining the logical flaw or goal deviation, and suggest how the Operator should correct the plan."
   },
   "modes": {
     "initial_plan": {
       "system": [
         "role",
-        "guidelines",
+        "thinking_process",
         "capabilities_operator",
         "output_protocol_initial",
         "examples_initial"
@@ -27,4 +27,4 @@
       "human": "Initial Goal: {{ initial_goal }}\n\n### Execution History\n{{ history_str }}\n\n### Operator's Thoughts Before Modifying\n{% if operator_native_thinking %}Native Thinking:\n{{ operator_native_thinking }}\n\n{% endif %}{% if operator_raw_thinking %}Raw Thinking:\n{{ operator_raw_thinking }}\n\n{% endif %}### Old Plan\n{{ content_before }}\n\n### Proposed New Plan\n{{ content_after }}\n\nPlease evaluate if the proposed new plan should be approved."
     }
   }
-}
+}

artemis/agents/summarizer/summarizer.md

@@ -13,16 +13,19 @@ You are the Summarizer Agent for a mobile automation system. Your task is to sum
    - **BANNED WORDS**: "successfully", "completed", "entered", "navigated to", "failed", "unsuccessful", "could not", "achieved".
 3. **Focus on Intent & Strategy**: The specific physical action (exact coordinates, swipes) is now recorded automatically. Focus primarily on the high-level Intent and reasoning. If you realize the Operator is on the wrong path and needs to backtrack or pivot, describe this as a shift in strategy.
 4. **Intended Context Transitions**: Describe the intended navigation path ("From Where -> Intended Where"). Do not declare that the transition actually happened. (e.g., "From the Home screen, I intended to open the Network Settings view").
-5. **Preserve Verifications & Missing Items**: If the Operator verified any state in this step, or explicitly mentioned what prerequisites or items are still missing to accomplish the goal, you MUST state these contents in detail in the summary.
+5. **Preserve Verifications & Iteration Progress**: If the Operator verified any state in this step, processed a specific candidate item/polling round, or explicitly mentioned what prerequisites or items are still missing, you MUST succinctly preserve these concrete progress details and verified items in the summary.
 6. **Objective Intervention Logs**: If the Failure Analyzer intervened, faithfully summarize what triggered the intervention and what the analyzer executed, without judging the outcome.
 
 # LOOP & STAGNATION DETECTION
 Use the `RECENT 10 STEPS HISTORY` to intelligently detect loops or stagnation:
-1. **Physical Action Loops**: Be vigilant for loops spanning multiple steps based on physical interactions. If you detect a valid action loop, explicitly mention the pattern and the current iteration count (e.g., "This is the 3rd cycle of attempting to access the video group folder").
-2. **Cognitive Stagnation**: While you generally ignore internal background tool calls (e.g., reading logs, checking states) when counting physical action loops, there is an exception: if you detect continuous multiple steps of purely internal tool calls without any physical actions, you MUST objectively record this strategic "stagnation" or "analysis loop".
+1. **Physical Action Loops vs. Planned Iterations**:
+   - *Planned Iterations / Monitoring (Valid Progress)*: If the Operator is systematically processing different candidate items in a loop, or executing planned periodic polling cycles with wait intervals under a `[Loop]` milestone, record this objectively as planned progress (e.g., "Inspected Candidate #2", "Completed Polling Round #2 for new messages"). Do NOT flag planned iterations across different items or spaced polling as abnormal loops.
+   - *Unintended Action Loops (Anomalies)*: If the Operator is repeatedly clicking the exact same target, failing to progress on the same item, or oscillating blindly between identical states without updating progress notes, explicitly record the anomalous loop pattern and iteration count (e.g., "This is the 3rd cycle of failing to open the same folder").
+2. **Cognitive Stagnation**: While you generally ignore internal background tool calls (e.g., reading logs, checking states) when counting physical action loops, if you detect continuous multiple steps of purely internal tool calls without any physical actions or note updates, objectively record this "stagnation" or "analysis loop".
 
 # INPUT STRUCTURE
 You will receive:
+- **CURRENT PLAN & TASK PLAN**: The current task plan including milestones, active subgoals, and live iteration checklists to help you understand the strategic context and loop phase.
 - **RECENT 10 STEPS HISTORY**: A brief history of the last 10 steps to help you detect loops and stagnation patterns.
 - **CHRONOLOGICAL STEP TRACE (CURRENT STEP)**: The step-by-step chronological history of the current execution turn, including:
   - `[Operator Monologue] / [Operator Native Thought]`: The operator's planning, intentions, progress counting, reasoning, verified states, and explicit mentions of missing prerequisites.

artemis/agents/validator/tool_declarations.py

@@ -17,8 +17,10 @@
 import ast
 import asyncio
 import base64
+import hashlib
 import inspect
 import json
+from pathlib import Path
 import time
 from typing import Any, Literal
 
@@ -35,7 +37,10 @@ from artemis.tools.explorer_tool import _run_explorer_logic
 from artemis.graph.state import State
 from artemis.tools.mobile.launch_app import find_package
 from artemis.utils.app_launch_utils import launch_app_with_retries
-from artemis.utils.coordinates import parse_swipe_parameters
+from artemis.utils.coordinates import (
+    compute_smart_swipe_coordinates,
+    parse_swipe_parameters,
+)
 from artemis.utils.logger import get_logger
 from artemis.utils.ocr_xml_fusion import fuse_ocr_with_xml
 from artemis.utils.visualization import format_minimal_list_with_elements
@@ -282,6 +287,31 @@ def normalize_coordinate_target(target: Any) -> int | list[int] | Any:
     return target
 
 
+def serialize_mobile_action_result(result: Any) -> dict[str, Any]:
+    """Serializes MobileActionExecutor execution results into a clean structured dict.
+
+    Extracts the post-action screenshot hash from the captured image to eliminate guesswork.
+    """
+    if not isinstance(result, tuple) or len(result) < 3:
+        return {"outcome": str(result) if result is not None else ""}
+    outcome, img_bytes, shot_path, *rest = result
+    xml_list = rest[0] if rest else None
+
+    post_image_name = None
+    if shot_path:
+        post_image_name = Path(shot_path).stem
+    elif img_bytes:
+        post_image_name = hashlib.sha256(img_bytes).hexdigest()
+
+    is_err = bool(outcome and ("Error" in outcome or "Failed" in outcome))
+    return {
+        "outcome": outcome,
+        "post_image_name": post_image_name,
+        "has_xml": bool(xml_list),
+        "status": "error" if is_err else "success",
+    }
+
+
 class MobileActionExecutor:
     """Encapsulates device action execution, observation capture, and result reporting."""
 
@@ -289,7 +319,7 @@ class MobileActionExecutor:
         self.ctx = ctx
         self.controller = controller or UnifiedMobileController(ctx)
 
-    @trace(type="tool", name="click")
+    @trace(type="action", name="click", serializer=serialize_mobile_action_result)
     async def exec_click(
         self,
         target: list[int] | Any,
@@ -327,7 +357,7 @@ class MobileActionExecutor:
 
         return outcome, img_bytes, shot_path, xml_list
 
-    @trace(type="tool", name="click_sequence")
+    @trace(type="action", name="click_sequence", serializer=serialize_mobile_action_result)
     async def exec_click_sequence(
         self,
         sequence: list[int | list[int] | str] | str,
@@ -407,7 +437,7 @@ class MobileActionExecutor:
         except Exception as e:
             return f"Error executing click sequence: {e}", None, None, None
 
-    @trace(type="tool", name="long_press")
+    @trace(type="action", name="long_press", serializer=serialize_mobile_action_result)
     async def exec_long_press(
         self,
         target: list[int] | Any,
@@ -449,7 +479,7 @@ class MobileActionExecutor:
 
         return outcome, img_bytes, shot_path, xml_list
 
-    @trace(type="tool", name="input_text")
+    @trace(type="action", name="input_text", serializer=serialize_mobile_action_result)
     async def exec_input_text(
         self,
         text: str,
@@ -484,11 +514,11 @@ class MobileActionExecutor:
 
         return outcome, img_bytes, shot_path, xml_list
 
-    @trace(type="tool", name="swipe")
+    @trace(type="action", name="swipe", serializer=serialize_mobile_action_result)
     async def exec_swipe(
         self,
         action: str | list[int] | Any = None,
-        duration: int = 400,
+        duration: int = 800,
         state: State | None = None,
         duration_ms: int | None = None,
         **kwargs: Any,
@@ -510,33 +540,16 @@ class MobileActionExecutor:
         try:
             if kind == "direction":
                 dir_name = target
-                dir_map = {
-                    "up": (
-                        int(width * 0.6),
-                        int(height * 0.8),
-                        int(width * 0.6),
-                        int(height * 0.2),
-                    ),
-                    "down": (
-                        int(width * 0.6),
-                        int(height * 0.2),
-                        int(width * 0.6),
-                        int(height * 0.8),
-                    ),
-                    "left": (
-                        int(width * 0.8),
-                        int(height * 0.5),
-                        int(width * 0.2),
-                        int(height * 0.5),
-                    ),
-                    "right": (
-                        int(width * 0.2),
-                        int(height * 0.5),
-                        int(width * 0.8),
-                        int(height * 0.5),
-                    ),
-                }
-                x1, y1, x2, y2 = dir_map[dir_name]
+                x1, y1, x2, y2, smart_dur = compute_smart_swipe_coordinates(
+                    direction=target,
+                    target=swipe_input.get("target"),
+                    indexed_elements=getattr(state, "indexed_elements", None) if state else None,
+                    ui_hierarchy=getattr(state, "ui_tree", None) if state else None,
+                    width=width,
+                    height=height,
+                    duration=final_duration,
+                )
+                final_duration = smart_dur
                 err = await self.controller.swipe_coords(x1, y1, x2, y2, final_duration)
                 if err:
                     return f"Error swiping {dir_name}: {err}", None, None, None
@@ -569,7 +582,7 @@ class MobileActionExecutor:
 
         return outcome, img_bytes, shot_path, xml_list
 
-    @trace(type="tool", name="press_key")
+    @trace(type="action", name="press_key", serializer=serialize_mobile_action_result)
     async def exec_press_key(
         self,
         key: str,
@@ -603,7 +616,7 @@ class MobileActionExecutor:
 
         return outcome, img_bytes, shot_path, xml_list
 
-    @trace(type="tool", name="manage_app")
+    @trace(type="action", name="manage_app", serializer=serialize_mobile_action_result)
     async def exec_manage_app(
         self,
         action: str,
@@ -649,7 +662,7 @@ class MobileActionExecutor:
 
         return outcome, img_bytes, shot_path, xml_list
 
-    @trace(type="tool", name="wait_for_delay")
+    @trace(type="action", name="wait_for_delay", serializer=serialize_mobile_action_result)
     async def exec_wait_for_delay(
         self,
         time_in_ms: int,
@@ -668,7 +681,7 @@ class MobileActionExecutor:
 
         return outcome, img_bytes, shot_path, xml_list
 
-    @trace(type="tool", name="wait_for_text")
+    @trace(type="action", name="wait_for_text", serializer=serialize_mobile_action_result)
     async def exec_wait_for_text(
         self,
         text: str,
@@ -979,24 +992,51 @@ INPUT_TEXT_TOOL = ToolDeclaration(
 SWIPE_TOOL = ToolDeclaration(
     name="swipe",
     description=(
-        "[ACTION] Scroll or swipe on the screen. The screen after swipe will be"
-        " returned automatically. Set duration >= 1000 to drag-and-drop."
+        "[ACTION] Perform a swipe, drag, or slider-adjustment gesture on the screen. The screen and UI hierarchy after swipe will be returned automatically.\n\n"
+        "• Directional Scrolling ('direction' or 'action'): Recommended for general browsing and standard page scrolling in most scenarios. Automatically computes safe swipe vectors and adaptive duration, retains a ~40% visual overlap anchor for zero-omission traversal, and prevents inertial flings. Supports scoping to a sub-container via 'target'. If it fails on certain custom layouts, fall back to specifying exact coordinates ('start' and 'end') directly.\n"
+        "• Precise Coordinate Gestures ('start', 'end' or coordinates list): Best for local, fine-grained interactions such as adjusting sliders/SeekBars (e.g., volume, brightness, progress bars), drag-and-drop / list reordering, or as a reliable fallback when directional scrolling fails on specific containers. Always drag slightly PAST the target position to overcome touch slop and reliably trigger the update. When setting a slider to Maximum (100%) or Minimum (0%), swipe fully to the extreme boundary."
     ),
     parameters={
         "type": "object",
         "properties": {
+            "direction": {
+                "type": "string",
+                "enum": ["up", "down", "left", "right"],
+                "description": (
+                    "Direction for scrolling and swiping: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
+                    " 'down' (drags top-to-bottom, scrolling up to reveal content above),"
+                    " 'left' (drags right-to-left, scrolling right),"
+                    " 'right' (drags left-to-right, scrolling left)."
+                ),
+            },
+            "start": {
+                "type": "array",
+                "items": {"type": "integer"},
+                "description": "Start normalized coordinates [start_x, start_y] in 0-1000 scale.",
+            },
+            "end": {
+                "type": "array",
+                "items": {"type": "integer"},
+                "description": "End normalized coordinates [end_x, end_y] in 0-1000 scale.",
+            },
+            "target": {
+                "description": "Optional target element index (e.g. 2) or container bounds [left, top, right, bottom] to scope the directional swipe within.",
+            },
             "action": {
                 "description": (
-                    "Swipe action direction string ('up', 'down', 'left', 'right') OR custom"
-                    " coordinates [start_x, start_y, end_x, end_y] in 0-1000 scale."
-                )
+                    "Backward-compatible swipe gesture: smart direction string ('up', 'down', 'left', 'right')"
+                    " OR precise custom coordinates [start_x, start_y, end_x, end_y] in 0-1000 scale."
+                ),
             },
             "duration": {
                 "type": "integer",
-                "description": "Swipe duration in milliseconds (default 400).",
+                "description": (
+                    "Optional swipe/drag duration in milliseconds (default 800). For drag-and-drop,"
+                    " list reordering, or sliding/adjusting sliders (e.g., volume, brightness, SeekBars),"
+                    " set duration >= 1000 (e.g. 1500). If omitted for directional swipe, duration is computed automatically."
+                ),
             },
         },
-        "required": ["action"],
     },
 )
 
@@ -1074,7 +1114,7 @@ MANAGE_APP_TOOL = ToolDeclaration(
 WAIT_FOR_DELAY_TOOL = ToolDeclaration(
     name="wait_for_delay",
     description=(
-        "[ACTION] Pause execution and wait for a fixed duration in milliseconds."
+        "[ACTION] Pause execution and wait for a specified duration in milliseconds."
         " The screen after pause will be returned automatically."
     ),
     parameters={
@@ -1082,7 +1122,11 @@ WAIT_FOR_DELAY_TOOL = ToolDeclaration(
         "properties": {
             "time_in_ms": {
                 "type": "integer",
-                "description": "The exact duration to pause in milliseconds (e.g., 2000).",
+                "description": (
+                    "The exact duration to pause in milliseconds (e.g., 2000 for 2s,"
+                    " 60000 for 1 min, 180000 for 3 mins, 300000 for 5 mins)."
+                    " Convert any required waiting duration into milliseconds."
+                ),
             }
         },
         "required": ["time_in_ms"],

artemis/agents/validator/validator.py

@@ -47,6 +47,7 @@ from artemis.context import ArtemisContext, DeviceContext, DevicePlatform
 from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.data_engine.trace import CURRENT_TRACE_ID, trace
 from artemis.graph.state import State
+from artemis.platform import platform
 from artemis.services.llm import get_llm
 from artemis.tools.mobile.launch_app import find_package
 from artemis.utils import app_launch_utils, image_diff, visualization
@@ -153,7 +154,7 @@ class ValidatorNode:
                     local_ctx = ArtemisContext(
                         trace_id="local-fallback",
                         device=DeviceContext(
-                            host_platform="LINUX",
+                            host_platform=platform.os_type.name,
                             mobile_platform=DevicePlatform.ANDROID,
                             device_id=device_id,
                             device_width=w,

artemis/agents/video_analyzer/video_analyzer.md

@@ -24,7 +24,7 @@ You must carefully choose the appropriate tool based on the nature of the task:
 
 - **Must use Visual Analysis (`spawn_sub_agent`)**:
   - **UI Interactions**: Identifying clicks, gestures, typing, navigation.
-  - **Visual State**: Reading on-screen text (OCR), checking UI status (toggles, dialogs, loading states).
+  - **Visual State**: Reading on-screen text, checking UI status (toggles, dialogs, loading states).
   - **Visual Content**: Identifying images, layout issues, animations.
 - **Can use Audio-Only Analysis (`analyze_audio_only`)**:
   - **Speech-centric**: Summarizing spoken content, identifying speakers, transcription.

artemis/agents/video_analyzer/video_analyzer.py

@@ -40,7 +40,7 @@ from langchain_core.messages import (
 from artemis.agents.video_analyzer.conflict_resolution import (
     ConflictResolutionService,
 )
-from artemis.config import settings
+from artemis.config import get_temp_dir, settings
 from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
 from artemis.context import ArtemisContext
 from artemis.controllers.controller_factory import get_controller
@@ -461,10 +461,10 @@ class VideoAnalyzer:
         utils_cfg = getattr(llm_config, "utils", None) if llm_config else None
         llm_cfg = getattr(utils_cfg, "video_analyzer", None) if utils_cfg else None
         model_str = (
-            llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.6-flash"
+            llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.7-flash"
         ).lower()
         self.model_name = (
-            llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.6-flash"
+            llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.7-flash"
         )
         if "/" in self.model_name:
             self.model_name = self.model_name.split("/")[-1]
@@ -1142,7 +1142,7 @@ class VideoAnalyzer:
                                         confidence = float(event.get("confidence_score", 0.0))
                                         relative_offset = max(0.0, verification_ts - actual_start)
 
-                                        out_dir = Path("/tmp/video_analyzer")
+                                        out_dir = get_temp_dir("video_analyzer")
                                         out_dir.mkdir(parents=True, exist_ok=True)
                                         existing_imgs = glob.glob(f"{out_dir}/img_*.jpg")
                                         max_idx = -1
@@ -1692,7 +1692,7 @@ class VideoAnalyzer:
             screenshot_path_str = None
             try:
                 rel_offset = max(0.0, v_ts - actual_start)
-                out_dir = Path("/tmp/video_analyzer")
+                out_dir = get_temp_dir("video_analyzer")
                 out_dir.mkdir(parents=True, exist_ok=True)
 
                 existing_imgs = glob.glob(f"{out_dir}/img_*.jpg")
@@ -1899,7 +1899,7 @@ class VideoAnalyzer:
                 if getattr(llm_cfg, "thinking_level", None) is not None:
                     thinking_level = llm_cfg.thinking_level
             else:
-                self.model_name = "gemini-3.6-flash"
+                self.model_name = "gemini-3.7-flash"
 
             # Track files for cleanup
             self.local_files_to_cleanup = set()

artemis/config/agent.py

@@ -18,7 +18,7 @@ import os
 from pathlib import Path
 from typing import Any
 
-from pydantic import AliasChoices, BaseModel, Field, model_validator
+from pydantic import BaseModel, Field, model_validator
 
 from artemis.config.constants import (
     AGENT_CONFIG_FILENAME,
@@ -289,7 +289,6 @@ class AgentGlobalConfig(BaseModel):
     denylisted_tools: dict[str, list[str]] = Field(
         default_factory=dict,
         description="List of denylisted tools per agent.",
-        validation_alias=AliasChoices("denylisted_tools", "blocklisted_tools"),
     )
     video_analyzer: VideoAnalyzerConfig = Field(
         default_factory=VideoAnalyzerConfig,

artemis/config/core/__init__.py

@@ -0,0 +1,24 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Artemis unified configuration core."""
+
+from artemis.config.core.engine import CascadingConfigEngine, config_engine
+from artemis.config.core.interpolator import interpolate_config_value
+
+__all__ = [
+    "CascadingConfigEngine",
+    "config_engine",
+    "interpolate_config_value",
+]

artemis/config/core/engine.py

@@ -0,0 +1,118 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Multi-source Cascading Configuration Engine."""
+
+import json
+from pathlib import Path
+import re
+from typing import Any
+
+from dotenv import load_dotenv
+
+from artemis.config.core.interpolator import interpolate_config_value
+from artemis.platform import platform
+from artemis.utils.logger import get_logger
+
+logger = get_logger(__name__)
+
+
+def _strip_json_comments(text: str) -> str:
+    """Strip // and /* */ comments from JSONC text while preserving string literals."""
+    pattern = r'//.*?$|/\*.*?\*/|\'(?:\\.|[^\\\'])*\'|"(?:\\.|[^\\"])*"'
+
+    def replacer(match: re.Match) -> str:
+        s = match.group(0)
+        if s.startswith("/"):
+            return ""
+        return s
+
+    return re.sub(pattern, replacer, text, flags=re.DOTALL | re.MULTILINE)
+
+
+def _deep_merge(base: dict[str, Any], override: dict[str, Any]) -> dict[str, Any]:
+    """Recursively deep merge two dictionaries."""
+    merged = base.copy()
+    for key, value in override.items():
+        if key in merged and isinstance(merged[key], dict) and isinstance(value, dict):
+            merged[key] = _deep_merge(merged[key], value)
+        else:
+            merged[key] = value
+    return merged
+
+
+class CascadingConfigEngine:
+    """Manages 5-tier configuration hierarchy with automatic cascade and interpolation."""
+
+    def __init__(self, workspace_root: Path | None = None):
+        self.workspace_root = workspace_root or Path.cwd()
+        self._load_environment_files()
+
+    def _load_environment_files(self) -> None:
+        """Load .env files from global config dir and workspace root."""
+        # 1. Global user .env
+        global_env = platform.paths.config_dir / ".env"
+        if global_env.exists():
+            load_dotenv(dotenv_path=global_env, override=False, encoding="utf-8")
+
+        # 2. Legacy ~/.artemis/.env fallback
+        legacy_env = platform.paths.data_dir / ".env"
+        if legacy_env.exists() and legacy_env != global_env:
+            load_dotenv(dotenv_path=legacy_env, override=False, encoding="utf-8")
+
+        # 3. Workspace .env
+        ws_env = self.workspace_root / ".env"
+        if ws_env.exists():
+            load_dotenv(dotenv_path=ws_env, override=True, encoding="utf-8")
+
+    def load_raw_config(
+        self,
+        cli_overrides: dict[str, Any] | None = None,
+    ) -> dict[str, Any]:
+        """Load and merge all config layers into a unified dictionary."""
+        config: dict[str, Any] = {}
+
+        # Tier 1: Global user configuration ($CONFIG_DIR/artemis.jsonc)
+        global_config_path = platform.paths.config_dir / "artemis.jsonc"
+        if global_config_path.exists():
+            try:
+                raw_text = global_config_path.read_text(encoding="utf-8")
+                cleaned = _strip_json_comments(raw_text)
+                config = _deep_merge(config, json.loads(cleaned))
+            except Exception as e:
+                logger.warning(f"Failed to load global config {global_config_path}: {e}")
+
+        # Tier 2: Workspace project configuration (config/artemis.jsonc)
+        ws_config_path = self.workspace_root / "config" / "artemis.jsonc"
+        if not ws_config_path.exists():
+            ws_config_path = self.workspace_root / "artemis.jsonc"
+
+        if ws_config_path.exists():
+            try:
+                raw_text = ws_config_path.read_text(encoding="utf-8")
+                cleaned = _strip_json_comments(raw_text)
+                config = _deep_merge(config, json.loads(cleaned))
+            except Exception as e:
+                logger.warning(f"Failed to load workspace config {ws_config_path}: {e}")
+
+        # Tier 3: CLI overrides
+        if cli_overrides:
+            config = _deep_merge(config, cli_overrides)
+
+        # Apply variable interpolation across merged dictionary
+        interpolated = interpolate_config_value(config, workspace_root=self.workspace_root)
+        return interpolated
+
+
+config_engine = CascadingConfigEngine()

artemis/config/core/interpolator.py

@@ -0,0 +1,78 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Dynamic configuration variable interpolation engine (${env:VAR}, ${path:NAME})."""
+
+import os
+from pathlib import Path
+import re
+from typing import Any
+
+from artemis.platform import platform
+
+_VAR_PATTERN = re.compile(r"\$\{([^}]+)\}")
+
+
+def _resolve_variable(expr: str, workspace_root: Path | None = None) -> str:
+    """Resolve a single ${namespace:key:-default} expression."""
+    expr = expr.strip()
+    namespace, _, rest = expr.partition(":")
+    if not rest:
+        # Defaults to environment variable lookup
+        return os.getenv(namespace, f"${{{expr}}}")
+
+    key, _, default = rest.partition(":-")
+
+    if namespace == "env":
+        return os.getenv(key, default)
+    elif namespace == "path":
+        paths = platform.paths
+        if key == "config_dir":
+            return str(paths.config_dir)
+        elif key == "data_dir":
+            return str(paths.data_dir)
+        elif key == "cache_dir":
+            return str(paths.cache_dir)
+        elif key == "logs_dir":
+            return str(paths.logs_dir)
+        elif key == "temp_dir":
+            return str(paths.temp_dir())
+        elif key == "workspace_root" or key == "root_dir":
+            return str(workspace_root or Path.cwd())
+        return default or f"${{{expr}}}"
+    elif namespace == "platform":
+        if key == "os_type":
+            return platform.os_type.value
+        elif key == "path_sep":
+            return platform.process.path_separator
+        return default or f"${{{expr}}}"
+
+    return os.getenv(expr, default or f"${{{expr}}}")
+
+
+def interpolate_config_value(val: Any, workspace_root: Path | None = None) -> Any:
+    """Recursively interpolate strings, lists, and dicts containing ${...} variables."""
+    if isinstance(val, str):
+        if "${" not in val:
+            return val
+
+        def replacer(match: re.Match) -> str:
+            return _resolve_variable(match.group(1), workspace_root)
+
+        return _VAR_PATTERN.sub(replacer, val)
+    elif isinstance(val, dict):
+        return {k: interpolate_config_value(v, workspace_root) for k, v in val.items()}
+    elif isinstance(val, list):
+        return [interpolate_config_value(item, workspace_root) for item in val]
+    return val

artemis/config/llm.py

@@ -227,10 +227,10 @@ def _expand_default_into_nodes(config_dict: dict) -> dict:
         "default",
         {
             "provider": "google",
-            "model": "gemini-3.6-flash",
+            "model": "gemini-3.7-flash",
             "fallback": {
                 "provider": "google",
-                "model": "gemini-3.5-flash",
+                "model": "gemini-3.6-flash",
             },
         },
     )

artemis/config/paths.py

@@ -27,6 +27,7 @@ from artemis.config.constants import (
     IPC_PORT_FILENAME,
     LS_ADDRESS_FILENAME,
 )
+from artemis.platform import platform
 
 # Project root directory (3 levels up from artemis/config/paths.py)
 ROOT_DIR = Path(__file__).resolve().parent.parent.parent
@@ -43,22 +44,10 @@ def get_app_dir() -> Path:
 
     Order of precedence:
     1. ARTEMIS_APP_DIR or ANTIGRAVITY_APP_DIR environment variable.
-    2. Legacy ~/.gemini/jetski directory if it exists.
-    3. Default ~/.artemis directory.
+    2. Legacy ~/.gemini/jetski or ~/.artemis directory if it exists.
+    3. OS-standard data directory via Platform Abstraction Layer (PAL).
     """
-    env_dir = os.getenv(ENV_ARTEMIS_APP_DIR) or os.getenv(ENV_ANTIGRAVITY_APP_DIR)
-    if env_dir:
-        path = Path(env_dir)
-        path.mkdir(parents=True, exist_ok=True)
-        return path
-
-    legacy_dir = Path(os.path.expanduser("~/.gemini/jetski"))
-    if legacy_dir.exists():
-        return legacy_dir
-
-    default_dir = Path(os.path.expanduser("~/.artemis"))
-    default_dir.mkdir(parents=True, exist_ok=True)
-    return default_dir
+    return platform.paths.resolve_app_dir()
 
 
 def get_default_traces_path() -> Path:
@@ -97,10 +86,7 @@ def get_temp_dir(subfolder: str | None = None) -> Path:
     Returns:
         Path object pointing to the initialized temporary directory.
     """
-    base_temp = get_app_dir() / "temp"
-    target_dir = base_temp / subfolder if subfolder else base_temp
-    target_dir.mkdir(parents=True, exist_ok=True)
-    return target_dir
+    return platform.paths.temp_dir(subfolder)
 
 
 def get_data_engine_db_path() -> Path:
@@ -132,7 +118,7 @@ def get_config_path(filename: str, default_bundled_path: Path | None = None) ->
     2. Central `config/` directory: `<ROOT_DIR>/config/<filename>`
     3. Project root directory: `<ROOT_DIR>/<filename>`
     4. Custom default bundled path (if provided and exists)
-    5. User application directory: `<APP_DIR>/<filename>`
+    5. User application/config directory via PAL: `<CONFIG_DIR>/<filename>` or `<APP_DIR>/<filename>`
     """
     env_var = f"ARTEMIS_{filename.upper().replace('.', '_').replace('-', '_')}"
     env_path_str = os.getenv(env_var)
@@ -155,6 +141,11 @@ def get_config_path(filename: str, default_bundled_path: Path | None = None) ->
     if default_bundled_path and default_bundled_path.exists():
         return default_bundled_path
 
+    # Check user PAL config directory
+    pal_config_path = platform.paths.config_dir / filename
+    if pal_config_path.exists():
+        return pal_config_path
+
     # Check user app directory
     app_dir_path = get_app_dir() / filename
     if app_dir_path.exists():

artemis/config/settings.py

@@ -27,17 +27,21 @@ from artemis.config.constants import (
     DEFAULT_EXPLORER_VERSION,
     DEFAULT_MODEL,
     DEFAULT_PROFILE,
+    ENV_ANTHROPIC_API_KEY,
     ENV_DATA_ENGINE_DB_PATH,
     ENV_GCP_API_KEY,
     ENV_GEMINI_API_KEY,
     ENV_GOOGLE_API_KEY,
+    ENV_OCR_API_KEY,
     ENV_OPEN_ROUTER_API_KEY,
     ENV_OPENAI_API_KEY,
+    ENV_VISION_API_KEY,
     ENV_XAI_API_KEY,
     ExplorerVersion,
 )
 from artemis.config.paths import (
     GLOBAL_APP_DIR,
+    ROOT_DIR,
     get_app_dir,
     get_data_engine_db_path,
     get_default_traces_path,
@@ -45,16 +49,48 @@ from artemis.config.paths import (
 )
 from artemis.utils.logger import get_logger
 
-# Load environment configuration from app dir if present, else current working dir
+# Load environment configuration from workspace root dir and app dir if present
+load_dotenv(dotenv_path=ROOT_DIR / ".env", verbose=True)
 _global_env = GLOBAL_APP_DIR / ".env"
-if _global_env.exists():
+if _global_env.exists() and _global_env.resolve() != (ROOT_DIR / ".env").resolve():
     load_dotenv(dotenv_path=_global_env, verbose=True)
-else:
-    load_dotenv(verbose=True)
 
 logger = get_logger(__name__)
 
 
+def is_placeholder_key(val: str | SecretStr | None) -> bool:
+    """Check if an API credential is empty or an unconfigured placeholder value."""
+    if val is None:
+        return True
+    raw_str = val.get_secret_value() if isinstance(val, SecretStr) else str(val)
+    raw_str = raw_str.strip().lower()
+    if not raw_str:
+        return True
+    if raw_str in (
+        "api_key",
+        "your_api_key",
+        "your_api_key_here",
+        "your_gemini_api_key_here",
+        "your_google_cloud_vision_api_key_here",
+        "your_openai_api_key_here",
+        "your_anthropic_api_key_here",
+        "your_openrouter_api_key_here",
+        "your_xai_api_key_here",
+        "none",
+        "empty",
+        "null",
+        "undefined",
+    ):
+        return True
+    if (
+        (raw_str.startswith("your_") and raw_str.endswith("_here"))
+        or (raw_str.startswith("<") and raw_str.endswith(">"))
+        or (raw_str.startswith("[") and raw_str.endswith("]"))
+    ):
+        return True
+    return False
+
+
 class Settings(BaseSettings):
     """Centralized ARTEMIS runtime settings loaded from environment and .env files."""
 
@@ -104,7 +140,24 @@ class Settings(BaseSettings):
 
     @model_validator(mode="after")
     def fallback_api_keys(self) -> "Settings":
-        """Normalize Google / Gemini / GCP and OCR API keys for seamless interoperability."""
+        """Normalize Google / Gemini / GCP and OCR API keys and filter placeholders."""
+        # Sanitize any placeholder values loaded from environment or .env
+        for attr in (
+            "OPENAI_API_KEY",
+            "GOOGLE_API_KEY",
+            "GEMINI_API_KEY",
+            "GCP_API_KEY",
+            "ANTHROPIC_API_KEY",
+            "XAI_API_KEY",
+            "OPEN_ROUTER_API_KEY",
+            "OCR_API_KEY",
+            "VISION_API_KEY",
+            "API_KEY",
+        ):
+            val = getattr(self, attr, None)
+            if val and is_placeholder_key(val):
+                setattr(self, attr, None)
+
         if not self.GOOGLE_API_KEY:
             if self.GEMINI_API_KEY:
                 self.GOOGLE_API_KEY = self.GEMINI_API_KEY
@@ -112,13 +165,8 @@ class Settings(BaseSettings):
                 self.GOOGLE_API_KEY = self.GCP_API_KEY
 
         # Fallback for OCR & Vision API keys
-        if not self.OCR_API_KEY:
-            if self.VISION_API_KEY:
-                self.OCR_API_KEY = self.VISION_API_KEY
-            elif self.API_KEY:
-                self.OCR_API_KEY = self.API_KEY
-        if not self.API_KEY and self.OCR_API_KEY:
-            self.API_KEY = self.OCR_API_KEY
+        if not self.OCR_API_KEY and self.VISION_API_KEY:
+            self.OCR_API_KEY = self.VISION_API_KEY
         return self
 
     def get_api_key(self, provider: str) -> SecretStr | None:
@@ -131,18 +179,22 @@ class Settings(BaseSettings):
             SecretStr containing the API key or None if not configured.
         """
         provider_lower = provider.lower()
+        key: SecretStr | None = None
         if provider_lower in ("google", "gemini", "vertexai"):
-            return self.GOOGLE_API_KEY or self.GEMINI_API_KEY or self.GCP_API_KEY
+            key = self.GOOGLE_API_KEY or self.GEMINI_API_KEY or self.GCP_API_KEY
         elif provider_lower in ("ocr", "vision", "google_vision"):
-            return self.OCR_API_KEY or self.VISION_API_KEY or self.API_KEY or self.GOOGLE_API_KEY
+            key = self.OCR_API_KEY or self.VISION_API_KEY
         elif provider_lower == "openai":
-            return self.OPENAI_API_KEY
-        elif provider_lower == "anthropic":
-            return self.ANTHROPIC_API_KEY
+            key = self.OPENAI_API_KEY
+        elif provider_lower in ("anthropic", "claude"):
+            key = self.ANTHROPIC_API_KEY
         elif provider_lower == "openrouter":
-            return self.OPEN_ROUTER_API_KEY
-        elif provider_lower == "xai":
-            return self.XAI_API_KEY
+            key = self.OPEN_ROUTER_API_KEY
+        elif provider_lower in ("xai", "grok"):
+            key = self.XAI_API_KEY
+
+        if key and not is_placeholder_key(key):
+            return key
         return None
 
     def set_api_key(self, provider: str, key: str, persist_to_env: bool = False) -> None:
@@ -161,7 +213,7 @@ class Settings(BaseSettings):
             self.GOOGLE_API_KEY = secret
             self.GEMINI_API_KEY = secret
             self.GCP_API_KEY = secret
-            env_key_name = ENV_GOOGLE_API_KEY
+            env_key_name = ENV_GEMINI_API_KEY
             os.environ[ENV_GOOGLE_API_KEY] = key
             os.environ[ENV_GEMINI_API_KEY] = key
             os.environ[ENV_GCP_API_KEY] = key
@@ -169,6 +221,10 @@ class Settings(BaseSettings):
             self.OPENAI_API_KEY = secret
             env_key_name = ENV_OPENAI_API_KEY
             os.environ[ENV_OPENAI_API_KEY] = key
+        elif provider_lower in ("anthropic", "claude"):
+            self.ANTHROPIC_API_KEY = secret
+            env_key_name = ENV_ANTHROPIC_API_KEY
+            os.environ[ENV_ANTHROPIC_API_KEY] = key
         elif provider_lower == "openrouter":
             self.OPEN_ROUTER_API_KEY = secret
             env_key_name = ENV_OPEN_ROUTER_API_KEY
@@ -177,18 +233,62 @@ class Settings(BaseSettings):
             self.XAI_API_KEY = secret
             env_key_name = ENV_XAI_API_KEY
             os.environ[ENV_XAI_API_KEY] = key
+        elif provider_lower in ("ocr", "vision", "google_vision"):
+            self.OCR_API_KEY = secret
+            self.VISION_API_KEY = secret
+            env_key_name = ENV_OCR_API_KEY
+            os.environ[ENV_OCR_API_KEY] = key
+            os.environ[ENV_VISION_API_KEY] = key
 
         if persist_to_env and env_key_name:
-            try:
-                env_file = get_app_dir() / ".env"
-                lines = []
-                if env_file.exists():
-                    lines = env_file.read_text(encoding="utf-8").splitlines()
-                lines = [line for line in lines if not line.startswith(f"{env_key_name}=")]
-                lines.append(f"{env_key_name}={key}")
-                env_file.write_text("\n".join(lines) + "\n", encoding="utf-8")
-            except Exception as e:
-                logger.warning(f"Could not persist {env_key_name} to .env: {e}")
+            target_env_files = [
+                ROOT_DIR / ".env",
+                get_app_dir() / ".env",
+            ]
+            seen_paths = set()
+            for env_file in target_env_files:
+                try:
+                    resolved = env_file.resolve()
+                    if resolved in seen_paths:
+                        continue
+                    seen_paths.add(resolved)
+
+                    lines = []
+                    if env_file.exists():
+                        lines = env_file.read_text(encoding="utf-8").splitlines()
+
+                    # Determine keys to update
+                    keys_to_update = [env_key_name]
+                    if provider_lower in ("google", "gemini", "vertexai"):
+                        keys_to_update = [ENV_GEMINI_API_KEY, ENV_GOOGLE_API_KEY, ENV_GCP_API_KEY]
+                    elif provider_lower in ("ocr", "vision", "google_vision"):
+                        keys_to_update = [ENV_OCR_API_KEY, ENV_VISION_API_KEY]
+
+                    new_lines = []
+                    updated_set = set()
+                    for line in lines:
+                        replaced = False
+                        for k in keys_to_update:
+                            if (
+                                line.startswith(f"{k}=")
+                                or line.startswith(f"#{k}=")
+                                or line.startswith(f"# {k}=")
+                            ):
+                                new_lines.append(f"{k}={key}")
+                                updated_set.add(k)
+                                replaced = True
+                                break
+                        if not replaced:
+                            new_lines.append(line)
+
+                    # Ensure the primary key is present if not replaced
+                    primary_key = keys_to_update[0]
+                    if primary_key not in updated_set:
+                        new_lines.append(f"{primary_key}={key}")
+
+                    env_file.write_text("\n".join(new_lines) + "\n", encoding="utf-8")
+                except Exception as e:
+                    logger.warning(f"Could not persist {env_key_name} to {env_file}: {e}")
 
 
 # Singleton instance

artemis/constants.py

@@ -12,7 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-RECURSION_LIMIT = 400
+RECURSION_LIMIT = 30000
 MAX_MESSAGES_IN_HISTORY = 25
 VALIDATOR_MESSAGES_KEY = "validator_messages"
 

artemis/context.py

@@ -75,7 +75,7 @@ class DeviceContext(BaseModel):
         extra="allow",
     )
 
-    host_platform: Literal["WINDOWS", "LINUX"]
+    host_platform: Literal["WINDOWS", "LINUX", "DARWIN", "MACOS"] | str
     mobile_platform: DevicePlatform
     device_id: str
 

artemis/controllers/unified_controller.py

@@ -206,12 +206,11 @@ class UnifiedMobileController:
             for _ in range(nb_chars):
                 await self._driver.press_key("delete")
             return True
-        # Safe & robust full clear: Move to End -> Shift+Home selection -> Delete -> Fallback backspaces
+        # Best-effort full clear: End -> Ctrl+A -> Delete
         try:
             clear_cmd = (
                 "input keyevent 123 && "
-                "input keyevent --meta 1 122 && "
-                "input keyevent 67 && "
+                "input keycombination 113 29 && input keyevent 67 && "
                 "input keyevent 67 67 67 67 67 67 67 67 67 67 67 67 67 67 67 67 67 67 67 67"
             )
             await self._driver.execute_shell(clear_cmd)

artemis/core/diagnostics/__init__.py

@@ -0,0 +1,62 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Artemis Diagnostics & System Readiness Package."""
+
+from artemis.core.diagnostics.emulator_manager import (
+    EmulatorLaunchStage,
+    EmulatorLaunchState,
+    emulator_manager,
+)
+from artemis.core.diagnostics.engine import ReadinessEngine, readiness_engine
+from artemis.core.diagnostics.probes.adb_probe import AdbDeviceProbe
+from artemis.core.diagnostics.probes.base import BaseProbe
+from artemis.core.diagnostics.probes.credentials_probe import (
+    LLMCredentialsProbe,
+    VisionOCRProbe,
+)
+from artemis.core.diagnostics.probes.runtime_probe import (
+    PythonRuntimeProbe,
+    SystemConfigProbe,
+)
+from artemis.core.diagnostics.probes.toolchain_probe import ToolchainProbe
+from artemis.core.diagnostics.schema import (
+    DeviceInfo,
+    ProbeAction,
+    ProbeCategory,
+    ProbeResult,
+    ProbeStatus,
+    SystemReadinessReport,
+)
+
+__all__ = [
+    "ReadinessEngine",
+    "readiness_engine",
+    "emulator_manager",
+    "EmulatorLaunchStage",
+    "EmulatorLaunchState",
+    "BaseProbe",
+    "PythonRuntimeProbe",
+    "SystemConfigProbe",
+    "AdbDeviceProbe",
+    "LLMCredentialsProbe",
+    "VisionOCRProbe",
+    "ToolchainProbe",
+    "DeviceInfo",
+    "ProbeAction",
+    "ProbeCategory",
+    "ProbeResult",
+    "ProbeStatus",
+    "SystemReadinessReport",
+]

artemis/core/diagnostics/emulator_manager.py

@@ -0,0 +1,480 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Android Emulator Launch & Boot Progress Lifecycle Manager."""
+
+import asyncio
+from collections import deque
+from enum import Enum
+import os
+from pathlib import Path
+import shutil
+import subprocess
+import threading
+import time
+from typing import Any
+
+from pydantic import BaseModel, Field
+
+from artemis.platform import OSType, platform
+from artemis.toolchain import toolchain
+from artemis.utils.logger import get_logger
+
+logger = get_logger(__name__)
+
+
+class EmulatorLaunchStage(str, Enum):
+    """Lifecycle stages of launching an Android Virtual Device."""
+
+    IDLE = "idle"
+    STARTING = "starting"  # Process spawned, verifying PID
+    WAITING_FOR_ADB = "waiting_for_adb"  # Process alive, waiting for emulator to appear in ADB
+    BOOTING = "booting"  # Visible in ADB, waiting for sys.boot_completed=1
+    READY = "ready"  # Boot complete and ready for automation
+    FAILED = "failed"  # Process crashed, lock error, or timeout
+    STOPPED = "stopped"  # Stopped by user
+
+
+class EmulatorLaunchState(BaseModel):
+    """State schema for real-time emulator launch tracking."""
+
+    avd_name: str | None = None
+    status: EmulatorLaunchStage = EmulatorLaunchStage.IDLE
+    pid: int | None = None
+    serial: str | None = None
+    stage_message: str = "Ready to launch"
+    progress_percent: int = 0
+    started_at: float | None = None
+    elapsed_seconds: int = 0
+    error: str | None = None
+    logs: list[str] = Field(default_factory=list)
+    can_retry: bool = True
+
+
+class EmulatorManager:
+    """Manages background emulator execution, stdout/stderr stream capture, and boot completion polling."""
+
+    def __init__(self):
+        self._current_state = EmulatorLaunchState()
+        self._proc: subprocess.Popen | None = None
+        self._log_buffer: deque[str] = deque(maxlen=100)
+        self._track_task: asyncio.Task | None = None
+        self._lock = asyncio.Lock()
+        self._reader_thread: threading.Thread | None = None
+
+    def _locate_emulator(self) -> str | None:
+        """Find the emulator binary path from PATH or standard SDK environments."""
+        resolved = toolchain.resolve("emulator")
+        if resolved:
+            return resolved
+
+        emu_path = shutil.which("emulator")
+        if emu_path:
+            return emu_path
+
+        sdk_candidates = [
+            os.environ.get("ANDROID_HOME"),
+            os.environ.get("ANDROID_SDK_ROOT"),
+            str(Path.home() / "Android" / "Sdk"),
+            str(Path.home() / "Android" / "sdk"),
+            str(Path.home() / "Library" / "Android" / "sdk"),
+            os.getenv("LOCALAPPDATA", "") + "/Android/Sdk" if os.getenv("LOCALAPPDATA") else None,
+            "/usr/lib/android-sdk",
+            "/opt/android-sdk",
+        ]
+        for base in sdk_candidates:
+            if base:
+                cand = (
+                    Path(base)
+                    / "emulator"
+                    / ("emulator.exe" if platform.os_type == OSType.WINDOWS else "emulator")
+                )
+                if cand.is_file():
+                    return str(cand)
+        return None
+
+    def _locate_adb(self) -> str:
+        """Find the adb binary path."""
+        resolved = toolchain.resolve("adb")
+        if resolved:
+            return resolved
+        return shutil.which("adb") or "adb"
+
+    def get_status(self) -> EmulatorLaunchState:
+        """Return current launch progress state snapshot."""
+        state = self._current_state.model_copy()
+        if state.started_at and state.status in (
+            EmulatorLaunchStage.STARTING,
+            EmulatorLaunchStage.WAITING_FOR_ADB,
+            EmulatorLaunchStage.BOOTING,
+        ):
+            state.elapsed_seconds = int(time.time() - state.started_at)
+        state.logs = list(self._log_buffer)
+        return state
+
+    def _stream_logs(self, proc: subprocess.Popen):
+        """Continuously read process stdout/stderr in a background thread."""
+        try:
+            if proc.stdout:
+                for line in iter(proc.stdout.readline, ""):
+                    if not line:
+                        break
+                    clean_line = line.strip()
+                    if clean_line:
+                        self._log_buffer.append(clean_line)
+        except Exception as e:
+            logger.debug(f"Log streaming finished: {e}")
+        finally:
+            if proc.stdout:
+                proc.stdout.close()
+
+    async def launch(self, avd_name: str) -> EmulatorLaunchState:
+        """Initiate background emulator launch and start asynchronous boot monitoring."""
+        clean_avd = avd_name.strip()
+        if not clean_avd:
+            self._current_state = EmulatorLaunchState(
+                status=EmulatorLaunchStage.FAILED,
+                error="AVD name cannot be empty.",
+                stage_message="Launch failed: AVD name cannot be empty.",
+            )
+            return self.get_status()
+
+        async with self._lock:
+            # If already launching the same AVD and process is alive
+            if (
+                self._current_state.status
+                in (
+                    EmulatorLaunchStage.STARTING,
+                    EmulatorLaunchStage.WAITING_FOR_ADB,
+                    EmulatorLaunchStage.BOOTING,
+                )
+                and self._current_state.avd_name == clean_avd
+                and self._proc is not None
+                and self._proc.poll() is None
+            ):
+                logger.info(f"AVD '{clean_avd}' is already in progress of launching.")
+                return self.get_status()
+
+            emu_path = self._locate_emulator()
+            if not emu_path:
+                self._current_state = EmulatorLaunchState(
+                    avd_name=clean_avd,
+                    status=EmulatorLaunchStage.FAILED,
+                    error="Android emulator executable not found in PATH or Android SDK. Please install Android SDK platform tools and emulator.",
+                    stage_message="Emulator binary not found.",
+                )
+                return self.get_status()
+
+            self._log_buffer.clear()
+            self._log_buffer.append(f"Starting emulator '{clean_avd}' using binary: {emu_path}")
+
+            try:
+                # Cancel previous tracker if any
+                if self._track_task and not self._track_task.done():
+                    self._track_task.cancel()
+
+                # Spawn emulator process capturing stdout & stderr
+                proc = subprocess.Popen(
+                    [emu_path, "-avd", clean_avd],
+                    stdout=subprocess.PIPE,
+                    stderr=subprocess.STDOUT,
+                    text=True,
+                    bufsize=1,
+                    start_new_session=True,
+                )
+                self._proc = proc
+                now = time.time()
+
+                # Start reader thread for stdout/stderr
+                self._reader_thread = threading.Thread(
+                    target=self._stream_logs, args=(proc,), daemon=True
+                )
+                self._reader_thread.start()
+
+                self._current_state = EmulatorLaunchState(
+                    avd_name=clean_avd,
+                    status=EmulatorLaunchStage.STARTING,
+                    pid=proc.pid,
+                    stage_message=f"Process spawned (PID: {proc.pid}). Initializing QEMU virtualization...",
+                    progress_percent=15,
+                    started_at=now,
+                    elapsed_seconds=0,
+                    error=None,
+                    logs=list(self._log_buffer),
+                )
+
+                # Launch async boot tracker task
+                self._track_task = asyncio.create_task(
+                    self._track_boot_lifecycle(clean_avd, proc, now)
+                )
+
+                return self.get_status()
+
+            except Exception as e:
+                logger.error(f"Failed to spawn emulator process: {e}")
+                self._current_state = EmulatorLaunchState(
+                    avd_name=clean_avd,
+                    status=EmulatorLaunchStage.FAILED,
+                    error=f"Failed to spawn emulator process: {e}",
+                    stage_message="Failed to spawn emulator process.",
+                    logs=list(self._log_buffer),
+                )
+                return self.get_status()
+
+    async def _track_boot_lifecycle(self, avd_name: str, proc: subprocess.Popen, started_at: float):
+        """Monitor emulator lifecycle from process execution to ADB connection and OS boot completion."""
+        adb_path = self._locate_adb()
+        detected_serial: str | None = None
+        max_wait_seconds = 180  # 3 minutes maximum boot timeout
+
+        try:
+            # Phase 1: Early crash detection (first 5 seconds)
+            for _ in range(5):
+                await asyncio.sleep(1)
+                poll_res = proc.poll()
+                if poll_res is not None:
+                    logs_str = "\n".join(list(self._log_buffer)[-10:])
+                    error_msg = f"Emulator process exited immediately with code {poll_res}."
+                    if "already running" in logs_str.lower() or "lock" in logs_str.lower():
+                        error_msg = f"AVD '{avd_name}' is locked by another running emulator instance. Try stopping existing instances or restart ADB."
+                    elif "panic" in logs_str.lower():
+                        error_msg = f"Emulator panic: {logs_str}"
+
+                    logger.error(f"[EmulatorManager] Early crash: {error_msg}")
+                    self._current_state = EmulatorLaunchState(
+                        avd_name=avd_name,
+                        status=EmulatorLaunchStage.FAILED,
+                        pid=proc.pid,
+                        error=error_msg,
+                        stage_message=f"Process exited prematurely (Exit code {poll_res})",
+                        started_at=started_at,
+                        elapsed_seconds=int(time.time() - started_at),
+                        logs=list(self._log_buffer),
+                    )
+                    return
+
+            # Phase 2: Waiting for ADB handshake
+            self._current_state.status = EmulatorLaunchStage.WAITING_FOR_ADB
+            self._current_state.progress_percent = 35
+            self._current_state.stage_message = (
+                "QEMU hypervisor started. Waiting for ADB connection..."
+            )
+
+            while time.time() - started_at < max_wait_seconds:
+                await asyncio.sleep(1.5)
+
+                # Check if process died
+                poll_res = proc.poll()
+                if poll_res is not None:
+                    logs_str = "\n".join(list(self._log_buffer)[-10:])
+                    self._current_state = EmulatorLaunchState(
+                        avd_name=avd_name,
+                        status=EmulatorLaunchStage.FAILED,
+                        pid=proc.pid,
+                        error=f"Emulator process terminated unexpectedly (exit code {poll_res}): {logs_str}",
+                        stage_message="Process terminated unexpectedly",
+                        started_at=started_at,
+                        elapsed_seconds=int(time.time() - started_at),
+                        logs=list(self._log_buffer),
+                    )
+                    return
+
+                # Check adb devices for emulator serial
+                try:
+                    p = await asyncio.create_subprocess_exec(
+                        adb_path,
+                        "devices",
+                        "-l",
+                        stdout=subprocess.PIPE,
+                        stderr=subprocess.PIPE,
+                    )
+                    stdout, _ = await p.communicate()
+                    out_text = stdout.decode(errors="replace")
+
+                    # Scan for emulator-*
+                    for line in out_text.splitlines():
+                        line = line.strip()
+                        if line.startswith("emulator-"):
+                            parts = line.split()
+                            if parts:
+                                detected_serial = parts[0]
+                                break
+                except Exception as e:
+                    logger.debug(f"ADB query error during boot tracking: {e}")
+
+                if detected_serial:
+                    break
+
+            if not detected_serial:
+                self._current_state = EmulatorLaunchState(
+                    avd_name=avd_name,
+                    status=EmulatorLaunchStage.FAILED,
+                    pid=proc.pid,
+                    error="Timeout waiting for emulator to connect to ADB.",
+                    stage_message="ADB handshake timed out.",
+                    started_at=started_at,
+                    elapsed_seconds=int(time.time() - started_at),
+                    logs=list(self._log_buffer),
+                )
+                return
+
+            # Phase 3: Android OS Booting (Polling sys.boot_completed)
+            self._current_state.status = EmulatorLaunchStage.BOOTING
+            self._current_state.serial = detected_serial
+            self._current_state.progress_percent = 65
+            self._current_state.stage_message = (
+                f"Connected to ADB ({detected_serial}). Android OS is booting up..."
+            )
+
+            while time.time() - started_at < max_wait_seconds:
+                await asyncio.sleep(2)
+
+                # Check if process died
+                poll_res = proc.poll()
+                if poll_res is not None:
+                    self._current_state = EmulatorLaunchState(
+                        avd_name=avd_name,
+                        status=EmulatorLaunchStage.FAILED,
+                        pid=proc.pid,
+                        error=f"Emulator process crashed during OS boot (exit code {poll_res}).",
+                        stage_message="Process crashed during OS boot",
+                        started_at=started_at,
+                        elapsed_seconds=int(time.time() - started_at),
+                        logs=list(self._log_buffer),
+                    )
+                    return
+
+                # Check sys.boot_completed
+                try:
+                    p = await asyncio.create_subprocess_exec(
+                        adb_path,
+                        "-s",
+                        detected_serial,
+                        "shell",
+                        "getprop sys.boot_completed",
+                        stdout=subprocess.PIPE,
+                        stderr=subprocess.PIPE,
+                    )
+                    stdout, _ = await p.communicate()
+                    boot_completed = stdout.decode(errors="replace").strip()
+
+                    if boot_completed == "1":
+                        logger.info(
+                            f"[EmulatorManager] Android system boot completed for {detected_serial} ({avd_name})"
+                        )
+                        self._current_state = EmulatorLaunchState(
+                            avd_name=avd_name,
+                            status=EmulatorLaunchStage.READY,
+                            pid=proc.pid,
+                            serial=detected_serial,
+                            stage_message=f"Android emulator '{avd_name}' is fully booted and ready!",
+                            progress_percent=100,
+                            started_at=started_at,
+                            elapsed_seconds=int(time.time() - started_at),
+                            error=None,
+                            logs=list(self._log_buffer),
+                        )
+                        try:
+                            from artemis.core.diagnostics.engine import readiness_engine
+
+                            readiness_engine.set_active_device_serial(detected_serial)
+                        except Exception as e:
+                            logger.debug(f"Failed to auto-select device serial: {e}")
+                        return
+                    else:
+                        # Gradual progress visual indicator
+                        elapsed = time.time() - started_at
+                        calc_progress = min(95, int(65 + (elapsed / 60) * 30))
+                        self._current_state.progress_percent = calc_progress
+                        self._current_state.stage_message = f"Android OS booting ({int(elapsed)}s)... Initializing system services..."
+                except Exception as e:
+                    logger.debug(f"Error querying boot_completed: {e}")
+
+            # Timeout after max_wait_seconds
+            self._current_state = EmulatorLaunchState(
+                avd_name=avd_name,
+                status=EmulatorLaunchStage.FAILED,
+                pid=proc.pid,
+                serial=detected_serial,
+                error=f"Android OS boot did not finish within {max_wait_seconds}s timeout.",
+                stage_message="Boot process timed out.",
+                started_at=started_at,
+                elapsed_seconds=int(time.time() - started_at),
+                logs=list(self._log_buffer),
+            )
+
+        except asyncio.CancelledError:
+            logger.info(f"[EmulatorManager] Boot tracker cancelled for '{avd_name}'")
+        except Exception as exc:
+            logger.error(f"[EmulatorManager] Unexpected exception in boot tracker: {exc}")
+            self._current_state = EmulatorLaunchState(
+                avd_name=avd_name,
+                status=EmulatorLaunchStage.FAILED,
+                pid=proc.pid if proc else None,
+                error=f"Unexpected error: {exc}",
+                stage_message="Unexpected error during boot tracking.",
+                started_at=started_at,
+                elapsed_seconds=int(time.time() - started_at),
+                logs=list(self._log_buffer),
+            )
+
+    async def stop(self) -> dict[str, Any]:
+        """Terminate the active emulator process."""
+        async with self._lock:
+            if self._track_task and not self._track_task.done():
+                self._track_task.cancel()
+
+            serial = self._current_state.serial
+            pid = self._current_state.pid
+            avd = self._current_state.avd_name
+
+            if serial:
+                try:
+                    adb_path = self._locate_adb()
+                    subprocess.run(
+                        [adb_path, "-s", serial, "emu", "kill"],
+                        stdout=subprocess.DEVNULL,
+                        stderr=subprocess.DEVNULL,
+                        timeout=5,
+                    )
+                except Exception:
+                    pass
+
+            if self._proc and self._proc.poll() is None:
+                try:
+                    self._proc.terminate()
+                    try:
+                        self._proc.wait(timeout=3)
+                    except subprocess.TimeoutExpired:
+                        self._proc.kill()
+                except Exception as e:
+                    logger.error(f"Error terminating emulator process {pid}: {e}")
+
+            self._current_state = EmulatorLaunchState(
+                avd_name=avd,
+                status=EmulatorLaunchStage.STOPPED,
+                stage_message="Emulator process stopped by user.",
+                logs=list(self._log_buffer),
+            )
+            return {"success": True, "message": f"Emulator '{avd}' stopped"}
+
+    def dismiss(self) -> dict[str, Any]:
+        """Reset emulator launch state back to idle."""
+        self._current_state = EmulatorLaunchState()
+        self._log_buffer.clear()
+        return {"success": True, "message": "State reset"}
+
+
+# Global singleton instance
+emulator_manager = EmulatorManager()

artemis/core/diagnostics/engine.py

@@ -0,0 +1,228 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""System Readiness & Diagnostic Orchestration Engine."""
+
+import asyncio
+import shutil
+import subprocess
+import time
+from typing import Any
+
+from artemis.core.diagnostics.probes.adb_probe import AdbDeviceProbe
+from artemis.core.diagnostics.probes.base import BaseProbe
+from artemis.core.diagnostics.probes.credentials_probe import (
+    LLMCredentialsProbe,
+    VisionOCRProbe,
+)
+from artemis.core.diagnostics.probes.runtime_probe import (
+    PythonRuntimeProbe,
+    SystemConfigProbe,
+)
+from artemis.core.diagnostics.probes.toolchain_probe import ToolchainProbe
+from artemis.core.diagnostics.schema import (
+    DeviceInfo,
+    ProbeCategory,
+    ProbeResult,
+    ProbeStatus,
+    SystemReadinessReport,
+)
+from artemis.platform import platform
+from artemis.toolchain import toolchain
+from artemis.utils.logger import get_logger
+
+logger = get_logger(__name__)
+
+
+class ReadinessEngine:
+    """Central orchestration engine executing modular readiness probes."""
+
+    def __init__(self):
+        self._probes: dict[str, BaseProbe] = {}
+        self._active_device_serial: str | None = None
+        self._python_probe = PythonRuntimeProbe()
+        self._config_probe = SystemConfigProbe()
+        self._toolchain_probe = ToolchainProbe()
+        self._credentials_probe = LLMCredentialsProbe()
+        self._ocr_probe = VisionOCRProbe()
+        self._adb_probe = AdbDeviceProbe()
+
+        # Register default core probes in logical lifecycle order
+        self.register_probe(self._python_probe)
+        self.register_probe(self._config_probe)
+        self.register_probe(self._toolchain_probe)
+        self.register_probe(self._credentials_probe)
+        self.register_probe(self._ocr_probe)
+        self.register_probe(self._adb_probe)
+
+    def register_probe(self, probe: BaseProbe) -> None:
+        """Register a new diagnostic probe."""
+        self._probes[probe.probe_id] = probe
+
+    def unregister_probe(self, probe_id: str) -> None:
+        """Remove a diagnostic probe."""
+        self._probes.pop(probe_id, None)
+
+    def set_active_device_serial(self, serial: str | None) -> None:
+        """Set user-selected active target device serial."""
+        self._active_device_serial = serial
+        if hasattr(self._adb_probe, "set_target_serial"):
+            self._adb_probe.set_target_serial(serial)
+
+    def get_active_device_serial(self) -> str | None:
+        """Get currently selected active device serial."""
+        return self._active_device_serial
+
+    async def run_probe(self, probe_id: str) -> ProbeResult | None:
+        """Execute a single specific probe by ID."""
+        probe = self._probes.get(probe_id)
+        if not probe:
+            return None
+        return await probe.probe()
+
+    async def run_all(self, categories: list[ProbeCategory] | None = None) -> SystemReadinessReport:
+        """Concurrently run all registered diagnostic probes and compile report."""
+        toolchain.clear_cache()
+        target_probes = [
+            probe
+            for probe in self._probes.values()
+            if categories is None or probe.category in categories
+        ]
+
+        # Concurrently execute probes
+        results: list[ProbeResult] = await asyncio.gather(
+            *[probe.probe() for probe in target_probes],
+            return_exceptions=False,
+        )
+
+        blockers = [r for r in results if r.is_blocker]
+        passed_blockers = [r for r in blockers if r.status == ProbeStatus.PASS]
+        overall_ready = len(blockers) > 0 and len(blockers) == len(passed_blockers)
+
+        # Extract active device info from ADB probe metadata if available
+        active_device: DeviceInfo | None = None
+        adb_result = next((r for r in results if r.id == "android_adb"), None)
+        if adb_result and adb_result.metadata.get("active_device"):
+            try:
+                active_device = DeviceInfo(**adb_result.metadata["active_device"])
+            except Exception:
+                pass
+
+        return SystemReadinessReport(
+            overall_ready=overall_ready,
+            blocker_count=len(blockers),
+            passed_blocker_count=len(passed_blockers),
+            probes=results,
+            active_device=active_device,
+            os_type=platform.os_type.value,
+            timestamp=time.time(),
+        )
+
+    async def restart_adb_server(self) -> dict[str, Any]:
+        """Execute adb kill-server && adb start-server to recover connectivity."""
+        adb_path = shutil.which("adb") or "adb"
+
+        def _restart_sync():
+            try:
+                subprocess.run(
+                    [adb_path, "kill-server"],
+                    stdout=subprocess.DEVNULL,
+                    stderr=subprocess.DEVNULL,
+                    timeout=5,
+                )
+                res = subprocess.run(
+                    [adb_path, "start-server"],
+                    stdout=subprocess.DEVNULL,
+                    stderr=subprocess.DEVNULL,
+                    timeout=5,
+                )
+                success = res.returncode == 0
+                return {
+                    "success": success,
+                    "message": "ADB server restarted successfully"
+                    if success
+                    else "Failed to restart ADB",
+                    "output": "",
+                }
+            except Exception as exc:
+                return {
+                    "success": False,
+                    "message": f"Error executing ADB restart: {exc}",
+                }
+
+        logger.info("[ReadinessEngine] Restarting ADB server...")
+        return await asyncio.to_thread(_restart_sync)
+
+    async def launch_emulator(self, avd_name: str) -> dict[str, Any]:
+        """Launch an Android emulator by AVD name in the background and track its lifecycle."""
+        from artemis.core.diagnostics.emulator_manager import emulator_manager
+
+        state = await emulator_manager.launch(avd_name)
+        return state.model_dump()
+
+    def get_emulator_status(self) -> dict[str, Any]:
+        """Query current status of background emulator launch."""
+        from artemis.core.diagnostics.emulator_manager import emulator_manager
+
+        return emulator_manager.get_status().model_dump()
+
+    async def stop_emulator(self) -> dict[str, Any]:
+        """Stop current running emulator."""
+        from artemis.core.diagnostics.emulator_manager import emulator_manager
+
+        return await emulator_manager.stop()
+
+    def dismiss_emulator(self) -> dict[str, Any]:
+        """Dismiss current emulator launch state."""
+        from artemis.core.diagnostics.emulator_manager import emulator_manager
+
+        return emulator_manager.dismiss()
+
+    async def connect_wireless_adb(self, host: str, port: int = 5555) -> dict[str, Any]:
+        """Connect to an Android device over Wi-Fi via adb connect."""
+        clean_host = host.strip()
+        if not clean_host:
+            return {"success": False, "message": "Host IP address cannot be empty"}
+        target = f"{clean_host}:{port}"
+        adb_path = shutil.which("adb") or "adb"
+
+        def _connect_sync():
+            try:
+                res = subprocess.run(
+                    [adb_path, "connect", target],
+                    capture_output=True,
+                    text=True,
+                    timeout=8,
+                )
+                output = (res.stdout + "\n" + res.stderr).strip()
+                success = "connected to" in output.lower() and "failed" not in output.lower()
+                return {
+                    "success": success,
+                    "target": target,
+                    "output": output,
+                    "message": f"Connected to {target}" if success else output,
+                }
+            except Exception as exc:
+                return {
+                    "success": False,
+                    "target": target,
+                    "message": f"Connection error: {exc}",
+                }
+
+        logger.info(f"[ReadinessEngine] Connecting to wireless ADB target '{target}'...")
+        return await asyncio.to_thread(_connect_sync)
+
+
+# Global singleton instance
+readiness_engine = ReadinessEngine()

artemis/core/diagnostics/probes/adb_probe.py

@@ -0,0 +1,449 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Android ADB & Device Connectivity Probe with AVD Detection."""
+
+import asyncio
+import os
+from pathlib import Path
+import shutil
+import subprocess
+from typing import Any
+
+from artemis.core.diagnostics.probes.base import BaseProbe
+from artemis.core.diagnostics.schema import (
+    DeviceInfo,
+    ProbeAction,
+    ProbeCategory,
+    ProbeResult,
+    ProbeStatus,
+)
+from artemis.platform import OSType, platform
+from artemis.toolchain import toolchain
+from artemis.utils.logger import get_logger
+
+logger = get_logger(__name__)
+
+
+class AdbDeviceProbe(BaseProbe):
+    """Deep inspection probe for Android Debug Bridge, connected mobile devices, and local AVD emulators."""
+
+    def __init__(self, target_serial: str | None = None):
+        self._target_serial = target_serial
+
+    @property
+    def probe_id(self) -> str:
+        return "android_adb"
+
+    @property
+    def category(self) -> ProbeCategory:
+        return ProbeCategory.DEVICE
+
+    @property
+    def is_blocker(self) -> bool:
+        return True
+
+    def set_target_serial(self, serial: str | None) -> None:
+        """Set or update preferred active device serial."""
+        self._target_serial = serial
+
+    def _locate_adb(self) -> str | None:
+        """Find the adb binary path from ToolchainResolver."""
+        return toolchain.resolve("adb", force_refresh=True)
+
+    def _locate_emulator(self) -> str | None:
+        """Find the emulator binary path from PATH or standard SDK environments."""
+        emu_path = shutil.which("emulator")
+        if emu_path:
+            return emu_path
+
+        sdk_candidates = [
+            os.environ.get("ANDROID_HOME"),
+            os.environ.get("ANDROID_SDK_ROOT"),
+            str(Path.home() / "Android" / "Sdk"),
+            str(Path.home() / "Android" / "sdk"),
+            str(Path.home() / "Library" / "Android" / "sdk"),
+            os.getenv("LOCALAPPDATA", "") + "/Android/Sdk" if os.getenv("LOCALAPPDATA") else None,
+            "/usr/lib/android-sdk",
+            "/opt/android-sdk",
+        ]
+        for base in sdk_candidates:
+            if base:
+                cand = (
+                    Path(base)
+                    / "emulator"
+                    / ("emulator.exe" if platform.os_type == OSType.WINDOWS else "emulator")
+                )
+                if cand.is_file():
+                    return str(cand)
+        return None
+
+    def _list_installed_avds(self, emulator_path: str | None = None) -> list[str]:
+        """Discover all locally created Android Virtual Devices (AVDs)."""
+        avd_names: list[str] = []
+
+        # Method 1: Scan ~/.android/avd/*.ini
+        avd_dir = Path.home() / ".android" / "avd"
+        if avd_dir.is_dir():
+            for ini_file in avd_dir.glob("*.ini"):
+                name = ini_file.stem
+                if name and name not in avd_names:
+                    avd_names.append(name)
+
+        return avd_names
+
+    async def _get_adb_version(self, adb_path: str) -> str:
+        """Query ADB version string."""
+        try:
+            proc = await asyncio.create_subprocess_exec(
+                adb_path,
+                "version",
+                stdout=subprocess.PIPE,
+                stderr=subprocess.PIPE,
+            )
+            stdout, _ = await proc.communicate()
+            if stdout:
+                first_line = stdout.decode(errors="replace").strip().splitlines()[0]
+                return first_line
+        except Exception as e:
+            logger.debug(f"Failed to read adb version: {e}")
+        return "Installed"
+
+    async def _parse_adb_devices(self, adb_path: str) -> list[DeviceInfo]:
+        """Execute `adb devices -l` and extract structured device metadata."""
+        devices: list[DeviceInfo] = []
+        try:
+            proc = await asyncio.create_subprocess_exec(
+                adb_path,
+                "devices",
+                "-l",
+                stdout=subprocess.PIPE,
+                stderr=subprocess.PIPE,
+            )
+            stdout, _ = await proc.communicate()
+            if proc.returncode != 0 or not stdout:
+                return devices
+
+            lines = [
+                line.strip()
+                for line in stdout.decode(errors="replace").splitlines()
+                if line.strip()
+            ]
+            for line in lines[1:]:  # Skip "List of devices attached"
+                parts = line.split()
+                if len(parts) < 2:
+                    continue
+
+                serial = parts[0]
+                state = parts[1]
+                extra_tokens = parts[2:]
+
+                model = None
+                product = None
+                for token in extra_tokens:
+                    if token.startswith("model:"):
+                        model = token.split(":", 1)[1].replace("_", " ")
+                    elif token.startswith("product:"):
+                        product = token.split(":", 1)[1]
+
+                is_emulator = (
+                    serial.startswith("emulator-") or "127.0.0.1" in serial or "localhost" in serial
+                )
+                if is_emulator and not model:
+                    model = "Android Emulator"
+
+                device_info = DeviceInfo(
+                    serial=serial,
+                    state=state,
+                    model=model,
+                    product=product,
+                    is_emulator=is_emulator,
+                )
+                devices.append(device_info)
+
+        except Exception as e:
+            logger.error(f"Error querying adb devices: {e}")
+
+        # For authorized devices, attempt non-blocking quick property enrichment
+        for dev in devices:
+            if dev.state == "device":
+                try:
+                    p_prop = await asyncio.create_subprocess_exec(
+                        adb_path,
+                        "-s",
+                        dev.serial,
+                        "shell",
+                        "getprop ro.build.version.release",
+                        stdout=subprocess.PIPE,
+                        stderr=subprocess.PIPE,
+                    )
+                    p_size = await asyncio.create_subprocess_exec(
+                        adb_path,
+                        "-s",
+                        dev.serial,
+                        "shell",
+                        "wm size",
+                        stdout=subprocess.PIPE,
+                        stderr=subprocess.PIPE,
+                    )
+                    out_prop, _ = await p_prop.communicate()
+                    out_size, _ = await p_size.communicate()
+
+                    if out_prop:
+                        dev.android_version = out_prop.decode(errors="replace").strip()
+                    if out_size:
+                        size_str = out_size.decode(errors="replace").strip()
+                        if "Physical size:" in size_str:
+                            dev.screen_resolution = size_str.split("Physical size:")[-1].strip()
+
+                    # Query installed application packages with timeout protection
+                    try:
+                        p_pkg = await asyncio.create_subprocess_exec(
+                            adb_path,
+                            "-s",
+                            dev.serial,
+                            "shell",
+                            "pm list packages",
+                            stdout=subprocess.PIPE,
+                            stderr=subprocess.PIPE,
+                        )
+                        out_pkg, _ = await asyncio.wait_for(p_pkg.communicate(), timeout=2.5)
+                        if out_pkg:
+                            pkgs = [
+                                line.split("package:", 1)[1].strip()
+                                for line in out_pkg.decode(errors="replace").splitlines()
+                                if line.startswith("package:")
+                                and line.split("package:", 1)[1].strip()
+                            ]
+                            dev.installed_packages = pkgs
+                    except Exception as e_pkg:
+                        logger.debug(f"Failed to query packages for {dev.serial}: {e_pkg}")
+                except Exception:
+                    pass
+
+        return devices
+
+    async def probe(self) -> ProbeResult:
+        """Run complete ADB, device, and local AVD connectivity probe."""
+        adb_path = self._locate_adb()
+        emulator_path = self._locate_emulator()
+        installed_avds = self._list_installed_avds(emulator_path)
+
+        # 1. ADB binary check
+        if not adb_path:
+            actions = []
+            if platform.os_type == OSType.WINDOWS:
+                actions.append(
+                    ProbeAction(
+                        action_type="command",
+                        label="Install via WinGet",
+                        payload="winget install Google.PlatformTools",
+                    )
+                )
+                actions.append(
+                    ProbeAction(
+                        action_type="command",
+                        label="Run PowerShell Setup",
+                        payload="powershell -ExecutionPolicy Bypass -File scripts/install_deps.ps1",
+                    )
+                )
+            elif platform.os_type == OSType.MACOS:
+                actions.append(
+                    ProbeAction(
+                        action_type="command",
+                        label="Install via Homebrew",
+                        payload="brew install android-platform-tools",
+                    )
+                )
+            else:
+                actions.append(
+                    ProbeAction(
+                        action_type="command",
+                        label="Linux Quick Install",
+                        payload="sudo apt-get install -y adb",
+                    )
+                )
+                actions.append(
+                    ProbeAction(
+                        action_type="command",
+                        label="One-Click Install All",
+                        payload="bash scripts/install_deps.sh",
+                    )
+                )
+
+            actions.append(
+                ProbeAction(
+                    action_type="link",
+                    label="Download Android Platform Tools",
+                    payload="https://developer.android.com/tools/releases/platform-tools",
+                )
+            )
+
+            return ProbeResult(
+                id=self.probe_id,
+                category=self.category,
+                title="Device / Emulator Connected",
+                status=ProbeStatus.FAIL,
+                is_blocker=self.is_blocker,
+                summary="ADB Not Found",
+                description="Android Debug Bridge (adb) is not installed or not in PATH.",
+                metadata={"installed": False},
+                actions=actions,
+            )
+
+        adb_ver = await self._get_adb_version(adb_path)
+        devices = await self._parse_adb_devices(adb_path)
+
+        # Normalize emulator command display
+        is_emu_in_path = shutil.which("emulator") is not None
+        emu_display_cmd = "emulator" if is_emu_in_path else (emulator_path or "emulator")
+
+        metadata: dict[str, Any] = {
+            "installed": True,
+            "adb_path": adb_path,
+            "adb_version": adb_ver,
+            "emulator_path": emulator_path,
+            "is_emulator_in_path": is_emu_in_path,
+            "installed_avds": installed_avds,
+            "device_count": len(devices),
+            "devices": [d.model_dump() for d in devices],
+        }
+
+        # 2. Case: No devices found
+        if not devices:
+            actions: list[ProbeAction] = []
+
+            # If user has installed AVDs, generate exact launch commands for each
+            if installed_avds:
+                for avd in installed_avds:
+                    actions.append(
+                        ProbeAction(
+                            action_type="command",
+                            label=f"Launch {avd}",
+                            payload=f"{emu_display_cmd} -avd {avd}",
+                        )
+                    )
+            else:
+                actions.append(
+                    ProbeAction(
+                        action_type="command",
+                        label="Start Default Emulator",
+                        payload=f"{emu_display_cmd} -avd Pixel_8_API_34",
+                    )
+                )
+
+            actions.append(
+                ProbeAction(
+                    action_type="hint",
+                    label="Connect via USB",
+                    payload="Connect your Android phone via USB cable and enable Developer Options -> USB Debugging.",
+                )
+            )
+
+            desc = "ADB is ready, but no active Android device or emulator was detected."
+            if installed_avds:
+                desc += f" Found {len(installed_avds)} installed emulator(s): {', '.join(installed_avds)}."
+
+            return ProbeResult(
+                id=self.probe_id,
+                category=self.category,
+                title="Device / Emulator Connected",
+                status=ProbeStatus.WARN,
+                is_blocker=self.is_blocker,
+                summary="No Device Found",
+                description=desc,
+                metadata=metadata,
+                actions=actions,
+            )
+
+        # 3. Check for authorized and ready devices
+        ready_devices = [d for d in devices if d.state == "device"]
+        unauthorized_devices = [d for d in devices if d.state == "unauthorized"]
+
+        if not ready_devices:
+            if unauthorized_devices:
+                unauth_serial = unauthorized_devices[0].serial
+                return ProbeResult(
+                    id=self.probe_id,
+                    category=self.category,
+                    title="Device / Emulator Connected",
+                    status=ProbeStatus.WARN,
+                    is_blocker=self.is_blocker,
+                    summary="Device Unauthorized",
+                    description=f"Device detected ({unauth_serial}), but USB debugging is not yet authorized.",
+                    metadata=metadata,
+                    actions=[
+                        ProbeAction(
+                            action_type="hint",
+                            label="Authorize USB Debugging",
+                            payload="Please unlock your Android phone and tap 'Allow' on the USB Debugging permission prompt.",
+                        ),
+                        ProbeAction(
+                            action_type="command",
+                            label="Restart ADB Server",
+                            payload=f"{adb_path} kill-server && {adb_path} start-server",
+                        ),
+                    ],
+                )
+            else:
+                first_serial = devices[0].serial
+                first_state = devices[0].state
+                return ProbeResult(
+                    id=self.probe_id,
+                    category=self.category,
+                    title="Device / Emulator Connected",
+                    status=ProbeStatus.WARN,
+                    is_blocker=self.is_blocker,
+                    summary="Device Booting",
+                    description=f"Device detected ({first_serial}, state: {first_state}). It is currently booting up or offline. Please wait a few seconds...",
+                    metadata=metadata,
+                    actions=[
+                        ProbeAction(
+                            action_type="hint",
+                            label="Device Booting",
+                            payload=f"Device {first_serial} is initializing. It will be ready automatically in a few seconds.",
+                        )
+                    ],
+                )
+
+        # 4. Ready device available
+        active_dev = ready_devices[0]
+        if self._target_serial:
+            matched = next((d for d in ready_devices if d.serial == self._target_serial), None)
+            if matched:
+                active_dev = matched
+
+        display_name = active_dev.model or active_dev.serial
+        if active_dev.screen_resolution:
+            display_name = f"{display_name} ({active_dev.screen_resolution})"
+
+        metadata["active_device"] = active_dev.model_dump()
+
+        return ProbeResult(
+            id=self.probe_id,
+            category=self.category,
+            title="Device / Emulator Connected",
+            status=ProbeStatus.PASS,
+            is_blocker=self.is_blocker,
+            summary="Connected",
+            description=f"ADB connected to {display_name}. Ready for perception and touch automation.",
+            metadata=metadata,
+            actions=[
+                ProbeAction(
+                    action_type="hint",
+                    label="Device Ready",
+                    payload=f"Serial: {active_dev.serial} | Android OS: {active_dev.android_version or 'Unknown'}",
+                )
+            ],
+        )

artemis/core/diagnostics/probes/base.py

@@ -0,0 +1,45 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Abstract Base Probe for Artemis System Diagnostics."""
+
+from abc import ABC, abstractmethod
+from artemis.core.diagnostics.schema import ProbeCategory, ProbeResult
+
+
+class BaseProbe(ABC):
+    """Abstract interface representing a system readiness probe."""
+
+    @property
+    @abstractmethod
+    def probe_id(self) -> str:
+        """Unique identifier for this probe."""
+        pass
+
+    @property
+    @abstractmethod
+    def category(self) -> ProbeCategory:
+        """Category to which this probe belongs."""
+        pass
+
+    @property
+    @abstractmethod
+    def is_blocker(self) -> bool:
+        """Whether a failure in this probe should block autonomous task execution."""
+        pass
+
+    @abstractmethod
+    async def probe(self) -> ProbeResult:
+        """Execute the probe inspection and return a structured ProbeResult."""
+        pass

artemis/core/diagnostics/probes/credentials_probe.py

@@ -0,0 +1,296 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""LLM & Multimodal Vision Credentials Readiness Probe."""
+
+from typing import Any
+from artemis.config import settings
+from artemis.core.diagnostics.probes.base import BaseProbe
+from artemis.core.diagnostics.schema import (
+    ProbeAction,
+    ProbeCategory,
+    ProbeResult,
+    ProbeStatus,
+)
+
+
+class LLMCredentialsProbe(BaseProbe):
+    """Probe verifying Gemini and multi-provider multimodal LLM credentials."""
+
+    @property
+    def probe_id(self) -> str:
+        return "gemini_api_key"
+
+    @property
+    def category(self) -> ProbeCategory:
+        return ProbeCategory.CREDENTIALS
+
+    @property
+    def is_blocker(self) -> bool:
+        return True
+
+    def _mask_key(self, key_str: str) -> str:
+        """Helper to safely mask an API credential for display."""
+        if len(key_str) > 10:
+            return f"{key_str[:6]}...{key_str[-4:]}"
+        return "***"
+
+    async def probe(self) -> ProbeResult:
+        import os
+
+        from artemis.config.settings import is_placeholder_key
+
+        gemini_key = settings.get_api_key("google")
+        openai_key = settings.get_api_key("openai")
+        claude_key = settings.get_api_key("anthropic")
+        openrouter_key = settings.get_api_key("openrouter")
+        xai_key = settings.get_api_key("xai")
+        ocr_key = settings.get_api_key("ocr")
+
+        configured_providers: list[dict[str, Any]] = []
+        api_keys_map: dict[str, str] = {}
+        if gemini_key and not is_placeholder_key(gemini_key):
+            g_val = gemini_key.get_secret_value()
+            configured_providers.append(
+                {
+                    "provider": "google",
+                    "label": "Gemini",
+                    "masked": self._mask_key(g_val),
+                    "raw_key": g_val,
+                    "key": g_val,
+                }
+            )
+            api_keys_map["google"] = g_val
+            api_keys_map["gemini"] = g_val
+        if openai_key and not is_placeholder_key(openai_key):
+            o_val = openai_key.get_secret_value()
+            configured_providers.append(
+                {
+                    "provider": "openai",
+                    "label": "ChatGPT",
+                    "masked": self._mask_key(o_val),
+                    "raw_key": o_val,
+                    "key": o_val,
+                }
+            )
+            api_keys_map["openai"] = o_val
+        if claude_key and not is_placeholder_key(claude_key):
+            c_val = claude_key.get_secret_value()
+            configured_providers.append(
+                {
+                    "provider": "anthropic",
+                    "label": "Claude",
+                    "masked": self._mask_key(c_val),
+                    "raw_key": c_val,
+                    "key": c_val,
+                }
+            )
+            api_keys_map["anthropic"] = c_val
+        if openrouter_key and not is_placeholder_key(openrouter_key):
+            or_val = openrouter_key.get_secret_value()
+            configured_providers.append(
+                {
+                    "provider": "openrouter",
+                    "label": "OpenRouter",
+                    "masked": self._mask_key(or_val),
+                    "raw_key": or_val,
+                    "key": or_val,
+                }
+            )
+            api_keys_map["openrouter"] = or_val
+        if xai_key and not is_placeholder_key(xai_key):
+            x_val = xai_key.get_secret_value()
+            configured_providers.append(
+                {
+                    "provider": "xai",
+                    "label": "xAI (Grok)",
+                    "masked": self._mask_key(x_val),
+                    "raw_key": x_val,
+                    "key": x_val,
+                }
+            )
+            api_keys_map["xai"] = x_val
+
+        # Detect any custom model endpoints or environment variables defined in files
+        for env_var, label, prov_id in [
+            ("DEEPSEEK_API_KEY", "DeepSeek", "deepseek"),
+            ("GROQ_API_KEY", "Groq", "groq"),
+            ("OPENAI_BASE_URL", "Custom OpenAI Endpoint", "custom"),
+            ("OLLAMA_BASE_URL", "Local Ollama", "ollama"),
+            ("VLLM_BASE_URL", "vLLM Endpoint", "vllm"),
+            ("VERTEX_AI_PROJECT", "Google Cloud Vertex AI", "vertexai"),
+        ]:
+            val = os.environ.get(env_var)
+            if val and val.strip() and not is_placeholder_key(val.strip()):
+                configured_providers.append(
+                    {
+                        "provider": prov_id,
+                        "label": label,
+                        "masked": self._mask_key(val.strip()),
+                        "raw_key": val.strip(),
+                        "key": val.strip(),
+                    }
+                )
+                api_keys_map[prov_id] = val.strip()
+
+        if ocr_key and not is_placeholder_key(ocr_key):
+            api_keys_map["ocr"] = ocr_key.get_secret_value()
+
+        current_active_key = (
+            gemini_key.get_secret_value()
+            if gemini_key
+            else (configured_providers[0]["key"] if configured_providers else "")
+        )
+
+        metadata = {
+            "configured_count": len(configured_providers),
+            "providers": configured_providers,
+            "has_ocr_key": ocr_key is not None,
+            "current_key": current_active_key,
+            "current_gemini_key": gemini_key.get_secret_value() if gemini_key else "",
+            "api_keys": api_keys_map,
+        }
+
+        # Case 1: Gemini API Key configured (Standard / Recommended)
+        if gemini_key:
+            masked = self._mask_key(gemini_key.get_secret_value())
+            return ProbeResult(
+                id=self.probe_id,
+                category=self.category,
+                title="Multimodal LLM API Key",
+                status=ProbeStatus.PASS,
+                is_blocker=self.is_blocker,
+                summary="Active (Gemini)",
+                description=f"Gemini multimodal API credential is active ({masked}) and ready for vision and reasoning.",
+                metadata=metadata,
+                actions=[
+                    ProbeAction(
+                        action_type="hint",
+                        label="Provider Active",
+                        payload="Gemini 2.5 Flash / Pro Multimodal Engine is enabled.",
+                    )
+                ],
+            )
+
+        # Case 2: Other LLM provider configured
+        if configured_providers:
+            first_p = configured_providers[0]
+            return ProbeResult(
+                id=self.probe_id,
+                category=self.category,
+                title="Multimodal LLM API Key",
+                status=ProbeStatus.PASS,
+                is_blocker=self.is_blocker,
+                summary=f"Active ({first_p['label']})",
+                description=f"{first_p['label']} multimodal API credential is active ({first_p['masked']}).",
+                metadata=metadata,
+                actions=[
+                    ProbeAction(
+                        action_type="hint",
+                        label="Provider Active",
+                        payload=f"{first_p['label']} multimodal vision endpoint is active.",
+                    )
+                ],
+            )
+
+        # Case 3: No LLM key configured
+        return ProbeResult(
+            id=self.probe_id,
+            category=self.category,
+            title="Multimodal LLM API Key",
+            status=ProbeStatus.FAIL,
+            is_blocker=self.is_blocker,
+            summary="Key Missing",
+            description="No Multimodal LLM credential (e.g. GEMINI_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY) found in environment or .env file.",
+            metadata=metadata,
+            actions=[
+                ProbeAction(
+                    action_type="command",
+                    label="Run Artemis Init",
+                    payload="artemis init",
+                ),
+                ProbeAction(
+                    action_type="link",
+                    label="Get Google AI Studio Key",
+                    payload="https://aistudio.google.com/app/apikey",
+                ),
+            ],
+        )
+
+
+class VisionOCRProbe(BaseProbe):
+    """Probe verifying optional Google Cloud Vision OCR credentials."""
+
+    @property
+    def probe_id(self) -> str:
+        return "vision_ocr_key"
+
+    @property
+    def category(self) -> ProbeCategory:
+        return ProbeCategory.CREDENTIALS
+
+    @property
+    def is_blocker(self) -> bool:
+        return False
+
+    def _mask_key(self, key_str: str) -> str:
+        """Helper to safely mask an API credential for display."""
+        if len(key_str) > 10:
+            return f"{key_str[:6]}...{key_str[-4:]}"
+        return "***"
+
+    async def probe(self) -> ProbeResult:
+        from artemis.utils.ocr_api import is_ocr_configured
+
+        ocr_key = settings.get_api_key("ocr")
+        is_configured = is_ocr_configured() and ocr_key is not None
+
+        if is_configured and ocr_key:
+            val = ocr_key.get_secret_value()
+            masked = self._mask_key(val)
+            return ProbeResult(
+                id=self.probe_id,
+                category=self.category,
+                title="Vision OCR API Key (Optional)",
+                status=ProbeStatus.PASS,
+                is_blocker=False,
+                summary="Active & Configured",
+                description=f"Google Cloud Vision OCR ({masked}) is active for image text recognition.",
+                metadata={"configured": True, "masked_key": masked, "key": val, "raw_key": val},
+                actions=[
+                    ProbeAction(
+                        action_type="hint",
+                        label="OCR Enabled",
+                        payload="OCR is active and will fuse text with UI XML hierarchy.",
+                    )
+                ],
+            )
+
+        return ProbeResult(
+            id=self.probe_id,
+            category=self.category,
+            title="Vision OCR API Key (Optional)",
+            status=ProbeStatus.PASS,
+            is_blocker=False,
+            summary="Not Configured (Optional)",
+            description="OCR_API_KEY is not set. Perception operates seamlessly in pure UI XML hierarchy mode.",
+            metadata={"configured": False},
+            actions=[
+                ProbeAction(
+                    action_type="hint",
+                    label="Standard XML Perception",
+                    payload="Artemis uses pure UI layout parsing and Set-of-Marks visual grounding.",
+                )
+            ],
+        )

artemis/core/diagnostics/probes/runtime_probe.py

@@ -0,0 +1,169 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Python Runtime and System Configuration Diagnostic Probes."""
+
+import platform
+import sys
+
+from artemis.config import parse_llm_config
+from artemis.core.diagnostics.probes.base import BaseProbe
+from artemis.core.diagnostics.schema import (
+    ProbeAction,
+    ProbeCategory,
+    ProbeResult,
+    ProbeStatus,
+)
+
+
+class PythonRuntimeProbe(BaseProbe):
+    """Probe inspecting Python runtime environment, version, and architecture."""
+
+    @property
+    def probe_id(self) -> str:
+        return "python_runtime"
+
+    @property
+    def category(self) -> ProbeCategory:
+        return ProbeCategory.RUNTIME
+
+    @property
+    def is_blocker(self) -> bool:
+        return True
+
+    async def probe(self) -> ProbeResult:
+        py_ver_tuple = sys.version_info
+        py_ver_str = f"{py_ver_tuple.major}.{py_ver_tuple.minor}.{py_ver_tuple.micro}"
+        is_supported = py_ver_tuple >= (3, 12)
+        arch = platform.machine()
+        executable = sys.executable
+
+        metadata = {
+            "version": py_ver_str,
+            "major": py_ver_tuple.major,
+            "minor": py_ver_tuple.minor,
+            "micro": py_ver_tuple.micro,
+            "executable": executable,
+            "architecture": arch,
+            "platform": sys.platform,
+            "is_supported": is_supported,
+        }
+
+        if is_supported:
+            return ProbeResult(
+                id=self.probe_id,
+                category=self.category,
+                title="Python Runtime Environment",
+                status=ProbeStatus.PASS,
+                is_blocker=self.is_blocker,
+                summary=f"Python {py_ver_str} Ready",
+                description=f"Python {py_ver_str} ({arch}, 64-bit) detected at {executable}. Satisfies requirement (>= 3.12).",
+                metadata=metadata,
+                actions=[
+                    ProbeAction(
+                        action_type="hint",
+                        label="Python Runtime OK",
+                        payload=f"Python {py_ver_str} ({arch})",
+                    )
+                ],
+            )
+        else:
+            return ProbeResult(
+                id=self.probe_id,
+                category=self.category,
+                title="Python Runtime Environment",
+                status=ProbeStatus.FAIL,
+                is_blocker=self.is_blocker,
+                summary=f"Python {py_ver_str} Unsupported",
+                description=f"Python {py_ver_str} detected. Artemis requires Python >= 3.12.",
+                metadata=metadata,
+                actions=[
+                    ProbeAction(
+                        action_type="link",
+                        label="Download Python 3.12+",
+                        payload="https://www.python.org/downloads/",
+                    ),
+                    ProbeAction(
+                        action_type="command",
+                        label="Install with uv",
+                        payload="uv python install 3.12",
+                    ),
+                ],
+            )
+
+
+class SystemConfigProbe(BaseProbe):
+    """Probe verifying config/artemis.jsonc and settings integrity."""
+
+    @property
+    def probe_id(self) -> str:
+        return "system_config"
+
+    @property
+    def category(self) -> ProbeCategory:
+        return ProbeCategory.RUNTIME
+
+    @property
+    def is_blocker(self) -> bool:
+        return True
+
+    async def probe(self) -> ProbeResult:
+        try:
+            cfg = parse_llm_config()
+            planner_model = getattr(getattr(cfg, "planner", None), "model", "Gemini 2.5 Flash")
+            metadata = {
+                "valid": True,
+                "config_file": "config/artemis.jsonc",
+                "planner_model": planner_model,
+            }
+            return ProbeResult(
+                id=self.probe_id,
+                category=self.category,
+                title="Artemis System Configuration",
+                status=ProbeStatus.PASS,
+                is_blocker=self.is_blocker,
+                summary="Config Valid",
+                description=f"System configuration loaded and validated (Planner model: {planner_model}).",
+                metadata=metadata,
+                actions=[
+                    ProbeAction(
+                        action_type="hint",
+                        label="Config Healthy",
+                        payload="config/artemis.jsonc is loaded and valid.",
+                    )
+                ],
+            )
+        except Exception as e:
+            metadata = {
+                "valid": False,
+                "config_file": "config/artemis.jsonc",
+                "error": str(e),
+            }
+            return ProbeResult(
+                id=self.probe_id,
+                category=self.category,
+                title="Artemis System Configuration",
+                status=ProbeStatus.FAIL,
+                is_blocker=self.is_blocker,
+                summary="Config Error",
+                description=f"Failed to parse config/artemis.jsonc: {e}",
+                metadata=metadata,
+                actions=[
+                    ProbeAction(
+                        action_type="command",
+                        label="Run Artemis Init",
+                        payload="artemis init",
+                    )
+                ],
+            )

artemis/core/diagnostics/probes/toolchain_probe.py

@@ -0,0 +1,139 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Toolchain and Video Recording Auxiliary Probe."""
+
+from artemis.core.diagnostics.probes.base import BaseProbe
+from artemis.core.diagnostics.schema import (
+    ProbeAction,
+    ProbeCategory,
+    ProbeResult,
+    ProbeStatus,
+)
+from artemis.platform import OSType, platform
+from artemis.toolchain import toolchain
+
+
+class ToolchainProbe(BaseProbe):
+    """Probe checking required multimedia and video recording toolchains (ffmpeg, scrcpy)."""
+
+    @property
+    def probe_id(self) -> str:
+        return "toolchain"
+
+    @property
+    def category(self) -> ProbeCategory:
+        return ProbeCategory.TOOLCHAIN
+
+    @property
+    def is_blocker(self) -> bool:
+        return False
+
+    async def probe(self) -> ProbeResult:
+        ffmpeg_path = toolchain.resolve("ffmpeg", force_refresh=True)
+        scrcpy_path = toolchain.resolve("scrcpy", force_refresh=True)
+
+        tools_installed = []
+        tools_missing = []
+        if ffmpeg_path:
+            tools_installed.append("FFmpeg")
+        else:
+            tools_missing.append("FFmpeg")
+
+        if scrcpy_path:
+            tools_installed.append("scrcpy")
+        else:
+            tools_missing.append("scrcpy")
+
+        metadata = {
+            "ffmpeg": ffmpeg_path is not None,
+            "scrcpy": scrcpy_path is not None,
+            "ffmpeg_path": ffmpeg_path,
+            "scrcpy_path": scrcpy_path,
+            "tools_installed": tools_installed,
+            "tools_missing": tools_missing,
+        }
+
+        if ffmpeg_path and scrcpy_path:
+            return ProbeResult(
+                id=self.probe_id,
+                category=self.category,
+                title="Video Recording Toolchain",
+                status=ProbeStatus.PASS,
+                is_blocker=self.is_blocker,
+                summary="Ready (FFmpeg + scrcpy)",
+                description=f"FFmpeg ({ffmpeg_path}) and scrcpy ({scrcpy_path}) are available. Live stream and high-speed video replays are active.",
+                metadata=metadata,
+                actions=[
+                    ProbeAction(
+                        action_type="hint",
+                        label="Toolchain Active",
+                        payload="FFmpeg and scrcpy are installed and active.",
+                    )
+                ],
+            )
+
+        missing_str = " & ".join(tools_missing)
+        actions = []
+
+        if platform.os_type == OSType.WINDOWS:
+            actions.append(
+                ProbeAction(
+                    action_type="command",
+                    label="Install via WinGet",
+                    payload="winget install Gyan.FFmpeg Genymobile.scrcpy",
+                )
+            )
+            actions.append(
+                ProbeAction(
+                    action_type="command",
+                    label="Run PowerShell Setup",
+                    payload="powershell -ExecutionPolicy Bypass -File scripts/install_deps.ps1",
+                )
+            )
+        elif platform.os_type == OSType.MACOS:
+            actions.append(
+                ProbeAction(
+                    action_type="command",
+                    label="Install via Homebrew",
+                    payload="brew install ffmpeg scrcpy",
+                )
+            )
+        else:
+            actions.append(
+                ProbeAction(
+                    action_type="command",
+                    label="Install on Linux",
+                    payload="sudo apt-get install -y ffmpeg scrcpy",
+                )
+            )
+            actions.append(
+                ProbeAction(
+                    action_type="command",
+                    label="One-Click Install Script",
+                    payload="bash scripts/install_deps.sh",
+                )
+            )
+
+        return ProbeResult(
+            id=self.probe_id,
+            category=self.category,
+            title="Video Recording Toolchain",
+            status=ProbeStatus.FAIL,
+            is_blocker=self.is_blocker,
+            summary=f"Missing {missing_str}",
+            description=f"Video toolchain is required for screen streaming and test replay recording. Missing: {missing_str}.",
+            metadata=metadata,
+            actions=actions,
+        )

artemis/core/diagnostics/schema.py

@@ -0,0 +1,114 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Schema definitions for Artemis System Diagnostics & Readiness Engine."""
+
+from enum import Enum
+from typing import Any, Literal
+from pydantic import BaseModel, Field
+
+
+class ProbeStatus(str, Enum):
+    """Execution status of a readiness probe."""
+
+    PASS = "pass"  # 🟢 Ready for execution
+    WARN = "warn"  # 🟡 Warning / Non-blocking prerequisite missing (e.g. no devices attached)
+    FAIL = "fail"  # 🔴 Blocking prerequisite failure (e.g. ADB missing, no API key)
+    SKIPPED = "skipped"  # ⚪ Skipped probe
+
+
+class ProbeCategory(str, Enum):
+    """Category of system readiness check."""
+
+    DEVICE = "device"  # Hardware & Device Driver (ADB, Emulator, Cloud device)
+    CREDENTIALS = "auth"  # LLM API Keys & OCR Credentials
+    TOOLCHAIN = "toolchain"  # External tools (FFmpeg, scrcpy, etc.)
+    RUNTIME = "runtime"  # Python runtime & Config health
+
+
+class ProbeAction(BaseModel):
+    """Actionable guidance payload for quick remediation or user guidance."""
+
+    action_type: Literal["command", "hint", "link"] = Field(
+        description="Type of remediation action ('command' to copy/run, 'hint' for guidance, 'link' for docs)"
+    )
+    label: str = Field(description="Display label for the action button or badge")
+    payload: str = Field(description="Command string to execute/copy or explanation hint")
+
+
+class DeviceInfo(BaseModel):
+    """Structured Android device / emulator metadata."""
+
+    serial: str = Field(description="Device serial number or emulator port ID (e.g. emulator-5554)")
+    state: str = Field(
+        default="device", description="Connection state ('device', 'unauthorized', 'offline')"
+    )
+    model: str | None = Field(
+        default=None, description="Device model name (e.g. Pixel 8, sdk_gphone64_arm64)"
+    )
+    product: str | None = Field(default=None, description="Product code name")
+    android_version: str | None = Field(
+        default=None, description="Android OS release version (e.g. 14)"
+    )
+    screen_resolution: str | None = Field(
+        default=None, description="Screen width x height (e.g. 1080x2400)"
+    )
+    is_emulator: bool = Field(default=False, description="Whether the device is an emulator")
+    installed_packages: list[str] = Field(
+        default_factory=list, description="List of recognized installed package names"
+    )
+
+
+class ProbeResult(BaseModel):
+    """Outcome of a single readiness probe check."""
+
+    id: str = Field(
+        description="Unique identifier for the probe (e.g. 'android_adb', 'gemini_api_key')"
+    )
+    category: ProbeCategory = Field(description="Category of the probe")
+    title: str = Field(description="Human-readable title (e.g. 'Device / Emulator Connected')")
+    status: ProbeStatus = Field(description="Current status (pass, warn, fail)")
+    is_blocker: bool = Field(
+        default=True, description="Whether passing this probe is strictly required to run tasks"
+    )
+    summary: str = Field(
+        description="Short badge summary text (e.g. 'Connected', 'No Device', 'Active & Ready')"
+    )
+    description: str = Field(description="Detailed explanation of the status")
+    metadata: dict[str, Any] = Field(
+        default_factory=dict, description="Arbitrary extra metadata (e.g. device list)"
+    )
+    actions: list[ProbeAction] = Field(
+        default_factory=list, description="List of actionable remediation steps"
+    )
+
+
+class SystemReadinessReport(BaseModel):
+    """Aggregated system readiness report across all probes."""
+
+    overall_ready: bool = Field(description="True if all blocker probes are passing")
+    blocker_count: int = Field(description="Total count of required blocker probes")
+    passed_blocker_count: int = Field(
+        description="Count of required blocker probes currently passing"
+    )
+    probes: list[ProbeResult] = Field(
+        default_factory=list, description="Ordered list of probe results"
+    )
+    active_device: DeviceInfo | None = Field(
+        default=None, description="Currently selected/active target device"
+    )
+    os_type: str = Field(
+        default="linux", description="Operating system type ('linux', 'darwin', 'windows')"
+    )
+    timestamp: float = Field(description="Unix timestamp of when the report was generated")

artemis/data_engine/engine.py

@@ -78,6 +78,7 @@ class DataEngine:
         self._pending_threads = []
         self._accumulated_logs = {}
         self._bg_task_to_trace_id = {}
+        self._trace_name_cache = {}
         # Background tasks are written directly to SQLite storage
 
         self.ipc_socket = None
@@ -429,8 +430,15 @@ class DataEngine:
             traces = self.storage.get_step_traces(step_id)
             generic_tools = []
             for t in traces:
-                if t.type == "tool":
-                    generic_tools.append(t.model_dump())
+                if t.type in ("tool", "action") or (t.type == "llm_call" and t.status == "failed"):
+                    t_dict = t.model_dump()
+                    if t.parent_trace_id and "agent_name" not in t_dict:
+                        parent_name = getattr(self, "_trace_name_cache", {}).get(
+                            t.parent_trace_id
+                        ) or getattr(self, "_trace_name_cache", {}).get(str(t.parent_trace_id))
+                        if parent_name:
+                            t_dict["agent_name"] = parent_name
+                    generic_tools.append(t_dict)
             step_dict["generic_tools"] = generic_tools
         except Exception as e:
             logger.error(f"Failed to fetch step traces for SSE: {e}")
@@ -486,6 +494,10 @@ class DataEngine:
         with self.lock:
             self._trace_counter = getattr(self, "_trace_counter", 0) + 1
             trace_ts = time.time() + (self._trace_counter * 1e-7)
+            if not hasattr(self, "_trace_name_cache"):
+                self._trace_name_cache = {}
+            self._trace_name_cache[trace_id] = name
+            self._trace_name_cache[str(trace_id)] = name
 
         trace = TraceRecord(
             trace_id=trace_id,
@@ -519,7 +531,14 @@ class DataEngine:
             thread.start()
 
         # Publish event
-        self._publish("trace_recorded", trace.model_dump())
+        trace_dict = trace.model_dump()
+        if parent_trace_id:
+            parent_name = getattr(self, "_trace_name_cache", {}).get(parent_trace_id) or getattr(
+                self, "_trace_name_cache", {}
+            ).get(str(parent_trace_id))
+            if parent_name:
+                trace_dict["agent_name"] = parent_name
+        self._publish("trace_recorded", trace_dict)
 
         return trace_id
 

artemis/drivers/android/adb_driver.py

@@ -22,7 +22,9 @@ from typing import Any, Literal
 
 from adbutils import AdbClient, AdbDevice
 from artemis.clients.ui_automator_client import UIAutomatorClient
+from artemis.config.paths import get_temp_dir
 from artemis.drivers.base import BaseDeviceDriver, KeyCode, ScreenData, SwipeDirection
+from artemis.toolchain import find_ffmpeg, find_scrcpy
 from artemis.utils.logger import get_logger
 
 logger = get_logger(__name__)
@@ -231,7 +233,7 @@ class AndroidAdbDriver(BaseDeviceDriver):
         start_y: int,
         end_x: int,
         end_y: int,
-        duration_ms: int = 400,
+        duration_ms: int = 800,
     ) -> bool:
         try:
             cmd = f"input swipe {start_x} {start_y} {end_x} {end_y} {duration_ms}"
@@ -245,31 +247,33 @@ class AndroidAdbDriver(BaseDeviceDriver):
     async def swipe_direction(
         self,
         direction: SwipeDirection | Literal["up", "down", "left", "right"],
-        duration_ms: int = 400,
+        duration_ms: int = 800,
     ) -> bool:
         dir_str = str(direction).lower()
-        mid_x = self._width // 2
+        mid_x = int(
+            self._width * 0.6
+        )  # Lock to 60% width to avoid edge gestures and alphabet fast-scroll sidebar
         mid_y = self._height // 2
 
         if dir_str == "up":
-            # Drag bottom to top -> scrolls down
+            # Drag bottom to top -> scrolls down (0.7 -> 0.3 leaves ~50-60% overlap, 800ms prevents fling)
             return await self.swipe(
-                mid_x, int(self._height * 0.75), mid_x, int(self._height * 0.25), duration_ms
+                mid_x, int(self._height * 0.7), mid_x, int(self._height * 0.3), duration_ms
             )
         elif dir_str == "down":
             # Drag top to bottom -> scrolls up
             return await self.swipe(
-                mid_x, int(self._height * 0.25), mid_x, int(self._height * 0.75), duration_ms
+                mid_x, int(self._height * 0.3), mid_x, int(self._height * 0.7), duration_ms
             )
         elif dir_str == "left":
             # Drag right to left -> scrolls right
             return await self.swipe(
-                int(self._width * 0.8), mid_y, int(self._width * 0.2), mid_y, duration_ms
+                int(self._width * 0.75), mid_y, int(self._width * 0.25), mid_y, duration_ms
             )
         elif dir_str == "right":
             # Drag left to right -> scrolls left
             return await self.swipe(
-                int(self._width * 0.2), mid_y, int(self._width * 0.8), mid_y, duration_ms
+                int(self._width * 0.25), mid_y, int(self._width * 0.75), mid_y, duration_ms
             )
         return False
 
@@ -407,14 +411,15 @@ class AndroidAdbDriver(BaseDeviceDriver):
 
     async def start_video_recording(self, output_dir: Path | None = None) -> None:
         """Starts screen recording via scrcpy in background."""
-        out_dir = output_dir or Path("/tmp")
+        out_dir = output_dir or get_temp_dir("recordings")
         out_dir.mkdir(parents=True, exist_ok=True)
         self._recording_mkv_path = out_dir / "recording.mkv"
         self._recording_output_path = out_dir / "recording.mp4"
         logger.info(f"Starting scrcpy video recording to {self._recording_mkv_path}...")
         try:
+            scrcpy_bin = find_scrcpy()
             cmd = [
-                "scrcpy",
+                scrcpy_bin,
                 "--serial",
                 self.device_id,
                 "--no-window",
@@ -449,10 +454,8 @@ class AndroidAdbDriver(BaseDeviceDriver):
 
         if mkv and mkv.exists() and mp4:
             try:
-                from artemis.utils.video import get_ffmpeg_path
-
                 proc = await asyncio.create_subprocess_exec(
-                    get_ffmpeg_path(),
+                    find_ffmpeg(),
                     "-y",
                     "-i",
                     str(mkv),

artemis/drivers/base.py

@@ -130,7 +130,7 @@ class BaseDeviceDriver(ABC):
         start_y: int,
         end_x: int,
         end_y: int,
-        duration_ms: int = 400,
+        duration_ms: int = 800,
     ) -> bool:
         """Performs a drag/swipe gesture from start to end pixel coordinates."""
         ...
@@ -139,7 +139,7 @@ class BaseDeviceDriver(ABC):
     async def swipe_direction(
         self,
         direction: SwipeDirection | Literal["up", "down", "left", "right"],
-        duration_ms: int = 400,
+        duration_ms: int = 800,
     ) -> bool:
         """Performs standard cardinal scrolling gesture across device viewport."""
         ...
@@ -210,7 +210,7 @@ class BaseDeviceDriver(ABC):
         self,
         start_norm: list[int],
         end_norm: list[int],
-        duration_ms: int = 400,
+        duration_ms: int = 800,
     ) -> bool:
         """Swipes between normalized (0-1000 scale) screen coordinates."""
         width, height = self.screen_size

artemis/engine/graph_runner.py

@@ -21,6 +21,7 @@ from artemis.core.state import ExecutionContextState, ExecutionStatus, StepRecor
 from artemis.drivers.mock.mock_driver import MockDeviceDriver
 from artemis.engine.base_runner import BaseRunner
 from artemis.graph.graph import get_graph
+from artemis.platform import platform
 from artemis.utils.logger import get_logger
 
 logger = get_logger(__name__)
@@ -48,7 +49,7 @@ class GraphRunner(BaseRunner):
                 if artemis_ctx is None:
                     artemis_ctx = ArtemisContext(
                         device=DeviceContext(
-                            host_platform="LINUX",
+                            host_platform=platform.os_type.name,
                             device_serial=self.driver.device_id,
                         )
                     )

artemis/engine/reactive_runner.py

@@ -23,6 +23,7 @@ from artemis.core.state import ExecutionContextState, ExecutionStatus, StepRecor
 from artemis.drivers.mock.mock_driver import MockDeviceDriver
 from artemis.engine.base_runner import BaseRunner
 from artemis.graph.state import State
+from artemis.platform import platform
 from artemis.utils.logger import get_logger
 
 logger = get_logger(__name__)
@@ -57,7 +58,7 @@ class ReactiveRunner(BaseRunner):
                 if artemis_ctx is None:
                     artemis_ctx = ArtemisContext(
                         device=DeviceContext(
-                            host_platform="LINUX",
+                            host_platform=platform.os_type.name,
                             device_serial=self.driver.device_id,
                         )
                     )

artemis/graph/graph.py

@@ -358,6 +358,61 @@ class CyFunctionDetector(metaclass=_CyFunctionDetectorMeta):
     pass
 
 
+def check_plan_mutation_rejections(
+    content_before: str, content_after: str, state: State | None = None
+) -> str | None:
+    """Checks if a task plan modification violates system safety or loop integrity rules."""
+    lines_after = content_after.splitlines()
+    top_level_after = [line for line in lines_after if line.startswith("- [")]
+
+    # 1. Incomplete nested subgoals check
+    if top_level_after and all(line.startswith("- [x]") for line in top_level_after):
+        has_incomplete = any(re.match(r"^\s*-\s*\[[ /]\]", line) for line in lines_after)
+        if has_incomplete:
+            return (
+                "Your changes to the task plan were not applied. Your changes would cause the"
+                " task to end completely, but there seem to be goals in the task plan"
+                " that have not been marked as completed. If you believe these goals"
+                " are already completed, please update their statuses."
+            )
+
+    # 2. Continuous monitoring / Loop milestone protection check
+    lines_before = content_before.splitlines()
+    top_level_before = [line for line in lines_before if line.startswith("- [")]
+
+    continuous_pattern = re.compile(r"\[Loop\].*?(?:continuous|until\s*manual)", re.IGNORECASE)
+    has_continuous_before = any(continuous_pattern.search(line) for line in top_level_before)
+
+    injected = getattr(state, "injected_instruction", None) if state else None
+    user_stopped = bool(
+        injected
+        and any(
+            w in str(injected).lower()
+            for w in ["stop", "finish", "end", "停止", "结束", "退出", "quit"]
+        )
+    )
+
+    if has_continuous_before and not user_stopped:
+        has_continuous_after = any(continuous_pattern.search(line) for line in top_level_after)
+        if not has_continuous_after:
+            return (
+                "Your changes to the task plan were rejected. You cannot delete an active [Loop] "
+                "continuous monitoring milestone or remove its [Loop] tag. Please keep the [Loop] "
+                "milestone active (in progress [/]) and record each check cycle as an indented "
+                "subtask."
+            )
+        for line in top_level_after:
+            if continuous_pattern.search(line) and line.startswith("- [x]"):
+                return (
+                    "Your changes to the task plan were rejected. Ongoing continuous monitoring "
+                    "milestones (exit condition: 'until manually stopped') must remain in progress "
+                    "([/]) and cannot be unilaterally marked as completed [x]. Please keep it "
+                    "active ([/]) and record each polling check as an indented subtask."
+                )
+
+    return None
+
+
 class NoteArgs(BaseModel):
     model_config = {"ignored_types": (CyFunctionDetector,)}
     key: str = Field(..., description=SAVE_NOTE_ARG_KEY_DESC)
@@ -392,36 +447,15 @@ def wrap_note_tool(ctx: ArtemisContext, original_tool):
             if task_plan_path.exists():
                 content_after = task_plan_path.read_text(encoding="utf-8")
 
-            lines = content_after.splitlines()
-            top_level_lines = [line for line in lines if line.startswith("- [")]
-            if top_level_lines and all(line.startswith("- [x]") for line in top_level_lines):
-                has_incomplete = any(re.match(r"^\s*-\s*\[[ /]\]", line) for line in lines)
-                if has_incomplete:
-                    logger.info(
-                        "Rejected: Top-level subgoals are complete but"
-                        " incomplete nested subgoals remain."
-                    )
-                    if task_plan_path.exists():
-                        task_plan_path.write_text(content_before, encoding="utf-8")
+            rejection_message = check_plan_mutation_rejections(content_before, content_after, state)
+            if rejection_message:
+                logger.info(f"Rejected task plan update: {rejection_message}")
+                if task_plan_path.exists():
+                    task_plan_path.write_text(content_before, encoding="utf-8")
 
-                    return Command(
-                        update={
-                            VALIDATOR_MESSAGES_KEY: [
-                                SystemMessage(
-                                    content=(
-                                        "Your changes to the task plan were not"
-                                        " applied. Your changes would cause the"
-                                        " task to end completely, but there"
-                                        " seem to be goals in the task plan"
-                                        " that have not been marked as"
-                                        " completed. If you believe these goals"
-                                        " are already completed, please update"
-                                        " their statuses."
-                                    )
-                                )
-                            ]
-                        }
-                    )
+                return Command(
+                    update={VALIDATOR_MESSAGES_KEY: [SystemMessage(content=rejection_message)]}
+                )
 
             threshold = (
                 ctx.execution_setup.planner_validation_threshold
@@ -558,36 +592,15 @@ def wrap_update_note_tool(ctx: ArtemisContext, original_tool):
             if task_plan_path.exists():
                 content_after = task_plan_path.read_text(encoding="utf-8")
 
-            lines = content_after.splitlines()
-            top_level_lines = [line for line in lines if line.startswith("- [")]
-            if top_level_lines and all(line.startswith("- [x]") for line in top_level_lines):
-                has_incomplete = any(re.match(r"^\s*-\s*\[[ /]\]", line) for line in lines)
-                if has_incomplete:
-                    logger.info(
-                        "Rejected: Top-level subgoals are complete but"
-                        " incomplete nested subgoals remain."
-                    )
-                    if task_plan_path.exists():
-                        task_plan_path.write_text(content_before, encoding="utf-8")
+            rejection_message = check_plan_mutation_rejections(content_before, content_after, state)
+            if rejection_message:
+                logger.info(f"Rejected task plan update via update_note: {rejection_message}")
+                if task_plan_path.exists():
+                    task_plan_path.write_text(content_before, encoding="utf-8")
 
-                    return Command(
-                        update={
-                            VALIDATOR_MESSAGES_KEY: [
-                                SystemMessage(
-                                    content=(
-                                        "Your changes to the task plan were not"
-                                        " applied. Your changes would cause the"
-                                        " task to end completely, but there"
-                                        " seem to be goals in the task plan"
-                                        " that have not been marked as"
-                                        " completed. If you believe these goals"
-                                        " are already completed, please update"
-                                        " their statuses."
-                                    )
-                                )
-                            ]
-                        }
-                    )
+                return Command(
+                    update={VALIDATOR_MESSAGES_KEY: [SystemMessage(content=rejection_message)]}
+                )
 
             threshold = (
                 ctx.execution_setup.planner_validation_threshold
@@ -710,6 +723,18 @@ async def get_graph(ctx: ArtemisContext) -> CompiledStateGraph:
 
         lines = task_plan_content.splitlines()
 
+        # Check for continuous monitoring loop that should not terminate prematurely
+        continuous_pattern = re.compile(r"\[Loop\].*?(?:continuous|until\s*manual)", re.IGNORECASE)
+        has_continuous_monitoring = any(continuous_pattern.search(line) for line in lines)
+        injected = getattr(state, "injected_instruction", None) if state else None
+        user_stopped = bool(
+            injected
+            and any(
+                w in str(injected).lower()
+                for w in ["stop", "finish", "end", "停止", "结束", "退出", "quit"]
+            )
+        )
+
         # Check for all completed
         top_level_lines = [line for line in lines if line.startswith("- [")]
 
@@ -719,6 +744,12 @@ async def get_graph(ctx: ArtemisContext) -> CompiledStateGraph:
 
         all_done = all(line.startswith("- [x]") for line in top_level_lines)
         if all_done:
+            if has_continuous_monitoring and not user_stopped:
+                logger.warning(
+                    "All subgoals marked [x] but task plan has active continuous monitoring"
+                    " without user stop signal. Continuing."
+                )
+                return "continue"
             logger.info("All subgoals are completed, ending the goal")
             return "end"
 

artemis/graph/perception.py

@@ -23,7 +23,7 @@ from artemis.controllers.unified_controller import UnifiedMobileController
 from artemis.data_engine.trace import trace
 from artemis.graph.state import State
 from artemis.utils.logger import get_logger
-from artemis.utils.ocr_api import perform_ocr
+from artemis.utils.ocr_api import is_ocr_configured, perform_ocr
 from artemis.utils.ocr_xml_fusion import (
     _crop_image_remove_status_bar,
     _detect_status_bar_height,
@@ -124,25 +124,35 @@ async def perception_node(state: State, ctx: ArtemisContext) -> dict:
         ctx.device.device_height = device_data.height
         logger.info(f"Updated context device dimensions: {device_data.width}x{device_data.height}")
 
-    # 3. Perform OCR
-    screen_height = device_data.height
-    status_bar_height = _detect_status_bar_height(xml_hierarchy, screen_height)
-    if status_bar_height > 0:
-        logger.info(
-            f"Detected status bar height: {status_bar_height}. Cropping screenshot for OCR."
-        )
-        cropped_b64, _, _ = await asyncio.to_thread(
-            _crop_image_remove_status_bar,
-            latest_screenshot_b64,
-            status_bar_height,
-        )
-        raw_ocr_results = await perform_ocr(cropped_b64)
-        ocr_results = _map_coordinates_back(raw_ocr_results, status_bar_height)
+    # 3. Perform OCR (if configured, else fallback to pure XML layout)
+    ocr_results = []
+    if is_ocr_configured():
+        screen_height = device_data.height
+        status_bar_height = _detect_status_bar_height(xml_hierarchy, screen_height)
+        try:
+            if status_bar_height > 0:
+                logger.info(
+                    f"Detected status bar height: {status_bar_height}. Cropping screenshot for OCR."
+                )
+                cropped_b64, _, _ = await asyncio.to_thread(
+                    _crop_image_remove_status_bar,
+                    latest_screenshot_b64,
+                    status_bar_height,
+                )
+                raw_ocr_results = await perform_ocr(cropped_b64)
+                ocr_results = _map_coordinates_back(raw_ocr_results, status_bar_height)
+            else:
+                logger.info("No status bar detected. Performing OCR on full screenshot.")
+                ocr_results = await perform_ocr(latest_screenshot_b64)
+        except Exception as ocr_err:
+            logger.warning(
+                f"Perception OCR execution encountered error: {ocr_err}. Proceeding with XML-only layout."
+            )
+            ocr_results = []
     else:
-        logger.info("No status bar detected. Performing OCR on full screenshot.")
-        ocr_results = await perform_ocr(latest_screenshot_b64)
+        logger.debug("OCR is not configured. Proceeding with XML-only layout.")
 
-    # 4. Fuse OCR with XML
+    # 4. Fuse OCR with XML (if ocr_results is empty, returns original xml_hierarchy intact)
     fused_xml = fuse_ocr_with_xml(xml_hierarchy, ocr_results)
 
     # 5. Save to Data Engine in background thread

artemis/interfaces/cli/commands/bench.py

@@ -1,157 +0,0 @@
-# Copyright 2026 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Benchmark and evaluation command (artemis bench)."""
-
-import asyncio
-from pathlib import Path
-import subprocess
-import sys
-from typing import Annotated
-
-from artemis.config import initialize_llm_config
-from artemis.sdk import Agent
-from artemis.sdk.builders import Builders
-from artemis.sdk.types.task import AgentProfile
-from artemis.utils.logger import get_logger
-import typer
-
-logger = get_logger(__name__)
-
-
-def check_and_start_emulator(adb_path: str = "adb") -> bool:
-    """Checks if Android emulator is running; optionally starts it."""
-    try:
-        res = subprocess.run([adb_path, "devices"], capture_output=True, text=True, timeout=5)
-        for line in res.stdout.splitlines():
-            if "emulator-" in line and "device" in line:
-                return True
-    except Exception:
-        pass
-    return False
-
-
-async def run_benchmark(
-    task_name: str | None = None,
-    profile_name: str = "pro",
-    console_port: int = 5554,
-    adb_path: str = "adb",
-) -> None:
-    """Runs AndroidWorld benchmark task evaluations."""
-    # Add android_world-main if present
-    workspace_root = Path(__file__).resolve().parent.parent.parent.parent.parent
-    aw_path = workspace_root / "android_world-main"
-    if aw_path.exists() and str(aw_path) not in sys.path:
-        sys.path.insert(0, str(aw_path))
-
-    try:
-        from android_world import registry
-        from android_world.env import env_launcher
-    except ImportError:
-        logger.error("AndroidWorld benchmark suite is not installed or available in path.")
-        raise typer.Exit(1)
-
-    task_registry = registry.TaskRegistry()
-    aw_registry = task_registry.get_registry(task_registry.ANDROID_WORLD_FAMILY)
-
-    if task_name and task_name not in aw_registry:
-        logger.error(f"Task '{task_name}' not found in AndroidWorld registry.")
-        available = sorted(list(aw_registry.keys()))
-        logger.info(f"Available tasks ({len(available)}): {', '.join(available[:10])}...")
-        raise typer.Exit(1)
-
-    logger.info(f"Initializing AndroidWorld environment on console port {console_port}...")
-    env = env_launcher.load_and_setup_env(
-        console_port=console_port,
-        emulator_setup=False,
-        adb_path=adb_path,
-    )
-
-    llm_config = initialize_llm_config()
-    profile = AgentProfile(name="default", llm_config=llm_config)
-    config = Builders.AgentConfig.with_default_profile(profile).build()
-
-    agent = Agent(config=config)
-    await agent.init()
-
-    tasks_to_run = [task_name] if task_name else list(aw_registry.keys())
-
-    try:
-        for t_name in tasks_to_run:
-            task_type = aw_registry[t_name]
-            params = task_type.generate_random_params()
-            task_instance = task_type(params)
-
-            logger.info(f"\n🎯 [Benchmark] Starting: {t_name} | Goal: {task_instance.goal}")
-            task_instance.initialize_task(env)
-
-            result = await agent.run_task(
-                goal=task_instance.goal,
-                name=f"bench_{t_name}",
-                profile=profile_name,
-            )
-            success = task_instance.is_successful(env)
-            logger.info(f"🏆 Task '{t_name}' Result: {result} | Benchmark Verified: {success}")
-    finally:
-        await agent.clean()
-        env.close()
-
-
-def bench_command(
-    task: Annotated[
-        str | None,
-        typer.Option(
-            "--task",
-            "-t",
-            help="Specific AndroidWorld task name to evaluate (e.g. Clock_1, Calculator_1).",
-        ),
-    ] = None,
-    profile: Annotated[
-        str,
-        typer.Option(
-            "--profile",
-            "-p",
-            help="Agent profile to evaluate ('flash' or 'pro').",
-        ),
-    ] = "pro",
-    adb_path: Annotated[
-        str,
-        typer.Option(
-            "--adb-path",
-            help="Path to ADB executable.",
-        ),
-    ] = "adb",
-    console_port: Annotated[
-        int,
-        typer.Option(
-            "--port",
-            help="Emulator console port.",
-        ),
-    ] = 5554,
-) -> None:
-    """Run AndroidWorld benchmark task evaluations against the agent."""
-    if not check_and_start_emulator(adb_path=adb_path):
-        typer.secho(
-            f"Warning: No active emulator detected on port {console_port}. Please start an emulator.",
-            fg=typer.colors.YELLOW,
-        )
-
-    asyncio.run(
-        run_benchmark(
-            task_name=task,
-            profile_name=profile,
-            console_port=console_port,
-            adb_path=adb_path,
-        )
-    )

artemis/interfaces/cli/commands/doctor.py

@@ -14,7 +14,7 @@
 
 """System, environment, and device diagnostics (artemis doctor)."""
 
-import shutil
+import platform as sys_platform
 import subprocess
 import sys
 
@@ -26,16 +26,21 @@ from artemis.config import (
     parse_llm_config,
     settings,
 )
+from artemis.platform import OSType, platform
+from artemis.toolchain import toolchain
+from artemis.utils.ocr_api import is_ocr_configured
 
 
 def doctor_command() -> None:
     """Run diagnostics to inspect system dependencies, device connectivity, and configuration."""
     console = Console()
     console.print()
+    os_name = sys_platform.system()
+    arch_name = sys_platform.machine()
     console.print(
         Panel(
-            "[bold cyan]☕ Artemis System & Environment Doctor[/bold cyan]\n"
-            "[dim]Checking prerequisites for autonomous mobile testing...[/dim]",
+            f"[bold cyan]☕ Artemis System & Environment Doctor[/bold cyan]\n"
+            f"[dim]Platform: {os_name} ({arch_name}) | PAL: {platform.os_type.value}[/dim]",
             expand=False,
         )
     )
@@ -53,7 +58,7 @@ def doctor_command() -> None:
         table.add_row(
             "Python Runtime",
             "[bold green]✔ OK[/bold green]",
-            f"Python {py_ver} (64-bit)",
+            f"Python {py_ver} ({arch_name}, 64-bit)",
         )
     else:
         all_passed = False
@@ -64,7 +69,7 @@ def doctor_command() -> None:
         )
 
     # 2. ADB Installation & Server
-    adb_path = shutil.which("adb")
+    adb_path = toolchain.resolve("adb")
     if adb_path:
         try:
             res = subprocess.run(
@@ -72,6 +77,8 @@ def doctor_command() -> None:
                 capture_output=True,
                 text=True,
                 timeout=5,
+                encoding="utf-8",
+                errors="replace",
             )
             adb_info = res.stdout.splitlines()[0] if res.stdout else "Installed"
             table.add_row(
@@ -81,10 +88,17 @@ def doctor_command() -> None:
             table.add_row("Android ADB", "[bold green]✔ OK[/bold green]", f"{adb_path}")
     else:
         all_passed = False
+        hint = (
+            "Run: winget install Google.PlatformTools"
+            if platform.os_type == OSType.WINDOWS
+            else "Run: brew install android-platform-tools"
+            if platform.os_type == OSType.MACOS
+            else "Run: sudo apt-get install -y adb"
+        )
         table.add_row(
             "Android ADB",
             "[bold red]✖ Missing[/bold red]",
-            "adb not found in PATH. Install Android Platform-Tools.",
+            f"adb not found in PATH or Android SDK. {hint}",
         )
 
     # 3. Connected Android Devices
@@ -96,6 +110,8 @@ def doctor_command() -> None:
                 capture_output=True,
                 text=True,
                 timeout=5,
+                encoding="utf-8",
+                errors="replace",
             )
             lines = [line_str.strip() for line_str in res.stdout.splitlines() if line_str.strip()]
             for line in lines[1:]:  # skip 'List of devices attached'
@@ -123,7 +139,7 @@ def doctor_command() -> None:
             table.add_row(
                 "Android Devices",
                 "[bold yellow]⚠ Unauthorized[/bold yellow]",
-                "Device detected but unauthorized. Check phone screen for USB Debugging permission prompt.",
+                "Device detected but unauthorized. Check phone screen for USB Debugging prompt.",
             )
     else:
         table.add_row(
@@ -160,11 +176,11 @@ def doctor_command() -> None:
         table.add_row(
             "LLM Providers",
             "[bold red]✖ Missing[/bold red]",
-            "No LLM key found in .env. Run [bold cyan]artemis init[/bold cyan] to configure in 10 seconds.",
+            "No LLM key found in .env. Run [bold cyan]artemis init[/bold cyan] to configure.",
         )
 
-    # 5. Vision OCR Key Check
-    if ocr_key:
+    # 5. Vision OCR Key Check (Optional)
+    if is_ocr_configured() and ocr_key:
         val = ocr_key.get_secret_value()
         masked = f"{val[:6]}...{val[-4:]}" if len(val) > 10 else "***"
         table.add_row(
@@ -175,8 +191,8 @@ def doctor_command() -> None:
     else:
         table.add_row(
             "Vision OCR",
-            "[bold yellow]⚠ Missing[/bold yellow]",
-            "OCR_API_KEY / API_KEY not set. Perception fast path will use fallback.",
+            "[bold cyan]○ Optional[/bold cyan]",
+            "OCR_API_KEY is not set. Perception operates in pure XML layout mode.",
         )
 
     # 6. Configuration File Health
@@ -197,33 +213,45 @@ def doctor_command() -> None:
         )
 
     # 7. Video Recording Tools (FFmpeg & scrcpy)
-    ffmpeg_path = shutil.which("ffmpeg")
-    scrcpy_path = shutil.which("scrcpy")
+    ffmpeg_path = toolchain.resolve("ffmpeg")
+    scrcpy_path = toolchain.resolve("scrcpy")
 
     if ffmpeg_path:
         table.add_row(
-            "FFmpeg Tools",
-            "[bold green]✔ Installed[/bold green]",
-            f"{ffmpeg_path} (Video encoding/trimming ready)",
+            "FFmpeg Tools", "[bold green]✔ Installed[/bold green]", f"{ffmpeg_path} (Video ready)"
         )
     else:
+        hint = (
+            "winget install Gyan.FFmpeg"
+            if platform.os_type == OSType.WINDOWS
+            else "brew install ffmpeg"
+            if platform.os_type == OSType.MACOS
+            else "sudo apt-get install -y ffmpeg"
+        )
         table.add_row(
             "FFmpeg Tools",
             "[dim]⚪ Optional[/dim]",
-            "Not found. Install ffmpeg for video compression and audio analysis.",
+            f"Not found. ({hint})",
         )
 
     if scrcpy_path:
         table.add_row(
             "Screen Recorder",
             "[bold green]✔ Installed[/bold green]",
-            f"{scrcpy_path} (Live screen recording ready)",
+            f"{scrcpy_path} (Live stream ready)",
         )
     else:
+        hint = (
+            "winget install Genymobile.scrcpy"
+            if platform.os_type == OSType.WINDOWS
+            else "brew install scrcpy"
+            if platform.os_type == OSType.MACOS
+            else "sudo apt-get install -y scrcpy"
+        )
         table.add_row(
             "Screen Recorder",
             "[dim]⚪ Optional[/dim]",
-            "scrcpy not found. Install scrcpy for --with-video-recording-tools.",
+            f"scrcpy not found. ({hint})",
         )
 
     console.print(table)

artemis/interfaces/cli/commands/mcp.py

@@ -14,13 +14,40 @@
 
 """Model Context Protocol (MCP) server command (artemis mcp)."""
 
+import json
 from typing import Annotated
 
+from mcp_server.base import mcp as agent_mcp
+from mcp_server.utils import env_utils
 from artemis.mcp.adb_server import mcp as adb_mcp
 from artemis.utils.logger import get_logger
+from rich.console import Console
+from rich.syntax import Syntax
 import typer
 
 logger = get_logger(__name__)
+console = Console()
+
+
+def _get_config_snippet(client: str, python_exe: str, project_root: str) -> dict:
+    """Generates MCP configuration dictionary for the specified client."""
+    config_body = {
+        "command": python_exe,
+        "args": ["-m", "mcp_server"],
+        "cwd": project_root,
+        "env": {
+            "PYTHONUNBUFFERED": "1",
+        },
+    }
+
+    if client == "cursor":
+        return {"mcpServers": {"artemis": config_body}}
+    elif client in ("claude", "claude_code", "claude_desktop"):
+        return {"mcpServers": {"artemis": config_body}}
+    elif client == "openclaw":
+        return {"plugins": {"artemis_mcp": {"enabled": True, "type": "mcp", "server": config_body}}}
+    else:  # generic
+        return {"mcpServers": {"artemis": config_body}}
 
 
 def mcp_command(
@@ -29,14 +56,89 @@ def mcp_command(
         typer.Option(
             "--type",
             "-t",
-            help="Type of MCP server to start ('adb' or 'xml').",
+            help="Type of MCP server to start: 'agent' (default, universal IDE mobile agent), 'adb' (raw adb), 'xml' (xml fuzzy search).",
         ),
-    ] = "adb",
+    ] = "agent",
+    transport: Annotated[
+        str,
+        typer.Option(
+            "--transport",
+            help="MCP transport protocol ('stdio' or 'sse').",
+        ),
+    ] = "stdio",
+    host: Annotated[
+        str,
+        typer.Option(
+            "--host",
+            help="Host address when running with SSE transport.",
+        ),
+    ] = "127.0.0.1",
+    port: Annotated[
+        int,
+        typer.Option(
+            "--port",
+            "-p",
+            help="Port number when running with SSE transport.",
+        ),
+    ] = 8001,
+    generate_config: Annotated[
+        str | None,
+        typer.Option(
+            "--generate-config",
+            "-g",
+            help="Output ready-to-use MCP configuration JSON for 'cursor', 'claude', 'openclaw', or 'all'.",
+        ),
+    ] = None,
 ) -> None:
-    """Launch a ARTEMIS Model Context Protocol (MCP) server."""
-    if server_type.lower() == "adb":
-        logger.info("Starting Artemis ADB MCP Server over stdio...")
-        adb_mcp.run(transport="stdio")
+    """Launch or configure the ARTEMIS Model Context Protocol (MCP) server."""
+    project_root = env_utils.get_project_root()
+    python_exe = env_utils.resolve_python_executable(project_root)
+
+    if generate_config:
+        client = generate_config.lower()
+        if client == "all":
+            all_configs = {
+                "cursor (.cursor/mcp.json)": _get_config_snippet(
+                    "cursor", python_exe, project_root
+                ),
+                "claude (claude_desktop_config.json)": _get_config_snippet(
+                    "claude", python_exe, project_root
+                ),
+                "openclaw (openclaw.json)": _get_config_snippet(
+                    "openclaw", python_exe, project_root
+                ),
+            }
+            json_str = json.dumps(all_configs, indent=2)
+        else:
+            snippet = _get_config_snippet(client, python_exe, project_root)
+            json_str = json.dumps(snippet, indent=2)
+
+        syntax = Syntax(json_str, "json", theme="monokai", line_numbers=False)
+        console.print(f"[bold cyan]MCP Configuration for {client.upper()}:[/bold cyan]")
+        console.print(syntax)
+        raise typer.Exit(0)
+
+    st = server_type.lower()
+    if st in ("agent", "mobile", "artemis", "default"):
+        logger.info(f"Starting Artemis Mobile Agent MCP Server over {transport}...")
+        if transport.lower() == "sse":
+            agent_mcp.run(transport="sse", host=host, port=port)
+        else:
+            agent_mcp.run(transport="stdio")
+    elif st == "adb":
+        logger.info(f"Starting Artemis ADB MCP Server over {transport}...")
+        if transport.lower() == "sse":
+            adb_mcp.run(transport="sse", host=host, port=port)
+        else:
+            adb_mcp.run(transport="stdio")
+    elif st == "xml":
+        from artemis.mcp.xml_search_server import mcp as xml_mcp
+
+        logger.info(f"Starting Artemis XML Fuzzy Search MCP Server over {transport}...")
+        if transport.lower() == "sse":
+            xml_mcp.run(transport="sse", host=host, port=port)
+        else:
+            xml_mcp.run(transport="stdio")
     else:
-        logger.error(f"Unsupported MCP server type: {server_type}")
+        logger.error(f"Unsupported MCP server type: {server_type}. Use 'agent', 'adb', or 'xml'.")
         raise typer.Exit(1)

artemis/interfaces/cli/commands/server.py

@@ -15,6 +15,7 @@
 """Server and cloud proxy commands (artemis server)."""
 
 import http.server
+import os
 import socketserver
 import subprocess
 import time
@@ -50,7 +51,7 @@ def get_identity_token() -> str:
 
 
 class CloudRunProxyHandler(http.server.BaseHTTPRequestHandler):
-    target_url: str = "https://artemis-cloud-brain-971929466682.us-central1.run.app"
+    target_url: str = os.getenv("ARTEMIS_CLOUD_URL", "http://127.0.0.1:8000")
 
     def log_message(self, format, *args):
         logger.info(f"[{self.log_date_time_string()}] {format % args}")
@@ -81,7 +82,9 @@ class CloudRunProxyHandler(http.server.BaseHTTPRequestHandler):
         token = get_identity_token()
         if token:
             headers["Authorization"] = f"Bearer {token}"
-        headers["X-Artemis-Token"] = "alice-token"
+        tenant_token = os.getenv("ARTEMIS_TENANT_TOKEN")
+        if tenant_token:
+            headers["X-Artemis-Token"] = tenant_token
 
         content_length = int(self.headers.get("Content-Length", 0))
         body = self.rfile.read(content_length) if content_length > 0 else None
@@ -119,8 +122,10 @@ def web_proxy_command(
     port: Annotated[int, typer.Option("--port", "-p", help="Local port to bind.")] = 8080,
     target: Annotated[
         str,
-        typer.Option("--target", "-t", help="Target Cloud Run URL."),
-    ] = "https://artemis-cloud-brain-971929466682.us-central1.run.app",
+        typer.Option(
+            "--target", "-t", help="Target Artemis Cloud URL.", envvar="ARTEMIS_CLOUD_URL"
+        ),
+    ] = "http://127.0.0.1:8000",
 ) -> None:
     """Start the local authenticated proxy for ARTEMIS Cloud Web Dashboard."""
     CloudRunProxyHandler.target_url = target

artemis/interfaces/cli/commands/ui.py

@@ -0,0 +1,114 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""UI Launch and Web Console CLI Command (artemis ui)."""
+
+import threading
+import time
+from typing import Annotated
+import urllib.request
+import webbrowser
+
+from rich.console import Console
+from rich.panel import Panel
+import typer
+
+
+def _poll_and_open_browser(url: str, stop_event: threading.Event, timeout: float = 15.0) -> None:
+    """Poll the UI server until it responds, then launch the user's default browser."""
+    start = time.time()
+    while not stop_event.is_set() and (time.time() - start < timeout):
+        try:
+            req = urllib.request.Request(url, headers={"User-Agent": "Artemis-HealthCheck/1.0"})
+            with urllib.request.urlopen(req, timeout=1.5) as resp:
+                if resp.status == 200:
+                    time.sleep(0.3)
+                    webbrowser.open(url)
+                    return
+        except Exception:
+            time.sleep(0.4)
+
+
+def ui_command(
+    host: Annotated[
+        str,
+        typer.Option("--host", "-h", help="Bind host address for the UI server."),
+    ] = "0.0.0.0",
+    port: Annotated[
+        int,
+        typer.Option("--port", "-p", help="Port to run the unified UI server on."),
+    ] = 8000,
+    open_browser: Annotated[
+        bool,
+        typer.Option(
+            "--open/--no-open",
+            help="Automatically open the Showcase UI in default web browser.",
+        ),
+    ] = True,
+    reload: Annotated[
+        bool,
+        typer.Option(
+            "--reload/--no-reload",
+            help="Enable uvicorn live auto-reload (for development).",
+        ),
+    ] = False,
+) -> None:
+    """Launch the unified Artemis Showcase UI & Admin Console in one click."""
+    console = Console()
+    local_url = f"http://localhost:{port}"
+
+    console.print()
+    msg = (
+        f"[bold cyan]✨ Artemis Autonomous Mobile Agent UI[/bold cyan]\n\n"
+        f"📱 [bold green]Showcase UI & Onboarding:[/bold green] [cyan]{local_url}[/cyan]\n"
+        f"🛠️ [bold magenta]Admin Debug Console:[/bold magenta]     [cyan]{local_url}/admin[/cyan]\n"
+        f"📖 [dim]API Documentation:[/dim]         [dim]{local_url}/docs[/dim]\n\n"
+        f"[dim]Press Ctrl+C to stop the server.[/dim]"
+    )
+    console.print(Panel(msg, title="🚀 Web Server Active", border_style="cyan", expand=False))
+    # Ensure .env exists in workspace root
+    from artemis.config.paths import ROOT_DIR
+
+    env_file = ROOT_DIR / ".env"
+    env_example = ROOT_DIR / ".env.example"
+    if not env_file.exists():
+        if env_example.exists():
+            try:
+                env_file.write_text(env_example.read_text(encoding="utf-8"), encoding="utf-8")
+            except Exception:
+                pass
+        else:
+            try:
+                env_file.touch()
+            except Exception:
+                pass
+
+    stop_event = threading.Event()
+    if open_browser:
+        t = threading.Thread(
+            target=_poll_and_open_browser,
+            args=(local_url, stop_event),
+            daemon=True,
+        )
+        t.start()
+
+    try:
+        import uvicorn
+        from apps.admin_console.server import app
+
+        uvicorn.run(app, host=host, port=port, reload=reload)
+    except KeyboardInterrupt:
+        console.print("\n[yellow]🛑 UI Server stopped.[/yellow]")
+    finally:
+        stop_event.set()

artemis/interfaces/cli/main.py

@@ -18,13 +18,13 @@ import importlib.metadata
 from typing import Annotated
 
 from artemis.interfaces.cli.commands.batch import batch_command
-from artemis.interfaces.cli.commands.bench import bench_command
 from artemis.interfaces.cli.commands.doctor import doctor_command
 from artemis.interfaces.cli.commands.init import init_command
 from artemis.interfaces.cli.commands.mcp import mcp_command
 from artemis.interfaces.cli.commands.run import run_command
 from artemis.interfaces.cli.commands.server import server_app
 from artemis.interfaces.cli.commands.trace import trace_app
+from artemis.interfaces.cli.commands.ui import ui_command
 from artemis.utils.logger import get_logger
 from rich.console import Console
 from rich.panel import Panel
@@ -41,6 +41,9 @@ app = typer.Typer(
 )
 
 # Register subcommands
+app.command(name="ui", help="Launch the unified Showcase UI & Admin Console in your browser.")(
+    ui_command
+)
 app.command(name="run", help="Execute an autonomous task on a mobile device.")(run_command)
 app.command(name="init", help="Interactive quickstart wizard to configure API keys & device.")(
     init_command
@@ -49,7 +52,6 @@ app.command(name="doctor", help="Check system prerequisites, device status, and
     doctor_command
 )
 app.command(name="batch", help="Execute a batch sequence of automation tasks.")(batch_command)
-app.command(name="bench", help="Run AndroidWorld benchmark task evaluations.")(bench_command)
 app.command(name="mcp", help="Start the Artemis Model Context Protocol (MCP) server.")(mcp_command)
 app.add_typer(server_app, name="server", help="Cloud Run proxy and web dashboard server.")
 app.add_typer(trace_app, name="trace", help="Inspect and query execution traces.")

artemis/llm/router.py

@@ -118,7 +118,7 @@ def _patch_langchain_google_genai():
         from langchain_google_genai.chat_models import ChatGoogleGenerativeAI
         from google.genai.types import ToolConfig
 
-        if getattr(ChatGoogleGenerativeAI, "_is_latte_patched", False):
+        if getattr(ChatGoogleGenerativeAI, "_is_artemis_patched", False):
             return
 
         original_process = ChatGoogleGenerativeAI._process_tool_config
@@ -169,7 +169,7 @@ def _patch_langchain_google_genai():
             return config
 
         ChatGoogleGenerativeAI._process_tool_config = patched_process_tool_config
-        ChatGoogleGenerativeAI._is_latte_patched = True
+        ChatGoogleGenerativeAI._is_artemis_patched = True
     except Exception as e:
         logger.warning(f"Could not patch ChatGoogleGenerativeAI._process_tool_config: {e}")
 

artemis/mcp/__init__.py

@@ -0,0 +1,20 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Internal MCP servers for ARTEMIS engine (ADB server, XML Fuzzy Search)."""
+
+from artemis.mcp.adb_server import mcp as adb_mcp
+from artemis.mcp.xml_search_server import mcp as xml_mcp
+
+__all__ = ["adb_mcp", "xml_mcp"]

artemis/mcp/adb_server.py

@@ -29,9 +29,17 @@ if _REPO_ROOT not in sys.path:
 from adbutils import AdbClient
 from mcp.server.fastmcp import Context, FastMCP
 
+try:
+    from mcp.server.fastmcp.server import Settings as FastMCPSettings
+
+    FastMCPSettings.model_rebuild()
+except Exception:
+    pass
+
 from artemis.clients.ui_automator_client import UIAutomatorClient
 from artemis.context import ArtemisContext, DeviceContext, DevicePlatform
 from artemis.controllers.unified_controller import UnifiedMobileController
+from artemis.platform import platform
 from artemis.utils.app_launch_utils import launch_app_with_retries
 
 # Redirect stdio to prevent logs from breaking MCP JSON-RPC protocol and causing deadlocks.
@@ -130,7 +138,7 @@ def _get_controller():
     ctx = ArtemisContext(
         trace_id="mcp-session",
         device=DeviceContext(
-            host_platform="LINUX",
+            host_platform=platform.os_type.name,
             mobile_platform=DevicePlatform.ANDROID,
             device_id=device_id,
             device_width=width,

artemis/mcp/xml_search_server.py

@@ -25,6 +25,13 @@ if _REPO_ROOT not in sys.path:
 # pylint: disable=wrong-import-position
 from mcp.server.fastmcp import FastMCP
 
+try:
+    from mcp.server.fastmcp.server import Settings as FastMCPSettings
+
+    FastMCPSettings.model_rebuild()
+except Exception:
+    pass
+
 from artemis.data_engine.storage import StorageManager
 
 mcp = FastMCP("Android_XML_Fuzzy_Search")
@@ -83,7 +90,7 @@ def serialize_node(node, max_depth=1):
 
 @mcp.tool()
 def search_ui(image_hash: str, query: str, threshold: float = 0.6) -> dict:
-    """Search for text in UI data (XML and OCR) retrieved from Data Engine by image hash.
+    """Search for text in UI data retrieved from Data Engine by image hash.
 
     Args:
         image_hash: The SHA-256 hash of the image.
@@ -153,7 +160,7 @@ def parse_bounds(bounds_str):
 
 @mcp.tool()
 def search_by_coordinates(image_hash: str, x: int, y: int) -> str:
-    """Search for elements in UI data (XML and OCR) retrieved from Data Engine by image hash that overlap with the given coordinates.
+    """Search for elements in UI data retrieved from Data Engine by image hash that overlap with the given coordinates.
 
     Args:
         image_hash: The SHA-256 hash of the image.
@@ -161,7 +168,7 @@ def search_by_coordinates(image_hash: str, x: int, y: int) -> str:
         y: The Y coordinate.
 
     Returns:
-        A string containing the matching nodes and OCR results.
+        A string containing the matching nodes.
     """
     db_path = os.getenv("DATA_ENGINE_DB_PATH")
     if not db_path:

artemis/platform/__init__.py

@@ -0,0 +1,61 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Platform Abstraction Layer (PAL) single entry point and global singleton."""
+
+import sys
+
+from artemis.platform.base import (
+    IPlatform,
+    IPlatformPaths,
+    IPlatformProcess,
+    OSType,
+)
+from artemis.platform.darwin import DarwinPlatform
+from artemis.platform.linux import LinuxPlatform
+from artemis.platform.windows import WindowsPlatform
+
+
+def _create_platform() -> IPlatform:
+    """Factory creating the appropriate platform instance for current runtime environment."""
+    if sys.platform == "win32":
+        p = WindowsPlatform()
+    elif sys.platform == "darwin":
+        p = DarwinPlatform()
+    else:
+        p = LinuxPlatform()
+
+    # Automatically initialize UTF-8 environment
+    p.process.setup_utf8_io()
+    return p
+
+
+# Global singleton PAL instance
+platform: IPlatform = _create_platform()
+
+
+def get_platform() -> IPlatform:
+    """Get the active platform abstraction instance."""
+    global platform
+    return platform
+
+
+__all__ = [
+    "platform",
+    "get_platform",
+    "OSType",
+    "IPlatform",
+    "IPlatformPaths",
+    "IPlatformProcess",
+]

artemis/platform/base.py

@@ -0,0 +1,104 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Platform Abstraction Layer (PAL) - Base Protocols and Types."""
+
+from enum import Enum
+from pathlib import Path
+from typing import Protocol
+
+
+class OSType(str, Enum):
+    """Operating System Type."""
+
+    LINUX = "linux"
+    MACOS = "darwin"
+    WINDOWS = "windows"
+    UNKNOWN = "unknown"
+
+
+class IPlatformPaths(Protocol):
+    """Standardized directory and path resolver for the target operating system."""
+
+    @property
+    def config_dir(self) -> Path:
+        """Directory for persistent user/app configuration files."""
+        ...
+
+    @property
+    def data_dir(self) -> Path:
+        """Directory for persistent data (traces, SQLite DBs, replays)."""
+        ...
+
+    @property
+    def cache_dir(self) -> Path:
+        """Directory for non-essential cached data."""
+        ...
+
+    @property
+    def logs_dir(self) -> Path:
+        """Directory for persistent runtime logs."""
+        ...
+
+    def temp_dir(self, subfolder: str | None = None) -> Path:
+        """Unified temporary files directory for screenshots, recordings, and scratchpads."""
+        ...
+
+    def resolve_app_dir(self) -> Path:
+        """Resolved central application root directory for backwards compatibility."""
+        ...
+
+
+class IPlatformProcess(Protocol):
+    """Platform-specific process management, signals, and stream handling."""
+
+    @property
+    def path_separator(self) -> str:
+        """Separator character used for PATH and PYTHONPATH (':' on POSIX, ';' on Windows)."""
+        ...
+
+    def terminate_process_tree(self, pid: int, timeout_seconds: float = 3.0) -> bool:
+        """Safely and recursively terminate a process and all its child processes."""
+        ...
+
+    def setup_utf8_io(self) -> None:
+        """Ensure standard I/O and process streams operate in UTF-8 mode."""
+        ...
+
+
+class IPlatform(Protocol):
+    """Unified Platform Abstraction Layer interface."""
+
+    @property
+    def os_type(self) -> OSType:
+        """Operating system type identifier."""
+        ...
+
+    @property
+    def paths(self) -> IPlatformPaths:
+        """Platform paths resolver."""
+        ...
+
+    @property
+    def process(self) -> IPlatformProcess:
+        """Platform process and execution manager."""
+        ...
+
+    def get_package_manager_name(self) -> str | None:
+        """Name of the detected preferred package manager (e.g. apt, brew, winget)."""
+        ...
+
+    def get_install_command(self, tool_name: str) -> str:
+        """Generate platform-specific installation command for a missing tool."""
+        ...

artemis/platform/darwin.py

@@ -0,0 +1,150 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""macOS Platform Implementation (Apple Filesystem Hierarchy & Homebrew Integration)."""
+
+import os
+from pathlib import Path
+import shutil
+import signal
+import tempfile
+
+from artemis.platform.base import IPlatform, IPlatformPaths, IPlatformProcess, OSType
+
+
+class DarwinPlatformPaths(IPlatformPaths):
+    """macOS paths adhering to Apple standard Library/Application Support structure."""
+
+    def __init__(self):
+        self._home = Path.home()
+
+    @property
+    def config_dir(self) -> Path:
+        p = self._home / "Library" / "Application Support" / "artemis"
+        p.mkdir(parents=True, exist_ok=True)
+        return p
+
+    @property
+    def data_dir(self) -> Path:
+        env_dir = os.getenv("ARTEMIS_APP_DIR") or os.getenv("ANTIGRAVITY_APP_DIR")
+        if env_dir:
+            p = Path(env_dir)
+            p.mkdir(parents=True, exist_ok=True)
+            return p
+
+        # Check legacy ~/.artemis or ~/.gemini/jetski directory if it exists
+        legacy_artemis = self._home / ".artemis"
+        if legacy_artemis.exists():
+            return legacy_artemis
+
+        legacy_jetski = self._home / ".gemini" / "jetski"
+        if legacy_jetski.exists():
+            return legacy_jetski
+
+        p = self._home / "Library" / "Application Support" / "artemis"
+        p.mkdir(parents=True, exist_ok=True)
+        return p
+
+    @property
+    def cache_dir(self) -> Path:
+        p = self._home / "Library" / "Caches" / "artemis"
+        p.mkdir(parents=True, exist_ok=True)
+        return p
+
+    @property
+    def logs_dir(self) -> Path:
+        p = self._home / "Library" / "Logs" / "artemis"
+        p.mkdir(parents=True, exist_ok=True)
+        return p
+
+    def temp_dir(self, subfolder: str | None = None) -> Path:
+        base = Path(tempfile.gettempdir()) / "artemis"
+        target = base / subfolder if subfolder else base
+        target.mkdir(parents=True, exist_ok=True)
+        return target
+
+    def resolve_app_dir(self) -> Path:
+        return self.data_dir
+
+
+class DarwinPlatformProcess(IPlatformProcess):
+    """macOS process management."""
+
+    @property
+    def path_separator(self) -> str:
+        return ":"
+
+    def terminate_process_tree(self, pid: int, timeout_seconds: float = 3.0) -> bool:
+        try:
+            import psutil
+
+            parent = psutil.Process(pid)
+            children = parent.children(recursive=True)
+            for child in children:
+                try:
+                    child.send_signal(signal.SIGTERM)
+                except (psutil.NoSuchProcess, psutil.AccessDenied):
+                    pass
+
+            parent.send_signal(signal.SIGTERM)
+            gone, alive = psutil.wait_procs(children + [parent], timeout=timeout_seconds)
+            for p in alive:
+                try:
+                    p.kill()
+                except (psutil.NoSuchProcess, psutil.AccessDenied):
+                    pass
+            return True
+        except Exception:
+            try:
+                os.kill(pid, signal.SIGKILL)
+                return True
+            except OSError:
+                return False
+
+    def setup_utf8_io(self) -> None:
+        pass
+
+
+class DarwinPlatform(IPlatform):
+    """macOS Platform Adapter."""
+
+    def __init__(self):
+        self._paths = DarwinPlatformPaths()
+        self._process = DarwinPlatformProcess()
+
+    @property
+    def os_type(self) -> OSType:
+        return OSType.MACOS
+
+    @property
+    def paths(self) -> IPlatformPaths:
+        return self._paths
+
+    @property
+    def process(self) -> IPlatformProcess:
+        return self._process
+
+    def get_package_manager_name(self) -> str | None:
+        if shutil.which("brew"):
+            return "brew"
+        return None
+
+    def get_install_command(self, tool_name: str) -> str:
+        pkg_map = {
+            "adb": "android-platform-tools",
+            "ffmpeg": "ffmpeg",
+            "scrcpy": "scrcpy",
+        }
+        pkg = pkg_map.get(tool_name, tool_name)
+        return f"brew install {pkg}"

artemis/platform/linux.py

@@ -0,0 +1,173 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Linux Platform Implementation (XDG Base Directory Specification & POSIX Process Model)."""
+
+import os
+from pathlib import Path
+import shutil
+import signal
+import tempfile
+
+from artemis.platform.base import IPlatform, IPlatformPaths, IPlatformProcess, OSType
+
+
+class LinuxPlatformPaths(IPlatformPaths):
+    """Linux paths adhering to the XDG Base Directory specification with legacy fallbacks."""
+
+    def __init__(self):
+        self._home = Path.home()
+
+    @property
+    def config_dir(self) -> Path:
+        xdg_config = os.getenv("XDG_CONFIG_HOME")
+        if xdg_config:
+            p = Path(xdg_config) / "artemis"
+        else:
+            p = self._home / ".config" / "artemis"
+        p.mkdir(parents=True, exist_ok=True)
+        return p
+
+    @property
+    def data_dir(self) -> Path:
+        # Check custom app dir override first
+        env_dir = os.getenv("ARTEMIS_APP_DIR") or os.getenv("ANTIGRAVITY_APP_DIR")
+        if env_dir:
+            p = Path(env_dir)
+            p.mkdir(parents=True, exist_ok=True)
+            return p
+
+        xdg_data = os.getenv("XDG_DATA_HOME")
+        if xdg_data:
+            p = Path(xdg_data) / "artemis"
+            p.mkdir(parents=True, exist_ok=True)
+            return p
+
+        # Check legacy ~/.gemini/jetski or ~/.artemis directory if it exists
+        legacy_jetski = self._home / ".gemini" / "jetski"
+        if legacy_jetski.exists():
+            return legacy_jetski
+
+        legacy_artemis = self._home / ".artemis"
+        if legacy_artemis.exists():
+            return legacy_artemis
+
+        p = self._home / ".local" / "share" / "artemis"
+        p.mkdir(parents=True, exist_ok=True)
+        return p
+
+    @property
+    def cache_dir(self) -> Path:
+        xdg_cache = os.getenv("XDG_CACHE_HOME")
+        if xdg_cache:
+            p = Path(xdg_cache) / "artemis"
+        else:
+            p = self._home / ".cache" / "artemis"
+        p.mkdir(parents=True, exist_ok=True)
+        return p
+
+    @property
+    def logs_dir(self) -> Path:
+        p = self.data_dir / "logs"
+        p.mkdir(parents=True, exist_ok=True)
+        return p
+
+    def temp_dir(self, subfolder: str | None = None) -> Path:
+        base = Path(tempfile.gettempdir()) / "artemis"
+        target = base / subfolder if subfolder else base
+        target.mkdir(parents=True, exist_ok=True)
+        return target
+
+    def resolve_app_dir(self) -> Path:
+        return self.data_dir
+
+
+class LinuxPlatformProcess(IPlatformProcess):
+    """Linux process management using POSIX signals and process groups."""
+
+    @property
+    def path_separator(self) -> str:
+        return ":"
+
+    def terminate_process_tree(self, pid: int, timeout_seconds: float = 3.0) -> bool:
+        try:
+            import psutil
+
+            parent = psutil.Process(pid)
+            children = parent.children(recursive=True)
+            for child in children:
+                try:
+                    child.send_signal(signal.SIGTERM)
+                except (psutil.NoSuchProcess, psutil.AccessDenied):
+                    pass
+
+            parent.send_signal(signal.SIGTERM)
+            gone, alive = psutil.wait_procs(children + [parent], timeout=timeout_seconds)
+            for p in alive:
+                try:
+                    p.kill()
+                except (psutil.NoSuchProcess, psutil.AccessDenied):
+                    pass
+            return True
+        except Exception:
+            try:
+                os.kill(pid, signal.SIGKILL)
+                return True
+            except OSError:
+                return False
+
+    def setup_utf8_io(self) -> None:
+        pass
+
+
+class LinuxPlatform(IPlatform):
+    """Linux Platform Adapter."""
+
+    def __init__(self):
+        self._paths = LinuxPlatformPaths()
+        self._process = LinuxPlatformProcess()
+
+    @property
+    def os_type(self) -> OSType:
+        return OSType.LINUX
+
+    @property
+    def paths(self) -> IPlatformPaths:
+        return self._paths
+
+    @property
+    def process(self) -> IPlatformProcess:
+        return self._process
+
+    def get_package_manager_name(self) -> str | None:
+        for pm in ("apt-get", "dnf", "pacman", "zypper"):
+            if shutil.which(pm):
+                return pm
+        return None
+
+    def get_install_command(self, tool_name: str) -> str:
+        pm = self.get_package_manager_name()
+        pkg_map = {
+            "adb": {"apt-get": "adb", "dnf": "android-tools", "pacman": "android-tools"},
+            "ffmpeg": {"apt-get": "ffmpeg", "dnf": "ffmpeg", "pacman": "ffmpeg"},
+            "scrcpy": {"apt-get": "scrcpy", "dnf": "scrcpy", "pacman": "scrcpy"},
+        }
+        pkg = pkg_map.get(tool_name, {}).get(pm, tool_name)
+        if pm == "apt-get":
+            return f"sudo apt-get install -y {pkg}"
+        elif pm == "dnf":
+            return f"sudo dnf install -y {pkg}"
+        elif pm == "pacman":
+            return f"sudo pacman -S --noconfirm {pkg}"
+        return f"Install {tool_name} using your system package manager."

artemis/platform/windows.py

@@ -0,0 +1,190 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Windows Platform Implementation (KnownFolders/AppData, Win32 Process Tree & UTF-8 Injection)."""
+
+import os
+from pathlib import Path
+import shutil
+import subprocess
+import sys
+import tempfile
+
+from artemis.platform.base import IPlatform, IPlatformPaths, IPlatformProcess, OSType
+
+
+class WindowsPlatformPaths(IPlatformPaths):
+    """Windows paths adhering to KnownFolders (%APPDATA% / %LOCALAPPDATA%)."""
+
+    def __init__(self):
+        self._home = Path.home()
+
+    @property
+    def config_dir(self) -> Path:
+        appdata = os.getenv("APPDATA")
+        if appdata:
+            p = Path(appdata) / "Artemis"
+        else:
+            p = self._home / "AppData" / "Roaming" / "Artemis"
+        p.mkdir(parents=True, exist_ok=True)
+        return p
+
+    @property
+    def data_dir(self) -> Path:
+        env_dir = os.getenv("ARTEMIS_APP_DIR") or os.getenv("ANTIGRAVITY_APP_DIR")
+        if env_dir:
+            p = Path(env_dir)
+            p.mkdir(parents=True, exist_ok=True)
+            return p
+
+        local_appdata = os.getenv("LOCALAPPDATA")
+        if local_appdata:
+            p = Path(local_appdata) / "Artemis"
+            p.mkdir(parents=True, exist_ok=True)
+            return p
+
+        legacy_jetski = self._home / ".gemini" / "jetski"
+        if legacy_jetski.exists():
+            return legacy_jetski
+
+        legacy_artemis = self._home / ".artemis"
+        if legacy_artemis.exists():
+            return legacy_artemis
+
+        p = self._home / "AppData" / "Local" / "Artemis"
+        p.mkdir(parents=True, exist_ok=True)
+        return p
+
+    @property
+    def cache_dir(self) -> Path:
+        p = self.data_dir / "cache"
+        p.mkdir(parents=True, exist_ok=True)
+        return p
+
+    @property
+    def logs_dir(self) -> Path:
+        p = self.data_dir / "logs"
+        p.mkdir(parents=True, exist_ok=True)
+        return p
+
+    def temp_dir(self, subfolder: str | None = None) -> Path:
+        base = Path(tempfile.gettempdir()) / "artemis"
+        target = base / subfolder if subfolder else base
+        target.mkdir(parents=True, exist_ok=True)
+        return target
+
+    def resolve_app_dir(self) -> Path:
+        return self.data_dir
+
+
+class WindowsPlatformProcess(IPlatformProcess):
+    """Windows process management with tree termination and UTF-8 console streams."""
+
+    @property
+    def path_separator(self) -> str:
+        return ";"
+
+    def terminate_process_tree(self, pid: int, timeout_seconds: float = 3.0) -> bool:
+        # 1. Try taskkill /F /T /PID on Windows (fastest and cleanest for process trees)
+        try:
+            res = subprocess.run(
+                ["taskkill", "/F", "/T", "/PID", str(pid)],
+                stdout=subprocess.DEVNULL,
+                stderr=subprocess.DEVNULL,
+                check=False,
+                timeout=timeout_seconds,
+            )
+            if res.returncode == 0:
+                return True
+        except Exception:
+            pass
+
+        # 2. Fallback to psutil
+        try:
+            import psutil
+
+            parent = psutil.Process(pid)
+            children = parent.children(recursive=True)
+            for child in children:
+                try:
+                    child.kill()
+                except (psutil.NoSuchProcess, psutil.AccessDenied):
+                    pass
+            parent.kill()
+            return True
+        except Exception:
+            return False
+
+    def setup_utf8_io(self) -> None:
+        """Configures Windows stdout/stderr to UTF-8 mode to prevent encoding crashes."""
+        if hasattr(sys.stdout, "reconfigure"):
+            try:
+                sys.stdout.reconfigure(encoding="utf-8", errors="replace")
+            except Exception:
+                pass
+        if hasattr(sys.stderr, "reconfigure"):
+            try:
+                sys.stderr.reconfigure(encoding="utf-8", errors="replace")
+            except Exception:
+                pass
+        os.environ["PYTHONUTF8"] = "1"
+
+
+class WindowsPlatform(IPlatform):
+    """Windows Platform Adapter."""
+
+    def __init__(self):
+        self._paths = WindowsPlatformPaths()
+        self._process = WindowsPlatformProcess()
+
+    @property
+    def os_type(self) -> OSType:
+        return OSType.WINDOWS
+
+    @property
+    def paths(self) -> IPlatformPaths:
+        return self._paths
+
+    @property
+    def process(self) -> IPlatformProcess:
+        return self._process
+
+    def get_package_manager_name(self) -> str | None:
+        for pm in ("winget", "choco", "scoop"):
+            if shutil.which(pm):
+                return pm
+        return None
+
+    def get_install_command(self, tool_name: str) -> str:
+        pm = self.get_package_manager_name() or "winget"
+        pkg_map = {
+            "adb": {
+                "winget": "winget install Google.PlatformTools",
+                "choco": "choco install adb",
+                "scoop": "scoop install adb",
+            },
+            "ffmpeg": {
+                "winget": "winget install Gyan.FFmpeg",
+                "choco": "choco install ffmpeg",
+                "scoop": "scoop install ffmpeg",
+            },
+            "scrcpy": {
+                "winget": "winget install Genymobile.scrcpy",
+                "choco": "choco install scrcpy",
+                "scoop": "scoop install scrcpy",
+            },
+        }
+        if tool_name in pkg_map and pm in pkg_map[tool_name]:
+            return pkg_map[tool_name][pm]
+        return f"winget install {tool_name} (or download official installer)"

artemis/runtime/__init__.py

@@ -0,0 +1,22 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Artemis cross-platform runtime and process supervisor subsystem."""
+
+from artemis.runtime.supervisor import ProcessSupervisor, process_supervisor
+
+__all__ = [
+    "ProcessSupervisor",
+    "process_supervisor",
+]

artemis/runtime/supervisor.py

@@ -0,0 +1,94 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Cross-platform Process Supervisor and Lifecycle Manager."""
+
+import asyncio
+from collections.abc import Sequence
+import os
+from pathlib import Path
+
+from artemis.platform import platform
+from artemis.utils.logger import get_logger
+
+logger = get_logger(__name__)
+
+
+class ProcessSupervisor:
+    """Manages process spawning, tree cleanup, signals, and UTF-8 stream handling."""
+
+    @staticmethod
+    def prepare_environment(extra_env: dict[str, str] | None = None) -> dict[str, str]:
+        """Prepare child process environment with guaranteed UTF-8 mode."""
+        env = os.environ.copy()
+        env["PYTHONUNBUFFERED"] = "1"
+        env["PYTHONUTF8"] = "1"
+        if extra_env:
+            env.update(extra_env)
+        return env
+
+    @staticmethod
+    async def spawn_async(
+        cmd: Sequence[str],
+        cwd: Path | str | None = None,
+        env: dict[str, str] | None = None,
+        capture_output: bool = True,
+    ) -> asyncio.subprocess.Process:
+        """Spawn an asynchronous subprocess with clean environment and cross-platform flags."""
+        prepared_env = ProcessSupervisor.prepare_environment(env)
+        cwd_str = str(cwd) if cwd else None
+
+        stdout_dest = asyncio.subprocess.PIPE if capture_output else None
+        stderr_dest = asyncio.subprocess.PIPE if capture_output else None
+
+        return await asyncio.create_subprocess_exec(
+            *cmd,
+            stdout=stdout_dest,
+            stderr=stderr_dest,
+            cwd=cwd_str,
+            env=prepared_env,
+        )
+
+    @staticmethod
+    def terminate_tree(pid: int, timeout_seconds: float = 3.0) -> bool:
+        """Recursively terminate a process tree."""
+        return platform.process.terminate_process_tree(pid, timeout_seconds=timeout_seconds)
+
+    @staticmethod
+    async def stop_process(
+        proc: asyncio.subprocess.Process | None,
+        timeout_seconds: float = 3.0,
+    ) -> None:
+        """Safely stop an asyncio Process and kill its children if necessary."""
+        if proc is None:
+            return
+
+        if proc.returncode is not None:
+            return
+
+        try:
+            if proc.pid:
+                ProcessSupervisor.terminate_tree(proc.pid, timeout_seconds=timeout_seconds)
+            else:
+                proc.terminate()
+            await asyncio.wait_for(proc.wait(), timeout=timeout_seconds)
+        except Exception as e:
+            logger.debug(f"Failed graceful stop for process {proc}: {e}")
+            try:
+                proc.kill()
+            except Exception:
+                pass
+
+
+process_supervisor: ProcessSupervisor = ProcessSupervisor()

artemis/sdk/agent.py

@@ -197,11 +197,11 @@ class Agent:
             client = genai.Client(api_key=key)
 
             # 2. Pre-warm LangChain client
-            chat = ChatGoogleGenerativeAI(model="gemini-3.6-flash", google_api_key=key)
+            chat = ChatGoogleGenerativeAI(model="gemini-3.7-flash", google_api_key=key)
 
             # Fire both calls concurrently in the background
             await asyncio.gather(
-                client.aio.models.count_tokens(model="gemini-3.6-flash", contents="ping"),
+                client.aio.models.count_tokens(model="gemini-3.7-flash", contents="ping"),
                 chat.ainvoke("ping"),
                 return_exceptions=True,
             )
@@ -1001,8 +1001,10 @@ class Agent:
             logger.warning("UIAutomator client not available, using default dimensions")
             device_width, device_height = 1080, 2340
 
+        from artemis.platform import platform as pal_platform
+
         return DeviceContext(
-            host_platform="WINDOWS" if host_platform == "Windows" else "LINUX",
+            host_platform=pal_platform.os_type.name,
             mobile_platform=platform,
             device_id=device_id,
             device_width=device_width,

artemis/sdk/examples/video_transcription_example.py

@@ -38,7 +38,8 @@ def get_video_capable_llm_config() -> LLMConfig:
     """Returns an LLM config with video_analyzer configured.
 
     The video_analyzer must use a video-capable Gemini model:
-    - gemini-3.6-flash (recommended - fast and capable)
+    - gemini-3.7-flash (recommended - fast and capable)
+    - gemini-3.6-flash
     - gemini-3-pro-preview
     - gemini-2.5-flash
     - gemini-2.5-pro
@@ -63,8 +64,8 @@ def get_video_capable_llm_config() -> LLMConfig:
             ),
             video_analyzer=LLMWithFallback(
                 provider="google",
-                model="gemini-3.6-flash",
-                fallback=LLM(provider="google", model="gemini-2.5-flash"),
+                model="gemini-3.7-flash",
+                fallback=LLM(provider="google", model="gemini-3.6-flash"),
             ),
         ),
     )

artemis/sdk/types/agent.py

@@ -25,7 +25,7 @@ from artemis.config import (
 from artemis.context import DevicePlatform
 from artemis.sdk.types.task import AgentProfile, TaskRequestCommon
 from artemis.utils.video import detect_video_tools_enabled
-from pydantic import AliasChoices, BaseModel, Field
+from pydantic import BaseModel, Field
 
 
 class _CyFunctionDetectorMeta(type):
@@ -125,10 +125,7 @@ class AgentConfig(BaseModel):
 
     explorer: ExplorerConfig = Field(default_factory=ExplorerConfig)
     explorer_versions: dict[str, Literal["flash", "pro", "ultra"]]
-    denylisted_tools: dict[str, list[str]] = Field(
-        default_factory=dict,
-        validation_alias=AliasChoices("denylisted_tools", "blocklisted_tools"),
-    )
+    denylisted_tools: dict[str, list[str]]
     enable_video_ledger: bool = True
 
     model_config = {"arbitrary_types_allowed": True}

artemis/services/llm.py

@@ -395,7 +395,7 @@ async def invoke_llm_with_timeout_message[T](
 
 # Backward compatible factory functions delegating to ModelFactory
 def get_google_llm(
-    model_name: str = "gemini-3.6-flash",
+    model_name: str = "gemini-3.7-flash",
     temperature: float | None = None,
     timeout: float | None = None,
     thinking_budget: int | None = None,
@@ -417,7 +417,7 @@ def get_google_llm(
 
 
 def get_vertex_llm(
-    model_name: str = "gemini-3.6-flash",
+    model_name: str = "gemini-3.7-flash",
     temperature: float | None = None,
     timeout: float | None = None,
     thinking_budget: int | None = None,

artemis/toolchain/__init__.py

@@ -0,0 +1,34 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Managed Toolchain and Binary Discovery Subsystem."""
+
+from artemis.toolchain.descriptors import TOOLS, ToolDescriptor
+from artemis.toolchain.resolver import ToolchainResolver, toolchain
+
+find_adb = toolchain.find_adb
+find_ffmpeg = toolchain.find_ffmpeg
+find_scrcpy = toolchain.find_scrcpy
+is_installed = toolchain.is_installed
+
+__all__ = [
+    "toolchain",
+    "ToolchainResolver",
+    "ToolDescriptor",
+    "TOOLS",
+    "find_adb",
+    "find_ffmpeg",
+    "find_scrcpy",
+    "is_installed",
+]

artemis/toolchain/descriptors.py

@@ -0,0 +1,86 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Toolchain descriptors and metadata for cross-platform binary discovery."""
+
+from collections.abc import Callable
+from dataclasses import dataclass
+
+
+def _get_embedded_adb() -> str | None:
+    """Fallback to adb binary bundled with adbutils if available."""
+    try:
+        import adbutils
+
+        if hasattr(adbutils, "adb_path"):
+            return adbutils.adb_path()
+    except Exception:
+        pass
+    return None
+
+
+def _get_embedded_ffmpeg() -> str | None:
+    """Fallback to ffmpeg binary bundled with imageio_ffmpeg if available."""
+    try:
+        import imageio_ffmpeg
+
+        if hasattr(imageio_ffmpeg, "get_ffmpeg_exe"):
+            return imageio_ffmpeg.get_ffmpeg_exe()
+    except Exception:
+        pass
+    return None
+
+
+@dataclass(frozen=True)
+class ToolDescriptor:
+    """Metadata and resolution strategies for an external binary/toolchain."""
+
+    name: str
+    binary_name: str
+    win_binary_name: str
+    sdk_relative_path: str | None = None
+    embedded_fallback: Callable[[], str | None] | None = None
+    description: str = ""
+
+
+# Standard supported toolchains
+TOOLS: dict[str, ToolDescriptor] = {
+    "adb": ToolDescriptor(
+        name="adb",
+        binary_name="adb",
+        win_binary_name="adb.exe",
+        sdk_relative_path="platform-tools/adb",
+        embedded_fallback=_get_embedded_adb,
+        description="Android Debug Bridge client",
+    ),
+    "ffmpeg": ToolDescriptor(
+        name="ffmpeg",
+        binary_name="ffmpeg",
+        win_binary_name="ffmpeg.exe",
+        embedded_fallback=_get_embedded_ffmpeg,
+        description="FFmpeg multimedia processing framework",
+    ),
+    "scrcpy": ToolDescriptor(
+        name="scrcpy",
+        binary_name="scrcpy",
+        win_binary_name="scrcpy.exe",
+        description="Screen Copy display and control tool",
+    ),
+    "uv": ToolDescriptor(
+        name="uv",
+        binary_name="uv",
+        win_binary_name="uv.exe",
+        description="Fast Python package manager",
+    ),
+}

artemis/toolchain/resolver.py

@@ -0,0 +1,178 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Multi-tier cross-platform binary resolution and discovery engine."""
+
+import os
+from pathlib import Path
+import shutil
+
+from artemis.platform import OSType, platform
+from artemis.toolchain.descriptors import TOOLS, ToolDescriptor
+
+
+class ToolchainResolver:
+    """Discovers, validates, and resolves platform-specific binaries with graceful fallbacks."""
+
+    def __init__(self):
+        self._cache: dict[str, str | None] = {}
+
+    def clear_cache(self, tool_name: str | None = None) -> None:
+        """Invalidate cached tool paths to force fresh discovery."""
+        if tool_name:
+            self._cache.pop(tool_name.lower(), None)
+        else:
+            self._cache.clear()
+
+    def resolve(self, tool_name: str, force_refresh: bool = False) -> str | None:
+        """Resolve absolute path to a tool across all discovery tiers."""
+        key = tool_name.lower()
+        if not force_refresh and key in self._cache:
+            cached_val = self._cache[key]
+            if cached_val is not None and Path(cached_val).exists():
+                return cached_val
+            # Cache miss or binary removed: proceed to re-discover
+
+        desc = TOOLS.get(key)
+        if desc is None:
+            # Generic binary lookup
+            res = shutil.which(tool_name)
+            if res and Path(res).exists():
+                self._cache[key] = str(Path(res).resolve())
+                return self._cache[key]
+            self._cache.pop(key, None)
+            return None
+
+        binary_name = (
+            desc.win_binary_name if platform.os_type == OSType.WINDOWS else desc.binary_name
+        )
+
+        # Tier 1: Explicit environment variable override (e.g. ARTEMIS_ADB_PATH)
+        env_var = f"ARTEMIS_{desc.name.upper()}_PATH"
+        env_override = os.getenv(env_var)
+        if env_override and Path(env_override).exists():
+            resolved = str(Path(env_override).resolve())
+            self._cache[key] = resolved
+            return resolved
+
+        # Tier 2: SDK Environment Variables (for Android tools)
+        if desc.sdk_relative_path:
+            for sdk_env in ("ANDROID_HOME", "ANDROID_SDK_ROOT"):
+                sdk_dir = os.getenv(sdk_env)
+                if sdk_dir:
+                    candidate = Path(sdk_dir) / desc.sdk_relative_path
+                    if platform.os_type == OSType.WINDOWS:
+                        candidate = candidate.with_suffix(".exe")
+                    if candidate.exists():
+                        resolved = str(candidate.resolve())
+                        self._cache[key] = resolved
+                        return resolved
+
+        # Tier 3: Common OS default installation paths
+        common_candidates = self._get_os_common_paths(desc)
+        for cand in common_candidates:
+            if cand.exists():
+                resolved = str(cand.resolve())
+                self._cache[key] = resolved
+                return resolved
+
+        # Tier 4: System PATH
+        which_path = shutil.which(binary_name) or shutil.which(desc.name)
+        if which_path:
+            resolved = str(Path(which_path).resolve())
+            self._cache[key] = resolved
+            return resolved
+
+        # Tier 5: Python library embedded binary fallback
+        if desc.embedded_fallback:
+            try:
+                embedded = desc.embedded_fallback()
+                if embedded and Path(embedded).exists():
+                    resolved = str(Path(embedded).resolve())
+                    self._cache[key] = resolved
+                    return resolved
+            except Exception:
+                pass
+
+        # Do not permanently cache negative lookups so dynamic installs are detected immediately
+        self._cache.pop(key, None)
+        return None
+
+    def _get_os_common_paths(self, desc: ToolDescriptor) -> list[Path]:
+        candidates: list[Path] = []
+        home = Path.home()
+
+        if platform.os_type == OSType.WINDOWS:
+            local_appdata = os.getenv("LOCALAPPDATA", "")
+            program_files = os.getenv("ProgramFiles", "C:\\Program Files")
+            program_data = os.getenv("ProgramData", "C:\\ProgramData")
+
+            if desc.name == "adb":
+                if local_appdata:
+                    candidates.append(
+                        Path(local_appdata) / "Android" / "Sdk" / "platform-tools" / "adb.exe"
+                    )
+                candidates.append(Path(program_files) / "Android" / "platform-tools" / "adb.exe")
+            elif desc.name == "scrcpy":
+                candidates.append(Path(program_data) / "chocolatey" / "bin" / "scrcpy.exe")
+                if local_appdata:
+                    candidates.append(
+                        Path(local_appdata) / "Microsoft" / "WinGet" / "Links" / "scrcpy.exe"
+                    )
+            elif desc.name == "ffmpeg":
+                candidates.append(Path(program_data) / "chocolatey" / "bin" / "ffmpeg.exe")
+                if local_appdata:
+                    candidates.append(
+                        Path(local_appdata) / "Microsoft" / "WinGet" / "Links" / "ffmpeg.exe"
+                    )
+
+        elif platform.os_type == OSType.MACOS:
+            if desc.name == "adb":
+                candidates.append(home / "Library" / "Android" / "sdk" / "platform-tools" / "adb")
+                candidates.append(Path("/opt/homebrew/bin/adb"))
+                candidates.append(Path("/usr/local/bin/adb"))
+            elif desc.name in ("ffmpeg", "scrcpy"):
+                candidates.append(Path(f"/opt/homebrew/bin/{desc.name}"))
+                candidates.append(Path(f"/usr/local/bin/{desc.name}"))
+
+        elif platform.os_type == OSType.LINUX:
+            if desc.name == "adb":
+                candidates.append(home / "Android" / "Sdk" / "platform-tools" / "adb")
+                candidates.append(Path("/usr/lib/android-sdk/platform-tools/adb"))
+                candidates.append(Path("/usr/bin/adb"))
+            elif desc.name in ("ffmpeg", "scrcpy"):
+                candidates.append(Path(f"/usr/bin/{desc.name}"))
+                candidates.append(Path(f"/usr/local/bin/{desc.name}"))
+
+        return candidates
+
+    def find_adb(self) -> str:
+        """Find ADB executable or fallback to default name."""
+        return self.resolve("adb") or "adb"
+
+    def find_ffmpeg(self) -> str:
+        """Find FFmpeg executable or fallback to default name."""
+        return self.resolve("ffmpeg") or "ffmpeg"
+
+    def find_scrcpy(self) -> str:
+        """Find scrcpy executable or fallback to default name."""
+        return self.resolve("scrcpy") or "scrcpy"
+
+    def is_installed(self, tool_name: str) -> bool:
+        """Check whether a tool is found on this system."""
+        return self.resolve(tool_name) is not None
+
+
+# Global toolchain resolver singleton
+toolchain: ToolchainResolver = ToolchainResolver()

artemis/tools/actions/device_actions.py

@@ -18,7 +18,10 @@ import asyncio
 from typing import Any, Literal
 from artemis.drivers.base import BaseDeviceDriver
 from artemis.tools.base import artemis_tool
-from artemis.utils.coordinates import parse_swipe_parameters
+from artemis.utils.coordinates import (
+    compute_smart_swipe_coordinates,
+    parse_swipe_parameters,
+)
 from artemis.utils.logger import get_logger
 from pydantic import BaseModel, Field
 
@@ -53,9 +56,7 @@ class SwipeArgs(BaseModel):
     direction: Literal["up", "down", "left", "right"] | None = Field(
         None,
         description=(
-            "Smart directional swipe ('up', 'down', 'left', 'right') for general page"
-            " scrolling/browsing. Automatically applies safe vectors and a ~40% visual"
-            " overlap anchor: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
+            "Direction for scrolling and swiping: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
             " 'down' (drags top-to-bottom, scrolling up to reveal content above),"
             " 'left' (drags right-to-left, scrolling right),"
             " 'right' (drags left-to-right, scrolling left)."
@@ -83,9 +84,9 @@ class SwipeArgs(BaseModel):
         ),
     )
     duration: int = Field(
-        400,
+        800,
         description=(
-            "Swipe/drag duration in milliseconds (default 400). For drag-and-drop, "
+            "Swipe/drag duration in milliseconds (default 800). For drag-and-drop, "
             "list reordering, or sliding/adjusting sliders (e.g. volume, brightness, SeekBars), "
             "set duration >= 1000 (e.g. 1500)."
         ),
@@ -101,7 +102,11 @@ class PressKeyArgs(BaseModel):
 
 class WaitForDelayArgs(BaseModel):
     seconds: float = Field(
-        ..., description="Duration in seconds to wait for animations, pages, or state updates."
+        ...,
+        description=(
+            "Duration in seconds to wait (e.g., 2 for 2s, 60 for 1 min, 180 for 3 mins)."
+            " Use whenever time needs to elapse for animations, loading, or scheduled delays."
+        ),
     )
 
 
@@ -182,7 +187,7 @@ async def swipe(
     direction: str | None = None,
     start: list[int] | None = None,
     end: list[int] | None = None,
-    duration: int = 400,
+    duration: int = 800,
     driver: BaseDeviceDriver = None,
     **kwargs: Any,
 ) -> str:
@@ -200,7 +205,15 @@ async def swipe(
     kind, target, dur = parse_swipe_parameters(params, default_duration=duration)
 
     if kind == "direction" and isinstance(target, str):
-        success = await driver.swipe_direction(target, duration_ms=dur)
+        width, height = driver.screen_size
+        sx, sy, ex, ey, smart_dur = compute_smart_swipe_coordinates(
+            direction=target,
+            target=params.get("target"),
+            width=width,
+            height=height,
+            duration=dur,
+        )
+        success = await driver.swipe(sx, sy, ex, ey, duration_ms=smart_dur)
         return f"Swiped {target}" if success else f"Swipe {target} failed"
     elif kind == "coords" and isinstance(target, list) and len(target) == 4:
         width, height = driver.screen_size

artemis/tools/base.py

@@ -57,6 +57,10 @@ class ArtemisTool:
         self.handler = handler
         self.category = category
 
+    def is_available(self, ctx: Any = None) -> bool:
+        """Determines if this tool is currently available and should be exposed to agents."""
+        return True
+
     async def execute(self, driver: BaseDeviceDriver, ctx: Any, **kwargs: Any) -> Any:
         """Executes the tool handler with injected driver and context."""
         # Inspect handler signature to only pass supported arguments
@@ -213,25 +217,32 @@ class ToolRegistry:
         return cls._tools.get(normalized)
 
     @classmethod
-    def list_tools(cls, category: ToolCategory | None = None) -> list[ArtemisTool]:
-        """Lists all registered tools, optionally filtered by category."""
+    def list_tools(
+        cls, category: ToolCategory | None = None, available_only: bool = True
+    ) -> list[ArtemisTool]:
+        """Lists all registered tools, optionally filtered by category and availability."""
+        tools = [t for t in cls._tools.values() if not available_only or t.is_available()]
         if category:
-            return [t for t in cls._tools.values() if t.category == category]
-        return list(cls._tools.values())
+            return [t for t in tools if t.category == category]
+        return list(tools)
 
     @classmethod
-    def get_langchain_tools(cls, ctx: Any, names: list[str] | None = None) -> list[BaseTool]:
+    def get_langchain_tools(
+        cls, ctx: Any, names: list[str] | None = None, available_only: bool = True
+    ) -> list[BaseTool]:
         """Returns LangChain tool wrappers for the requested or all registered tools."""
         tools = (
             [cls._tools[n] for n in names if n in cls._tools]
             if names
             else list(cls._tools.values())
         )
+        if available_only:
+            tools = [t for t in tools if t.is_available(ctx)]
         return [t.to_langchain_tool(ctx) for t in tools]
 
     @classmethod
     def get_genai_declarations(
-        cls, names: list[str] | None = None
+        cls, names: list[str] | None = None, available_only: bool = True
     ) -> list[genai_types.FunctionDeclaration]:
         """Returns Google GenAI function declarations for registered tools."""
         tools = (
@@ -239,6 +250,8 @@ class ToolRegistry:
             if names
             else list(cls._tools.values())
         )
+        if available_only:
+            tools = [t for t in tools if t.is_available()]
         return [t.to_genai_declaration() for t in tools]
 
     @classmethod

artemis/tools/index.py

@@ -26,6 +26,8 @@ def get_tools_from_wrappers(
     """Instantiate and wrap LangChain tools from a list of ToolWrappers."""
     tools: list[BaseTool] = []
     for wrapper in wrappers:
+        if wrapper.is_available_fn is not None and not wrapper.is_available_fn(ctx):
+            continue
         if isinstance(wrapper, CompositeToolWrapper):
             comp_tools = wrapper.composite_tools_fn_getter(ctx)
             for t in comp_tools:

artemis/tools/mobile/exec_tools.py

@@ -29,7 +29,10 @@ from artemis.graph.state import State
 from artemis.tools.base import ArtemisTool, ToolCategory, ToolRegistry
 from artemis.tools.tool_wrapper import ToolWrapper
 from artemis.tools.types import CyFunctionDetector
-from artemis.utils.coordinates import parse_swipe_parameters
+from artemis.utils.coordinates import (
+    compute_smart_swipe_coordinates,
+    parse_swipe_parameters,
+)
 from artemis.utils.logger import get_logger
 
 logger = get_logger(__name__)
@@ -84,9 +87,7 @@ class SwipeArgs(BaseModel):
     direction: Literal["up", "down", "left", "right"] | None = Field(
         None,
         description=(
-            "Smart directional swipe ('up', 'down', 'left', 'right') for general page"
-            " scrolling/browsing. Automatically applies safe vectors and a ~40% visual"
-            " overlap anchor: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
+            "Direction for scrolling and swiping: 'up' (drags bottom-to-top, scrolling down to reveal content below),"
             " 'down' (drags top-to-bottom, scrolling up to reveal content above),"
             " 'left' (drags right-to-left, scrolling right),"
             " 'right' (drags left-to-right, scrolling left)."
@@ -106,6 +107,12 @@ class SwipeArgs(BaseModel):
             " local interactions (e.g. adjusting sliders, SeekBars, fine range selection, or drag-and-drop)."
         ),
     )
+    target: int | list[int] | str | None = Field(
+        None,
+        description=(
+            "Optional target element index (e.g. 2) or container bounds [left, top, right, bottom] to scope the directional swipe within."
+        ),
+    )
     action: Literal["up", "down", "left", "right"] | list[int] | None = Field(
         None,
         description=(
@@ -114,9 +121,9 @@ class SwipeArgs(BaseModel):
         ),
     )
     duration: int = Field(
-        400,
+        800,
         description=(
-            "Optional swipe/drag duration in milliseconds (default 400). For drag-and-drop,"
+            "Optional swipe/drag duration in milliseconds (default 800). For drag-and-drop,"
             " list reordering, or sliding/adjusting sliders (e.g., volume, brightness, SeekBars),"
             " set duration >= 1000 (e.g. 1500). If omitted for directional swipe, duration is computed automatically."
         ),
@@ -438,16 +445,19 @@ def get_input_text_tool(ctx: ArtemisContext) -> BaseTool:
 SWIPE_DOCSTRING = (
     "[ACTION] Perform a swipe, drag, or slider-adjustment gesture on the"
     " screen.\n\n"
-    "• Smart Directional Scrolling ('direction'): Best for general browsing"
-    " and page scrolling. Automatically computes safe swipe vectors (locking"
-    " axes to prevent edge/sidebar mis-clicks), retains a ~40% visual overlap"
-    " anchor for zero-omission traversal, and prevents inertial flings.\n"
+    "• Directional Scrolling ('direction'): Recommended for general"
+    " browsing and standard page scrolling in most scenarios. Automatically"
+    " computes safe swipe vectors and adaptive duration, retains a ~40% visual overlap"
+    " anchor for zero-omission traversal, and prevents inertial flings. Supports"
+    " scoping to a sub-container via 'target'. If it fails on certain custom layouts,"
+    " fall back to specifying exact coordinates ('start' and 'end') directly.\n"
     "• Precise Coordinate Gestures ('start', 'end'): Best for local,"
     " fine-grained interactions such as adjusting sliders/SeekBars (e.g.,"
-    " volume, brightness, progress bars) or drag-and-drop / list reordering."
-    " Always drag slightly PAST the target position to overcome touch slop"
-    " and reliably trigger the update. When setting a slider to Maximum (100%)"
-    " or Minimum (0%), swipe fully to the extreme boundary."
+    " volume, brightness, progress bars), drag-and-drop / list reordering,"
+    " or as a reliable fallback when directional scrolling fails on specific"
+    " containers. Always drag slightly PAST the target position to overcome"
+    " touch slop and reliably trigger the update. When setting a slider to"
+    " Maximum (100%) or Minimum (0%), swipe fully to the extreme boundary."
 )
 
 
@@ -471,7 +481,7 @@ class SwipeTool(ArtemisTool):
         direction: Literal["up", "down", "left", "right"] | str | None = None,
         start: list[int] | None = None,
         end: list[int] | None = None,
-        duration: int = 400,
+        duration: int = 800,
         tool_call_id: str | None = None,
         state: State | None = None,
         **kwargs: Any,
@@ -488,7 +498,7 @@ class SwipeTool(ArtemisTool):
         if duration is not None:
             params["duration"] = duration
 
-        kind, target, dur = parse_swipe_parameters(params, default_duration=duration or 400)
+        kind, target, dur = parse_swipe_parameters(params, default_duration=duration or 800)
 
         tcid = tool_call_id if tool_call_id is not None else kwargs.get("tool_call_id")
         st = state if state is not None else kwargs.get("state")
@@ -534,38 +544,18 @@ class SwipeTool(ArtemisTool):
                 controller = UnifiedMobileController(ctx)
 
                 if kind == "direction" and isinstance(target, str):
-                    if target == "up":
-                        coords = [
-                            width // 2,
-                            int(height * 0.8),
-                            width // 2,
-                            int(height * 0.2),
-                        ]
-                    elif target == "down":
-                        coords = [
-                            width // 2,
-                            int(height * 0.2),
-                            width // 2,
-                            int(height * 0.8),
-                        ]
-                    elif target == "left":
-                        coords = [
-                            int(width * 0.8),
-                            height // 2,
-                            int(width * 0.2),
-                            height // 2,
-                        ]
-                    elif target == "right":
-                        coords = [
-                            int(width * 0.2),
-                            height // 2,
-                            int(width * 0.8),
-                            height // 2,
-                        ]
-                    else:
-                        raise ValueError(f"Invalid direction: {target}")
-
-                    start_x, start_y, end_x, end_y = coords
+                    indexed_elems = getattr(state, "indexed_elements", None) if state else None
+                    ui_hier = getattr(state, "ui_tree", None) if state else None
+                    start_x, start_y, end_x, end_y, smart_dur = compute_smart_swipe_coordinates(
+                        direction=target,
+                        target=params.get("target"),
+                        indexed_elements=indexed_elems,
+                        ui_hierarchy=ui_hier,
+                        width=width,
+                        height=height,
+                        duration=dur,
+                    )
+                    dur = smart_dur
                 elif kind == "coords" and isinstance(target, list) and len(target) == 4:
                     nx1, ny1, nx2, ny2 = target
                     start_x = int(max(0, min(width - 1, nx1 * width / 1000)))

artemis/tools/mobile/ocr.py

@@ -30,7 +30,7 @@ from artemis.graph.state import State
 from artemis.tools.base import ArtemisTool, ToolCategory, ToolRegistry
 from artemis.tools.tool_wrapper import ToolWrapper
 from artemis.utils.logger import get_logger
-from artemis.utils.ocr_api import perform_ocr
+from artemis.utils.ocr_api import is_ocr_configured, perform_ocr
 
 
 class OcrArgs(BaseModel):
@@ -144,6 +144,9 @@ class OcrRecognitionTool(ArtemisTool):
             category=category,
         )
 
+    def is_available(self, ctx: Any = None) -> bool:
+        return is_ocr_configured()
+
     # pylint: disable=too-many-arguments,too-many-positional-arguments
     async def execute(
         self,
@@ -207,4 +210,5 @@ ocr_recognition_wrapper = ToolWrapper(
     tool_fn_getter=get_ocr_tool,
     on_success_fn=lambda output: f"OCR Recognition successful. Results:\n{output}",
     on_failure_fn=lambda error: f"OCR Recognition failed: {error}",
+    is_available_fn=lambda ctx: is_ocr_configured(),
 )

artemis/tools/tool_wrapper.py

@@ -40,6 +40,7 @@ class ToolWrapper(BaseModel):
     tool_fn_getter: Callable[[ArtemisContext], BaseTool]
     on_success_fn: Callable[..., str]
     on_failure_fn: Callable[..., str]
+    is_available_fn: Callable[[ArtemisContext], bool] | None = None
 
 
 class CompositeToolWrapper(ToolWrapper):

artemis/tools/types.py

@@ -62,7 +62,7 @@ class Target(BaseModel):
 
     @model_validator(mode="after")
     def _default_indices(self):
-        # Treat empty strings like "not provided"
+        # Treat empty strings like “not provided”
         if (
             self.resource_id is not None and self.resource_id != ""
         ) and self.resource_id_index is None:

artemis/utils/coordinates.py

@@ -20,7 +20,7 @@ from typing import Any
 
 def parse_swipe_parameters(
     args: Any,
-    default_duration: int | None = 400,
+    default_duration: int | None = 800,
 ) -> tuple[str | None, str | list[int] | None, int | None]:
     """Parses swipe/drag/scroll parameters from various possible input formats into a standardized tuple:
 
@@ -130,6 +130,188 @@ def parse_swipe_parameters(
     return None, None, duration_int
 
 
+def _extract_bounds(val: Any) -> list[int] | None:
+    """Safely parses bounds representation into [left, top, right, bottom]."""
+    if val is None:
+        return None
+    if isinstance(val, (list, tuple)) and len(val) == 4:
+        try:
+            return [int(float(x)) for x in val]
+        except (ValueError, TypeError):
+            return None
+    if isinstance(val, str):
+        match = re.match(r"\[(\-?\d+),(\-?\d+)\]\[(\-?\d+),(\-?\d+)\]", val.strip())
+        if match:
+            return [int(x) for x in match.groups()]
+        nums = re.findall(r"-?\d+", val)
+        if len(nums) == 4:
+            return [int(x) for x in nums]
+    return None
+
+
+def resolve_scrollable_container_bounds(
+    target: Any = None,
+    indexed_elements: list[dict[str, Any]] | None = None,
+    ui_hierarchy: list[dict[str, Any]] | None = None,
+    screen_width: int = 1080,
+    screen_height: int = 2400,
+) -> list[int]:
+    """Resolves the best-effort bounding box [left, top, right, bottom] for scrolling.
+
+    Heuristic Priority:
+    1. Explicit target element bounds (if target index/element/bounds passed).
+    2. Primary active scrollable container (RecyclerView, ScrollView, ListView, WebView, ViewPager).
+    3. Fallback to full screen [0, 0, screen_width, screen_height].
+    """
+    # 1. Target element reference
+    if target is not None:
+        direct_b = _extract_bounds(target)
+        if direct_b and len(direct_b) == 4:
+            return direct_b
+
+        target_idx = None
+        if isinstance(target, (int, float)):
+            target_idx = int(target)
+        elif isinstance(target, str) and target.strip().isdigit():
+            target_idx = int(target.strip())
+
+        if target_idx is not None and indexed_elements:
+            if 1 <= target_idx <= len(indexed_elements):
+                elem = indexed_elements[target_idx - 1]
+                b = _extract_bounds(elem.get("bounds"))
+                if b and len(b) == 4:
+                    return b
+
+    # 2. Look for primary scrollable container in ui_hierarchy or indexed_elements
+    candidates: list[list[int]] = []
+
+    def scan_hierarchy(nodes: list[dict[str, Any]]):
+        for node in nodes:
+            if not isinstance(node, dict):
+                continue
+            is_scroll = (
+                node.get("scrollable") is True or str(node.get("scrollable", "")).lower() == "true"
+            )
+            cls_name = str(node.get("class") or node.get("className") or "").lower()
+            if not is_scroll:
+                for kw in ("scrollview", "recyclerview", "listview", "webview", "viewpager"):
+                    if kw in cls_name:
+                        is_scroll = True
+                        break
+            if is_scroll:
+                b = _extract_bounds(node.get("bounds"))
+                if b and len(b) == 4:
+                    w = b[2] - b[0]
+                    h = b[3] - b[1]
+                    if w >= int(screen_width * 0.25) and h >= int(screen_height * 0.15):
+                        candidates.append(b)
+            children = node.get("children")
+            if isinstance(children, list) and children:
+                scan_hierarchy(children)
+
+    if ui_hierarchy:
+        scan_hierarchy(ui_hierarchy)
+    elif indexed_elements:
+        for elem in indexed_elements:
+            b = _extract_bounds(elem.get("bounds"))
+            cls_name = str(elem.get("class") or elem.get("className") or "").lower()
+            if any(
+                kw in cls_name
+                for kw in ("scrollview", "recyclerview", "listview", "webview", "viewpager")
+            ):
+                if b and len(b) == 4:
+                    candidates.append(b)
+
+    if candidates:
+
+        def area(b: list[int]) -> int:
+            return max(0, b[2] - b[0]) * max(0, b[3] - b[1])
+
+        candidates.sort(key=area, reverse=True)
+        return candidates[0]
+
+    return [0, 0, screen_width, screen_height]
+
+
+def compute_smart_swipe_coordinates(
+    direction: str,
+    target: Any = None,
+    indexed_elements: list[dict[str, Any]] | None = None,
+    ui_hierarchy: list[dict[str, Any]] | None = None,
+    width: int = 1080,
+    height: int = 2400,
+    duration: int | None = None,
+) -> tuple[int, int, int, int, int]:
+    """Computes best-effort container-aware intelligent swipe coordinates and duration.
+
+    Returns: (start_x, start_y, end_x, end_y, duration_ms)
+    """
+    container = resolve_scrollable_container_bounds(
+        target=target,
+        indexed_elements=indexed_elements,
+        ui_hierarchy=ui_hierarchy,
+        screen_width=width,
+        screen_height=height,
+    )
+
+    c_left = max(0, min(width - 1, container[0]))
+    c_top = max(0, min(height - 1, container[1]))
+    c_right = max(c_left + 50, min(width, container[2]))
+    c_bottom = max(c_top + 50, min(height, container[3]))
+
+    c_width = c_right - c_left
+    c_height = c_bottom - c_top
+
+    dir_str = str(direction or "up").strip().lower()
+
+    if dir_str == "up":
+        # Drag upwards -> reveal content below
+        sx = ex = int(c_left + c_width * 0.60)
+        sy = int(c_top + c_height * 0.70)
+        ey = int(c_top + c_height * 0.30)
+    elif dir_str == "down":
+        # Drag downwards -> reveal content above
+        sx = ex = int(c_left + c_width * 0.60)
+        sy = int(c_top + c_height * 0.30)
+        ey = int(c_top + c_height * 0.70)
+    elif dir_str == "left":
+        # Drag leftwards -> page right
+        sy = ey = int(c_top + c_height * 0.50)
+        sx = int(c_left + c_width * 0.75)
+        ex = int(c_left + c_width * 0.25)
+    elif dir_str == "right":
+        # Drag rightwards -> page left
+        sy = ey = int(c_top + c_height * 0.50)
+        sx = int(c_left + c_width * 0.25)
+        ex = int(c_left + c_width * 0.75)
+    else:
+        # Default up
+        sx = ex = int(c_left + c_width * 0.60)
+        sy = int(c_top + c_height * 0.70)
+        ey = int(c_top + c_height * 0.30)
+
+    # Clamp coordinates to screen bounds
+    sx = max(0, min(width - 1, sx))
+    ex = max(0, min(width - 1, ex))
+    sy = max(0, min(height - 1, sy))
+    ey = max(0, min(height - 1, ey))
+
+    if duration is None:
+        dist = ((ex - sx) ** 2 + (ey - sy) ** 2) ** 0.5
+        if dist <= 300:
+            duration_ms = 350
+        elif dist <= 600:
+            duration_ms = 550
+        elif dist <= 1000:
+            duration_ms = 800
+        else:
+            duration_ms = 900
+    else:
+        duration_ms = int(duration)
+
+    return sx, sy, ex, ey, duration_ms
+
+
 def normalize_point(x: int, y: int, width: int, height: int) -> list[int]:
     """Convert physical pixels to normalized 0-1000 coordinates."""
     nx = int(round(x * 1000.0 / width))
@@ -216,13 +398,13 @@ def normalize_step_actions(step_dict: dict[str, Any]) -> dict[str, Any]:
                     item["coordinates"] = target
             elif kind == "direction" and isinstance(target, str):
                 if target == "up":
-                    norm = [600, 800, 600, 200]
+                    norm = [600, 700, 600, 300]
                 elif target == "down":
-                    norm = [600, 200, 600, 800]
+                    norm = [600, 300, 600, 700]
                 elif target == "left":
-                    norm = [800, 500, 200, 500]
+                    norm = [750, 500, 250, 500]
                 elif target == "right":
-                    norm = [200, 500, 800, 500]
+                    norm = [250, 500, 750, 500]
                 else:
                     norm = None
                 if norm:
@@ -255,6 +437,43 @@ def normalize_step_actions(step_dict: dict[str, Any]) -> dict[str, Any]:
                     item["normalized_end_coordinates"] = [nx2, ny2]
             else:
                 item["normalized_coordinates"] = coords
+
+        seq = item.get("sequence") or item.get("targets")
+        if seq:
+            if isinstance(seq, str):
+                try:
+                    seq = json.loads(seq)
+                except Exception:
+                    try:
+                        import ast
+
+                        seq = ast.literal_eval(seq)
+                    except Exception:
+                        pass
+                if isinstance(seq, (list, tuple)):
+                    item["sequence"] = list(seq)
+            if isinstance(item.get("sequence"), list) and "normalized_sequence" not in item:
+                norm_seq = []
+                for pt in item["sequence"]:
+                    if (
+                        isinstance(pt, (list, tuple))
+                        and len(pt) == 2
+                        and all(isinstance(c, (int, float)) for c in pt)
+                    ):
+                        if any(c > 1000 for c in pt) or item.get("target_bounds"):
+                            norm_seq.append(
+                                [
+                                    int(round(pt[0] * 1000.0 / width)),
+                                    int(round(pt[1] * 1000.0 / height)),
+                                ]
+                            )
+                        else:
+                            norm_seq.append([int(round(pt[0])), int(round(pt[1]))])
+                    elif isinstance(pt, int):
+                        norm_seq.append(pt)
+                if norm_seq:
+                    item["normalized_sequence"] = norm_seq
+
         return item
 
     action_taken = step_dict.get("action_taken")
@@ -326,16 +545,26 @@ def normalize_step_actions(step_dict: dict[str, Any]) -> dict[str, Any]:
                     and isinstance(action_taken[0], dict)
                     else (action_taken if isinstance(action_taken, dict) else {})
                 )
-                # Enrich missing coordinates/target/input from action_taken
-                if "target" not in args and "coordinates" not in args:
-                    if first_act.get("coordinates"):
-                        args["coordinates"] = first_act["coordinates"]
-                    if first_act.get("args", {}).get("target"):
-                        args["target"] = first_act["args"]["target"]
-                if "normalized_coordinates" not in args and first_act.get("normalized_coordinates"):
-                    args["normalized_coordinates"] = first_act["normalized_coordinates"]
-                if "text" not in args and first_act.get("text"):
-                    args["text"] = first_act["text"]
+
+                is_seq_tool = (
+                    raw_name in ("click_sequence", "tap_sequence")
+                    or "sequence" in args
+                    or "targets" in args
+                )
+
+                # Enrich missing coordinates/target/input from action_taken only if not a multi-point sequence action
+                if not is_seq_tool:
+                    if "target" not in args and "coordinates" not in args:
+                        if first_act.get("coordinates"):
+                            args["coordinates"] = first_act["coordinates"]
+                        if first_act.get("args", {}).get("target"):
+                            args["target"] = first_act["args"]["target"]
+                    if "normalized_coordinates" not in args and first_act.get(
+                        "normalized_coordinates"
+                    ):
+                        args["normalized_coordinates"] = first_act["normalized_coordinates"]
+                    if "text" not in args and first_act.get("text"):
+                        args["text"] = first_act["text"]
 
                 args = _enrich_action_item(args)
                 payload["args"] = args

artemis/utils/credentials_validator.py

@@ -0,0 +1,194 @@
+# Copyright 2026 Google LLC
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+"""Validation utility to verify LLM and Vision API keys against live provider endpoints."""
+
+import httpx
+
+from artemis.utils.logger import get_logger
+
+logger = get_logger(__name__)
+
+# Minimal 1x1 PNG transparent pixel base64 for Vision API testing
+_TINY_PNG_B64 = "iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAYAAAAfFcSJAAAADUlEQVR42mNk+M9QDwADhgGAWjR9awAAAABJRU5ErkJggg=="
+
+
+def _extract_error_message(resp: httpx.Response) -> str:
+    """Extract a human-friendly error message from a provider HTTP response."""
+    try:
+        data = resp.json()
+        if isinstance(data, dict):
+            # Google style: {"error": {"message": "..."}}
+            err_obj = data.get("error")
+            if isinstance(err_obj, dict):
+                msg = err_obj.get("message")
+                if msg:
+                    return str(msg)
+            elif isinstance(err_obj, str):
+                return err_obj
+
+            # OpenAI / Anthropic / OpenRouter style
+            if "message" in data:
+                return str(data["message"])
+            if "detail" in data:
+                return str(data["detail"])
+    except Exception:
+        pass
+
+    # Fallback to response text or HTTP status
+    text = resp.text.strip()
+    if text and len(text) < 200:
+        return text
+    return f"HTTP {resp.status_code}: {resp.reason_phrase}"
+
+
+async def validate_api_key(
+    provider: str,
+    api_key: str,
+    base_url: str | None = None,
+    timeout: float = 12.0,
+) -> tuple[bool, str]:
+    """Tests if the provided API key is valid and usable with the corresponding provider.
+
+    Args:
+        provider: Provider identifier (e.g. 'google', 'gemini', 'ocr', 'openai', 'anthropic', etc.)
+        api_key: Secret API key string.
+        base_url: Optional custom endpoint URL.
+        timeout: Request timeout in seconds.
+
+    Returns:
+        tuple[bool, str]: (is_valid, descriptive_message)
+    """
+    clean_provider = provider.strip().lower()
+    clean_key = api_key.strip()
+
+    if not clean_key:
+        return False, "API key cannot be empty."
+
+    # Placeholders / default template values should be rejected
+    if clean_key in (
+        "API_KEY",
+        "your_gemini_api_key_here",
+        "your_google_cloud_vision_api_key_here",
+        "your_openai_api_key_here",
+        "your_anthropic_api_key_here",
+    ):
+        return False, "Please enter a real API key instead of the placeholder."
+
+    try:
+        async with httpx.AsyncClient(timeout=timeout) as client:
+            if clean_provider in ("google", "gemini", "vertexai", "vertex"):
+                # Test Google Gemini via generativelanguage API
+                url = f"https://generativelanguage.googleapis.com/v1beta/models?key={clean_key}&pageSize=1"
+                resp = await client.get(url)
+                if resp.status_code == 200:
+                    return True, "Google Gemini API key verified successfully."
+                err_msg = _extract_error_message(resp)
+                return False, f"Gemini API verification failed ({resp.status_code}): {err_msg}"
+
+            elif clean_provider in ("ocr", "vision", "google_vision"):
+                # Test Google Cloud Vision OCR API
+                url = f"https://vision.googleapis.com/v1/images:annotate?key={clean_key}"
+                payload = {
+                    "requests": [
+                        {
+                            "image": {"content": _TINY_PNG_B64},
+                            "features": [{"type": "TEXT_DETECTION"}],
+                        }
+                    ]
+                }
+                resp = await client.post(
+                    url, json=payload, headers={"Content-Type": "application/json"}
+                )
+                if resp.status_code == 200:
+                    return True, "Google Cloud Vision OCR API key verified successfully."
+                err_msg = _extract_error_message(resp)
+                return False, f"Vision OCR API verification failed ({resp.status_code}): {err_msg}"
+
+            elif clean_provider == "openai":
+                endpoint = (
+                    f"{base_url.rstrip('/')}/models"
+                    if base_url
+                    else "https://api.openai.com/v1/models"
+                )
+                resp = await client.get(endpoint, headers={"Authorization": f"Bearer {clean_key}"})
+                if resp.status_code == 200:
+                    return True, "OpenAI API key verified successfully."
+                err_msg = _extract_error_message(resp)
+                return False, f"OpenAI API verification failed ({resp.status_code}): {err_msg}"
+
+            elif clean_provider in ("anthropic", "claude"):
+                url = "https://api.anthropic.com/v1/models"
+                resp = await client.get(
+                    url,
+                    headers={
+                        "x-api-key": clean_key,
+                        "anthropic-version": "2023-06-01",
+                    },
+                )
+                if resp.status_code == 200:
+                    return True, "Anthropic Claude API key verified successfully."
+                err_msg = _extract_error_message(resp)
+                return False, f"Anthropic API verification failed ({resp.status_code}): {err_msg}"
+
+            elif clean_provider == "openrouter":
+                url = "https://openrouter.ai/api/v1/auth/key"
+                resp = await client.get(url, headers={"Authorization": f"Bearer {clean_key}"})
+                if resp.status_code == 200:
+                    return True, "OpenRouter API key verified successfully."
+                err_msg = _extract_error_message(resp)
+                return False, f"OpenRouter API verification failed ({resp.status_code}): {err_msg}"
+
+            elif clean_provider in ("xai", "grok"):
+                url = "https://api.x.ai/v1/models"
+                resp = await client.get(url, headers={"Authorization": f"Bearer {clean_key}"})
+                if resp.status_code == 200:
+                    return True, "xAI API key verified successfully."
+                err_msg = _extract_error_message(resp)
+                return False, f"xAI API verification failed ({resp.status_code}): {err_msg}"
+
+            elif clean_provider in ("ollama", "vllm", "custom"):
+                if base_url:
+                    endpoint = f"{base_url.rstrip('/')}/models"
+                    headers = (
+                        {"Authorization": f"Bearer {clean_key}"} if clean_key != "EMPTY" else {}
+                    )
+                    resp = await client.get(endpoint, headers=headers)
+                    if resp.status_code == 200:
+                        return True, f"Custom endpoint ({base_url}) verified successfully."
+                    err_msg = _extract_error_message(resp)
+                    return (
+                        False,
+                        f"Custom endpoint verification failed ({resp.status_code}): {err_msg}",
+                    )
+                return True, "Custom endpoint configured."
+
+            else:
+                # Unknown provider - fallback to basic length check
+                if len(clean_key) >= 8:
+                    return True, f"Credentials for {provider} configured."
+                return False, f"Invalid API key format for {provider}."
+
+    except httpx.TimeoutException:
+        logger.warning(f"Timeout verifying API key for {provider}")
+        return (
+            False,
+            f"Connection to {provider.capitalize()} API timed out. Please check network/proxy settings.",
+        )
+    except httpx.RequestError as exc:
+        logger.warning(f"Network error verifying API key for {provider}: {exc}")
+        return False, f"Network error connecting to {provider.capitalize()} API: {exc}"
+    except Exception as exc:
+        logger.error(f"Unexpected error validating API key for {provider}: {exc}")
+        return False, f"Validation error: {exc}"

artemis/utils/ocr_api.py

@@ -30,13 +30,33 @@ def get_http_client() -> httpx.AsyncClient:
     return _HTTP_CLIENT
 
 
+def is_ocr_configured() -> bool:
+    """Checks if a valid Google Vision OCR API key is configured."""
+    ocr_secret = settings.get_api_key("ocr")
+    if ocr_secret and ocr_secret.get_secret_value().strip():
+        val = ocr_secret.get_secret_value().strip()
+        if val not in ("API_KEY", "your_google_cloud_vision_api_key_here"):
+            return True
+    for env_var in ("OCR_API_KEY", "VISION_API_KEY"):
+        val = os.environ.get(env_var)
+        if (
+            val
+            and val.strip()
+            and val.strip() not in ("API_KEY", "your_google_cloud_vision_api_key_here")
+        ):
+            return True
+    return False
+
+
 async def perform_ocr(
-    screenshot_b64: str, client: httpx.AsyncClient = None
+    screenshot_b64: str,
+    client: httpx.AsyncClient | None = None,
 ) -> list[dict[str, Any]]:
-    """Calls Google Vision API for OCR using a persistent client to reuse TLS handshakes.
+    """Calls Google Vision API for text recognition on an image.
 
     Args:
         screenshot_b64: Base64 encoded screenshot image.
+        client: Optional persistent httpx.AsyncClient.
 
     Returns:
         A list of dictionaries containing detected text and position vertices.
@@ -46,10 +66,9 @@ async def perform_ocr(
         (ocr_secret.get_secret_value() if ocr_secret else None)
         or os.environ.get("OCR_API_KEY")
         or os.environ.get("VISION_API_KEY")
-        or os.environ.get("API_KEY")
-        or os.environ.get("GOOGLE_API_KEY")
-        or "API_KEY"
     )
+    if not api_key:
+        return []
 
     url = f"https://vision.googleapis.com/v1/images:annotate?key={api_key}"
     headers = {"Content-Type": "application/json"}
@@ -62,13 +81,9 @@ async def perform_ocr(
         ]
     }
 
-    try:
-        active_client = client if client is not None else get_http_client()
-        response = await active_client.post(url, json=data, headers=headers, timeout=30.0)
-        return _parse_ocr_response(response)
-    except Exception as e:
-        logger.error(f"Failed to call Vision API for OCR: {e}")
-        raise e
+    active_client = client if client is not None else get_http_client()
+    response = await active_client.post(url, json=data, headers=headers, timeout=30.0)
+    return _parse_ocr_response(response)
 
 
 def _parse_ocr_response(response: httpx.Response) -> list[dict[str, Any]]:

artemis/utils/visualization.py

@@ -583,35 +583,88 @@ def draw_action_overlay_on_image(
 
         # 2. Click Sequence
         elif action in ("click_sequence", "tap_sequence"):
-            sequence = action_args.get("sequence") or action_args.get("targets") or []
+            sequence = (
+                action_args.get("sequence")
+                or action_args.get("targets")
+                or action_args.get("coordinates")
+                or []
+            )
+            if isinstance(sequence, str):
+                sequence_str = sequence.strip()
+                try:
+                    import json
+
+                    sequence = json.loads(sequence_str)
+                except Exception:
+                    try:
+                        import ast
+
+                        sequence = ast.literal_eval(sequence_str)
+                    except Exception:
+                        pass
+
             if isinstance(sequence, list):
-                for idx, coord in enumerate(sequence, 1):
+                pts = []
+                for coord in sequence:
                     pt = _resolve_coordinates(coord, width, height)
                     if pt:
-                        x, y = pt
-                        draw.ellipse(
-                            [x - dot_radius, y - dot_radius, x + dot_radius, y + dot_radius],
-                            fill="red",
-                            outline="white",
-                            width=line_width,
+                        pts.append(pt)
+
+                # Draw connecting lines/arrows between consecutive sequence points
+                for i in range(len(pts) - 1):
+                    p1, p2 = pts[i], pts[i + 1]
+                    draw.line([p1, p2], fill="white", width=line_width + 3)
+                    draw.line([p1, p2], fill="red", width=line_width)
+
+                    # Arrow head
+                    dx = p2[0] - p1[0]
+                    dy = p2[1] - p1[1]
+                    angle = math.atan2(dy, dx)
+                    arrow_len = max(16, int(base_dim * 0.025))
+                    arrow_angle = math.pi / 6
+                    x1 = p2[0] - arrow_len * math.cos(angle - arrow_angle)
+                    y1 = p2[1] - arrow_len * math.sin(angle - arrow_angle)
+                    x2 = p2[0] - arrow_len * math.cos(angle + arrow_angle)
+                    y2 = p2[1] - arrow_len * math.sin(angle + arrow_angle)
+                    draw.polygon([p2, (x1, y1), (x2, y2)], fill="red", outline="white")
+
+                # Draw numbered circular touch points
+                for idx, (x, y) in enumerate(pts, 1):
+                    # Outer aura
+                    draw.ellipse(
+                        [
+                            x - dot_radius * 1.3,
+                            y - dot_radius * 1.3,
+                            x + dot_radius * 1.3,
+                            y + dot_radius * 1.3,
+                        ],
+                        outline="white",
+                        width=line_width + 1,
+                    )
+                    # Colored circle
+                    draw.ellipse(
+                        [x - dot_radius, y - dot_radius, x + dot_radius, y + dot_radius],
+                        fill="red",
+                        outline="white",
+                        width=line_width,
+                    )
+                    label = str(idx)
+                    try:
+                        font = ImageFont.load_default(size=int(dot_radius * 1.2))
+                        draw.text(
+                            (x - dot_radius // 3, y - dot_radius // 2),
+                            label,
+                            fill="white",
+                            font=font,
+                            stroke_width=2,
+                            stroke_fill="black",
+                        )
+                    except Exception:
+                        draw.text(
+                            (x - 4, y - 6),
+                            label,
+                            fill="white",
                         )
-                        label = str(idx)
-                        try:
-                            font = ImageFont.load_default(size=int(dot_radius * 1.2))
-                            draw.text(
-                                (x - dot_radius // 3, y - dot_radius // 2),
-                                label,
-                                fill="white",
-                                font=font,
-                                stroke_width=2,
-                                stroke_fill="black",
-                            )
-                        except Exception:
-                            draw.text(
-                                (x - 4, y - 6),
-                                label,
-                                fill="white",
-                            )
 
         # 3. Swipe / Drag / Scroll
         elif action in ("swipe", "drag", "scroll"):