feat: add Chinese UI, Responses models, and mirrored Windows setup

Localize the web console and persist custom model and capability settings.
Use OpenAI Responses exclusively and move runtime limits into environment configuration.
Bootstrap uv and the virtual environment on Windows, support optional China mirrors,
and package source distributions without environments, build output, or credentials.

Update regression tests and repository quality checks for the new behavior.

artemis/agents/checker/checker.json

@@ -1,5 +1,5 @@
 {
-  "system": "You are a precise, evidence-driven verification expert (Checker). You audit declared check items against recorded execution evidence and issue verdicts. You never operate the device, never modify any state, and never negotiate with the executing agent — you only observe, probe read-only state where allowed, and judge.",
+  "system": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。\n\nYou are a precise, evidence-driven verification expert (Checker). You audit declared check items against recorded execution evidence and issue verdicts. You never operate the device, never modify any state, and never negotiate with the executing agent — you only observe, probe read-only state where allowed, and judge.",
   "base_rules": "# Verdict Rules (always apply)\n- Quote every check item verbatim in `item_text` — never paraphrase.\n- Every verdict MUST carry concrete evidence: a specific probe output, a specific note excerpt, or a specific on-screen/screenshot observation. A `failed` verdict without concrete evidence is invalid and will be downgraded.\n- `inconclusive` means the evidence needed for a judgment was never captured or is genuinely ambiguous. When evidence was never collected, output `inconclusive` and state explicitly that the evidence is missing — do NOT output `failed` for lack of evidence.\n- Judge only what the check item declares. Do not invent additional criteria and do not relax the declared ones.",
   "verify_semantics": "# `verify` items (completion acceptance criteria)\n- A `verify` item asks: was the anchored subgoal actually accomplished to this standard?\n- A `failed` verify verdict can send its milestone back for repair, so it must include a `suggestion`: a concrete, actionable hint telling the executor what is missing or wrong.\n- The release decision (whether execution proceeds) is computed outside of you and is fail-open on errors; your job is only the truthful verdict value.",
   "assert_semantics": "# `assert` items (test assertions)\n- An `assert` item is a test expectation, not an instruction. A failed assertion is a legitimate, valuable test result: record it truthfully with evidence. Nothing will (or should) try to repair it.\n- Never mark an assert `failed` merely because evidence is absent — absence of captured evidence is `inconclusive` with an explicit note that the evidence is missing.\n- Precondition first: before judging an assert, confirm from the execution history (step replay, notes, anchor evidence) that the action(s) meant to trigger the asserted behavior were actually and correctly performed by the executor (the toggle was really tapped, the intended screen was really reached).\n- If that precondition is not established — the executor never performed the trigger, performed the wrong action, or the evidence cannot show it — output `inconclusive` and state explicitly that the precondition failed / the triggering action is not confirmed as executed. Never output `failed` here: a failed assert is attributed to the device under test, and an execution problem must not be reported as a device defect.\n- Output `failed` only when the precondition is established AND the device state, probe output, or recorded evidence still contradicts the expectation. The evidence must cite the concrete step(s) that performed the trigger plus the probe output / observation that contradicts the expectation, so the verdict reads as a device-defect diagnosis.\n- `suggestion` is not needed for assert verdicts.",

artemis/agents/diagnoser/diagnoser.json

@@ -1,6 +1,6 @@
 {
   "system_prompt": [
-    "# Diagnoser",
+    "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。\n\n# Diagnoser",
     "",
     "## Objective",
     "You are an expert AI diagnoser. Your objective is to pinpoint the root cause of mobile execution failures by cross-referencing visual evidence (videos/screenshots) with system telemetry (logs). Precisely answer the `Diagnostic Query` based on the provided context.",

artemis/agents/diagnoser/diagnoser.py

@@ -22,6 +22,8 @@ import uuid
 
 from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage, ToolMessage
 
+from artemis.llm.structured import content_to_text
+
 from artemis.core.tool_failure import is_tool_failure
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import (
@@ -31,7 +33,7 @@ from artemis.data_engine.trace import (
     trace_langchain_tool,
 )
 from artemis.graph.state import State
-from artemis.llm.google import usage_from_message
+from artemis.llm.usage import usage_from_message
 from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
 from artemis.tools.command_tool import get_run_short_adb_command_tool
 from artemis.tools.diagnoser_submit_answer_tool import get_submit_answer_tool
@@ -568,7 +570,7 @@ class Diagnoser:
                 response = await self._invoke_model(llm, current_messages, context_components)
 
                 if response.content:
-                    outcome = response.content
+                    outcome = content_to_text(response.content)
 
                 if not response.tool_calls:
                     break

artemis/agents/explorer/constants.py

@@ -23,7 +23,9 @@ Explorer tier the user configured.  Tier behavior lives in
 ASK_EXPLORER_TOOL_NAME = "ask_explorer"
 
 #: Model used when the LLM configuration carries no Explorer entry.
-DEFAULT_EXPLORER_MODEL = "gemini-3.8-flash"
+from artemis.config.settings import settings
+
+DEFAULT_EXPLORER_MODEL = settings.OPENAI_MODEL
 
 ASK_EXPLORER_DESCRIPTION = (
     "[EXPLORER] Ask the UI Explorer to locate elements on the current screen"

artemis/agents/explorer/explorer.json

@@ -6,7 +6,7 @@
 // detect_objects, get_ocr_list, inspect_region, ask_image_processor.
 // submit_answer is always available.
 {
-  "IDENTITY": "You are the agentic UI Explorer: given a phone screenshot and a query, you locate the matching UI element(s) and return their normalized [nx, ny] coordinates on a 0-1000 scale.",
+  "IDENTITY": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。\n\nYou are the agentic UI Explorer: given a phone screenshot and a query, you locate the matching UI element(s) and return their normalized [nx, ny] coordinates on a 0-1000 scale.",
   "OPERATING PRINCIPLES": [
     "**Confidence-Driven Cost-Benefit Strategy**: Maximize localization accuracy while minimizing turns and tool calls. Every turn costs latency; stop exploring as soon as further calls would yield negligible accuracy gains.",
     "**Direct Submit (High Confidence)**: If the initial annotated screenshot and the marked UI element list already make the target coordinates obvious, call `submit_answer` immediately in the first turn.",

artemis/agents/explorer/explorer.py

@@ -17,19 +17,16 @@
 The :class:`Explorer` class combines the following modules:
 
 - ``tiers``: engine, tools and budgets for flash / pro / ultra.
-- ``tool_declarations``: static tool schemas (universal + native).
+- ``tool_declarations``: Responses function tool schemas.
 - ``perception_tools``: the ``exec_*`` perception/vision tool method group.
 - ``universal_runner``: the LangChain-based ``_run_universal`` loop.
 - ``run_setup``: flash mode and the pre-loop setup phases of ``run``.
-- ``native_runner``: the native Gemini reasoning loop of ``run``.
 """
 
 import json
 import os
 from typing import Any, Literal
 
-from google import genai
-from google.genai import types
 
 from artemis.agents.explorer.constants import DEFAULT_EXPLORER_MODEL
 from artemis.agents.explorer.geometry import (
@@ -38,18 +35,16 @@ from artemis.agents.explorer.geometry import (
     norm_to_pixel,
     pixel_to_norm,
 )
-from artemis.agents.explorer.native_runner import NativeRunnerMixin
 from artemis.agents.explorer.perception_tools import PerceptionToolsMixin
 from artemis.agents.explorer.run_setup import RunSetupMixin
 from artemis.agents.explorer.screen_index import normalize_text, ScreenElement, ScreenIndex
 from artemis.agents.explorer.tiers import ExplorerTier, get_tier
-from artemis.agents.explorer.tool_declarations import NATIVE_EXPLORER_TOOL_DECLARATIONS
+from artemis.agents.explorer.tool_declarations import UNIVERSAL_EXPLORER_TOOLS
 from artemis.agents.explorer.universal_runner import UniversalRunnerMixin
 from artemis.config import settings
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import trace
 from artemis.graph.state import State
-from artemis.llm.google import is_gemini_model, strip_provider_prefix
 from artemis.utils.logger import get_logger
 from artemis.utils.ocr_api import is_ocr_configured
 
@@ -57,28 +52,25 @@ logger = get_logger(__name__)
 
 __all__ = ["Explorer"]
 
-#: Environment override for Gemini explicit context caching ("true" / "false").
-CACHING_ENV_VAR = "ARTEMIS_EXPLORER_CACHING"
-
 #: How far outside its labeled element (as a fraction of the screen, per side)
 #: a candidate's point may fall and still inherit that element's bounds.
 LABEL_TOLERANCE_RATIO = 0.02
 
 
-class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, NativeRunnerMixin):
+class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin):
     """Locates UI elements on a screenshot for the ``ask_explorer`` tool.
 
     The tier (flash / pro / ultra) is applied per ``run`` call from
     :data:`artemis.agents.explorer.tiers.EXPLORER_TIERS`; it decides the
     engine, the turn budget, the exposed perception tools and the caching
     default.  Construction is side-effect free: no network client is created
-    until the native Gemini branch actually needs one.
+    until the Responses execution loop needs one.
     """
 
     DENYLIST_TEMPLATE = (
         "\n# TOOL DENYLIST\n- The following tools are denylisted and cannot be used: {tools}\n"
     )
-    TOOLS = NATIVE_EXPLORER_TOOL_DECLARATIONS
+    TOOLS = UNIVERSAL_EXPLORER_TOOLS
 
     def __init__(self, ctx: ArtemisContext):
         self.ctx = ctx
@@ -107,7 +99,6 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
         self.denylisted_tools = self._compute_denylist(None)
 
         self.model_name = self._resolve_model_name()
-        self.use_native_gemini = self._detect_native_engine()
 
     # ------------------------------------------------------------------ #
     # Engine selection (no side effects)
@@ -119,37 +110,7 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
         llm_cfg = getattr(llm_config, "explorer", None) if llm_config else None
         model = getattr(llm_cfg, "model", None) if llm_cfg else None
         model_name = model if isinstance(model, str) and model else DEFAULT_EXPLORER_MODEL
-        return strip_provider_prefix(model_name)
-
-    def _detect_native_engine(self) -> bool:
-        """True when the native google-genai SDK should drive the reasoning loop.
-
-        A Gemini model alone is not enough: without a Google API key (or a
-        client already shared on the context) the SDK cannot be constructed,
-        so such configurations run through the universal LangChain engine.
-        """
-        if not is_gemini_model(self.model_name):
-            return False
-        if getattr(self.ctx, "_genai_client", None) is not None:
-            return True
-        api_key = getattr(settings, "GOOGLE_API_KEY", None)
-        return bool(api_key and api_key.get_secret_value())
-
-    def _ensure_genai_client(self):
-        """Returns the context-level GenAI client, creating it on first use.
-
-        Only the native branch calls this, so universal configurations never
-        touch ``genai`` (and never fail on a missing Google key).  The client
-        is stored on the context for connection pooling across agents.
-        """
-        client = getattr(self.ctx, "_genai_client", None)
-        if client is None:
-            logger.info("Initializing GenAI client on context for connection pooling (Explorer)")
-            api_key = getattr(settings, "GOOGLE_API_KEY", None)
-            client = genai.Client(api_key=api_key.get_secret_value() if api_key else None)
-            self.ctx._genai_client = client
-        self.client = client
-        return client
+        return model_name
 
     # ------------------------------------------------------------------ #
     # Tier application
@@ -180,32 +141,6 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
         self.max_turns = tier.max_turns
         self.denylisted_tools = self._compute_denylist(tier)
 
-    def _resolve_caching(self, tier: ExplorerTier, enable_caching: bool | None) -> bool:
-        """Resolves Gemini explicit context caching for this run.
-
-        Precedence: explicit argument, ``ARTEMIS_EXPLORER_CACHING`` env
-        override, the agent configuration, the execution setup, the global
-        settings, then the tier default.  Configuration values are only
-        trusted when they are real booleans so a mocked or malformed config
-        (``None``, sentinel objects) falls through instead of forcing a value.
-        """
-        if isinstance(enable_caching, bool):
-            return enable_caching
-        env_value = os.getenv(CACHING_ENV_VAR, "").strip().lower()
-        if env_value in ("true", "false"):
-            return env_value == "true"
-        for source in (
-            getattr(self.ctx, "agent_config", None),
-            getattr(self.ctx, "execution_setup", None),
-        ):
-            value = getattr(getattr(source, "explorer", None), "caching", None)
-            if isinstance(value, bool):
-                return value
-        value = getattr(settings, "EXPLORER_CACHING", None)
-        if isinstance(value, bool):
-            return value
-        return tier.caching
-
     def _hidden_tool_names(self) -> set[str]:
         """Denylisted tools plus OCR when it is unconfigured, evaluated at call time.
 
@@ -218,34 +153,14 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
             hidden.add("get_ocr_list")
         return hidden
 
-    def get_exposed_tools(self, only_submit: bool = False) -> list[types.FunctionDeclaration]:
+    def get_exposed_tools(self, only_submit: bool = False) -> list[dict[str, Any]]:
         """Native tool declarations the model may see for the current tier."""
         hidden = self._hidden_tool_names()
-        tools = [tool for tool in self.TOOLS if tool.name not in hidden]
+        tools = [tool for tool in self.TOOLS if tool["function"]["name"] not in hidden]
         if only_submit:
-            tools = [tool for tool in tools if tool.name == "submit_answer"]
+            tools = [tool for tool in tools if tool["function"]["name"] == "submit_answer"]
         return tools
 
-    def _prune_historical_images(self, contents, keep_last=1):
-        image_parts = []
-        for content in contents:
-            for part in content.parts:
-                if getattr(part, "inline_data", None) or getattr(part, "file_data", None):
-                    image_parts.append(part)
-        if len(image_parts) <= keep_last:
-            return
-        to_keep_ids = {id(p) for p in image_parts[-keep_last:]}
-        for content in contents:
-            content.parts = [
-                types.Part.from_text(
-                    text=("[Image pruned to maintain visual focus on latest state]")
-                )
-                if (getattr(p, "inline_data", None) or getattr(p, "file_data", None))
-                and id(p) not in to_keep_ids
-                else p
-                for p in content.parts
-            ]
-
     # ------------------------------------------------------------------ #
     # Label bounds and the structural pre-pass
     # ------------------------------------------------------------------ #
@@ -434,30 +349,13 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
             fused_xml, screenshot_path, minimal_list, self.image_name
         )
 
-        if not self.use_native_gemini:
-            return await self._run_universal(
-                query=query,
-                context_feedback=context_feedback,
-                screenshot_path=img_to_read,
-                state=state,
-                minimal_list=minimal_list,
-                prompt_template=prompt_template,
-                max_turns=self.max_turns,
-            )
-
-        client = self._ensure_genai_client()
-        model_name, temperature, thinking_level, _fallback_model = self._resolve_model_params()
-        return await self._run_native(
-            client=client,
+        return await self._run_universal(
             query=query,
             context_feedback=context_feedback,
+            screenshot_path=img_to_read,
+            state=state,
             minimal_list=minimal_list,
-            img_to_read=img_to_read,
-            enable_caching=self._resolve_caching(tier, enable_caching),
             prompt_template=prompt_template,
-            model_name=model_name,
-            temperature=temperature,
-            thinking_level=thinking_level,
             max_turns=self.max_turns,
         )
 

artemis/agents/explorer/native_runner.py

@@ -1,878 +0,0 @@
-# Copyright 2026 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Native Gemini execution loop of ``Explorer.run``.
-
-Split out of ``artemis.agents.explorer.explorer``: the native-SDK reasoning
-loop and its named phases (model invocation, submit_answer interception and
-validation, tool dispatching, failure mapping, resource cleanup), packaged as
-a mixin consumed by ``Explorer``.  Every google-genai call goes through the
-SDK's ``client.aio`` surface so uploads, cache management and generation
-never block the event loop.  This module also owns
-:func:`_generate_content_with_reliability`, the shared reliability wrapper
-for native google-genai model calls.
-"""
-
-import asyncio
-from collections.abc import Awaitable, Callable
-import json
-import os
-import time
-from typing import TYPE_CHECKING, Any
-
-from google.genai import types
-
-from artemis.agents.explorer.geometry import is_valid_norm_point
-from artemis.agents.explorer.tiers import SUBMIT_TOOL
-from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
-from artemis.data_engine.trace import TraceSpan
-from artemis.llm.reliability import (
-    LLMExhaustedError,
-    LLMPermanentError,
-    classify_failure,
-    retry_policy_for,
-)
-from artemis.services.llm import _record_llm_event, _record_llm_retry
-from artemis.utils.logger import get_logger
-
-logger = get_logger(__name__)
-
-#: Loads a local image as a ``Part`` (inline bytes or a File API reference).
-ImagePartGetter = Callable[[str], Awaitable[types.Part]]
-
-
-async def _generate_content_with_reliability(operation, *, label: str = "Explorer model call"):
-    """Run one native google-genai model call under the shared reliability layer.
-
-    ``operation`` must return a fresh awaitable on each call. Non-retryable
-    failures raise ``LLMPermanentError``; exhausted retries raise
-    ``LLMExhaustedError`` according to the shared category policy.
-    """
-    attempt = 0
-    while True:
-        try:
-            return await operation()
-        except Exception as call_err:
-            attempt += 1
-            failure = classify_failure(call_err)
-            if not failure.retryable:
-                logger.error(f"{label} permanently failed [{failure.category.value}]: {call_err}")
-                _record_llm_event(
-                    "llm_gave_up",
-                    {
-                        "source": "explorer",
-                        "error": str(call_err)[:1000],
-                        "category": failure.category.value,
-                        "retryable": False,
-                    },
-                    status="failed",
-                )
-                raise LLMPermanentError(
-                    f"{label} failed [{failure.category.value}]: {call_err}",
-                    failure=failure,
-                    cause=call_err,
-                ) from call_err
-            policy = retry_policy_for(failure.category)
-            if attempt >= policy.max_attempts:
-                logger.error(
-                    f"{label} exhausted {attempt} attempt(s) [{failure.category.value}]: {call_err}"
-                )
-                _record_llm_event(
-                    "llm_gave_up",
-                    {
-                        "source": "explorer",
-                        "error": str(call_err)[:1000],
-                        "category": failure.category.value,
-                        "retryable": True,
-                        "attempts": attempt,
-                    },
-                    status="failed",
-                )
-                raise LLMExhaustedError(
-                    f"{label} exhausted {attempt} attempt(s)"
-                    f" [{failure.category.value}]: {call_err}",
-                    failure=failure,
-                    cause=call_err,
-                ) from call_err
-            delay = policy.delay_for(attempt)
-            logger.warning(
-                f"{label} failed [{failure.category.value}] on attempt"
-                f" {attempt}/{policy.max_attempts}: {call_err}."
-                f" Retrying in {delay:.2f}s..."
-            )
-            _record_llm_retry(
-                str(call_err),
-                delay,
-                attempt=attempt,
-                max_retries=policy.max_attempts,
-                source="explorer",
-            )
-            await asyncio.sleep(delay)
-
-
-def _bare_tool_name(name: str) -> str:
-    """Strips a namespace prefix (``ns:tool``) some models add to tool names."""
-    return name.split(":")[-1] if ":" in name else name
-
-
-class NativeRunnerMixin:
-    """Native Gemini SDK reasoning loop of :class:`Explorer`."""
-
-    # This mixin is composed into ``Explorer``.  Declaring the attributes and
-    # sibling-mixin methods it consumes makes that host contract visible to
-    # static analysis without adding runtime shims or changing MRO behavior.
-    if TYPE_CHECKING:
-        from artemis.context import ArtemisContext
-
-        ctx: ArtemisContext
-        denylisted_tools: set[str]
-        turn_latencies: list[float]
-        turn_cached_tokens: list[int]
-        trace_history: list[dict[str, Any]]
-
-        def get_exposed_tools(
-            self, only_submit: bool = False
-        ) -> list[types.FunctionDeclaration]: ...
-
-        def _prune_historical_images(self, contents: list, keep_last: int = 1) -> None: ...
-
-        def _enrich_candidates(self, candidates: list[Any]) -> list[Any]: ...
-
-        async def exec_ask_perception_tool(
-            self,
-            search_query: str,
-            nx: int,
-            ny: int,
-            detect_queries: list[str],
-        ) -> dict[str, Any]: ...
-
-        async def exec_detect_objects(
-            self, queries: list[str], target_image_id: str = "img_0"
-        ) -> dict[str, Any]: ...
-
-        async def exec_ask_image_processor(
-            self, instruction: str, target_image_id: str = "img_0"
-        ) -> dict[str, Any]: ...
-
-        async def exec_get_ocr_list(self) -> dict[str, Any]: ...
-
-        async def exec_inspect_region(
-            self,
-            x_min: int,
-            y_min: int,
-            x_max: int,
-            y_max: int,
-            zoom_factor: float = 2.0,
-        ) -> dict[str, Any]: ...
-
-    def _make_image_part_getter(
-        self, client, use_file_api: bool, uploaded_files: list
-    ) -> ImagePartGetter:
-        """Builds the image-part loader used for screenshots and tool images."""
-
-        async def get_image_part(file_path: str) -> types.Part:
-            if use_file_api:
-                file_ref = await client.aio.files.upload(file=file_path)
-                uploaded_files.append(file_ref)
-                return types.Part(
-                    file_data=types.FileData(
-                        file_uri=file_ref.uri,
-                        mime_type=file_ref.mime_type or "image/jpeg",
-                    )
-                )
-            with open(file_path, "rb") as f:
-                img_bytes = f.read()
-            return types.Part.from_bytes(data=img_bytes, mime_type="image/jpeg")
-
-        return get_image_part
-
-    def _build_initial_contents(
-        self, query: str, context_feedback: str, minimal_list: str, initial_part: types.Part
-    ) -> list[types.Content]:
-        """Builds the initial user content with the operator request and screenshot."""
-        return [
-            types.Content(
-                role="user",
-                parts=[
-                    types.Part.from_text(
-                        text=f"""Operator Request:
-- Query: {query}
-- Context Feedback: {context_feedback}
-
-Initial marked UI elements list (corresponding to numbers ①, ②... in the image):
-{minimal_list}
-"""
-                    ),
-                    initial_part,
-                ],
-            )
-        ]
-
-    async def _create_cache_resource(
-        self, client, model_name: str, prompt_template: str, contents: list
-    ):
-        """Creates the explicit cache resource; returns None on failure."""
-        try:
-            logger.info("Creating cache resource for Explorer...")
-            cached_content = await client.aio.caches.create(
-                model=model_name,
-                config=types.CreateCachedContentConfig(
-                    contents=[contents[0]],
-                    system_instruction=prompt_template,
-                    tools=[types.Tool(function_declarations=self.get_exposed_tools())],
-                    tool_config=types.ToolConfig(
-                        function_calling_config=types.FunctionCallingConfig(
-                            mode=types.FunctionCallingConfigMode.ANY
-                        ),
-                        include_server_side_tool_invocations=True,
-                    ),
-                    ttl="300s",
-                ),
-            )
-            logger.info(f"Cache resource created successfully: {cached_content.name}")
-            return cached_content
-        except Exception as cache_err:
-            logger.error(f"Failed to create cache resource: {cache_err}")
-            return None
-
-    async def _run_native(
-        self,
-        *,
-        client,
-        query: str,
-        context_feedback: str,
-        minimal_list: str,
-        img_to_read: str,
-        enable_caching: bool,
-        prompt_template: str,
-        model_name: str,
-        temperature,
-        thinking_level,
-        max_turns: int,
-    ) -> str:
-        """Runs the native Gemini path: cache setup, reasoning loop, cleanup."""
-        use_file_api = os.getenv("ARTEMIS_USE_FILE_API", "false").lower() == "true"
-        uploaded_files: list = []
-        get_image_part = self._make_image_part_getter(client, use_file_api, uploaded_files)
-
-        cached_content = None
-        try:
-            initial_part = await get_image_part(img_to_read)
-            contents = self._build_initial_contents(
-                query, context_feedback, minimal_list, initial_part
-            )
-
-            if enable_caching:
-                cached_content = await self._create_cache_resource(
-                    client, model_name, prompt_template, contents
-                )
-
-            agent_outcome = await self._native_loop(
-                client=client,
-                contents=contents,
-                get_image_part=get_image_part,
-                cached_content=cached_content,
-                prompt_template=prompt_template,
-                model_name=model_name,
-                temperature=temperature,
-                thinking_level=thinking_level,
-                max_turns=max_turns,
-            )
-
-        except Exception as e:
-            agent_outcome = self._format_native_failure(e, model_name)
-        finally:
-            await self._cleanup_native_resources(client, cached_content, uploaded_files)
-
-        return agent_outcome
-
-    async def _native_loop(
-        self,
-        *,
-        client,
-        contents: list,
-        get_image_part: ImagePartGetter,
-        cached_content,
-        prompt_template: str,
-        model_name: str,
-        temperature,
-        thinking_level,
-        max_turns: int,
-    ) -> str:
-        """Main native reasoning loop: one model turn per iteration."""
-        turn = 0
-        agent_outcome = ""
-        self.turn_latencies = []
-        self.turn_cached_tokens = []
-        self.trace_history = []
-
-        while turn < max_turns:
-            turn += 1
-            logger.info(f"Turn {turn}/{max_turns}: Invoking Native Gemini SDK for Explorer...")
-
-            self._prune_historical_images(contents, keep_last=1)
-
-            is_final_turn = turn == max_turns
-            if is_final_turn:
-                warning_msg = (
-                    "\n[WARNING] This is your final iteration. You MUST"
-                    " call 'submit_answer' to submit your final result. No"
-                    " other tools are available."
-                )
-                contents.append(
-                    types.Content(
-                        role="user",
-                        parts=[types.Part.from_text(text=warning_msg)],
-                    )
-                )
-
-            response, thinking_parts, text_parts = await self._invoke_native_model(
-                client=client,
-                contents=contents,
-                cached_content=cached_content,
-                is_final_turn=is_final_turn,
-                prompt_template=prompt_template,
-                model_name=model_name,
-                temperature=temperature,
-                thinking_level=thinking_level,
-            )
-
-            turn_record = {
-                "iteration": turn,
-                "thoughts": ("\n".join(thinking_parts) if thinking_parts else ""),
-                "tool_calls": [],
-            }
-
-            function_calls = response.function_calls
-            if not function_calls:
-                self._handle_missing_tool_calls(
-                    contents, response, turn_record, text_parts, max_turns, turn
-                )
-                continue
-
-            # Record the model's function call request in history
-            contents.append(self._model_content(response, function_calls))
-
-            tool_response_parts: list[types.Part] = []
-
-            # Intercept and validate submit_answer to manage the task lifecycle
-            submit_outcome = self._process_submit_answer(
-                function_calls, turn_record, tool_response_parts
-            )
-            if submit_outcome is not None:
-                agent_outcome = submit_outcome
-                break
-
-            await self._dispatch_tool_calls(
-                function_calls, get_image_part, turn_record, tool_response_parts
-            )
-
-            if tool_response_parts:
-                # Native Gemini SDK allows the 'user' role to return mixed parts
-                contents.append(types.Content(role="user", parts=tool_response_parts))
-
-            self.trace_history.append(turn_record)
-
-        if turn >= max_turns and not agent_outcome:
-            agent_outcome = (
-                "Error: Explorer reached maximum iterations without a conclusive answer."
-            )
-
-        return agent_outcome
-
-    @staticmethod
-    def _model_content(response, function_calls: list) -> types.Content:
-        """The model turn to record: the SDK's content when present, else a rebuild."""
-        candidates = getattr(response, "candidates", None)
-        if candidates:
-            content = getattr(candidates[0], "content", None)
-            if content:
-                return content
-        return types.Content(
-            role="model",
-            parts=[types.Part(function_call=fc) for fc in function_calls],
-        )
-
-    def _build_generate_config(
-        self,
-        *,
-        cached_content,
-        is_final_turn: bool,
-        prompt_template: str,
-        temperature,
-        thinking_level,
-    ) -> types.GenerateContentConfig:
-        """Builds the per-turn generation config (cached vs full tool config)."""
-        thinking_config = (
-            types.ThinkingConfig(thinking_level=thinking_level) if thinking_level else None
-        )
-        if cached_content and not is_final_turn:
-            return types.GenerateContentConfig(
-                cached_content=cached_content.name,
-                temperature=temperature,
-                safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
-                thinking_config=thinking_config,
-            )
-        return types.GenerateContentConfig(
-            system_instruction=prompt_template,
-            temperature=temperature,
-            safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
-            tools=[
-                types.Tool(function_declarations=self.get_exposed_tools(only_submit=is_final_turn))
-            ],
-            tool_config=types.ToolConfig(
-                function_calling_config=types.FunctionCallingConfig(
-                    mode=types.FunctionCallingConfigMode.ANY
-                ),
-                include_server_side_tool_invocations=True,
-            ),
-            thinking_config=thinking_config,
-        )
-
-    @staticmethod
-    def _extract_response_texts(response) -> tuple[list[str], list[str]]:
-        """Extracts model thoughts and plain text parts from a response."""
-        thinking_parts = []
-        text_parts = []
-        candidates = getattr(response, "candidates", None)
-        if not candidates and isinstance(response, dict):
-            candidates = response.get("candidates")
-
-        if candidates and len(candidates) > 0:
-            candidate = candidates[0]
-            content = getattr(candidate, "content", None)
-            if not content and isinstance(candidate, dict):
-                content = candidate.get("content")
-
-            parts = getattr(content, "parts", None)
-            if not parts and isinstance(content, dict):
-                parts = content.get("parts")
-
-            if parts:
-                for part in parts:
-                    is_thought = False
-                    part_text = None
-                    if isinstance(part, dict):
-                        is_thought = part.get("thought", False)
-                        part_text = part.get("text", None)
-                    else:
-                        is_thought = getattr(part, "thought", False)
-                        part_text = getattr(part, "text", None)
-
-                    if is_thought and part_text:
-                        thinking_parts.append(part_text)
-                    elif part_text:
-                        text_parts.append(part_text)
-        return thinking_parts, text_parts
-
-    async def _invoke_native_model(
-        self,
-        *,
-        client,
-        contents: list,
-        cached_content,
-        is_final_turn: bool,
-        prompt_template: str,
-        model_name: str,
-        temperature,
-        thinking_level,
-    ):
-        """Invokes the model for one turn under a trace span; returns response and texts."""
-        ctx = self.ctx
-        start_turn = time.perf_counter()
-        with TraceSpan(name="gemini_explorer_call", ctx=ctx) as span:
-            generate_config = self._build_generate_config(
-                cached_content=cached_content,
-                is_final_turn=is_final_turn,
-                prompt_template=prompt_template,
-                temperature=temperature,
-                thinking_level=thinking_level,
-            )
-
-            response = await _generate_content_with_reliability(
-                lambda: asyncio.wait_for(
-                    client.aio.models.generate_content(
-                        model=model_name,
-                        contents=contents,
-                        config=generate_config,
-                    ),
-                    timeout=180,
-                ),
-            )
-            end_turn = time.perf_counter()
-            turn_dur = end_turn - start_turn
-            self.turn_latencies.append(turn_dur)
-
-            cached_tokens = 0
-            if getattr(response, "usage_metadata", None) and getattr(
-                response.usage_metadata,
-                "cached_content_token_count",
-                None,
-            ):
-                cached_tokens = response.usage_metadata.cached_content_token_count
-            self.turn_cached_tokens.append(cached_tokens)
-            span.result = (
-                f"Function calls: {len(response.function_calls)}"
-                if response.function_calls
-                else "Final answer"
-            )
-            # Extract and log model thoughts and text if present
-            thinking_parts, text_parts = self._extract_response_texts(response)
-            if thinking_parts:
-                span.payload["explorer_thought"] = "\n".join(thinking_parts)
-            if text_parts:
-                span.payload["explorer_text"] = "\n".join(text_parts)
-
-        return response, thinking_parts, text_parts
-
-    def _handle_missing_tool_calls(
-        self,
-        contents: list,
-        response,
-        turn_record: dict,
-        text_parts: list[str],
-        max_turns: int,
-        turn: int,
-    ) -> None:
-        """Handles a turn where the model produced plain text instead of tool calls.
-
-        Responses without candidates (safety blocks, empty replies) are
-        recorded as a synthetic model text part so the conversation history
-        stays well-formed for the retry.
-        """
-        logger.warning(f"Explorer turn {turn}: Model hallucinated plain text. Forcing retry.")
-        model_text = "\n".join(text_parts) if text_parts else ""
-        turn_record["tool_calls"].append(
-            {
-                "name": "hallucinated_plain_text",
-                "args": {"text": model_text},
-                "response": {
-                    "error": (
-                        "Forced retry because model failed to call submit_answer or any other tool"
-                    )
-                },
-            }
-        )
-        self.trace_history.append(turn_record)
-
-        candidates = getattr(response, "candidates", None)
-        model_content = getattr(candidates[0], "content", None) if candidates else None
-        if not model_content:
-            model_content = types.Content(
-                role="model",
-                parts=[types.Part.from_text(text=model_text or "(no response)")],
-            )
-        contents.append(model_content)
-        contents.append(
-            types.Content(
-                role="user",
-                parts=[
-                    types.Part.from_text(
-                        text=(
-                            "You have not called 'submit_answer'"
-                            " to submit your final answer yet. If"
-                            " you are not certain about the final"
-                            " answer, you can continue exploring"
-                            f" {max_turns - turn} more"
-                            " time(s)."
-                        )
-                    )
-                ],
-            )
-        )
-
-    @staticmethod
-    def _validate_submit_args(function_calls: list, args: dict, candidates: list) -> list[str]:
-        """Validates a submit_answer call; returns the list of validation errors."""
-        errors = []
-
-        if len(function_calls) > 1:
-            errors.append(
-                "The tool 'submit_answer' MUST be called"
-                " individually in your final turn without any other"
-                " parallel tool calls. Please remove other tool"
-                " calls and re-submit."
-            )
-
-        fallback_message = args.get("fallback_message", "")
-        if not candidates and not fallback_message:
-            errors.append(
-                "The candidates list is empty. You must provide at"
-                " least one candidate matching the query, or"
-                " explain observations via fallback_message."
-            )
-
-        for i, cand in enumerate(candidates):
-            label = cand.get("label")
-            coords = cand.get("coords")
-            if not label:
-                errors.append(f"Candidate at index {i} is missing a 'label'.")
-            if not coords or not isinstance(coords, list) or len(coords) != 2:
-                errors.append(
-                    f"Candidate '{label or i}' must have 'coords'"
-                    " as an array of exactly 2 integers `[nx,"
-                    " ny]`."
-                )
-            elif not is_valid_norm_point(coords):
-                try:
-                    nx, ny = int(coords[0]), int(coords[1])
-                except (ValueError, TypeError):
-                    errors.append(f"Candidate '{label or i}' coordinates are invalid integers.")
-                else:
-                    errors.append(
-                        f"Candidate '{label or i}' coordinates"
-                        f" `[{nx}, {ny}]` must strictly be in"
-                        " the `[0-1000]` normalized scale"
-                        " range inclusive."
-                    )
-
-        return errors
-
-    def _process_submit_answer(
-        self, function_calls: list, turn_record: dict, tool_response_parts: list
-    ) -> str | None:
-        """Intercepts submit_answer; returns the final outcome JSON when valid.
-
-        On validation failure the error is fed back as a tool response and
-        None is returned so the ReAct loop can continue and self-correct.
-        """
-        submit_call = next(
-            (fc for fc in function_calls if _bare_tool_name(fc.name) == SUBMIT_TOOL),
-            None,
-        )
-        if not submit_call:
-            return None
-
-        args = submit_call.args or {}
-        candidates = args.get("candidates", [])
-        errors = self._validate_submit_args(function_calls, args, candidates)
-
-        if errors:
-            logger.warning(f"Explorer submit_answer validation failed: {errors}")
-            turn_record["tool_calls"].append(
-                {
-                    "name": SUBMIT_TOOL,
-                    "args": args,
-                    "response": {"error": ("Validation Failed:\n" + "\n".join(errors))},
-                }
-            )
-            tool_response_parts.append(
-                types.Part.from_function_response(
-                    name=SUBMIT_TOOL,
-                    response={
-                        "error": (
-                            "Validation Failed:\n"
-                            + "\n".join(errors)
-                            + "\nPlease correct these formatting"
-                            " errors and re-submit using"
-                            " submit_answer."
-                        )
-                    },
-                )
-            )
-            # Allow the ReAct loop to continue so the model can self-correct
-            return None
-
-        logger.info("Explorer submit_answer validation passed successfully!")
-        # Candidates submitted by a registered label inherit that element's
-        # bounds; the rest of the model's arguments are passed through as is.
-        outcome_args = dict(args)
-        outcome_args["candidates"] = self._enrich_candidates(candidates)
-        agent_outcome = json.dumps(outcome_args, ensure_ascii=False)
-        turn_record["tool_calls"].append(
-            {"name": SUBMIT_TOOL, "args": args, "response": {"result": "success"}}
-        )
-        self.trace_history.append(turn_record)
-        return agent_outcome
-
-    @staticmethod
-    async def _append_tool_response_with_images(
-        name: str,
-        res: dict,
-        get_image_part: ImagePartGetter,
-        tool_response_parts: list,
-        *,
-        result_key: str = "result",
-    ) -> None:
-        """Appends a tool result part plus every annotated image it produced.
-
-        Image loading failures are logged and skipped: the textual result is
-        still delivered, so one unreadable annotation cannot sink the turn.
-        """
-        tool_response_parts.append(
-            types.Part.from_function_response(name=name, response={result_key: res.get("text")})
-        )
-        image_paths = [p for p in [res.get("image_path"), *(res.get("image_paths") or [])] if p]
-        for img_p in image_paths:
-            try:
-                tool_response_parts.append(await get_image_part(img_p))
-            except Exception as e:
-                logger.warning(f"Failed to load image response part for {img_p}: {e}")
-
-    async def _execute_native_tool(
-        self,
-        name: str,
-        args: dict,
-        get_image_part: ImagePartGetter,
-        tool_call_trace: dict,
-        tool_response_parts: list,
-    ) -> None:
-        """Executes one non-submit tool call and appends its response parts."""
-        if name == "ask_perception_tool":
-            search_query = args.get("search_query")
-            nx = args.get("nx")
-            ny = args.get("ny")
-            detect_queries = args.get("detect_queries")
-            res = await self.exec_ask_perception_tool(
-                search_query=search_query if isinstance(search_query, str) else "",
-                nx=nx if isinstance(nx, int) else -1,
-                ny=ny if isinstance(ny, int) else -1,
-                detect_queries=detect_queries if isinstance(detect_queries, list) else [],
-            )
-            tool_call_trace["response"] = res
-            await self._append_tool_response_with_images(
-                name, res, get_image_part, tool_response_parts, result_key="text"
-            )
-        elif name == "detect_objects":
-            res = await self.exec_detect_objects(
-                queries=args.get("queries") or [],
-                target_image_id=args.get("target_image_id", "img_0"),
-            )
-            tool_call_trace["response"] = res
-            await self._append_tool_response_with_images(
-                name, res, get_image_part, tool_response_parts
-            )
-        elif name == "ask_image_processor":
-            res = await self.exec_ask_image_processor(
-                instruction=args.get("instruction") or "",
-                target_image_id=args.get("target_image_id", "img_0"),
-            )
-            tool_call_trace["response"] = res
-            await self._append_tool_response_with_images(
-                name, res, get_image_part, tool_response_parts
-            )
-        elif name == "get_ocr_list":
-            res = await self.exec_get_ocr_list()
-            tool_call_trace["response"] = res
-            await self._append_tool_response_with_images(
-                name, res, get_image_part, tool_response_parts
-            )
-        elif name == "inspect_region":
-            res = await self.exec_inspect_region(
-                x_min=args["x_min"],
-                y_min=args["y_min"],
-                x_max=args["x_max"],
-                y_max=args["y_max"],
-                zoom_factor=args.get("zoom_factor", 2.0),
-            )
-            tool_call_trace["response"] = res
-            await self._append_tool_response_with_images(
-                name, res, get_image_part, tool_response_parts
-            )
-        else:
-            tool_call_trace["response"] = {"error": f"Tool {name} not found"}
-            tool_response_parts.append(
-                types.Part.from_function_response(
-                    name=name,
-                    response={"error": f"Tool {name} not found"},
-                )
-            )
-
-    async def _dispatch_tool_calls(
-        self,
-        function_calls: list,
-        get_image_part: ImagePartGetter,
-        turn_record: dict,
-        tool_response_parts: list,
-    ) -> None:
-        """Dispatches all non-submit tool calls sequentially with tracing.
-
-        Denylisted names are refused here as well, even though they are
-        absent from the declarations, because models occasionally call tools
-        they were never offered.
-        """
-        for fc in function_calls:
-            name = _bare_tool_name(fc.name)
-            if name == SUBMIT_TOOL:
-                continue
-            args = fc.args or {}
-            tool_call_trace = {"name": name, "args": args}
-            if name in self.denylisted_tools:
-                logger.warning(
-                    f"Explorer attempted to call denylisted tool '{name}'. Blocking execution."
-                )
-                error = f"Tool '{name}' is denylisted and unavailable."
-                tool_call_trace["response"] = {"error": error}
-                turn_record["tool_calls"].append(tool_call_trace)
-                tool_response_parts.append(
-                    types.Part.from_function_response(name=fc.name, response={"error": error})
-                )
-                continue
-
-            logger.info(f"Explorer executing tool '{name}' sequentially...")
-
-            try:
-                await self._execute_native_tool(
-                    name, args, get_image_part, tool_call_trace, tool_response_parts
-                )
-            except Exception as e:
-                logger.error(f"Explorer tool {name} execution failed: {e}")
-                tool_call_trace["response"] = {"error": str(e)}
-                tool_response_parts.append(
-                    types.Part.from_function_response(name=name, response={"error": str(e)})
-                )
-            finally:
-                turn_record["tool_calls"].append(tool_call_trace)
-
-    def _format_native_failure(self, e: Exception, model_name: str) -> str:
-        """Maps a loop failure to the user-facing fallback outcome JSON."""
-        logger.error(f"Explorer execution loop failed: {e}")
-
-        err_msg = str(e)
-        if "preempted" in err_msg.lower():
-            clean_msg = (
-                "Model request was preempted by the server due to high"
-                " demand. Please try again later."
-            )
-        elif "overloaded" in err_msg.lower() or "503" in err_msg:
-            clean_msg = "Model service is currently overloaded. Please try again later."
-        elif "quota" in err_msg.lower() or "429" in err_msg:
-            clean_msg = "Model API quota exceeded or rate limited."
-        elif "thinkingconfig" in err_msg.lower():
-            clean_msg = (
-                "Model configuration error: ThinkingConfig is not"
-                f" supported for model {model_name}."
-            )
-        else:
-            clean_msg = f"Explorer agent execution failed: {err_msg}"
-
-        return json.dumps(
-            {"candidates": [], "fallback_message": clean_msg},
-            ensure_ascii=False,
-        )
-
-    async def _cleanup_native_resources(self, client, cached_content, uploaded_files: list) -> None:
-        """Deletes the cache resource and File API uploads created for this run."""
-        if cached_content:
-            try:
-                logger.info(f"Deleting cache resource: {cached_content.name}")
-                await client.aio.caches.delete(name=cached_content.name)
-            except Exception as cleanup_cache_err:
-                logger.warning(f"Failed to delete cache resource: {cleanup_cache_err}")
-        for file_ref in uploaded_files:
-            try:
-                await client.aio.files.delete(name=file_ref.name)
-            except Exception as cleanup_err:
-                logger.warning(f"Failed to delete uploaded file {file_ref.name}: {cleanup_err}")
-        logger.info(f"Explorer turn latencies: {self.turn_latencies}")

artemis/agents/explorer/tool_declarations.py

@@ -14,14 +14,9 @@
 
 """Static tool declarations for the Explorer agent.
 
-Split out of ``artemis.agents.explorer.explorer`` as pure data:
-``UNIVERSAL_EXPLORER_TOOLS`` (OpenAI-style dict schemas for the universal
-LangChain path) and ``NATIVE_EXPLORER_TOOL_DECLARATIONS`` (google-genai
-``FunctionDeclaration`` objects for the native Gemini path).
+OpenAI function schemas consumed by the Responses model adapter.
 """
 
-from google.genai import types
-
 UNIVERSAL_EXPLORER_TOOLS = [
     {
         "type": "function",
@@ -183,212 +178,3 @@ UNIVERSAL_EXPLORER_TOOLS = [
         },
     },
 ]
-
-
-NATIVE_EXPLORER_TOOL_DECLARATIONS = [
-    types.FunctionDeclaration(
-        name="ask_perception_tool",
-        description=(
-            "[Perception] Concurrently executes and awaits three sub-tasks"
-            " in parallel, including UI-tree text search, coordinate audit,"
-            " and visual object detection on the screen.\n\nOutput"
-            " Format:\nReturns:'text' (str): A single string consolidating"
-            " results with 0-1000 normalized coordinates, including UI"
-            " search matches labeled [X],"
-            " coordinate search matches labeled [X], and visually detected"
-            " object targets labeled [D].\n(Annotated screenshots are also"
-            " added to the context containing corresponding colored label"
-            " dots)."
-        ),
-        parameters=types.Schema(
-            type=types.Type.OBJECT,
-            properties={
-                "search_query": types.Schema(
-                    type=types.Type.STRING,
-                    description=("Required. Text query to search for in UI elements."),
-                ),
-                "nx": types.Schema(
-                    type=types.Type.INTEGER,
-                    description=(
-                        "Required. Normalized X coordinate (0-1000 range)"
-                        " to audit for overlapping elements."
-                    ),
-                ),
-                "ny": types.Schema(
-                    type=types.Type.INTEGER,
-                    description=(
-                        "Required. Normalized Y coordinate (0-1000 range)"
-                        " to audit for overlapping elements."
-                    ),
-                ),
-                "detect_queries": types.Schema(
-                    type=types.Type.ARRAY,
-                    items=types.Schema(type=types.Type.STRING),
-                    description=(
-                        "Required. List of query strings to visually detect on the screen."
-                    ),
-                ),
-            },
-            required=["search_query", "nx", "ny", "detect_queries"],
-        ),
-    ),
-    types.FunctionDeclaration(
-        name="detect_objects",
-        description=(
-            "[Perception] Locates elements, shapes and more using a"
-            " VLM. Returns their normalized coordinates (in a [0, 1000]"
-            " scale) labeled with [D1], [D2] etc. in the text output, along"
-            " with an annotated image containing corresponding visual label"
-            " dots."
-        ),
-        parameters=types.Schema(
-            type=types.Type.OBJECT,
-            properties={
-                "target_image_id": types.Schema(
-                    type=types.Type.STRING,
-                    description="The ID of the image to process.",
-                ),
-                "queries": types.Schema(
-                    type=types.Type.ARRAY,
-                    items=types.Schema(type=types.Type.STRING),
-                    description=("A list of single query strings to detect on the screen."),
-                ),
-            },
-            required=["queries", "target_image_id"],
-        ),
-    ),
-    types.FunctionDeclaration(
-        name="get_ocr_list",
-        description=(
-            "[Perception] Retrieves all text elements detected on the"
-            " original, unprocessed screen via OCR. Returns their"
-            " normalized coordinates (in a [0, 1000] scale) labeled with"
-            " [O1], [O2] etc. in the text output, along with an annotated"
-            " image containing corresponding visual label dots."
-        ),
-        parameters=types.Schema(
-            type=types.Type.OBJECT,
-            properties={},
-        ),
-    ),
-    types.FunctionDeclaration(
-        name="ask_image_processor",
-        description=(
-            "[Transformation] A scripting tool for pixel-level image"
-            " processing. CAUTION: This tool is slow and expensive. Returns"
-            " a summary of the operations performed and new image IDs,"
-            " along with new labels and screen coordinates."
-        ),
-        parameters=types.Schema(
-            type=types.Type.OBJECT,
-            properties={
-                "target_image_id": types.Schema(
-                    type=types.Type.STRING,
-                    description=(
-                        "The ID of the image in the Image Pool to start"
-                        " from (e.g. 'img_0', 'img_1')."
-                    ),
-                ),
-                "instruction": types.Schema(
-                    type=types.Type.STRING,
-                    description=(
-                        "1-2 precise commands describing a simple task."
-                        " Include all necessary coordinates and other"
-                        " parameters.\n"
-                    ),
-                ),
-            },
-            required=["instruction", "target_image_id"],
-        ),
-    ),
-    types.FunctionDeclaration(
-        name="submit_answer",
-        description=(
-            "[Finalization] Submits the final list of ranked candidate UI"
-            " elements matching the search query, or a fallback message if"
-            " no elements were found, to complete the search task. Fails"
-            " validation if: (1) candidates is empty and no"
-            " fallback_message is provided, (2) coordinates are not exactly"
-            " 2 integers [nx, ny], or (3) coordinates lie outside"
-            " normalized [0, 1000] range."
-        ),
-        parameters=types.Schema(
-            type=types.Type.OBJECT,
-            properties={
-                "candidates": types.Schema(
-                    type=types.Type.ARRAY,
-                    items=types.Schema(
-                        type=types.Type.OBJECT,
-                        properties={
-                            "label": types.Schema(
-                                type=types.Type.STRING,
-                                description=(
-                                    "Reference label (e.g. X1, O2, D3, or a number like 3)"
-                                ),
-                            ),
-                            "coords": types.Schema(
-                                type=types.Type.ARRAY,
-                                items=types.Schema(type=types.Type.INTEGER),
-                                description=("[nx, ny] coordinates in normalized 0-1000 scale"),
-                            ),
-                            "description": types.Schema(
-                                type=types.Type.STRING,
-                                description=("Brief description of this candidate element"),
-                            ),
-                        },
-                        required=["label", "coords"],
-                    ),
-                    description=(
-                        "A ranked list of candidate UI elements that match"
-                        " the query, sorted from highest to lowest"
-                        " confidence (maximum 10 candidates). Leave empty"
-                        " if no matching elements were found on the screen."
-                    ),
-                ),
-                "fallback_message": types.Schema(
-                    type=types.Type.STRING,
-                    description=(
-                        "Optional fallback explanation if target elements"
-                        " were not found or if there are special"
-                        " observations."
-                    ),
-                ),
-            },
-            required=["candidates"],
-        ),
-    ),
-    types.FunctionDeclaration(
-        name="inspect_region",
-        description=(
-            "[Perception] Crops and resizes a bounding box region of the"
-            " original screenshot. Returns the cropped and resized image"
-            " directly, without any image ID or coordinate transform."
-        ),
-        parameters=types.Schema(
-            type=types.Type.OBJECT,
-            properties={
-                "x_min": types.Schema(
-                    type=types.Type.INTEGER,
-                    description="Left X coordinate (0-1000).",
-                ),
-                "y_min": types.Schema(
-                    type=types.Type.INTEGER,
-                    description="Top Y coordinate (0-1000).",
-                ),
-                "x_max": types.Schema(
-                    type=types.Type.INTEGER,
-                    description="Right X coordinate (0-1000).",
-                ),
-                "y_max": types.Schema(
-                    type=types.Type.INTEGER,
-                    description="Bottom Y coordinate (0-1000).",
-                ),
-                "zoom_factor": types.Schema(
-                    type=types.Type.NUMBER,
-                    description=("Zoom factor to upscale the cropped region (1.0 to 4.0)."),
-                ),
-            },
-            required=["x_min", "y_min", "x_max", "y_max", "zoom_factor"],
-        ),
-    ),
-]

artemis/agents/explorer/universal_runner.py

@@ -30,6 +30,8 @@ from langchain_core.messages import (
     ToolMessage,
 )
 
+from artemis.llm.structured import content_to_text
+
 from artemis.agents.explorer.geometry import is_valid_norm_point
 from artemis.agents.explorer.tiers import SUBMIT_TOOL
 from artemis.agents.explorer.tool_declarations import UNIVERSAL_EXPLORER_TOOLS
@@ -128,7 +130,8 @@ class UniversalRunnerMixin:
                     return json.dumps(
                         {
                             "candidates": [],
-                            "fallback_message": str(response.content) or "No candidates found.",
+                            "fallback_message": content_to_text(response.content)
+                            or "No candidates found.",
                         },
                         ensure_ascii=False,
                     )

artemis/agents/flash/flash_runner.md

@@ -1,3 +1,11 @@
+# 输出语言要求
+所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。
+保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
+遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。
+
+每次调用设备操作工具前，用一小段简体中文说明当前观察、即将执行的动作和目的；操作后仅在获得结果或新观察时说明实际变化。不要只在最终答复中使用中文。工具参数中的自由描述文本也使用简体中文，精确引用的界面文字仍保留原文。
+
+
 # ROLE & OBJECTIVE
 You are an autonomous and highly efficient Android Device Execution Agent. Your goal is to accomplish the user's objective on the device.
 

artemis/agents/flash/flash_summarizer.md

@@ -1,11 +1,17 @@
+# 输出语言要求
+所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。
+保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
+遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。
+
+
 # ROLE & OBJECTIVE
 You are the Step Summarizer for an Android UI automation agent. Your task is to synthesize the Before/After screenshots and physical action into a concise, high-information-density, strictly objective first-person historical memory.
 
 ---
 
 # PERSPECTIVE & FORMAT CONSTRAINTS
-1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **"I"** (e.g., "In Step {{ step_number }}, I tapped...", "I swiped up on... and observed..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
-2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 sentences, 60–100 words).
+1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **“我”** (e.g., "In Step {{ step_number }}, I tapped...", "I swiped up on... and observed..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
+2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 句，约 100–200 个汉字；保留关键原文证据，不为凑字数添加内容).
 3. **No Lists or Formatting**: Do NOT use bullet points, numbered lists, markdown headers, bold labels, or line breaks in your output. Your output must NEVER contain `---` separators or section-marker lines — those belong to the input, not the summary.
 
 ---
@@ -13,7 +19,7 @@ You are the Step Summarizer for an Android UI automation agent. Your task is to
 # CORE OBSERVATION RULES
 1. **Zero Subjective Validation (STRICT)**:
    - Absolutely avoid declaring semantic goal achievement, subjective success, or final failure. In historical compression, subjective assumptions cause future turns to falsely believe a subgoal was definitively accomplished or permanently blocked.
-   - **STRICTLY BANNED WORDS**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
+   - **STRICTLY BANNED WORDS (including Chinese equivalents such as “成功完成”、“已达成”、“失败” and “不可能”)**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
 2. **Action & Visual Delta Structure**:
    - **Target & Action**: Identify what control/button was targeted (referencing the red visual indicator on the BEFORE screen) and the physical action performed.
    - **Objective Transition**: Describe the physical screen change (e.g., page transitioned, modal dialog opened, list scrolled revealing new items, checkbox/toggle toggled, keyboard appeared).

artemis/agents/flash/flash_summarizer_single.md

@@ -1,3 +1,9 @@
+# 输出语言要求
+所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。
+保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
+遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。
+
+
 # ROLE & OBJECTIVE
 You are the Step Summarizer for an Android UI automation agent. Exactly ONE screenshot is available for this step — the decision frame captured when the action was issued. There is NO after-action screenshot. Your task is to describe, strictly objectively and in first person, what THIS screen showed and where the physical action landed.
 
@@ -11,8 +17,8 @@ You are the Step Summarizer for an Android UI automation agent. Exactly ONE scre
 ---
 
 # PERSPECTIVE & FORMAT CONSTRAINTS
-1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **"I"** (e.g., "In Step {{ step_number }}, I tapped... on a screen showing..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
-2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 sentences, 60–100 words).
+1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **“我”** (e.g., "In Step {{ step_number }}, I tapped... on a screen showing..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
+2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 句，约 100–200 个汉字；保留关键原文证据，不为凑字数添加内容).
 3. **No Lists or Formatting**: Do NOT use bullet points, numbered lists, markdown headers, bold labels, or line breaks. Your output must NEVER contain `---` separators or section-marker lines — those belong to the input, not the summary.
 
 ---
@@ -20,7 +26,7 @@ You are the Step Summarizer for an Android UI automation agent. Exactly ONE scre
 # CORE OBSERVATION RULES
 1. **Zero Subjective Validation (STRICT)**:
    - Absolutely avoid declaring semantic goal achievement, subjective success, or final failure. In historical compression, subjective assumptions cause future turns to falsely believe a subgoal was definitively accomplished or permanently blocked.
-   - **STRICTLY BANNED WORDS**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
+   - **STRICTLY BANNED WORDS (including Chinese equivalents such as “成功完成”、“已达成”、“失败” and “不可能”)**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
 2. **Preserve Critical Data**: Faithfully transcribe visible text, toast alerts, error banners, verification codes, prices, account names, or tracking numbers that appear on this screen.
 
 ---

artemis/agents/flash/runner.py

@@ -14,8 +14,7 @@
 
 """Universal Multi-Model FlashRunner for Artemis.
 
-Executes autonomous reactive mobile workflows across Google Gemini,
-OpenAI GPT-4o/o3, Anthropic Claude 3.5/3.7, and OpenRouter endpoints.
+Executes autonomous reactive mobile workflows through OpenAI Responses.
 
 Conversation shape (shared with the Pro operator, history redesign §3.2):
 the prompt is built every turn from a session :class:`TranscriptLedger` —
@@ -38,6 +37,7 @@ active window until its turn is chunked, whose user lines then carry it on.
 """
 
 import asyncio
+from artemis.config.settings import settings
 import base64
 from dataclasses import dataclass, field
 import json
@@ -83,7 +83,7 @@ from artemis.memory.transcript import PRO_UI_LIST_MARKER, TranscriptLedger, mark
 from artemis.services.llm import (
     RobustChatModelWrapper,
     acomplete,
-    get_google_llm,
+    get_openai_llm,
     get_llm,
     invoke_llm_with_timeout_message,
 )
@@ -284,7 +284,9 @@ class FlashRunner:
         except Exception as e:
             logger.warning(f"Failed to get operator LLM from config, using default: {e}")
 
-            return RobustChatModelWrapper(get_google_llm(model_name="gemini-2.5-flash"), self.ctx)
+            return RobustChatModelWrapper(
+                get_openai_llm(model_name=settings.OPENAI_MODEL), self.ctx
+            )
 
     def _render_system_prompt(self, tools_declaration: list) -> str:
         """Renders the system prompt from the flash_runner.md template.

artemis/agents/flash/summarizer.py

@@ -26,6 +26,7 @@ frame, the ``flash_summarizer.md`` neutral-wording contract, and versioned
 """
 
 import asyncio
+from artemis.config.settings import settings
 import base64
 from pathlib import Path
 from typing import Any
@@ -36,7 +37,7 @@ from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
 
 from artemis.context import ArtemisContext
 from artemis.memory.step_memory import JobKey, StepMemoryService
-from artemis.services.llm import RobustChatModelWrapper, get_google_llm, get_llm
+from artemis.services.llm import RobustChatModelWrapper, get_openai_llm, get_llm
 from artemis.services.token_meter import record_llm_usage
 from artemis.utils.logger import get_logger
 from artemis.utils.task_tree import format_actions_clean
@@ -162,15 +163,15 @@ class VisualStepSummarizer(StepMemoryService):
         )
 
         # Initialize lightweight VLM: prioritize explicit model_name
-        target_model = model_name or "gemini-2.5-flash-lite"
+        target_model = model_name or settings.OPENAI_MODEL
         self._model_name = target_model
         try:
             if model_name:
-                self._llm = get_google_llm(model_name=target_model, temperature=0.0)
+                self._llm = get_openai_llm(model_name=target_model, temperature=0.0)
             else:
                 self._llm = get_llm(ctx, name="summarizer", is_utils=True)
         except Exception:
-            self._llm = get_google_llm(model_name=target_model, temperature=0.0)
+            self._llm = get_openai_llm(model_name=target_model, temperature=0.0)
         try:
             configured = getattr(self._llm, "model", None) or getattr(self._llm, "model_name", None)
             if isinstance(configured, str) and configured:
@@ -341,7 +342,7 @@ class VisualStepSummarizer(StepMemoryService):
         """Meter one raw-model lens call as an ``llm_usage`` trace, best-effort.
 
         Gateway-wrapped models already meter at the wrapper exit; only the raw
-        ``get_google_llm`` bypass needs explicit metering here. Lens prompts
+        ``get_openai_llm`` bypass needs explicit metering here. Lens prompts
         are tiny and must not overwrite the session's ``last_prompt_tokens``
         (the compaction thresholds' live context base), hence
         ``update_last_prompt=False``.

artemis/agents/history_analyzer/history_analyzer.py

@@ -25,6 +25,8 @@ from pathlib import Path
 from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
 from langchain_core.tools import BaseTool
 
+from artemis.llm.structured import content_to_text
+
 from artemis.core.tool_failure import is_tool_failure
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import TraceSpan, trace
@@ -139,7 +141,7 @@ class HistoryAnalyzer:
                 return (
                     response.content.strip()
                     if isinstance(response.content, str)
-                    else str(response.content)
+                    else content_to_text(response.content)
                 )
 
             # Process tool calls

artemis/agents/log_analyzer/log_analyzer.py

@@ -51,6 +51,8 @@ class SpawnLogReaderArgs(BaseModel):
 
 from langgraph.prebuilt import InjectedState
 
+from artemis.llm.structured import content_to_text
+
 from artemis.core.tool_failure import ToolFailure, is_tool_failure
 from artemis.context import ArtemisContext
 from artemis.data_engine.trace import (
@@ -224,7 +226,7 @@ class LogAnalyzerNode:
             response = await invoke_llm_with_timeout_message(acomplete(llm, current_messages))
 
             if response.content:
-                outcome = response.content
+                outcome = content_to_text(response.content)
 
             if not response.tool_calls:
                 break

artemis/agents/log_analyzer/output_analyzer.py

@@ -16,6 +16,8 @@ from pathlib import Path
 from typing import Annotated
 from langchain_core.messages import HumanMessage, SystemMessage, ToolMessage
 from langchain_core.tools import tool
+from artemis.llm.structured import content_to_text
+
 from artemis.core.tool_failure import ToolFailure, is_tool_failure
 from artemis.context import ArtemisContext
 from artemis.services.llm import get_llm, invoke_llm_with_timeout_message
@@ -131,7 +133,7 @@ class TaskOutputAnalyzerNode:
             response = await invoke_llm_with_timeout_message(llm.ainvoke(current_messages))
 
             if response.content:
-                outcome = response.content
+                outcome = content_to_text(response.content)
 
             if not response.tool_calls:
                 break

artemis/agents/operator/operator.json

@@ -1,3 +1,3 @@
 {
-  "main_template": "# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n[% if \"wait_for_delay\" in available_tools %]- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n[% endif %]- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.\n  2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading state. This boundary probe must remain unidirectional; never reverse direction merely to prove completion. If the task declares an earlier exit condition (for example, find one verified match or process N items), that explicit condition may end the traversal without reaching the list boundary.\n  3. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.\n  4. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed with confirmed anchor continuity and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for \"double-checking\", even if older execution history has been pruned. Resume from the last recorded anchor and direction. Only the single bounded recovery above, or a refreshed/mutated list, permits limited re-verification; otherwise continue the single pass over new content.\n- **Dynamic Monitoring & Reactive Loop Protocol (Monitor-Trigger-Action)**: When executing an active milestone matching `[Loop] <Goal> into note <TargetNote> (Exit: <ExitCondition>; Interval: <Interval>)` or `[Loop:continuous] <Goal> into note <TargetNote> (Interval: <Interval>)` (the milestone grammar itself is defined under Task Plan Grammar below), run this 5-step cyclic state machine:\n  - **Step 1 (Baseline Sync)**: On the initial cycle, observe all visible entries/metrics in the primary monitoring view and save the full snaLine truncated
+  "main_template": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。\n\n每次调用设备操作工具前，用一小段简体中文说明当前观察、即将执行的动作和目的；操作后仅在获得结果或新观察时说明实际变化。不要只在最终答复中使用中文。工具参数中的自由描述文本也使用简体中文，精确引用的界面文字仍保留原文。\n\n# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n[% if \"wait_for_delay\" in available_tools %]- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n[% endif %]- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n  1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.\n  2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading state. This boundary probe must remain unidirectional; never reverse direction merely to prove completion. If the task declares an earlier exit condition (for example, find one verified match or process N items), that explicit condition may end the traversal without reaching the list boundary.\n  3. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtLine truncated
 }

artemis/agents/operator/operator.py

@@ -29,7 +29,7 @@ from artemis.data_engine.trace import (
 from artemis.graph.state import State
 from artemis.graph.visibility import strict_state
 from artemis.mcp.action_specs import OPERATOR_SHELL_ORDER, operator_shell_tool
-from artemis.llm.google import usage_from_message
+from artemis.llm.usage import usage_from_message
 from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
 from artemis.tools.command_tool import (
     analyze_task_output_wrapper,

artemis/agents/outputter/outputter.md

@@ -1,3 +1,9 @@
+# 输出语言要求
+所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。
+保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
+遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。
+
+
 # Context & Inputs
 
 ## User's Initial Goal

artemis/agents/outputter/outputter.py

@@ -20,6 +20,8 @@ from pathlib import Path
 
 from jinja2 import Template
 from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
+from artemis.llm.structured import content_to_text
+
 from artemis.core.tool_failure import ToolFailure, is_tool_failure
 from artemis.config import OutputConfig
 from artemis.context import ArtemisContext
@@ -265,7 +267,7 @@ async def outputter(
         messages.append(response)
 
         if not response.tool_calls:
-            raw_answer = response.content
+            raw_answer = content_to_text(response.content)
             break
 
         for tc in response.tool_calls:

artemis/agents/planner/planner.json

@@ -1,13 +1,13 @@
 {
   "blocks": {
-    "role": "## Role\nYou are a Senior Test Engineer leading a team of mobile execution agents. Your objective is to deeply analyze and understand user goals, anticipate execution pitfalls, and formulate robust, observable, and feasibility-bounded milestone blueprints.",
+    "role": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。\n\n## Role\nYou are a Senior Test Engineer leading a team of mobile execution agents. Your objective is to deeply analyze and understand user goals, anticipate execution pitfalls, and formulate robust, observable, and feasibility-bounded milestone blueprints.",
     "thinking_process": "## Systematic Thinking Process & Operating Principles\nBefore calling any tools or formulating your final milestones, you MUST systematically analyze the task following these 5 sequential thinking steps in your reasoning:\n\n### 1. Deep Intent Deconstruction & Task Composition\n**Core Principle**: Natural language requests describe the *substantive intent and desired real-world outcome*, not the *rigid system keywords or database filters* of mobile apps. Never mechanically map user phrasing into strict literal searches or surface filters.\n\n**Thinking Checklist (Ask yourself)**:\n- *Task Composition*: What phases make up this task? Is it a sequence of one-off linear steps (e.g., open app, configure settings), an iterative loop over multiple items (e.g., inspecting a list of search results), or a periodic polling loop (e.g., refreshing to monitor new messages, waiting for battery level)?\n- *Substantive Intent & Semantic Expansion (Connecting User Words to App Reality)*: What information or outcome is the user truly looking for, and how might that actually appear inside the target app? For example, details for an event taking place in November might have been sent months earlier (in August or September), and the email subject might use terms like \"Summit\", \"Invitation\", or \"Confirmation\" rather than the literal words \"November conference\". Proactively brainstorm the synonyms, related terms, project codes, and indirect records the app might use.\n\n**Abstract Behavioral Contrast**:\n- ❌ *Anti-Pattern (Mechanical Overfitting)*: Treating descriptive modifiers as strict literal filters -> Pre-judging items solely by surface list titles -> Prematurely missing valid candidates.\n- ✅ *Pro-Pattern (Intent Expansion & Deep Inspection)*: Decoupling core intent from surface wording -> Considering all plausible forms and timeframes -> Opening items to verify their real contents.\n\n### 2. Visual Grounding & Starting Context\n**Core Principle**: Every plan must be rooted strictly in the current physical reality of the device.\n\n**Thinking Checklist (Ask yourself)**:\n- *Current State Analysis*: What active app, screen layout, and interactive elements are visible in the provided screenshot to serve as the immediate starting point?\n\n### 3. Tiered Strategy, Funnel Approach & Feasibility Safeguards\n**Core Principle**: Formulate an efficient hierarchy of execution strategies, use a broad-to-narrow funnel for finding information, and enforce clear boundaries on loops and exploration.\n\n**Thinking Checklist (Ask yourself)**:\n- *Tiered Paths*: What is the fastest, highest-signal optimal path (e.g., targeted in-app search, direct shortcuts)? If that fails or yields no results, what is the pragmatic fallback path (e.g., browsing categories, chronological scanning)?\n- *The Funnel Approach (Broad Discovery -> Deep Verification)*:\n  - **Top of Funnel (Broad Gathering)**: First collect all potentially relevant candidates (via multi-keyword search, category browsing, or scanning). Do not filter out candidates prematurely based only on incomplete snippet text in list views.\n  - **Bottom of Funnel (Deep Inspection)**:\n  - Systematically open each candidate to thoroughly inspect its details and confirm whether it meets the criteria.\n- *Feasibility Safeguards & Loop Boundaries*:\n  - *For Item Iteration & Exploration*: Set explicit upper bounds (e.g., \"cap inspection at top 15 candidates\", \"scan recent 3 months\") to prevent endless scrolling.\n  - *For Periodic Monitoring / Polling*: Specify the check action (e.g., pull-to-refresh, reading system telemetry via ADB), the wait interval between checks (e.g., check every 5 minutes to conserve resources), and the exit criteria (e.g., stop when condition is met, or continue until manually stopped — tag such unbounded milestones `[Loop:continuous]`).\n\n### 4. Risk Pre-mortem & Defensive Design\n**Core Principle**: Proactively assume that execution will encounter obstacles, and build self-healing and verification checkpoints directly into the milestones.\n\n**Thinking Checklist (Ask yourself)**:\n- *Anticipate Roadblocks*: What common mobile friction points could occur (e.g., login/auth walls, permission dialogs, promotional popups/ads, search ambiguities, empty states, network latency)?\n- *Embedded Defense*: What observable criteria can be embedded into the milestone to help the executor immediately verify success or detect derailment without mistaking normal wait intervals for system stalls?\n\n### 5. Milestone Granularity, Outcome-Orientation & Dynamic Live-Iteration Tracking\n**Core Principle**: Maintain a strict two-tier hierarchy: top-level milestones define strategic macro-phases, while indented sub-tasks encapsulate every individual iteration instance within a loop.\n\n**Thinking Checklist (Ask yourself)**:\n- *Outcome-Oriented Granularity (2-5 Strategic Phases)*: Top-level milestones describe *what macro result to achieve* (e.g., \Line truncated
     "capabilities_operator": "## Execution Capabilities\n\nThe execution agent primarily interacts with the mobile device and analyzes screen recordings and logs. It can perform the following types of actions:\n- **UI Interaction**: Tapping, swiping, and entering text on the screen.\n- **App Management**: Launching and stopping specific applications.\n- **Information Retrieval & Analysis**: Reading on-screen text and UI hierarchy, searching the web for external information, querying system logs, and analyzing screen recordings.\n- **Terminal & Background Operations**: Executing arbitrary ADB shell commands on the device for exploration. It is capable of running long-running ADB commands asynchronously in the background as tasks.\n\n### Capability Boundaries & Planning Rules\n- **Cross-App Operations**: When a task requires interacting with multiple applications, split the operations into distinct milestones for each app. Avoid mixing actions for different apps in a single milestone.",
     "output_protocol_initial": "## Task Plan & Scratchpad Protocol\n\n1. **Articulate Thinking**: Begin by reasoning through the 5-step **Systematic Thinking Process** to establish your strategy, funnel approach, bounds, and pre-mortem checks.\n2. **Designate Notes in Milestones & Pre-generate Answer Cards**: If the task involves collecting findings, logging periodic monitoring events, recording candidates, or analytical summaries, designate clear, descriptive note names directly in the milestone descriptions (e.g., `candidates_list`, `monitoring_log`, `schedule_summary`). You can and should invoke the `save_note` tool to pre-generate these structured template files (answer card / scratchpad skeletons with Markdown headers, sections, or tables) before submitting `task_plan`. This provides downstream executors with a clear, ready-to-fill schema.\n3. **Submit Plan**: Submit your initial plan by invoking the `save_note` tool with `key='task_plan'`. Avoid writing the plan only in your text response, as it will only be recognized by the system if submitted via the tool.\n\n### Plan Formatting Rules\n- Format the plan strictly as a Markdown task list starting with `- [ ]` for uncompleted items.\n{{ plan_grammar }}\n- For linear macro-milestones: `- [ ] <Phase Description>`\n- For bounded repetitive, iterative, or periodic polling milestones, tag the item with `[Loop]` and declare its Exit Boundary and Interval (e.g., `- [ ] [Loop] <Overall Loop / Monitoring Goal> (Exit: <Condition>; Interval: <e.g., every 5 minutes>)`).\n- For unbounded continuous monitoring that only the user can stop, tag the item with `[Loop:continuous]` instead (e.g., `- [ ] [Loop:continuous] <Monitoring Goal> (Interval: <e.g., every 5 minutes>)`); never declare a self-serve exit condition for it.\n- **Strict Two-Tier Hierarchy & Loop Subtask Contract**:\n  1. **Subtask Nesting Rule**: ALL individual iteration instances, polling checks, candidate inspections, or baseline setups MUST be formatted as 2-space indented sub-tasks (`  - [ ]`) directly nested under their parent `[Loop]` milestone.\n  2. **Mandatory First-Instance Initialization**: Every `[Loop]` milestone MUST be initialized with its first concrete child sub-task (e.g., `  - [ ] Inspect Candidate #1: ...` or `  - [ ] Polling Check #1 (Baseline & Initial Sync): ...`).\n  3. **No Flattened Cycles**: NEVER elevate individual checks, recurring polling cycles, or candidate inspections to top-level milestones outside the `[Loop]`. Top-level items are reserved exclusively for strategic macro-phases.\n- Ensure the content of the `task_plan` note contains only the task list, excluding introductions, explanations, conversational text, or headers.",
     "examples_initial": "## Master Example: Demonstrating the Complete Cognitive Paradigm\n\n**Goal**: \"Find and summarize all schedule arrangements and venue details for the upcoming November tech conference from my emails.\"\n\n### Naive / Incorrect Approaches:\n1. *Flawed Semantics & Too Granular*:\n- [ ] Tap search icon in Gmail\n- [ ] Type \"November\"\n- [ ] Open first email and tap back\n*(Fails: Micromanages atomic UI taps; assumes emails for November arrangements must have \"November\" in the title or be received in November)*\n2. *Vague & Unbounded Trap*:\n- [ ] Open email and check all emails for conference details\n*(Fails: Unbounded scan trap; causes infinite scrolling and step budget exhaustion on mobile)*\n\n---\n\n### Correct Approach (Full Cognitive Thinking & Robust Plan):\n\n#### Step-by-Step Thinking:\n1. **Intent & Semantic Expansion**:\n   - *Core Objective*: Extract confirmed schedule dates, sessions, and venue info for the November tech conference.\n   - *Semantic Insight*: Conference communications are often sent months in advance (e.g., ticket confirmations or registration emails in August/September). The subject line may say \"Global Developer Summit\", \"Tech Expo Agenda\", or \"Registration Confirmed\" rather than literally \"November conference\".\n   - *Task Composition*: Linear setup (open email app and search) -> Iterative candidate inspection loop (examine each promising email) -> Final consolidation.\n2. **Visual Grounding**:\n   - The device screenshot shows the Home screen with the Gmail app icon visible. The plan must start by opening the email client.\n3. **Tiered Strategy, Funnel Approach & Safeguards**:\n   - *Tier 1 (Optimal)*: Use in-app search with rich semantic keywords (\"conference\", \"summit\", \"schedule\", \"agenda\", \"venue\", \"November\").\n   - *Tier 2 (Fallback)*: If search is unavailable or yields zero results, fallback to chronological scanning of recent 3 months.\n   - *Funnel (Broad Gathering -> Deep Inspection)*: Gather candidate emails from search results; do not discard candidates based solely on list previews. Open each candidate thread to inspect full body details.\n   - *Feasibility Boundary*: Cap the candidate email inspection pool at top 15 emails to prevent infinite scrolling.\n4. **Risk Pre-mortem**:\n   - *Risks*: Promotional marketing spam; conflicting drafts with preliminary dates; cancelled sessions.\n   - *Defensive Check*: Instruct downstream executor to filter out promotional newsletters, verify session timestamps, and confirm final venue address.\n5. **Milestone Granularity & Scratchpad Blueprint**:\n   - Focus milestones on business outcomes (search, inspect candidates, consolidate), avoiding micromanaged click-by-click steps.\n   - Use indented sub-tasks to guide the loop of inspecting each candidate email, initializing with Item #1 and adding subsequent items as they are identified. This prevents the agent from losing state or re-inspecting the same items.\n   - Designate note `november_conference_summary` as the destination for recording findings and pre-generate the structured template note.\n\n#### Tool Invocation:\n1. Pre-generate designated answer card note template:\n`save_note(key='november_conference_summary', content='# November Tech Conference Summary\\n\\n## Key Details\\n- Conference Name: \\n- Dates: \\n- Venue / Address: \\n\\n## Daily Schedule & Sessions\\n(To be populated during email inspection)\\n')`\n\n2. Submit milestone plan:\n`save_note(key='task_plan', content='- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\\n  - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\\n- [ ] Review and consolidate all extracted information in `november_conference_summary` into a clear, finalized conference itinerary')`\n\n#### Content of `task_plan` Note:\n- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\n  - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\n- [ ] ReviLine truncated
     "check_generation": "## Declaring Verification Standards (check lines)\nYou may attach check lines to the plan. {% if midway_checks %}An independent Checker judges each line in the background when its milestone completes, and a failed verify reopens that milestone for repair{% if final_check %}; a final review at task exit then audits the whole run against the user's goal{% endif %}.{% else %}A final review at task exit judges every line once, from the recorded evidence around each milestone's completion; there is no midway repair loop.{% endif %}\n\n- **User-declared expectations become `- assert:` lines.** Whenever the user's goal states an explicit expectation (\"should show X\", \"must not crash\", \"the toast Y should appear\"), declare it as an assert line preserving the user's wording verbatim. Expectations about the whole run or the final state use the `@end` suffix (task-level lines at the end of the plan).\n- **`- verify:` lines only for milestones that genuinely need acceptance checks** — completion points of irreversible operations, or milestones that directly contribute to the final goal. Do NOT attach a verify line to every subgoal: restraint is an explicit requirement (checking cost scales with the number of check lines).\n- **Do not fabricate assertions** for subgoals that have nothing verifiable. Rewrite transient expectations as persistently probeable state wherever possible (e.g., \"alarm exists in the alarm list\" instead of \"a toast flashed\") — evidence for transient moments only exists if the execution history happened to capture it.\n- **Ordering requirements are post-hoc.** If the user demands \"A must be confirmed before B runs\", still generate the assertion preserving their semantics, but annotate in the plan that ordering is verified retroactively from history — the system performs post-hoc audits and CANNOT block B from executing. Never silently promise enforcement that does not exist.",
     "check_audit": "- **Check Standard Protection**: Check lines (`- verify:` / `- assert:`) declared in the plan are the task's verification standards. FLAG any proposed plan that deletes, weakens, or rewords existing check lines — softening a declared standard is never an acceptable adaptation. Adding new check lines is allowed.",
-    "role_validator": "## Role\nYou are the Task Plan Reviewer, a fast advisory second opinion. The Operator has modified the top-level task plan and the change is already applied. Your job is to spot a concrete concern — goal drift, a broken loop contract, a weakened check standard, a logical dead end — and, if there is one, state it with its reason so the Operator can decide. You never block or roll back anything.",
+    "role_validator": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文，包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式，不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留，可另附中文解释；不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构；结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据，不编造操作结果。\n\n## Role\nYou are the Task Plan Reviewer, a fast advisory second opinion. The Operator has modified the top-level task plan and the change is already applied. Your job is to spot a concrete concern — goal drift, a broken loop contract, a weakened check standard, a logical dead end — and, if there is one, state it with its reason so the Operator can decide. You never block or roll back anything.",
     "validator_guidelines": "## Validation Principles\n- **Goal Alignment**: Ensure the new plan ultimately accomplishes the `Initial Goal` requested by the user. If the Operator has abandoned the core objective or fundamentally altered it, flag the plan.\n- **Logical Coherence**: Verify that the proposed milestones form a logical, achievable sequence to reach the goal. Support valid ongoing iteration/polling subgoals and trigger branches without misconstruing periodic checks as infinite loops. Flag the plan if it introduces contradictory steps, unhandled deadlocks, or obvious logical fallacies.\n{{ plan_grammar }}\n- **Loop & Monitoring Protection**: If the existing plan contains an active `[Loop]` or `[Loop:continuous]` milestone, you MUST FLAG any proposed plan that deletes the milestone or removes its loop tag, as well as any plan that marks a `[Loop:continuous]` continuous monitoring milestone as completed (`[x]`) without an explicit external user stop signal.\n- **Adaptive Acceptance**: The Operator operates in a dynamic environment and may encounter unexpected UI states or dynamic event triggers. You should approve alternative strategies, extra intermediate steps, loop status tracking, or necessary workarounds as long as they serve the primary goal effectively.\n- **Advisory Verdict & Constructive Feedback**: Your verdict is advisory — the change stays applied whether or not you approve it. Approve unless you have a concrete, evidence-backed concern; a vague unease is not a concern. If you flag the plan, give one or two sentences naming the flaw or goal deviation and the correction you suggest, concrete enough for the Operator to weigh it against its own observations."
   },
   "modes": {

artemis/agents/validator/tool_declarations.py

@@ -23,7 +23,6 @@ import base64
 import json
 from typing import Any, Literal
 
-from google.genai import types as genai_types
 from langchain_core.messages import ToolMessage
 from pydantic import BaseModel, Field
 
@@ -99,44 +98,6 @@ def prune_intermediate_screenshots(messages: list[Any]) -> None:
     if not messages:
         return
 
-    # Check for google.genai.types.Content objects
-    if hasattr(messages[0], "parts"):
-        user_indices_with_images = []
-        for idx in range(1, len(messages)):
-            content = messages[idx]
-            if getattr(content, "role", None) == "user" and getattr(content, "parts", None):
-                for part in content.parts:
-                    if getattr(part, "inline_data", None) is not None:
-                        user_indices_with_images.append(idx)
-                        break
-
-        if len(user_indices_with_images) > 1:
-            indices_to_prune = user_indices_with_images[:-1]
-            for idx in indices_to_prune:
-                content = messages[idx]
-                new_parts = []
-                for part in content.parts:
-                    if getattr(part, "inline_data", None) is not None:
-                        try:
-                            new_parts.append(
-                                genai_types.Part.from_text(
-                                    text=(
-                                        "[Screenshot of intermediate step omitted for performance]"
-                                    )
-                                )
-                            )
-                        except Exception as exc:
-                            logger.debug(
-                                "Could not build placeholder text part for pruned screenshot;"
-                                " dropping it: %s",
-                                exc,
-                                exc_info=True,
-                            )
-                    else:
-                        new_parts.append(part)
-                content.parts = new_parts
-        return
-
     # LangChain BaseMessage format
     last_img_msg_idx = -1
     for idx in range(len(messages) - 1, -1, -1):

artemis/agents/video_analyzer/__init__.py

@@ -12,4 +12,4 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-"""Video analyzer utility for analyzing video content with Gemini models."""
+"""Video analyzer utility for analyzing video content with OpenAI Responses models."""

artemis/agents/video_analyzer/audio_runner.py

@@ -12,7 +12,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-"""Native Gemini execution path for audio-only segment analysis.
+"""Responses execution path for audio-only segment analysis.
 
 Extracted from ``video_analyzer.py`` as a pure structural split.  Patchable
 collaborators are looked up late through the facade module so
@@ -23,24 +23,12 @@ keep working.
 import asyncio
 from pathlib import Path
 
-from google.genai import types
 
 from artemis.agents.video_analyzer import video_analyzer as _va
-from artemis.agents.video_analyzer.chunk_conversation import (
-    _MISSING_SUMMARY_ERROR,
-    _NO_TOOL_CALL_ERROR,
-    _SUBMIT_NOW_WARNING,
-    _WRONG_TOOL_ERROR,
-    SUB_AGENT_MAX_TURNS,
-    _append_rejection,
-    _clean_text,
-    _tool_name,
-)
 from artemis.agents.video_analyzer.reliability import (
     SubAgentAnswerExhausted,
     classify_video_failure,
 )
-from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
 from artemis.data_engine.trace import TraceSpan
 from artemis.llm.reliability import retry_policy_for
 from artemis.utils.logger import get_logger
@@ -72,7 +60,7 @@ async def exec_analyze_audio_only(
         return "Error: start_time must be a numeric recording-relative timestamp."
     if not str(specific_query or "").strip():
         return "Error: specific_query must describe the audio evidence to find."
-    max_retries = analyzer.native_max_retries
+    max_retries = analyzer.max_retries
     current_start = round(start_time, 1) if isinstance(start_time, (int, float)) else start_time
     current_end = round(end_time, 1) if isinstance(end_time, (int, float)) else end_time
 
@@ -134,37 +122,25 @@ async def exec_analyze_audio_only(
                 analyzer, result, media.actual_start, start_time, end_time, specific_query
             )
 
-            if not analyzer.use_native_gemini:
-                audio_result = await analyzer._exec_analyze_audio_universal(
-                    audio_path=media.audio_path,
-                    start_time=current_start,
-                    end_time=current_end,
-                    actual_start=media.actual_start,
-                    prompt_with_context=media.prompt_with_context,
-                    specific_query=specific_query,
-                )
-                summary, analysis = analyzer._parse_chunk_result(audio_result)
-                analyzer.blackboard.complete_segment(
-                    current_start,
-                    current_end,
-                    specific_query,
-                    lease_owner,
-                    summary,
-                    analysis,
-                    modality="audio",
-                )
-                return audio_result
-
-            return await _run_native_audio_conversation(
-                analyzer,
-                media,
+            audio_result = await analyzer._exec_analyze_audio_universal(
+                audio_path=media.audio_path,
+                start_time=current_start,
+                end_time=current_end,
+                actual_start=media.actual_start,
+                prompt_with_context=media.prompt_with_context,
+                specific_query=specific_query,
+            )
+            summary, analysis = analyzer._parse_chunk_result(audio_result)
+            analyzer.blackboard.complete_segment(
                 current_start,
                 current_end,
-                start_time,
-                end_time,
                 specific_query,
                 lease_owner,
+                summary,
+                analysis,
+                modality="audio",
             )
+            return audio_result
 
         except Exception as e:
             # Same contract as the video sub-agent loop: classification
@@ -232,49 +208,6 @@ async def _finish_failed_audio(
 ):
     """Records terminal audio failure, attempting the universal fallback first."""
     failure = classify_video_failure(error)
-    if (
-        analyzer.use_native_gemini
-        and failure.should_fallback
-        and media.audio_path is not None
-        and media.actual_start is not None
-        and media.prompt_with_context is not None
-    ):
-        try:
-            logger.warning(
-                f"Native audio analysis failed [{failure.category.value}]; using universal fallback"
-            )
-            _va._record_llm_event(
-                "llm_fallback",
-                {
-                    "reason": "native_audio_analysis_failed",
-                    "category": failure.category.value,
-                    "error": str(error)[:500],
-                },
-            )
-            fallback_result = await analyzer._exec_analyze_audio_universal(
-                audio_path=media.audio_path,
-                start_time=current_start,
-                end_time=current_end,
-                actual_start=media.actual_start,
-                prompt_with_context=media.prompt_with_context,
-                specific_query=specific_query,
-                force_fallback=True,
-            )
-            summary, analysis = analyzer._parse_chunk_result(fallback_result)
-            analyzer.blackboard.complete_segment(
-                current_start,
-                current_end,
-                specific_query,
-                lease_owner,
-                summary,
-                analysis,
-                modality="audio",
-            )
-            return fallback_result
-        except Exception as fallback_error:
-            error = fallback_error
-            failure = classify_video_failure(error)
-            retryable = None
     analyzer.blackboard.fail_segment(
         current_start,
         current_end,
@@ -366,178 +299,3 @@ def _build_audio_prompt(
         )
     prompt_with_context += f"{specific_query}"
     return prompt_with_context
-
-
-async def _run_native_audio_conversation(
-    analyzer,
-    media: _AudioMedia,
-    current_start: float,
-    current_end: float,
-    start_time: float,
-    end_time: float | None,
-    specific_query: str,
-    lease_owner: str,
-) -> str:
-    """Uploads the audio and drives the native Gemini audio agent to a committed answer."""
-    file = None
-    try:
-        with TraceSpan(name="upload_audio_to_gemini") as span:
-            file = await analyzer.upload_and_poll_file(media.audio_path)
-            span.result = f"Uploaded {file.name}"
-        logger.info(f"Invoking Gemini for audio-only task with model {analyzer.model_name}...")
-
-        sub_agent_contents = [
-            types.Content(
-                role="user",
-                parts=[
-                    types.Part.from_uri(file_uri=file.uri, mime_type=file.mime_type)
-                    if hasattr(file, "uri")
-                    else file,
-                    types.Part.from_text(text=media.prompt_with_context),
-                ],
-            )
-        ]
-
-        (
-            final_summary,
-            final_analysis,
-            final_confidence_score,
-        ) = await _drive_audio_agent_loop(analyzer, sub_agent_contents)
-
-        end_val = end_time if end_time is not None else "unknown"
-        analyzer._record_blackboard_entry(
-            {
-                "start": start_time,
-                "end": end_val,
-                "target": specific_query,
-                "summary": final_summary,
-                "confidence_score": final_confidence_score,
-            },
-            modality="audio",
-        )
-
-        end_str = f"{end_time:.1f}s" if end_time is not None else "unknown"
-        audio_result = (
-            f"[from {start_time:.1f}s to {end_str}] Summary:"
-            f" {final_summary} Analysis: {final_analysis}"
-        )
-        analyzer.blackboard.complete_segment(
-            current_start,
-            current_end,
-            specific_query,
-            lease_owner,
-            final_summary,
-            final_analysis,
-            modality="audio",
-        )
-        return audio_result
-    finally:
-        if file:
-            logger.info(f"Cleaning up cloud file {file.name}...")
-            try:
-                await asyncio.wait_for(
-                    analyzer.client.aio.files.delete(name=file.name),
-                    timeout=30,
-                )
-                analyzer.cloud_files_to_cleanup.discard(file.name)
-            except Exception as ce:
-                logger.error(f"Failed to delete cloud file {file.name}: {ce}")
-
-
-async def _drive_audio_agent_loop(analyzer, sub_agent_contents: list) -> tuple[str, str, float]:
-    """Return a validated (summary, analysis, confidence_score).
-
-    Raise ``SubAgentAnswerExhausted`` if no answer is accepted within the turn limit.
-    """
-    rejection = _NO_TOOL_CALL_ERROR
-
-    for turn in range(1, SUB_AGENT_MAX_TURNS + 1):
-        if turn == SUB_AGENT_MAX_TURNS - 1:
-            sub_agent_contents.append(
-                types.Content(
-                    role="user",
-                    parts=[types.Part.from_text(text=_SUBMIT_NOW_WARNING)],
-                )
-            )
-
-        with TraceSpan(name="gemini_generate_audio_content") as span:
-            response = await asyncio.wait_for(
-                analyzer.client.aio.models.generate_content(
-                    model=analyzer.model_name,
-                    contents=sub_agent_contents,
-                    config=types.GenerateContentConfig(
-                        system_instruction=analyzer.audio_system_prompt,
-                        tools=[types.Tool(function_declarations=[analyzer.submit_answer_tool])],
-                        safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
-                    ),
-                ),
-                timeout=analyzer.model_call_timeout_seconds,
-            )
-
-        full_text = response.text or ""
-        function_calls = response.function_calls or []
-
-        if function_calls:
-            answer, rejection = _handle_audio_function_calls(
-                sub_agent_contents, response, function_calls, full_text
-            )
-            if answer is not None:
-                return answer
-        else:
-            rejection = _NO_TOOL_CALL_ERROR
-            sub_agent_contents.append(
-                types.Content(
-                    role="model",
-                    parts=[types.Part.from_text(text=full_text)],
-                )
-            )
-
-    raise SubAgentAnswerExhausted(SUB_AGENT_MAX_TURNS, rejection)
-
-
-def _validate_audio_answer(summary: str, confidence_score) -> str | None:
-    """Return a validation error for the audio answer, or None if valid."""
-    if not summary:
-        return _MISSING_SUMMARY_ERROR
-    if (
-        not isinstance(confidence_score, (int, float))
-        or isinstance(confidence_score, bool)
-        or not (0.0 <= float(confidence_score) <= 1.0)
-    ):
-        return _BAD_CONFIDENCE_ERROR
-    return None
-
-
-def _handle_audio_function_calls(
-    sub_agent_contents: list,
-    response,
-    function_calls: list,
-    full_text: str,
-) -> tuple[tuple[str, str, float] | None, str | None]:
-    """Return (answer, rejection), appending any rejection to the conversation."""
-    original_parts = response.candidates[0].content.parts
-    for fc in function_calls:
-        if _tool_name(fc.name) != "submit_answer":
-            continue
-        args = dict(fc.args or {})
-        summary = _clean_text(args.get("summary"))
-        analysis = _clean_text(args.get("analysis"))
-
-        rejection = _validate_audio_answer(summary, args.get("confidence_score"))
-        if rejection is not None:
-            _append_rejection(
-                sub_agent_contents, original_parts, function_calls, full_text, fc.name, rejection
-            )
-            return None, rejection
-
-        return (summary, analysis or "No analysis provided.", float(args["confidence_score"])), None
-
-    _append_rejection(
-        sub_agent_contents,
-        original_parts,
-        function_calls,
-        full_text,
-        function_calls[0].name,
-        _WRONG_TOOL_ERROR,
-    )
-    return None, _WRONG_TOOL_ERROR

artemis/agents/video_analyzer/chunk_agentic.py

@@ -1,455 +0,0 @@
-# Copyright 2026 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Analyze video chunks through the Gemini Interactions API.
-
-``processing: "agentic"`` lets the model search and revisit an uploaded clip.
-Validation errors are returned as function results, with video context retained
-through ``previous_interaction_id``. Only accepted answers reach the blackboard.
-
-The stream consumer reconstructs function arguments from step deltas. Completion
-events may omit steps, so it fetches the interaction when reconstruction fails.
-"""
-
-from __future__ import annotations
-
-import asyncio
-import copy
-from dataclasses import dataclass, field
-import json
-from types import SimpleNamespace
-from typing import Any
-
-from artemis.agents.video_analyzer import gemini_files
-from artemis.agents.video_analyzer import video_analyzer as _va
-from artemis.agents.video_analyzer.chunk_conversation import (
-    _NO_TOOL_CALL_ERROR,
-    _SUBMIT_NOW_WARNING,
-    _WRONG_TOOL_ERROR,
-    SUB_AGENT_MAX_TURNS,
-    SubAgentAnswerExhausted,
-    _ChunkMedia,
-    _clean_text,
-    _persist_timeline_events,
-    _tool_name,
-    _validate_answer,
-)
-from artemis.agents.video_analyzer.universal_tools import UNIVERSAL_SUBMIT_ANSWER_TOOL
-from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan
-from artemis.llm.google import normalize_usage
-from artemis.utils.logger import get_logger
-
-logger = get_logger(__name__)
-
-STREAM_START_TIMEOUT_SECONDS = 60.0
-INTERACTION_READ_TIMEOUT_SECONDS = 30.0
-
-# Interaction statuses after which no follow-up turn can be sent
-# (``google.genai`` ``InteractionStatus``).
-_TERMINAL_FAILURE_STATUSES = frozenset({"failed", "cancelled", "incomplete", "budget_exceeded"})
-
-
-def build_interactions_submit_tool() -> dict[str, Any]:
-    """The ``submit_answer`` declaration in Interactions API (flat JSON schema) form.
-
-    Derived from the universal (OpenAI-style) declaration so the schema is
-    written once; ``timeline_events`` is required to match the native path.
-    """
-    function = UNIVERSAL_SUBMIT_ANSWER_TOOL["function"]
-    parameters = copy.deepcopy(function["parameters"])
-    required = list(parameters.get("required", []))
-    if "timeline_events" not in required:
-        required.append("timeline_events")
-    parameters["required"] = required
-    return {
-        "type": "function",
-        "name": function["name"],
-        "description": function["description"],
-        "parameters": parameters,
-    }
-
-
-def _steps(interaction: Any) -> list:
-    return list(getattr(interaction, "steps", None) or [])
-
-
-def _find_function_call(interaction: Any):
-    """The last ``function_call`` step of an interaction, if any."""
-    for step in reversed(_steps(interaction)):
-        if getattr(step, "type", None) == "function_call":
-            return step
-    return None
-
-
-def _raise_if_failed(interaction: Any) -> None:
-    """Raises when an interaction ended in a terminal failure state.
-
-    A follow-up with ``previous_interaction_id`` onto a failed interaction is
-    rejected by the API, so the failure surfaces here as an exception and
-    flows through the shared retry / classification layer instead.
-    """
-    status = str(getattr(interaction, "status", None) or "").lower()
-    errors = list(getattr(interaction, "errors", None) or [])
-    if status not in _TERMINAL_FAILURE_STATUSES and not errors:
-        return
-    details = "; ".join(
-        f"[{getattr(error, 'code', None)}] {getattr(error, 'message', error)}" for error in errors
-    )
-    raise RuntimeError(
-        f"Gemini Interactions interaction {getattr(interaction, 'id', None)} ended with"
-        f" status {status or 'unknown'}" + (f": {details}" if details else "")
-    )
-
-
-async def run_agentic_chunk_conversation(
-    analyzer,
-    media: _ChunkMedia,
-    current_start: float,
-    current_end: float,
-    start_time: float,
-    end_time: float | None,
-    specific_query: str,
-    lease_owner: str,
-) -> str:
-    """Uploads the chunk and drives the agentic sub-agent to a committed answer."""
-    file = None
-    try:
-        with TraceSpan(name="upload_video_to_gemini") as span:
-            file = await analyzer.upload_and_poll_file(media.compressed_path)
-            span.result = f"Uploaded {file.name}"
-        logger.info(
-            f"Invoking Gemini agentic video understanding for sub-agent task with model"
-            f" {analyzer.model_name} (streaming)..."
-        )
-        trace_id = CURRENT_TRACE_ID.get()
-
-        if analyzer.ctx.data_engine and trace_id:
-            analyzer.ctx.data_engine.record_trace(
-                type="llm_call",
-                name="video_sub_agent",
-                payload={
-                    "contents": [
-                        f"file://{file.name}",
-                        media.prompt_with_context,
-                    ],
-                    "system_instruction": analyzer.sub_system_prompt,
-                    "video_processing": "agentic",
-                },
-                parent_trace_id=trace_id,
-            )
-
-        final_summary, final_analysis = await _drive_agentic_loop(
-            analyzer,
-            file,
-            trace_id,
-            media,
-            start_time,
-            end_time,
-            specific_query,
-        )
-
-        end_str = f"{current_end:.1f}s" if current_end is not None else "unknown"
-        chunk_result = (
-            f"[from {current_start:.1f}s to {end_str}] Summary:"
-            f" {final_summary} Analysis: {final_analysis}"
-        )
-        analyzer.blackboard.complete_segment(
-            current_start,
-            current_end,
-            specific_query,
-            lease_owner,
-            final_summary,
-            final_analysis,
-        )
-        return chunk_result
-    finally:
-        if file:
-            await gemini_files.delete_cloud_file(
-                analyzer.client, file.name, analyzer.cloud_files_to_cleanup
-            )
-
-
-def _initial_request(analyzer, file, prompt: str, tool: dict[str, Any]) -> dict[str, Any]:
-    return {
-        "model": analyzer.model_name,
-        "input": [
-            {
-                "type": "video",
-                "uri": file.uri,
-                "mime_type": file.mime_type,
-                "processing": "agentic",
-            },
-            {"type": "text", "text": prompt},
-        ],
-        "system_instruction": analyzer.sub_system_prompt,
-        "tools": [tool],
-        "stream": True,
-    }
-
-
-def _follow_up_request(
-    analyzer, previous_id: str, steps: list[dict[str, Any]], tool: dict[str, Any]
-) -> dict[str, Any]:
-    """A stateful follow-up turn: the server keeps the video context.
-
-    Only the conversation state is inherited through
-    ``previous_interaction_id``; ``system_instruction`` and ``tools`` are
-    interaction-scoped and must be re-sent on every turn.
-    """
-    return {
-        "model": analyzer.model_name,
-        "previous_interaction_id": previous_id,
-        "input": steps,
-        "system_instruction": analyzer.sub_system_prompt,
-        "tools": [tool],
-        "stream": True,
-    }
-
-
-def _function_result(call_id: str, error: str) -> dict[str, Any]:
-    return {
-        "type": "function_result",
-        "call_id": call_id,
-        "name": "submit_answer",
-        "result": {"error": error},
-    }
-
-
-async def _drive_agentic_loop(
-    analyzer,
-    file,
-    trace_id,
-    media: _ChunkMedia,
-    start_time: float,
-    end_time: float | None,
-    specific_query: str,
-) -> tuple[str, str]:
-    """Return a validated answer and persist its timeline events.
-
-    Raise ``SubAgentAnswerExhausted`` if no answer is accepted within the turn limit.
-    """
-    tool = build_interactions_submit_tool()
-    request = _initial_request(analyzer, file, media.prompt_with_context, tool)
-    rejection = _NO_TOOL_CALL_ERROR
-
-    for turn in range(1, SUB_AGENT_MAX_TURNS + 1):
-        interaction = await _stream_agentic_turn(analyzer, request, trace_id)
-        interaction_id = getattr(interaction, "id", None) or ""
-        call = _find_function_call(interaction)
-
-        if call is None:
-            rejection = _NO_TOOL_CALL_ERROR
-            steps = [{"type": "text", "text": _SUBMIT_NOW_WARNING}]
-        elif _tool_name(getattr(call, "name", None)) != "submit_answer":
-            rejection = _WRONG_TOOL_ERROR
-            steps = [_function_result(call.id, rejection)]
-        else:
-            args = dict(getattr(call, "arguments", None) or {})
-            timeline_events = args.get("timeline_events", [])
-            if not isinstance(timeline_events, list):
-                timeline_events = []
-            summary = _clean_text(args.get("summary"))
-            analysis = _clean_text(args.get("analysis"))
-
-            rejection = _validate_answer(summary, timeline_events, start_time, end_time)
-            if rejection is None:
-                await _persist_timeline_events(
-                    analyzer, timeline_events, media, start_time, end_time, specific_query
-                )
-                return summary, analysis or "No analysis provided."
-            steps = [_function_result(call.id, rejection)]
-
-        if turn < SUB_AGENT_MAX_TURNS:
-            request = _follow_up_request(analyzer, interaction_id, steps, tool)
-
-    raise SubAgentAnswerExhausted(SUB_AGENT_MAX_TURNS, rejection)
-
-
-async def _stream_agentic_turn(analyzer, request: dict[str, Any], trace_id):
-    """Runs one traced streaming Interactions turn and returns the completed interaction."""
-    with TraceSpan(name="gemini_agentic_video_call") as span:
-        span.payload = {
-            "model": request.get("model"),
-            "video_processing": "agentic",
-            "previous_interaction_id": request.get("previous_interaction_id"),
-        }
-        stream = await asyncio.wait_for(
-            analyzer.client.aio.interactions.create(**request),
-            timeout=STREAM_START_TIMEOUT_SECONDS,
-        )
-        interaction = await asyncio.wait_for(
-            _consume_agentic_stream(analyzer, span, stream, trace_id),
-            timeout=analyzer.agentic_call_timeout_seconds,
-        )
-        call = _find_function_call(interaction)
-        span.result = (
-            f"Function call: {_tool_name(getattr(call, 'name', None))}"
-            if call is not None
-            else f"Status: {getattr(interaction, 'status', 'unknown')}"
-        )
-    return interaction
-
-
-@dataclass
-class _PendingStep:
-    """A step under construction while its start/delta/stop events stream in."""
-
-    kind: str
-    call_id: str | None = None
-    name: str | None = None
-    seed_arguments: dict[str, Any] = field(default_factory=dict)
-    arguments_json: list[str] = field(default_factory=list)
-    text: list[str] = field(default_factory=list)
-
-    def finish(self):
-        """Materializes the step in the shape the SDK's typed steps expose.
-
-        Returns ``None`` for a function call whose streamed arguments are not
-        valid JSON: a half-built call must not stand in for the real one, so
-        the step is dropped and the interaction is re-read by id instead.
-        """
-        if self.kind == "function_call":
-            arguments: dict[str, Any] = dict(self.seed_arguments)
-            raw = "".join(self.arguments_json)
-            if raw.strip():
-                try:
-                    parsed = json.loads(raw)
-                except ValueError:
-                    logger.warning(
-                        f"Could not parse streamed arguments for {self.name}: {raw[:200]!r};"
-                        " dropping the step"
-                    )
-                    return None
-                if isinstance(parsed, dict):
-                    arguments = parsed
-            return SimpleNamespace(
-                type="function_call", id=self.call_id, name=self.name, arguments=arguments
-            )
-        if self.kind == "model_output":
-            return SimpleNamespace(
-                type="model_output",
-                content=[SimpleNamespace(type="text", text="".join(self.text))],
-            )
-        return SimpleNamespace(type=self.kind, id=self.call_id)
-
-
-async def _consume_agentic_stream(analyzer, span, stream, trace_id):
-    """Consumes Interactions SSE events, rebuilding steps and mirroring progress.
-
-    Returns an interaction-shaped object (``id`` / ``status`` / ``steps`` /
-    ``usage``). The interaction is re-read by id when the stream ended
-    without a completion event, or when it stopped at ``requires_action``
-    without a parsable function call.
-    """
-    data_engine = analyzer.ctx.data_engine if trace_id else None
-    interaction_id: str | None = None
-    status: str | None = None
-    completed_steps: list | None = None
-    usage: dict[str, int] | None = None
-    pending: dict[int, _PendingStep] = {}
-    steps: list = []
-    processing_calls = 0
-
-    def _emit(text: str, *, thinking: bool) -> None:
-        if data_engine and text:
-            data_engine.stream_output(trace_id, text, is_thinking=thinking)
-
-    async for event in stream:
-        data = getattr(event, "data", event)
-        event_type = getattr(data, "event_type", None)
-
-        if event_type == "interaction.created":
-            created = getattr(data, "interaction", None)
-            interaction_id = getattr(created, "id", None) or interaction_id
-        elif event_type == "step.start":
-            step = getattr(data, "step", None)
-            index = getattr(data, "index", None)
-            kind = str(getattr(step, "type", None) or "")
-            pending[index] = _PendingStep(
-                kind=kind,
-                call_id=getattr(step, "id", None) or getattr(step, "call_id", None),
-                name=getattr(step, "name", None),
-                seed_arguments=dict(getattr(step, "arguments", None) or {}),
-            )
-            if kind == "processing_call":
-                processing_calls += 1
-                _emit(
-                    f"Inspecting the recording (video lookup {processing_calls})\n",
-                    thinking=True,
-                )
-        elif event_type == "step.delta":
-            delta = getattr(data, "delta", None)
-            delta_type = getattr(delta, "type", None)
-            current = pending.get(getattr(data, "index", None))
-            if delta_type == "arguments_delta":
-                fragment = getattr(delta, "arguments", None)
-                if current is not None and isinstance(fragment, str):
-                    current.arguments_json.append(fragment)
-            elif delta_type == "text":
-                text = getattr(delta, "text", None)
-                if current is not None and isinstance(text, str):
-                    current.text.append(text)
-                _emit(text or "", thinking=False)
-            elif delta_type == "thought_summary":
-                content = getattr(delta, "content", None)
-                _emit(getattr(content, "text", None) or "", thinking=True)
-        elif event_type == "step.stop":
-            current = pending.pop(getattr(data, "index", None), None)
-            finished = current.finish() if current is not None else None
-            if finished is not None:
-                steps.append(finished)
-            usage = normalize_usage(getattr(data, "usage", None)) or usage
-        elif event_type == "interaction.completed":
-            completed = getattr(data, "interaction", None)
-            interaction_id = getattr(completed, "id", None) or interaction_id
-            status = getattr(completed, "status", None) or status
-            completed_steps = getattr(completed, "steps", None) or None
-            usage = normalize_usage(getattr(completed, "usage", None)) or usage
-            _raise_if_failed(completed)
-        elif event_type == "interaction.status_update":
-            status = getattr(data, "status", None) or status
-        elif event_type == "error":
-            error = getattr(data, "error", None)
-            raise RuntimeError(
-                "Gemini Interactions stream error"
-                f" [{getattr(error, 'code', None)}]: {getattr(error, 'message', None)}"
-            )
-
-    for current in pending.values():
-        finished = current.finish()
-        if finished is not None:
-            steps.append(finished)
-    span.payload["processing_calls"] = processing_calls
-    if completed_steps:
-        steps = list(completed_steps)
-
-    needs_reread = status is None or (
-        status == "requires_action" and _find_function_call(SimpleNamespace(steps=steps)) is None
-    )
-    if needs_reread:
-        if not interaction_id:
-            raise RuntimeError("Gemini Interactions stream ended without an interaction")
-        logger.info(f"Re-reading interaction {interaction_id} (status={status})")
-        fetched = await asyncio.wait_for(
-            analyzer.client.aio.interactions.get(interaction_id),
-            timeout=INTERACTION_READ_TIMEOUT_SECONDS,
-        )
-        _raise_if_failed(fetched)
-        status = getattr(fetched, "status", None) or status
-        steps = _steps(fetched) or steps
-        usage = normalize_usage(getattr(fetched, "usage", None)) or usage
-
-    if usage:
-        span.payload["usage_metadata"] = usage
-    return SimpleNamespace(id=interaction_id, status=status, steps=steps, usage=usage)

artemis/agents/video_analyzer/chunk_conversation.py

@@ -1,491 +0,0 @@
-# Copyright 2026 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Native Gemini sub-agent conversation and submit_answer handling for chunks.
-
-Extracted from ``video_analyzer.py`` as a pure structural split.  Patchable
-collaborators are looked up late through the facade module so
-``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")`` targets
-keep working.
-"""
-
-import asyncio
-import glob
-from pathlib import Path
-import re
-
-from google.genai import types
-
-from artemis.agents.video_analyzer import gemini_files
-from artemis.agents.video_analyzer import video_analyzer as _va
-from artemis.agents.video_analyzer.reliability import SubAgentAnswerExhausted
-from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
-from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan
-from artemis.llm.google import normalize_usage
-from artemis.utils.logger import get_logger
-
-logger = get_logger(__name__)
-
-#: Turns the chunk sub-agent gets to produce an accepted ``submit_answer``
-#: (shared by the static and agentic paths).
-SUB_AGENT_MAX_TURNS = 2
-
-_SUBMIT_NOW_WARNING = "[WARNING] You must call the submit_answer tool now, time is running out."
-_WRONG_TOOL_ERROR = "Tool not recognized. Please use submit_answer."
-_NO_TOOL_CALL_ERROR = "The model replied with text only and never called submit_answer."
-_MISSING_SUMMARY_ERROR = (
-    "submit_answer was called without a 'summary'. Call it again with a non-empty"
-    " 'summary' (and 'analysis') describing what you observed."
-)
-
-
-def _tool_name(name: str | None) -> str:
-    name = name or ""
-    return name.split(":")[-1] if ":" in name else name
-
-
-def _clean_text(value) -> str:
-    return str(value or "").strip().replace("\n", " ")
-
-
-def _validate_answer(
-    summary: str, timeline_events: list, start_time: float, end_time: float | None
-) -> str | None:
-    """Return a validation error for submit_answer, or None if valid."""
-    if not summary:
-        return _MISSING_SUMMARY_ERROR
-    return _validate_timeline_events(timeline_events, start_time, end_time)
-
-
-class _ChunkMedia:
-    """Mutable holder for per-attempt media artifacts shared with failure handling."""
-
-    __slots__ = ("path", "compressed_path", "actual_start", "prompt_with_context")
-
-    def __init__(self):
-        self.path: Path | None = None
-        self.compressed_path: Path | None = None
-        self.actual_start: float | None = None
-        self.prompt_with_context: str | None = None
-
-
-async def _run_native_chunk_conversation(
-    analyzer,
-    media: _ChunkMedia,
-    current_start: float,
-    current_end: float,
-    start_time: float,
-    end_time: float | None,
-    specific_query: str,
-    lease_owner: str,
-) -> str:
-    """Uploads the chunk and drives the native Gemini sub-agent to a committed answer."""
-    file = None
-    try:
-        with TraceSpan(name="upload_video_to_gemini") as span:
-            file = await analyzer.upload_and_poll_file(media.compressed_path)
-            span.result = f"Uploaded {file.name}"
-        logger.info(
-            f"Invoking Gemini for sub-agent task with model {analyzer.model_name} (streaming)..."
-        )
-        trace_id = CURRENT_TRACE_ID.get()
-
-        if analyzer.ctx.data_engine and trace_id:
-            analyzer.ctx.data_engine.record_trace(
-                type="llm_call",
-                name="video_sub_agent",
-                payload={
-                    "contents": [
-                        f"file://{file.name}",
-                        media.prompt_with_context,
-                    ],
-                    "system_instruction": analyzer.sub_system_prompt,
-                },
-                parent_trace_id=trace_id,
-            )
-
-        sub_agent_contents = [
-            types.Content(
-                role="user",
-                parts=[
-                    types.Part.from_uri(file_uri=file.uri, mime_type=file.mime_type)
-                    if hasattr(file, "uri")
-                    else file,
-                    types.Part.from_text(text=media.prompt_with_context),
-                ],
-            )
-        ]
-
-        final_summary, final_analysis = await _drive_sub_agent_loop(
-            analyzer,
-            sub_agent_contents,
-            trace_id,
-            media,
-            start_time,
-            end_time,
-            specific_query,
-        )
-
-        end_str = f"{current_end:.1f}s" if current_end is not None else "unknown"
-        chunk_result = (
-            f"[from {current_start:.1f}s to {end_str}] Summary:"
-            f" {final_summary} Analysis: {final_analysis}"
-        )
-        analyzer.blackboard.complete_segment(
-            current_start,
-            current_end,
-            specific_query,
-            lease_owner,
-            final_summary,
-            final_analysis,
-        )
-        return chunk_result
-    finally:
-        if file:
-            await gemini_files.delete_cloud_file(
-                analyzer.client, file.name, analyzer.cloud_files_to_cleanup
-            )
-
-
-async def _drive_sub_agent_loop(
-    analyzer,
-    sub_agent_contents: list,
-    trace_id,
-    media: _ChunkMedia,
-    start_time: float,
-    end_time: float | None,
-    specific_query: str,
-) -> tuple[str, str]:
-    """Return a validated answer and persist its timeline events.
-
-    Raise ``SubAgentAnswerExhausted`` if no answer is accepted within the turn limit.
-    """
-    rejection = _NO_TOOL_CALL_ERROR
-
-    for turn in range(1, SUB_AGENT_MAX_TURNS + 1):
-        if turn == SUB_AGENT_MAX_TURNS - 1:
-            sub_agent_contents.append(
-                types.Content(
-                    role="user",
-                    parts=[types.Part.from_text(text=_SUBMIT_NOW_WARNING)],
-                )
-            )
-
-        full_text, function_calls, original_parts = await _stream_sub_agent_turn(
-            analyzer, sub_agent_contents, trace_id
-        )
-
-        if function_calls:
-            answer, rejection = await _handle_sub_agent_calls(
-                analyzer,
-                sub_agent_contents,
-                function_calls,
-                original_parts,
-                full_text,
-                media,
-                start_time,
-                end_time,
-                specific_query,
-            )
-            if answer is not None:
-                return answer
-        else:
-            rejection = _NO_TOOL_CALL_ERROR
-            sub_agent_contents.append(
-                types.Content(
-                    role="model",
-                    parts=[types.Part.from_text(text=full_text)],
-                )
-            )
-
-    raise SubAgentAnswerExhausted(SUB_AGENT_MAX_TURNS, rejection)
-
-
-async def _stream_sub_agent_turn(
-    analyzer, sub_agent_contents: list, trace_id
-) -> tuple[str, list, list]:
-    """Starts one streamed Gemini turn and consumes it fully."""
-    with TraceSpan(name="gemini_stream_content_main") as span:
-        stream = await asyncio.wait_for(
-            analyzer.client.aio.models.generate_content_stream(
-                model=analyzer.model_name,
-                contents=sub_agent_contents,
-                config=types.GenerateContentConfig(
-                    system_instruction=analyzer.sub_system_prompt,
-                    tools=[types.Tool(function_declarations=[analyzer.submit_answer_tool])],
-                    safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
-                ),
-            ),
-            timeout=60,
-        )
-
-    full_text, function_calls, original_parts = await asyncio.wait_for(
-        _read_sub_agent_stream(analyzer, span, stream, trace_id),
-        timeout=analyzer.model_call_timeout_seconds,
-    )
-    return full_text, function_calls, original_parts
-
-
-async def _read_sub_agent_stream(analyzer, span, target_stream, current_trace_id):
-    """Consumes a streamed response, mirroring text/thoughts into the trace."""
-    text = ""
-    function_calls = []
-    accumulated_parts = []
-    async for chunk in target_stream:
-        usage = normalize_usage(getattr(chunk, "usage_metadata", None))
-        if usage:
-            span.payload["usage_metadata"] = usage
-
-        chunk_text = chunk.text or ""
-        if chunk_text:
-            text += chunk_text
-            if analyzer.ctx.data_engine and current_trace_id:
-                analyzer.ctx.data_engine.stream_output(current_trace_id, chunk_text)
-        if chunk.function_calls:
-            function_calls.extend(chunk.function_calls)
-
-        candidates = getattr(chunk, "candidates", None)
-        if candidates and isinstance(candidates, list) and len(candidates) > 0:
-            content = getattr(candidates[0], "content", None)
-            parts = getattr(content, "parts", None) if content else None
-            if parts:
-                for part in parts:
-                    accumulated_parts.append(part)
-                    if getattr(part, "thought", False) and part.text:
-                        if analyzer.ctx.data_engine and current_trace_id:
-                            analyzer.ctx.data_engine.stream_output(
-                                current_trace_id,
-                                part.text,
-                                is_thinking=True,
-                            )
-        else:
-            if chunk.function_calls:
-                for fc in chunk.function_calls:
-                    accumulated_parts.append(types.Part(function_call=fc))
-
-    return text, function_calls, accumulated_parts
-
-
-async def _handle_sub_agent_calls(
-    analyzer,
-    sub_agent_contents: list,
-    function_calls: list,
-    original_parts: list,
-    full_text: str,
-    media: _ChunkMedia,
-    start_time: float,
-    end_time: float | None,
-    specific_query: str,
-) -> tuple[tuple[str, str] | None, str | None]:
-    """Return ``(answer, rejection)`` after processing one turn's tool calls.
-
-    Persist events for accepted answers; append rejected calls to the conversation
-    with an error for the model to correct.
-    """
-    for fc in function_calls:
-        if _tool_name(fc.name) != "submit_answer":
-            continue
-        args = dict(fc.args or {})
-        timeline_events = args.get("timeline_events", [])
-        if not isinstance(timeline_events, list):
-            timeline_events = []
-        summary = _clean_text(args.get("summary"))
-        analysis = _clean_text(args.get("analysis"))
-
-        rejection = _validate_answer(summary, timeline_events, start_time, end_time)
-        if rejection is not None:
-            _append_rejection(
-                sub_agent_contents, original_parts, function_calls, full_text, fc.name, rejection
-            )
-            return None, rejection
-
-        await _persist_timeline_events(
-            analyzer, timeline_events, media, start_time, end_time, specific_query
-        )
-        return (summary, analysis or "No analysis provided."), None
-
-    _append_rejection(
-        sub_agent_contents,
-        original_parts,
-        function_calls,
-        full_text,
-        function_calls[0].name,
-        _WRONG_TOOL_ERROR,
-    )
-    return None, _WRONG_TOOL_ERROR
-
-
-def _validate_timeline_events(
-    timeline_events: list, start_time: float, end_time: float | None
-) -> str | None:
-    """Returns the first validation error message for the submitted events, if any."""
-    for event in timeline_events:
-        if (
-            "confidence_score" not in event
-            or not isinstance(
-                event["confidence_score"],
-                (int, float),
-            )
-            or isinstance(event["confidence_score"], bool)
-            or not (0.0 <= float(event["confidence_score"]) <= 1.0)
-        ):
-            return (
-                "Error: 'confidence_score' is"
-                " missing or out of bounds in"
-                " one of the timeline_events."
-                " It must be a float between"
-                " 0.0 and 1.0 inclusive. Fix"
-                " the payload and call"
-                " submit_answer again."
-            )
-
-        if (
-            "verification_timestamp_secs" not in event
-            or not isinstance(
-                event["verification_timestamp_secs"],
-                (int, float),
-            )
-            or isinstance(event["verification_timestamp_secs"], bool)
-        ):
-            return (
-                "Error:"
-                " 'verification_timestamp_secs'"
-                " is missing or invalid in one"
-                " of the timeline_events. Fix"
-                " the payload and call"
-                " submit_answer again."
-            )
-
-        verification_ts = float(
-            event.get(
-                "verification_timestamp_secs",
-                start_time,
-            )
-        )
-        confidence = float(event.get("confidence_score", 0.0))  # noqa: F841
-
-        if verification_ts < start_time or (end_time is not None and verification_ts > end_time):
-            return (
-                "Error:"
-                " 'verification_timestamp_secs'"
-                f" {verification_ts} must fall"
-                " within the start_time"
-                f" ({start_time}) and end_time"
-                f" ({end_time}) boundaries. Fix"
-                " the payload and call"
-                " submit_answer again."
-            )
-    return None
-
-
-def _append_rejection(
-    sub_agent_contents: list,
-    original_parts: list,
-    function_calls: list,
-    full_text: str,
-    response_name: str,
-    error_msg: str,
-) -> None:
-    """Appends the model turn plus an error function response to the conversation."""
-    sub_agent_contents.append(
-        types.Content(
-            role="model",
-            parts=original_parts
-            if original_parts
-            else [types.Part(function_call=fc) for fc in function_calls]
-            + ([types.Part.from_text(text=full_text)] if full_text else []),
-        )
-    )
-    sub_agent_contents.append(
-        types.Content(
-            role="user",
-            parts=[
-                types.Part.from_function_response(
-                    name=response_name,
-                    response={"error": (error_msg)},
-                )
-            ],
-        )
-    )
-
-
-async def _persist_timeline_events(
-    analyzer,
-    timeline_events: list,
-    media: _ChunkMedia,
-    start_time: float,
-    end_time: float | None,
-    specific_query: str,
-) -> None:
-    """Captures verification screenshots and records validated events on the blackboard."""
-    for event in timeline_events:
-        verification_ts = float(
-            event.get(
-                "verification_timestamp_secs",
-                start_time,
-            )
-        )
-        confidence = float(event.get("confidence_score", 0.0))
-        relative_offset = max(0.0, verification_ts - media.actual_start)
-
-        out_dir = _va.get_temp_dir("video_analyzer")
-        out_dir.mkdir(parents=True, exist_ok=True)
-        existing_imgs = glob.glob(f"{out_dir}/img_*.jpg")
-        max_idx = -1
-        for img in existing_imgs:
-            match = re.search(r"img_(\d+)\.jpg$", img)
-            if match:
-                max_idx = max(max_idx, int(match.group(1)))
-        next_idx = max_idx + 1
-        screenshot_output_path = out_dir / f"img_{next_idx}.jpg"
-        while screenshot_output_path.exists():
-            next_idx += 1
-            screenshot_output_path = out_dir / f"img_{next_idx}.jpg"
-        screenshot_output_path.touch()
-        analyzer.local_files_to_cleanup.add(screenshot_output_path)
-        ffmpeg_cmd = [
-            "ffmpeg",
-            "-ss",
-            str(relative_offset),
-            "-i",
-            str(media.path),
-            "-vframes",
-            "1",
-            "-q:v",
-            "2",
-            "-y",
-            str(screenshot_output_path),
-        ]
-        try:
-            proc = await asyncio.create_subprocess_exec(
-                *ffmpeg_cmd,
-                stdout=asyncio.subprocess.PIPE,
-                stderr=asyncio.subprocess.PIPE,
-            )
-            await proc.communicate()
-        except Exception as e:
-            logger.warning(f"Failed to extract frame: {e}")
-
-        screenshot_path_str = str(screenshot_output_path)
-        entry = {
-            "start": event.get("start_time", start_time),
-            "end": event.get("end_time", end_time),
-            "target": specific_query,
-            "summary": event.get("transcription", ""),
-            "confidence_score": confidence,
-        }
-        if Path(screenshot_output_path).exists():
-            entry["screenshot"] = str(screenshot_path_str)
-
-        analyzer._record_blackboard_entry(entry)

artemis/agents/video_analyzer/chunk_runner.py

@@ -12,29 +12,23 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-"""Native Gemini execution path for a single video chunk sub-agent.
+"""Responses execution path for a single video chunk sub-agent.
 
 Extracted from ``video_analyzer.py`` as a pure structural split.  Patchable
 collaborators (``get_controller``, ``compress_video_for_api``, semaphores,
 LLM event recorders, ...) are looked up late through the facade module so
 ``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")`` targets
 keep working.  The sub-agent conversation itself lives in
-``chunk_conversation``.
+``universal_engine``.
 """
 
 import asyncio
 from pathlib import Path
 
 from artemis.agents.video_analyzer import video_analyzer as _va
-from artemis.agents.video_analyzer.chunk_agentic import run_agentic_chunk_conversation
-from artemis.agents.video_analyzer.chunk_conversation import (
-    _ChunkMedia,
-    _run_native_chunk_conversation,
-)
 from artemis.agents.video_analyzer.reliability import (
     AgenticVideoDegraded,
     classify_video_failure,
-    is_agentic_rejection,
 )
 from artemis.llm.reliability import retry_policy_for
 from artemis.utils.logger import get_logger
@@ -42,11 +36,23 @@ from artemis.utils.logger import get_logger
 logger = get_logger(__name__)
 
 
+class _ChunkMedia:
+    """Mutable holder for per-attempt media artifacts shared with failure handling."""
+
+    __slots__ = ("path", "compressed_path", "actual_start", "prompt_with_context")
+
+    def __init__(self):
+        self.path: Path | None = None
+        self.compressed_path: Path | None = None
+        self.actual_start: float | None = None
+        self.prompt_with_context: str | None = None
+
+
 async def exec_single_chunk(
     analyzer, start_time: float, end_time: float | None, specific_query: str
 ) -> str:
     """Analyzes one closed chunk interval, with claim/lease and retry handling."""
-    max_retries = analyzer.native_max_retries
+    max_retries = analyzer.max_retries
     current_start = round(start_time, 1) if isinstance(start_time, (int, float)) else start_time
     current_end = round(end_time, 1) if isinstance(end_time, (int, float)) else end_time
 
@@ -85,7 +91,6 @@ async def exec_single_chunk(
     while attempt <= max_retries:
         # The mode this attempt was launched in: a parallel chunk may degrade
         # the run while this one is still in flight.
-        attempt_mode = analyzer.video_processing
         try:
             logger.info(
                 f"Attempt {attempt + 1}/{max_retries + 1} for sub-agent:"
@@ -108,49 +113,25 @@ async def exec_single_chunk(
             )
             _ensure_fits_static_ceiling(analyzer, current_start, current_end)
 
-            if not analyzer.use_native_gemini:
-                chunk_result = await analyzer._exec_single_chunk_universal(
-                    compressed_path=media.compressed_path,
-                    raw_path=media.path,
-                    start_time=current_start,
-                    end_time=current_end,
-                    actual_start=media.actual_start,
-                    prompt_with_context=media.prompt_with_context,
-                    specific_query=specific_query,
-                )
-                summary, analysis = analyzer._parse_chunk_result(chunk_result)
-                analyzer.blackboard.complete_segment(
-                    current_start,
-                    current_end,
-                    specific_query,
-                    lease_owner,
-                    summary,
-                    analysis,
-                )
-                return chunk_result
-
-            if analyzer.video_processing == "agentic":
-                return await run_agentic_chunk_conversation(
-                    analyzer,
-                    media,
-                    current_start,
-                    current_end,
-                    start_time,
-                    end_time,
-                    specific_query,
-                    lease_owner,
-                )
-
-            return await _run_native_chunk_conversation(
-                analyzer,
-                media,
+            chunk_result = await analyzer._exec_single_chunk_universal(
+                compressed_path=media.compressed_path,
+                raw_path=media.path,
+                start_time=current_start,
+                end_time=current_end,
+                actual_start=media.actual_start,
+                prompt_with_context=media.prompt_with_context,
+                specific_query=specific_query,
+            )
+            summary, analysis = analyzer._parse_chunk_result(chunk_result)
+            analyzer.blackboard.complete_segment(
                 current_start,
                 current_end,
-                start_time,
-                end_time,
                 specific_query,
                 lease_owner,
+                summary,
+                analysis,
             )
+            return chunk_result
 
         except Exception as e:
             # Retry decisions are owned by the shared classification layer.
@@ -159,22 +140,6 @@ async def exec_single_chunk(
             # stream output from the failed attempt is discarded, never
             # surfaced as a complete answer.
             failure = classify_video_failure(e)
-            if attempt_mode == "agentic" and is_agentic_rejection(failure, e):
-                # Switch to static and let the coordinator resize the interval.
-                _degrade_agentic_to_static(analyzer, e, failure)
-                return await _finish_failed_chunk(
-                    analyzer,
-                    media,
-                    current_start,
-                    current_end,
-                    specific_query,
-                    lease_owner,
-                    AgenticVideoDegraded(
-                        f"Agentic video processing rejected ({e}); re-plan"
-                        f" {current_start:.1f}s-{current_end:.1f}s at the static chunk size"
-                    ),
-                    retryable=False,
-                )
             attempts_so_far = attempt + 1
             if not failure.retryable:
                 logger.warning(
@@ -229,26 +194,6 @@ async def exec_single_chunk(
     )
 
 
-def _degrade_agentic_to_static(analyzer, error: Exception, failure) -> None:
-    """Switches the rest of this run to static video understanding (idempotent)."""
-    if analyzer.video_processing != "agentic":
-        return
-    logger.warning(
-        f"Agentic video processing rejected ({error}); the rest of this run"
-        " analyzes video in static mode"
-    )
-    _va._record_llm_event(
-        "llm_fallback",
-        {
-            "reason": "agentic_video_rejected",
-            "category": failure.category.value,
-            "error": str(error)[:500],
-        },
-    )
-    analyzer.video_processing = "static"
-    analyzer.chunk_size_seconds = analyzer.static_chunk_size_seconds
-
-
 def _ensure_fits_static_ceiling(analyzer, current_start: float, current_end: float) -> None:
     """Refuses to analyze a clip sized for the agentic ceiling once the run is static.
 
@@ -280,48 +225,6 @@ async def _finish_failed_chunk(
 ):
     """Records terminal failure, attempting the universal fallback first when eligible."""
     failure = classify_video_failure(error)
-    if (
-        analyzer.use_native_gemini
-        and failure.should_fallback
-        and media.path is not None
-        and media.compressed_path is not None
-        and media.actual_start is not None
-        and media.prompt_with_context is not None
-    ):
-        try:
-            logger.warning("Native video chunk exhausted retries; using universal fallback")
-            _va._record_llm_event(
-                "llm_fallback",
-                {
-                    "reason": "native_video_chunk_failed",
-                    "category": failure.category.value,
-                    "error": str(error)[:500],
-                },
-            )
-            fallback_result = await analyzer._exec_single_chunk_universal(
-                compressed_path=media.compressed_path,
-                raw_path=media.path,
-                start_time=current_start,
-                end_time=current_end,
-                actual_start=media.actual_start,
-                prompt_with_context=media.prompt_with_context,
-                specific_query=specific_query,
-                force_fallback=True,
-            )
-            summary, analysis = analyzer._parse_chunk_result(fallback_result)
-            analyzer.blackboard.complete_segment(
-                current_start,
-                current_end,
-                specific_query,
-                lease_owner,
-                summary,
-                analysis,
-            )
-            return fallback_result
-        except Exception as fallback_error:
-            error = fallback_error
-            failure = classify_video_failure(error)
-            retryable = None
     analyzer.blackboard.fail_segment(
         current_start,
         current_end,

artemis/agents/video_analyzer/gemini_files.py

@@ -1,162 +0,0 @@
-# Copyright 2026 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Gemini File API helpers: upload/poll lifecycle and abandoned-file cleanup."""
-
-import asyncio
-from datetime import datetime
-from pathlib import Path
-import time
-
-from artemis.utils.logger import get_logger
-
-try:
-    from datetime import UTC
-except ImportError:
-    UTC = UTC
-
-logger = get_logger(__name__)
-
-_LAST_CLEANUP_TIME = 0.0
-_CLEANUP_LOCK = asyncio.Lock()
-
-
-async def cleanup_abandoned_gemini_files(client) -> None:
-    """Scan and delete remaining cloud video files whose creation time has elapsed standard TTL."""
-    global _LAST_CLEANUP_TIME
-    if _CLEANUP_LOCK.locked():
-        return
-
-    async with _CLEANUP_LOCK:
-        now_ts = time.time()
-        if now_ts - _LAST_CLEANUP_TIME < 3600:
-            return
-        _LAST_CLEANUP_TIME = now_ts
-
-    try:
-        pager = await asyncio.wait_for(client.aio.files.list(), timeout=30)
-        files = []
-        async for f in pager:
-            files.append(f)
-        now = datetime.now(UTC)
-        files_to_delete = []
-
-        for f in files:
-            display_name = getattr(f, "display_name", "") or ""
-
-            if (
-                display_name.startswith("compressed_")
-                or display_name.startswith("audio_")
-                or "artemis" in display_name
-            ):
-                create_time = getattr(f, "created_at", None) or getattr(f, "create_time", None)
-                if not create_time:
-                    continue
-
-                if isinstance(create_time, str):
-                    try:
-                        parsed_time = datetime.fromisoformat(create_time.replace("Z", "+00:00"))
-                    except Exception:
-                        continue
-                elif isinstance(create_time, datetime):
-                    parsed_time = create_time
-                else:
-                    continue
-
-                if parsed_time.tzinfo is None:
-                    parsed_time = parsed_time.replace(tzinfo=UTC)
-
-                age_seconds = (now - parsed_time).total_seconds()
-
-                if age_seconds > 7200:
-                    files_to_delete.append(f)
-                    logger.info(
-                        f"Marked expired cloud asset for deletion: {f.name}"
-                        f" ({display_name}), Age: {age_seconds / 3600:.1f}h"
-                    )
-
-        if files_to_delete:
-            logger.info(f"Purging {len(files_to_delete)} expired cloud assets in parallel...")
-            tasks = [
-                asyncio.wait_for(client.aio.files.delete(name=f.name), timeout=30)
-                for f in files_to_delete
-            ]
-            await asyncio.gather(*tasks, return_exceptions=True)
-    except Exception as e:
-        logger.warning(f"Routine cloud maintenance skipped: {e}")
-
-
-async def delete_cloud_file(
-    client, file_name: str, cloud_files_to_cleanup: set, *, timeout: float = 30.0
-) -> None:
-    """Best-effort deletion of an uploaded file; failures are logged, never raised."""
-    logger.info(f"Cleaning up cloud file {file_name}...")
-    try:
-        await asyncio.wait_for(client.aio.files.delete(name=file_name), timeout=timeout)
-        cloud_files_to_cleanup.discard(file_name)
-    except Exception as ce:
-        logger.error(f"Failed to delete cloud file {file_name}: {ce}")
-
-
-async def upload_and_poll_file(client, compressed_path: Path, cloud_files_to_cleanup: set) -> any:
-    """Upload a media file to the Gemini File API and poll until it is ACTIVE."""
-    logger.info(f"Uploading {compressed_path} to Gemini File API...")
-    file_size_mb = compressed_path.stat().st_size / (1024 * 1024)
-    upload_timeout = max(30.0, min(120.0, file_size_mb * 2.0))
-
-    file = await asyncio.wait_for(
-        client.aio.files.upload(file=compressed_path),
-        timeout=upload_timeout,
-    )
-    cloud_files_to_cleanup.add(file.name)
-
-    max_wait = max(60, min(180, 60 + int(file_size_mb * 2)))
-
-    wait_interval = 0.5
-    start_wait = time.time()
-    retry_count = 0
-    max_retries = 3
-
-    while True:
-        try:
-            f_state = await asyncio.wait_for(client.aio.files.get(name=file.name), timeout=20)
-            retry_count = 0
-        except Exception as poll_error:
-            retry_count += 1
-            logger.warning(
-                f"Temporary issue polling file {file.name} (attempt"
-                f" {retry_count}/{max_retries}): {poll_error}"
-            )
-            if retry_count > max_retries:
-                raise RuntimeError(
-                    f"Failed to poll Gemini File API after {max_retries} attempts: {poll_error}"
-                )
-            await asyncio.sleep(2.0)
-            continue
-
-        if f_state.state.name == "ACTIVE":
-            break
-        elif f_state.state.name == "FAILED":
-            raise RuntimeError(f"Gemini File API processing failed for {file.name}")
-
-        if time.time() - start_wait > max_wait:
-            raise TimeoutError(
-                f"Gemini File API processing timeout for {file.name}"
-                f" (Waited {max_wait}s for size {file_size_mb:.1f}MB)"
-            )
-
-        logger.info(f"File {file.name} is {f_state.state.name}, waiting {wait_interval}s...")
-        await asyncio.sleep(wait_interval)
-        wait_interval = min(3.0, wait_interval * 1.5)
-    return file

artemis/agents/video_analyzer/native_coordinator.py

@@ -1,541 +0,0 @@
-# Copyright 2026 Google LLC
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-#     http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-"""Native Gemini coordinator loop for the video analyzer main agent.
-
-Extracted from ``video_analyzer.py`` as a pure structural split.  Patchable
-collaborators are looked up late through the facade module so
-``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")`` targets
-keep working.
-"""
-
-import asyncio
-import os
-from pathlib import Path
-import shutil
-import tempfile
-
-from google.genai import types
-
-from artemis.agents.video_analyzer import video_analyzer as _va
-from artemis.agents.video_analyzer.reliability import classify_video_failure
-from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
-from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan
-from artemis.llm.google import normalize_usage
-from artemis.utils.logger import get_logger
-
-logger = get_logger(__name__)
-
-
-class _DummyResponse:
-    """Aggregated view over a consumed stream, shaped like a full response."""
-
-    def __init__(self, text, function_calls, parts):
-        self.text = text
-        self.function_calls = function_calls
-        self.candidates = [
-            types.Candidate(
-                content=types.Content(
-                    role="model",
-                    parts=parts if parts else ([types.Part.from_text(text=text)] if text else []),
-                )
-            )
-        ]
-
-
-async def run_native(
-    analyzer,
-    ctx,
-    time_description: str,
-    purpose: str,
-    system_prompt: str,
-    temperature: float,
-    thinking_level,
-) -> tuple[str, str]:
-    """Drives the native Gemini coordinator loop to a final answer."""
-    tools_declaration = analyzer.tools_declaration
-
-    contents = [
-        types.Content(
-            role="user",
-            parts=[
-                types.Part.from_text(
-                    text=(f"Time description: {time_description}\nPurpose: {purpose}")
-                )
-            ],
-        )
-    ]
-
-    agent_outcome = ""
-    status = "success"
-    max_iterations = 10
-    iterations = 0
-    last_ledger_index = 0
-
-    try:
-        while iterations < max_iterations:
-            iterations += 1
-            logger.info(f"Iteration {iterations}: Invoking Native Gemini SDK...")
-
-            analyzer._clean_blackboard()
-
-            last_ledger_index = _append_ledger_diff(analyzer, contents, last_ledger_index)
-
-            is_final_turn = iterations == max_iterations
-            _append_turn_guidance(contents, iterations, max_iterations, is_final_turn)
-
-            response = await _stream_main_turn(
-                analyzer,
-                ctx,
-                contents,
-                system_prompt,
-                temperature,
-                thinking_level,
-                tools_declaration,
-                is_final_turn,
-            )
-
-            function_calls = response.function_calls
-            if not function_calls:
-                agent_outcome = response.text
-                break
-
-            if response.candidates and response.candidates[0].content:
-                contents.append(response.candidates[0].content)
-            else:
-                contents.append(
-                    types.Content(
-                        role="model",
-                        parts=[types.Part(function_call=fc) for fc in function_calls],
-                    )
-                )
-
-            tool_response_parts = await _execute_native_tool_calls(analyzer, function_calls)
-
-            if tool_response_parts:
-                contents.append(types.Content(role="user", parts=tool_response_parts))
-
-        if iterations >= max_iterations and not agent_outcome:
-            agent_outcome = (
-                "Error: Video analyzer agent reached maximum iterations"
-                " without settling on a final answer."
-            )
-            status = "error"
-
-    except Exception as e:
-        logger.error(f"Video analyzer failed: {e}")
-        failure = classify_video_failure(e)
-        if failure.should_fallback:
-            logger.warning(
-                f"Native coordinator failed with {failure.category.value}; "
-                "continuing through universal fallback"
-            )
-            agent_outcome, status = await analyzer._run_universal(
-                time_description,
-                purpose,
-                system_prompt,
-                force_fallback=True,
-            )
-        else:
-            agent_outcome = f"Video analysis failed: {e}"
-            status = "error"
-    finally:
-        analyzer._record_reliability_metrics()
-        await _cleanup_native_run(analyzer)
-
-    return agent_outcome, status
-
-
-def _append_ledger_diff(analyzer, contents: list, last_ledger_index: int) -> int:
-    """Appends new blackboard entries (and proofs) since the last coordinator turn."""
-    new_entries = analyzer.blackboard_entries[last_ledger_index:]
-    if analyzer.enable_ledger and new_entries:
-        lines = []
-        image_parts = []
-        for e in new_entries:
-            line = f"[{e['start']}s - {e['end']}s] SUMMARY: {e['summary']}"
-            screenshot_file = e.get("screenshot") or e.get("screenshot_path")
-            if screenshot_file and Path(screenshot_file).exists():
-                line += f" PROOF: {Path(screenshot_file).name}"
-                image_parts.append(
-                    types.Part.from_text(text=f"PROOF: {Path(screenshot_file).name}")
-                )
-                image_parts.append(
-                    types.Part.from_bytes(
-                        data=Path(screenshot_file).read_bytes(),
-                        mime_type="image/jpeg",
-                    )
-                )
-            lines.append(line)
-        diff_text = "NEW BLACKBOARD ENTRIES ADDED: " + " | ".join(lines)
-        contents.append(
-            types.Content(
-                role="user",
-                parts=[types.Part.from_text(text=diff_text)],
-            )
-        )
-        if image_parts:
-            contents.append(types.Content(role="user", parts=image_parts))
-        last_ledger_index = len(analyzer.blackboard_entries)
-    return last_ledger_index
-
-
-def _append_turn_guidance(
-    contents: list, iterations: int, max_iterations: int, is_final_turn: bool
-) -> None:
-    """Appends per-turn steering messages (final-turn notice or progress nudge)."""
-    if is_final_turn:
-        contents.append(
-            types.Content(
-                role="user",
-                parts=[
-                    types.Part.from_text(
-                        text=(
-                            "This is your final iteration; all"
-                            " tools are stripped, and you must"
-                            " provide your final answer directly."
-                        )
-                    )
-                ],
-            )
-        )
-    else:
-        if iterations > 1:
-            contents.append(
-                types.Content(
-                    role="user",
-                    parts=[
-                        types.Part.from_text(
-                            text=(
-                                "You have not completed the video"
-                                " analysis yet (iteration"
-                                f" {iterations} of"
-                                f" {max_iterations})."
-                            )
-                        )
-                    ],
-                )
-            )
-
-
-def _record_main_llm_trace(
-    ctx, trace_id, contents: list, system_prompt: str, tools_declaration, is_final_turn: bool
-) -> None:
-    """Records the sanitized coordinator LLM call into the trace store."""
-    serialized_contents = []
-    for c in contents:
-        if hasattr(c, "model_dump"):
-            dumped = c.model_dump()
-            if isinstance(dumped, dict) and "parts" in dumped and dumped["parts"]:
-                for p in dumped["parts"]:
-                    if (
-                        isinstance(p, dict)
-                        and "inline_data" in p
-                        and p["inline_data"]
-                        and "data" in p["inline_data"]
-                    ):
-                        p["inline_data"]["data"] = "<image_bytes_sanitized>"
-            serialized_contents.append(dumped)
-        else:
-            serialized_contents.append(str(c))
-
-    ctx.data_engine.record_trace(
-        type="llm_call",
-        name="video_analyzer_main",
-        payload={
-            "contents": serialized_contents,
-            "system_instruction": system_prompt,
-            "tools": (
-                [t.name for t in tools_declaration]
-                if (tools_declaration and not is_final_turn)
-                else []
-            ),
-        },
-        parent_trace_id=trace_id,
-    )
-
-
-async def _stream_main_turn(
-    analyzer,
-    ctx,
-    contents: list,
-    system_prompt: str,
-    temperature: float,
-    thinking_level,
-    tools_declaration,
-    is_final_turn: bool,
-):
-    """Runs one traced, retry-wrapped streaming turn of the main coordinator."""
-    with TraceSpan(name="gemini_main_agent_call") as span:
-        trace_id = CURRENT_TRACE_ID.get()
-
-        if ctx.data_engine and trace_id:
-            _record_main_llm_trace(
-                ctx, trace_id, contents, system_prompt, tools_declaration, is_final_turn
-            )
-
-        async def run_stream():
-            return await _consume_main_stream(
-                analyzer,
-                ctx,
-                span,
-                trace_id,
-                contents,
-                system_prompt,
-                temperature,
-                thinking_level,
-                tools_declaration,
-                is_final_turn,
-            )
-
-        response = await _va._invoke_with_retry(
-            lambda: asyncio.wait_for(run_stream(), timeout=300),
-            "Native Gemini video coordinator",
-        )
-        span.result = (
-            f"Function calls: {len(response.function_calls)}"
-            if response.function_calls
-            else "Final answer"
-        )
-    return response
-
-
-async def _consume_main_stream(
-    analyzer,
-    ctx,
-    span,
-    trace_id,
-    contents: list,
-    system_prompt: str,
-    temperature: float,
-    thinking_level,
-    tools_declaration,
-    is_final_turn: bool,
-) -> _DummyResponse:
-    """Starts the coordinator stream and aggregates it into a response object."""
-    stream = await asyncio.wait_for(
-        analyzer.client.aio.models.generate_content_stream(
-            model=analyzer.model_name,
-            contents=contents,
-            config=types.GenerateContentConfig(
-                system_instruction=system_prompt,
-                temperature=temperature,
-                tools=[]
-                if is_final_turn
-                else [types.Tool(function_declarations=tools_declaration)],
-                safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
-                **(
-                    {"thinking_config": types.ThinkingConfig(thinking_level=thinking_level)}
-                    if thinking_level
-                    else {}
-                ),
-            ),
-        ),
-        timeout=60,
-    )
-
-    text = ""
-    accumulated_parts = []
-    function_calls = []
-
-    async for chunk in stream:
-        usage = normalize_usage(getattr(chunk, "usage_metadata", None))
-        if usage:
-            span.payload["usage_metadata"] = usage
-
-        chunk_text = chunk.text or ""
-        text += chunk_text
-        if ctx.data_engine and trace_id and chunk_text:
-            ctx.data_engine.stream_output(trace_id, chunk_text, is_thinking=False)
-
-        if chunk.function_calls:
-            function_calls.extend(chunk.function_calls)
-
-        candidates = getattr(chunk, "candidates", None)
-        if candidates and isinstance(candidates, list) and len(candidates) > 0:
-            content = getattr(candidates[0], "content", None)
-            parts = getattr(content, "parts", None) if content else None
-            if parts:
-                for part in parts:
-                    accumulated_parts.append(part)
-                    if getattr(part, "thought", False) and part.text:
-                        if ctx.data_engine and trace_id:
-                            ctx.data_engine.stream_output(
-                                trace_id,
-                                part.text,
-                                is_thinking=True,
-                            )
-        else:
-            if chunk.function_calls:
-                for fc in chunk.function_calls:
-                    accumulated_parts.append(types.Part(function_call=fc))
-
-    return _DummyResponse(text, function_calls, accumulated_parts)
-
-
-async def _execute_native_tool_calls(analyzer, function_calls: list) -> list:
-    """Executes coordinator tool calls (parallel where safe) and collects responses."""
-    sub_agent_calls = [
-        fc
-        for fc in function_calls
-        if (fc.name.split(":")[-1] if ":" in fc.name else fc.name) == "spawn_sub_agent"
-    ]
-    audio_only_calls = [
-        fc
-        for fc in function_calls
-        if (fc.name.split(":")[-1] if ":" in fc.name else fc.name) == "analyze_audio_only"
-    ]
-    other_calls = [
-        fc
-        for fc in function_calls
-        if (fc.name.split(":")[-1] if ":" in fc.name else fc.name)
-        not in ["spawn_sub_agent", "analyze_audio_only"]
-    ]
-
-    tool_response_parts = []
-
-    if sub_agent_calls:
-        await _run_spawn_calls(analyzer, sub_agent_calls, tool_response_parts)
-
-    if audio_only_calls:
-        await _run_audio_calls(analyzer, audio_only_calls, tool_response_parts)
-
-    await _run_other_calls(analyzer, other_calls, tool_response_parts)
-
-    return tool_response_parts
-
-
-async def _run_spawn_calls(analyzer, sub_agent_calls: list, tool_response_parts: list) -> None:
-    """Runs spawn_sub_agent calls in parallel and appends their responses."""
-    logger.info(f"Executing {len(sub_agent_calls)} sub-agent calls in parallel...")
-
-    async def bounded_spawn(fc):
-        # exec_spawn_sub_agent limits each actual model leaf.
-        # Holding the same semaphore around the coordinator
-        # would deadlock when all slots wait for child slots.
-        return await analyzer.exec_spawn_sub_agent(
-            fc.args.get("start_time"),
-            fc.args.get("end_time"),
-            fc.args.get("specific_query"),
-        )
-
-    tasks = [bounded_spawn(fc) for fc in sub_agent_calls]
-    results = await asyncio.gather(*tasks, return_exceptions=True)
-
-    for fc, result in zip(sub_agent_calls, results):
-        if isinstance(result, Exception):
-            error_msg = f"Sub-agent failed. Error: {result}"
-            logger.error(error_msg)
-            tool_response_parts.append(
-                types.Part.from_function_response(name=fc.name, response={"error": error_msg})
-            )
-        else:
-            tool_response_parts.append(
-                types.Part.from_function_response(name=fc.name, response={"result": result})
-            )
-
-
-async def _run_audio_calls(analyzer, audio_only_calls: list, tool_response_parts: list) -> None:
-    """Runs analyze_audio_only calls in parallel and appends their responses."""
-    logger.info(f"Executing {len(audio_only_calls)} analyze_audio_only calls in parallel...")
-
-    async def bounded_audio_spawn(fc):
-        async with _va.API_SEMAPHORE:
-            return await analyzer.exec_analyze_audio_only(
-                fc.args.get("start_time"),
-                fc.args.get("end_time"),
-                fc.args.get("specific_query"),
-            )
-
-    tasks = [bounded_audio_spawn(fc) for fc in audio_only_calls]
-    results = await asyncio.gather(*tasks, return_exceptions=True)
-
-    for fc, result in zip(audio_only_calls, results):
-        if isinstance(result, Exception):
-            error_msg = f"Audio analysis failed. Error: {result}"
-            logger.error(error_msg)
-            tool_response_parts.append(
-                types.Part.from_function_response(name=fc.name, response={"error": error_msg})
-            )
-        else:
-            tool_response_parts.append(
-                types.Part.from_function_response(name=fc.name, response={"result": result})
-            )
-
-
-async def _run_other_calls(analyzer, other_calls: list, tool_response_parts: list) -> None:
-    """Runs remaining coordinator tool calls sequentially."""
-    for fc in other_calls:
-        name = fc.name.split(":")[-1] if ":" in fc.name else fc.name
-        args = fc.args
-        logger.info(f"Executing tool '{name}' sequentially...")
-
-        if name == "extract_segment_metadata":
-            try:
-                res = await analyzer.exec_extract_segment_metadata(
-                    args.get("start_time"), args.get("end_time")
-                )
-                tool_response_parts.append(
-                    types.Part.from_function_response(name=name, response={"result": res})
-                )
-            except Exception as e:
-                tool_response_parts.append(
-                    types.Part.from_function_response(name=name, response={"error": str(e)})
-                )
-
-
-async def _cleanup_native_run(analyzer) -> None:
-    """Releases remaining cloud files and temporary local media after a run."""
-    if analyzer.cloud_files_to_cleanup:
-        logger.info(
-            f"Cleaning up {len(analyzer.cloud_files_to_cleanup)} remaining"
-            " cloud files in parallel..."
-        )
-        tasks = [
-            asyncio.wait_for(analyzer.client.aio.files.delete(name=name), timeout=30)
-            for name in list(analyzer.cloud_files_to_cleanup)
-        ]
-        await asyncio.gather(*tasks, return_exceptions=True)
-        analyzer.cloud_files_to_cleanup.clear()
-
-    if os.environ.get("KEEP_VIDEOS") or os.environ.get("ARTEMIS_DEBUG"):
-        logger.info("Skipping cleanup of local video files and directories due to debug mode.")
-    else:
-        for path in list(analyzer.local_files_to_cleanup):
-            try:
-                if path.exists():
-                    logger.info(f"Cleaning up local file {path}...")
-                    path.unlink()
-            except Exception as e:
-                logger.error(f"Failed to delete local file {path}: {e}")
-
-        for dir_path in list(analyzer.local_dirs_to_cleanup):
-            try:
-                resolved = dir_path.resolve()
-                system_dirs = {
-                    Path("/").resolve(),
-                    Path("/tmp").resolve(),
-                    Path("/var/tmp").resolve(),
-                    Path.home().resolve(),
-                    Path(tempfile.gettempdir()).resolve(),
-                }
-                if resolved in system_dirs:
-                    logger.debug(f"Skipping deletion of root/system directory: {dir_path}")
-                    continue
-
-                if dir_path.exists():
-                    logger.info(f"Cleaning up local directory {dir_path}...")
-                    shutil.rmtree(dir_path, ignore_errors=True)
-            except Exception as e:
-                logger.error(f"Failed to delete local directory {dir_path}: {e}")

artemis/agents/video_analyzer/universal_engine.py

@@ -35,6 +35,8 @@ from langchain_core.messages import (
     ToolMessage,
 )
 
+from artemis.llm.structured import content_to_text
+
 from artemis.agents.video_analyzer import video_analyzer as _va
 from artemis.agents.video_analyzer.reliability import (
     SubAgentAnswerExhausted,
@@ -241,7 +243,7 @@ def _extract_submit_payload(response) -> tuple[str, str, list]:
         return final_summary, final_analysis, list(args.get("timeline_events") or [])
 
     raw_content = getattr(response, "content", None)
-    text_content = str(raw_content).strip() if raw_content else ""
+    text_content = content_to_text(raw_content).strip()
     if text_content:
         return text_content[:200].replace("\n", " "), text_content, []
 
@@ -435,7 +437,7 @@ async def run_universal(
             messages.append(response)
 
             if not getattr(response, "tool_calls", None):
-                agent_outcome = str(response.content)
+                agent_outcome = content_to_text(response.content)
                 break
 
             for tc in response.tool_calls:

artemis/agents/video_analyzer/universal_tools.py

@@ -14,14 +14,11 @@
 
 """Tool schema declarations for the video analyzer.
 
-Holds both the OpenAI-style function schemas used by the universal
-(LangChain) engine and the builders for the native Gemini
-``FunctionDeclaration`` equivalents.  Pure data/declarations only.
+OpenAI function schemas for the Responses video analysis engine.
 """
 
 from typing import Any
 
-from google.genai import types
 
 UNIVERSAL_SUBMIT_ANSWER_TOOL: dict[str, Any] = {
     "type": "function",
@@ -172,152 +169,3 @@ UNIVERSAL_MAIN_TOOLS: list[dict[str, Any]] = [
         },
     },
 ]
-
-
-def build_native_tools_declaration() -> list[types.FunctionDeclaration]:
-    """Builds the native Gemini declarations for the coordinator tools."""
-    return [
-        types.FunctionDeclaration(
-            name="extract_segment_metadata",
-            description=(
-                "Retrieves metadata (such as duration_seconds and"
-                " file_size_mb) for a specific video segment. Does not"
-                " return the video content or perform content analysis."
-            ),
-            parameters=types.Schema(
-                type=types.Type.OBJECT,
-                properties={
-                    "start_time": types.Schema(
-                        type=types.Type.NUMBER,
-                        description="Start time in seconds",
-                    ),
-                    "end_time": types.Schema(
-                        type=types.Type.NUMBER,
-                        description=(
-                            "End time in seconds (optional, defaults to latest available time)"
-                        ),
-                    ),
-                },
-                required=["start_time"],
-            ),
-        ),
-        types.FunctionDeclaration(
-            name="spawn_sub_agent",
-            description=(
-                "Spawns a sub-agent to analyze both visual and audio"
-                " content of a video segment. Parallelizes chunk analysis"
-                " across workers. Returns combined summary and analysis."
-            ),
-            parameters=types.Schema(
-                type=types.Type.OBJECT,
-                properties={
-                    "start_time": types.Schema(
-                        type=types.Type.NUMBER,
-                        description="Start time in seconds",
-                    ),
-                    "end_time": types.Schema(
-                        type=types.Type.NUMBER,
-                        description=(
-                            "End time in seconds. Use the maximum available time if not specified."
-                        ),
-                    ),
-                    "specific_query": types.Schema(
-                        type=types.Type.STRING,
-                        description=("Specific query or intent for the sub-agent"),
-                    ),
-                },
-                required=["start_time", "specific_query"],
-            ),
-        ),
-        types.FunctionDeclaration(
-            name="analyze_audio_only",
-            description=(
-                "Extracts and analyzes only the audio track from a video"
-                " segment. More efficient for non-visual tasks."
-            ),
-            parameters=types.Schema(
-                type=types.Type.OBJECT,
-                properties={
-                    "start_time": types.Schema(
-                        type=types.Type.NUMBER,
-                        description="Start time in seconds",
-                    ),
-                    "end_time": types.Schema(
-                        type=types.Type.NUMBER,
-                        description="End time in seconds (optional)",
-                    ),
-                    "specific_query": types.Schema(
-                        type=types.Type.STRING,
-                        description=("Specific query or intent for the audio content"),
-                    ),
-                },
-                required=["start_time", "specific_query"],
-            ),
-        ),
-    ]
-
-
-def build_submit_answer_declaration() -> types.FunctionDeclaration:
-    """Builds the native Gemini declaration for the sub-agent submit_answer tool."""
-    return types.FunctionDeclaration(
-        name="submit_answer",
-        description=("Submit the final findings for the requested segment analysis."),
-        parameters=types.Schema(
-            type=types.Type.OBJECT,
-            properties={
-                "summary": types.Schema(
-                    type=types.Type.STRING,
-                    description=("A short paragraph summarizing of your findings."),
-                ),
-                "analysis": types.Schema(
-                    type=types.Type.STRING,
-                    description="A short paragraph of detailed analysis.",
-                ),
-                "timeline_events": types.Schema(
-                    type=types.Type.ARRAY,
-                    description=(
-                        "List of detected events matching the query. If no"
-                        " relevant events occurred in the segment, return"
-                        " an empty array."
-                    ),
-                    items=types.Schema(
-                        type=types.Type.OBJECT,
-                        properties={
-                            "start_time": types.Schema(
-                                type=types.Type.NUMBER,
-                                description="Event start time (seconds)",
-                            ),
-                            "end_time": types.Schema(
-                                type=types.Type.NUMBER,
-                                description="Event end time (seconds)",
-                            ),
-                            "transcription": types.Schema(
-                                type=types.Type.STRING,
-                                description=(
-                                    "A concise transcription of what happened in the event.  "
-                                ),
-                            ),
-                            "confidence_score": types.Schema(
-                                type=types.Type.NUMBER,
-                                description=("Confidence score between 0 and 1"),
-                            ),
-                            "verification_timestamp_secs": types.Schema(
-                                type=types.Type.NUMBER,
-                                description=(
-                                    "Exact timestamp (seconds) for verification screenshot."
-                                ),
-                            ),
-                        },
-                        required=[
-                            "start_time",
-                            "end_time",
-                            "transcription",
-                            "confidence_score",
-                            "verification_timestamp_secs",
-                        ],
-                    ),
-                ),
-            },
-            required=["summary", "analysis", "timeline_events"],
-        ),
-    )

artemis/agents/video_analyzer/video_analyzer.py

@@ -14,9 +14,8 @@
 
 """VideoAnalyzer facade: entry point and shared runtime state.
 
-The execution engines live in sibling modules (``chunk_native``,
-``audio_native``, ``universal_engine``, ``native_coordinator``,
-``gemini_files``, ``universal_tools``) and resolve their patchable
+The execution engines live in sibling modules (``chunk_runner``,
+``audio_runner``, ``universal_engine``, ``universal_tools``) and resolve their patchable
 collaborators through THIS module's namespace, so existing
 ``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")``
 targets keep working.  Keep the imports below in place even when the facade
@@ -36,8 +35,6 @@ import tempfile
 import time
 from typing import Any
 
-from google import genai
-from google.genai import types
 from langchain_core.messages import (
     BaseMessage,
     HumanMessage,
@@ -49,9 +46,6 @@ from artemis.agents.video_analyzer.conflict_resolution import (
     ConflictResolutionService,
 )
 from artemis.agents.video_analyzer.blackboard import get_video_blackboard
-from artemis.agents.video_analyzer.gemini_files import (
-    cleanup_abandoned_gemini_files,
-)
 from artemis.agents.video_analyzer.reliability import (
     AgenticVideoDegraded,
     VideoCircuitBreaker,
@@ -60,19 +54,11 @@ from artemis.agents.video_analyzer.reliability import (
 from artemis.agents.video_analyzer.universal_tools import (
     UNIVERSAL_MAIN_TOOLS,
     UNIVERSAL_SUBMIT_ANSWER_TOOL,
-    build_native_tools_declaration,
-    build_submit_answer_declaration,
 )
 from artemis.config import get_temp_dir, settings
-from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
 from artemis.context import ArtemisContext
 from artemis.controllers.controller_factory import get_controller
 from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan, trace
-from artemis.llm.google import (
-    is_gemini_model,
-    resolve_video_processing,
-    strip_provider_prefix,
-)
 from artemis.llm.reliability import retry_policy_for
 from artemis.services.llm import _record_llm_event, _record_llm_retry, get_llm
 from artemis.utils.logger import get_logger
@@ -159,16 +145,8 @@ class VideoAnalyzer:
             getattr(agent_config, "enable_video_ledger", True),
         )
         self.static_chunk_size_seconds = float(
-            getattr(self.video_config, "chunk_size_seconds", 60.0)
+            getattr(self.video_config, "chunk_size_seconds", settings.ARTEMIS_VIDEO_CHUNK_SECONDS)
         )
-        self.agentic_chunk_size_seconds = float(
-            getattr(self.video_config, "agentic_chunk_size_seconds", 600.0)
-        )
-        self.agentic_call_timeout_seconds = float(
-            getattr(self.video_config, "agentic_call_timeout_seconds", 300.0)
-        )
-        # Effective chunk ceiling; re-derived once the engine picks a
-        # video-processing mode (agentic sub-agents take long clips).
         self.chunk_size_seconds = self.static_chunk_size_seconds
         self.video_processing = "static"
         self.min_chunk_seconds = float(getattr(self.video_config, "min_chunk_seconds", 4.0))
@@ -178,9 +156,15 @@ class VideoAnalyzer:
         self.max_dense_action_frames = int(
             getattr(self.video_config, "max_dense_action_frames", 24)
         )
-        self.native_max_retries = int(getattr(self.video_config, "native_max_retries", 1))
+        self.max_retries = int(
+            getattr(self.video_config, "max_retries", settings.ARTEMIS_VIDEO_MAX_RETRIES)
+        )
         self.model_call_timeout_seconds = float(
-            getattr(self.video_config, "model_call_timeout_seconds", 120.0)
+            getattr(
+                self.video_config,
+                "model_call_timeout_seconds",
+                settings.ARTEMIS_VIDEO_TIMEOUT_SECONDS,
+            )
         )
         breaker = getattr(ctx, "_video_circuit_breaker", None)
         if breaker is None:
@@ -210,7 +194,6 @@ class VideoAnalyzer:
             audio_prompt_path.read_text(encoding="utf-8") if audio_prompt_path.exists() else ""
         )
 
-        self._init_tools()
         self._init_engine()
 
     def _get_universal_llm(self, *, use_fallback: bool = False):
@@ -406,65 +389,13 @@ class VideoAnalyzer:
         analysis = match.group(2).strip() if match.group(2) else None
         return summary, analysis
 
-    def _init_tools(self) -> None:
-        """Initializes native and sub-agent tool declarations."""
-        self.tools_declaration = build_native_tools_declaration()
-        self.submit_answer_tool = build_submit_answer_declaration()
-
     def _init_engine(self) -> None:
-        """Initializes model engine and decides whether to use native Gemini or Universal path."""
-        ctx = self.ctx
-        llm_config = getattr(ctx, "llm_config", None)
+        llm_config = getattr(self.ctx, "llm_config", None)
         utils_cfg = getattr(llm_config, "utils", None) if llm_config else None
         llm_cfg = getattr(utils_cfg, "video_analyzer", None) if utils_cfg else None
-        self.model_name = strip_provider_prefix(
-            llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.8-flash"
-        )
-
-        has_google_key = bool(
-            settings.GOOGLE_API_KEY and settings.GOOGLE_API_KEY.get_secret_value()
-        )
-
-        # If client is already set on ctx or explicit Gemini configuration
-        self.client = getattr(ctx, "_genai_client", None)
-        if self.client is not None:
-            self.use_native_gemini = True
-        elif has_google_key and is_gemini_model(self.model_name):
-            try:
-                self.client = genai.Client(api_key=settings.GOOGLE_API_KEY.get_secret_value())
-                ctx._genai_client = self.client
-                self.use_native_gemini = True
-                asyncio.create_task(cleanup_abandoned_gemini_files(self.client))
-            except Exception as e:
-                logger.warning(
-                    f"Failed to initialize native Gemini client: {e}. Using universal engine."
-                )
-                self.use_native_gemini = False
-        else:
-            self.use_native_gemini = False
-        self._resolve_video_processing()
-
-    def _resolve_video_processing(self) -> None:
-        """Picks static vs agentic video understanding for the sub-agents.
-
-        Agentic processing (Gemini Interactions API) needs the native client
-        and a supporting model; the configured knob (``auto`` / ``agentic`` /
-        ``static``) decides the rest. The chunk ceiling follows the mode
-        because agentic sub-agents are built to take long clips.
-        """
-        configured = getattr(self.video_config, "processing", "auto")
-        mode = "static"
-        if self.use_native_gemini:
-            mode = resolve_video_processing(configured, self.model_name)
-        self.video_processing = mode
-        self.chunk_size_seconds = (
-            self.agentic_chunk_size_seconds if mode == "agentic" else self.static_chunk_size_seconds
-        )
-        if mode == "agentic":
-            logger.info(
-                f"Video sub-agents use agentic video understanding on {self.model_name}"
-                f" (chunk ceiling {self.chunk_size_seconds:.0f}s)"
-            )
+        self.model_name = getattr(llm_cfg, "model", None) or settings.OPENAI_MODEL
+        self.video_processing = "static"
+        self.chunk_size_seconds = self.static_chunk_size_seconds
 
     @property
     def blackboard_ledger(self) -> str:
@@ -476,13 +407,6 @@ class VideoAnalyzer:
             lines.append(f"{e['start']}s - {e['end']}s: {e['summary']}")
         return " | ".join(lines)
 
-    async def upload_and_poll_file(self, compressed_path: Path) -> any:
-        from artemis.agents.video_analyzer import gemini_files
-
-        return await gemini_files.upload_and_poll_file(
-            self.client, compressed_path, self.cloud_files_to_cleanup
-        )
-
     def get_overlapping_warnings(self, start_time: float, end_time: float | None) -> list[dict]:
         warnings = []
         if not isinstance(start_time, (int, float)) or (
@@ -739,17 +663,17 @@ class VideoAnalyzer:
     async def _exec_single_chunk(
         self, start_time: float, end_time: float | None, specific_query: str
     ) -> str:
-        from artemis.agents.video_analyzer import chunk_native
+        from artemis.agents.video_analyzer import chunk_runner
 
-        return await chunk_native.exec_single_chunk(self, start_time, end_time, specific_query)
+        return await chunk_runner.exec_single_chunk(self, start_time, end_time, specific_query)
 
     @trace(type="tool", name="analyze_audio_only")
     async def exec_analyze_audio_only(
         self, start_time: float, end_time: float | None, specific_query: str
     ) -> str:
-        from artemis.agents.video_analyzer import audio_native
+        from artemis.agents.video_analyzer import audio_runner
 
-        return await audio_native.exec_analyze_audio_only(
+        return await audio_runner.exec_analyze_audio_only(
             self, start_time, end_time, specific_query
         )
 
@@ -837,15 +761,12 @@ class VideoAnalyzer:
             utils_cfg = getattr(llm_config, "utils", None) if llm_config else None
             llm_cfg = getattr(utils_cfg, "video_analyzer", None) if utils_cfg else None
             temperature = 0.2
-            thinking_level = None
             if llm_cfg:
-                self.model_name = strip_provider_prefix(llm_cfg.model)
+                self.model_name = llm_cfg.model
                 if getattr(llm_cfg, "temperature", None) is not None:
                     temperature = llm_cfg.temperature
-                if getattr(llm_cfg, "thinking_level", None) is not None:
-                    thinking_level = llm_cfg.thinking_level
             else:
-                self.model_name = "gemini-3.8-flash"
+                self.model_name = settings.OPENAI_MODEL
 
             # Track files for cleanup
             self.local_files_to_cleanup = set()
@@ -868,24 +789,11 @@ class VideoAnalyzer:
                 " proceeding.\n"
             )
 
-        if not self.use_native_gemini:
-            try:
-                return await self._run_universal(
-                    time_description=time_description,
-                    purpose=purpose,
-                    system_prompt=system_prompt,
-                )
-            finally:
-                await self._cleanup_universal_resources()
-
-        from artemis.agents.video_analyzer import native_coordinator
-
-        return await native_coordinator.run_native(
-            self,
-            ctx,
-            time_description,
-            purpose,
-            system_prompt,
-            temperature,
-            thinking_level,
-        )
+        try:
+            return await self._run_universal(
+                time_description=time_description,
+                purpose=purpose,
+                system_prompt=system_prompt,
+            )
+        finally:
+            await self._cleanup_universal_resources()