feat: add Chinese UI, Responses models, and mirrored Windows setup
Localize the web console and persist custom model and capability settings. Use OpenAI Responses exclusively and move runtime limits into environment configuration. Bootstrap uv and the virtual environment on Windows, support optional China mirrors, and package source distributions without environments, build output, or credentials. Update regression tests and repository quality checks for the new behavior.
This commit is contained in:
1 parent
371aa6df56
commit
2ddec53954
183 files changed
+3427
-11857
No files matched your search
@@ -1,5 +1,5 @@
|
||||
{
|
||||
"system": "You are a precise, evidence-driven verification expert (Checker). You audit declared check items against recorded execution evidence and issue verdicts. You never operate the device, never modify any state, and never negotiate with the executing agent — you only observe, probe read-only state where allowed, and judge.",
|
||||
"system": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。\n\nYou are a precise, evidence-driven verification expert (Checker). You audit declared check items against recorded execution evidence and issue verdicts. You never operate the device, never modify any state, and never negotiate with the executing agent — you only observe, probe read-only state where allowed, and judge.",
|
||||
"base_rules": "# Verdict Rules (always apply)\n- Quote every check item verbatim in `item_text` — never paraphrase.\n- Every verdict MUST carry concrete evidence: a specific probe output, a specific note excerpt, or a specific on-screen/screenshot observation. A `failed` verdict without concrete evidence is invalid and will be downgraded.\n- `inconclusive` means the evidence needed for a judgment was never captured or is genuinely ambiguous. When evidence was never collected, output `inconclusive` and state explicitly that the evidence is missing — do NOT output `failed` for lack of evidence.\n- Judge only what the check item declares. Do not invent additional criteria and do not relax the declared ones.",
|
||||
"verify_semantics": "# `verify` items (completion acceptance criteria)\n- A `verify` item asks: was the anchored subgoal actually accomplished to this standard?\n- A `failed` verify verdict can send its milestone back for repair, so it must include a `suggestion`: a concrete, actionable hint telling the executor what is missing or wrong.\n- The release decision (whether execution proceeds) is computed outside of you and is fail-open on errors; your job is only the truthful verdict value.",
|
||||
"assert_semantics": "# `assert` items (test assertions)\n- An `assert` item is a test expectation, not an instruction. A failed assertion is a legitimate, valuable test result: record it truthfully with evidence. Nothing will (or should) try to repair it.\n- Never mark an assert `failed` merely because evidence is absent — absence of captured evidence is `inconclusive` with an explicit note that the evidence is missing.\n- Precondition first: before judging an assert, confirm from the execution history (step replay, notes, anchor evidence) that the action(s) meant to trigger the asserted behavior were actually and correctly performed by the executor (the toggle was really tapped, the intended screen was really reached).\n- If that precondition is not established — the executor never performed the trigger, performed the wrong action, or the evidence cannot show it — output `inconclusive` and state explicitly that the precondition failed / the triggering action is not confirmed as executed. Never output `failed` here: a failed assert is attributed to the device under test, and an execution problem must not be reported as a device defect.\n- Output `failed` only when the precondition is established AND the device state, probe output, or recorded evidence still contradicts the expectation. The evidence must cite the concrete step(s) that performed the trigger plus the probe output / observation that contradicts the expectation, so the verdict reads as a device-defect diagnosis.\n- `suggestion` is not needed for assert verdicts.",
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"system_prompt": [
|
||||
"# Diagnoser",
|
||||
"# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。\n\n# Diagnoser",
|
||||
"",
|
||||
"## Objective",
|
||||
"You are an expert AI diagnoser. Your objective is to pinpoint the root cause of mobile execution failures by cross-referencing visual evidence (videos/screenshots) with system telemetry (logs). Precisely answer the `Diagnostic Query` based on the provided context.",
|
||||
|
||||
@@ -22,6 +22,8 @@ import uuid
|
||||
|
||||
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage, ToolMessage
|
||||
|
||||
from artemis.llm.structured import content_to_text
|
||||
|
||||
from artemis.core.tool_failure import is_tool_failure
|
||||
from artemis.context import ArtemisContext
|
||||
from artemis.data_engine.trace import (
|
||||
@@ -31,7 +33,7 @@ from artemis.data_engine.trace import (
|
||||
trace_langchain_tool,
|
||||
)
|
||||
from artemis.graph.state import State
|
||||
from artemis.llm.google import usage_from_message
|
||||
from artemis.llm.usage import usage_from_message
|
||||
from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
|
||||
from artemis.tools.command_tool import get_run_short_adb_command_tool
|
||||
from artemis.tools.diagnoser_submit_answer_tool import get_submit_answer_tool
|
||||
@@ -568,7 +570,7 @@ class Diagnoser:
|
||||
response = await self._invoke_model(llm, current_messages, context_components)
|
||||
|
||||
if response.content:
|
||||
outcome = response.content
|
||||
outcome = content_to_text(response.content)
|
||||
|
||||
if not response.tool_calls:
|
||||
break
|
||||
|
||||
@@ -23,7 +23,9 @@ Explorer tier the user configured. Tier behavior lives in
|
||||
ASK_EXPLORER_TOOL_NAME = "ask_explorer"
|
||||
|
||||
#: Model used when the LLM configuration carries no Explorer entry.
|
||||
DEFAULT_EXPLORER_MODEL = "gemini-3.8-flash"
|
||||
from artemis.config.settings import settings
|
||||
|
||||
DEFAULT_EXPLORER_MODEL = settings.OPENAI_MODEL
|
||||
|
||||
ASK_EXPLORER_DESCRIPTION = (
|
||||
"[EXPLORER] Ask the UI Explorer to locate elements on the current screen"
|
||||
|
||||
@@ -6,7 +6,7 @@
|
||||
// detect_objects, get_ocr_list, inspect_region, ask_image_processor.
|
||||
// submit_answer is always available.
|
||||
{
|
||||
"IDENTITY": "You are the agentic UI Explorer: given a phone screenshot and a query, you locate the matching UI element(s) and return their normalized [nx, ny] coordinates on a 0-1000 scale.",
|
||||
"IDENTITY": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。\n\nYou are the agentic UI Explorer: given a phone screenshot and a query, you locate the matching UI element(s) and return their normalized [nx, ny] coordinates on a 0-1000 scale.",
|
||||
"OPERATING PRINCIPLES": [
|
||||
"**Confidence-Driven Cost-Benefit Strategy**: Maximize localization accuracy while minimizing turns and tool calls. Every turn costs latency; stop exploring as soon as further calls would yield negligible accuracy gains.",
|
||||
"**Direct Submit (High Confidence)**: If the initial annotated screenshot and the marked UI element list already make the target coordinates obvious, call `submit_answer` immediately in the first turn.",
|
||||
|
||||
@@ -17,19 +17,16 @@
|
||||
The :class:`Explorer` class combines the following modules:
|
||||
|
||||
- ``tiers``: engine, tools and budgets for flash / pro / ultra.
|
||||
- ``tool_declarations``: static tool schemas (universal + native).
|
||||
- ``tool_declarations``: Responses function tool schemas.
|
||||
- ``perception_tools``: the ``exec_*`` perception/vision tool method group.
|
||||
- ``universal_runner``: the LangChain-based ``_run_universal`` loop.
|
||||
- ``run_setup``: flash mode and the pre-loop setup phases of ``run``.
|
||||
- ``native_runner``: the native Gemini reasoning loop of ``run``.
|
||||
"""
|
||||
|
||||
import json
|
||||
import os
|
||||
from typing import Any, Literal
|
||||
|
||||
from google import genai
|
||||
from google.genai import types
|
||||
|
||||
from artemis.agents.explorer.constants import DEFAULT_EXPLORER_MODEL
|
||||
from artemis.agents.explorer.geometry import (
|
||||
@@ -38,18 +35,16 @@ from artemis.agents.explorer.geometry import (
|
||||
norm_to_pixel,
|
||||
pixel_to_norm,
|
||||
)
|
||||
from artemis.agents.explorer.native_runner import NativeRunnerMixin
|
||||
from artemis.agents.explorer.perception_tools import PerceptionToolsMixin
|
||||
from artemis.agents.explorer.run_setup import RunSetupMixin
|
||||
from artemis.agents.explorer.screen_index import normalize_text, ScreenElement, ScreenIndex
|
||||
from artemis.agents.explorer.tiers import ExplorerTier, get_tier
|
||||
from artemis.agents.explorer.tool_declarations import NATIVE_EXPLORER_TOOL_DECLARATIONS
|
||||
from artemis.agents.explorer.tool_declarations import UNIVERSAL_EXPLORER_TOOLS
|
||||
from artemis.agents.explorer.universal_runner import UniversalRunnerMixin
|
||||
from artemis.config import settings
|
||||
from artemis.context import ArtemisContext
|
||||
from artemis.data_engine.trace import trace
|
||||
from artemis.graph.state import State
|
||||
from artemis.llm.google import is_gemini_model, strip_provider_prefix
|
||||
from artemis.utils.logger import get_logger
|
||||
from artemis.utils.ocr_api import is_ocr_configured
|
||||
|
||||
@@ -57,28 +52,25 @@ logger = get_logger(__name__)
|
||||
|
||||
__all__ = ["Explorer"]
|
||||
|
||||
#: Environment override for Gemini explicit context caching ("true" / "false").
|
||||
CACHING_ENV_VAR = "ARTEMIS_EXPLORER_CACHING"
|
||||
|
||||
#: How far outside its labeled element (as a fraction of the screen, per side)
|
||||
#: a candidate's point may fall and still inherit that element's bounds.
|
||||
LABEL_TOLERANCE_RATIO = 0.02
|
||||
|
||||
|
||||
class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, NativeRunnerMixin):
|
||||
class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin):
|
||||
"""Locates UI elements on a screenshot for the ``ask_explorer`` tool.
|
||||
|
||||
The tier (flash / pro / ultra) is applied per ``run`` call from
|
||||
:data:`artemis.agents.explorer.tiers.EXPLORER_TIERS`; it decides the
|
||||
engine, the turn budget, the exposed perception tools and the caching
|
||||
default. Construction is side-effect free: no network client is created
|
||||
until the native Gemini branch actually needs one.
|
||||
until the Responses execution loop needs one.
|
||||
"""
|
||||
|
||||
DENYLIST_TEMPLATE = (
|
||||
"\n# TOOL DENYLIST\n- The following tools are denylisted and cannot be used: {tools}\n"
|
||||
)
|
||||
TOOLS = NATIVE_EXPLORER_TOOL_DECLARATIONS
|
||||
TOOLS = UNIVERSAL_EXPLORER_TOOLS
|
||||
|
||||
def __init__(self, ctx: ArtemisContext):
|
||||
self.ctx = ctx
|
||||
@@ -107,7 +99,6 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
|
||||
self.denylisted_tools = self._compute_denylist(None)
|
||||
|
||||
self.model_name = self._resolve_model_name()
|
||||
self.use_native_gemini = self._detect_native_engine()
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# Engine selection (no side effects)
|
||||
@@ -119,37 +110,7 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
|
||||
llm_cfg = getattr(llm_config, "explorer", None) if llm_config else None
|
||||
model = getattr(llm_cfg, "model", None) if llm_cfg else None
|
||||
model_name = model if isinstance(model, str) and model else DEFAULT_EXPLORER_MODEL
|
||||
return strip_provider_prefix(model_name)
|
||||
|
||||
def _detect_native_engine(self) -> bool:
|
||||
"""True when the native google-genai SDK should drive the reasoning loop.
|
||||
|
||||
A Gemini model alone is not enough: without a Google API key (or a
|
||||
client already shared on the context) the SDK cannot be constructed,
|
||||
so such configurations run through the universal LangChain engine.
|
||||
"""
|
||||
if not is_gemini_model(self.model_name):
|
||||
return False
|
||||
if getattr(self.ctx, "_genai_client", None) is not None:
|
||||
return True
|
||||
api_key = getattr(settings, "GOOGLE_API_KEY", None)
|
||||
return bool(api_key and api_key.get_secret_value())
|
||||
|
||||
def _ensure_genai_client(self):
|
||||
"""Returns the context-level GenAI client, creating it on first use.
|
||||
|
||||
Only the native branch calls this, so universal configurations never
|
||||
touch ``genai`` (and never fail on a missing Google key). The client
|
||||
is stored on the context for connection pooling across agents.
|
||||
"""
|
||||
client = getattr(self.ctx, "_genai_client", None)
|
||||
if client is None:
|
||||
logger.info("Initializing GenAI client on context for connection pooling (Explorer)")
|
||||
api_key = getattr(settings, "GOOGLE_API_KEY", None)
|
||||
client = genai.Client(api_key=api_key.get_secret_value() if api_key else None)
|
||||
self.ctx._genai_client = client
|
||||
self.client = client
|
||||
return client
|
||||
return model_name
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# Tier application
|
||||
@@ -180,32 +141,6 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
|
||||
self.max_turns = tier.max_turns
|
||||
self.denylisted_tools = self._compute_denylist(tier)
|
||||
|
||||
def _resolve_caching(self, tier: ExplorerTier, enable_caching: bool | None) -> bool:
|
||||
"""Resolves Gemini explicit context caching for this run.
|
||||
|
||||
Precedence: explicit argument, ``ARTEMIS_EXPLORER_CACHING`` env
|
||||
override, the agent configuration, the execution setup, the global
|
||||
settings, then the tier default. Configuration values are only
|
||||
trusted when they are real booleans so a mocked or malformed config
|
||||
(``None``, sentinel objects) falls through instead of forcing a value.
|
||||
"""
|
||||
if isinstance(enable_caching, bool):
|
||||
return enable_caching
|
||||
env_value = os.getenv(CACHING_ENV_VAR, "").strip().lower()
|
||||
if env_value in ("true", "false"):
|
||||
return env_value == "true"
|
||||
for source in (
|
||||
getattr(self.ctx, "agent_config", None),
|
||||
getattr(self.ctx, "execution_setup", None),
|
||||
):
|
||||
value = getattr(getattr(source, "explorer", None), "caching", None)
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
value = getattr(settings, "EXPLORER_CACHING", None)
|
||||
if isinstance(value, bool):
|
||||
return value
|
||||
return tier.caching
|
||||
|
||||
def _hidden_tool_names(self) -> set[str]:
|
||||
"""Denylisted tools plus OCR when it is unconfigured, evaluated at call time.
|
||||
|
||||
@@ -218,34 +153,14 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
|
||||
hidden.add("get_ocr_list")
|
||||
return hidden
|
||||
|
||||
def get_exposed_tools(self, only_submit: bool = False) -> list[types.FunctionDeclaration]:
|
||||
def get_exposed_tools(self, only_submit: bool = False) -> list[dict[str, Any]]:
|
||||
"""Native tool declarations the model may see for the current tier."""
|
||||
hidden = self._hidden_tool_names()
|
||||
tools = [tool for tool in self.TOOLS if tool.name not in hidden]
|
||||
tools = [tool for tool in self.TOOLS if tool["function"]["name"] not in hidden]
|
||||
if only_submit:
|
||||
tools = [tool for tool in tools if tool.name == "submit_answer"]
|
||||
tools = [tool for tool in tools if tool["function"]["name"] == "submit_answer"]
|
||||
return tools
|
||||
|
||||
def _prune_historical_images(self, contents, keep_last=1):
|
||||
image_parts = []
|
||||
for content in contents:
|
||||
for part in content.parts:
|
||||
if getattr(part, "inline_data", None) or getattr(part, "file_data", None):
|
||||
image_parts.append(part)
|
||||
if len(image_parts) <= keep_last:
|
||||
return
|
||||
to_keep_ids = {id(p) for p in image_parts[-keep_last:]}
|
||||
for content in contents:
|
||||
content.parts = [
|
||||
types.Part.from_text(
|
||||
text=("[Image pruned to maintain visual focus on latest state]")
|
||||
)
|
||||
if (getattr(p, "inline_data", None) or getattr(p, "file_data", None))
|
||||
and id(p) not in to_keep_ids
|
||||
else p
|
||||
for p in content.parts
|
||||
]
|
||||
|
||||
# ------------------------------------------------------------------ #
|
||||
# Label bounds and the structural pre-pass
|
||||
# ------------------------------------------------------------------ #
|
||||
@@ -434,30 +349,13 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
|
||||
fused_xml, screenshot_path, minimal_list, self.image_name
|
||||
)
|
||||
|
||||
if not self.use_native_gemini:
|
||||
return await self._run_universal(
|
||||
query=query,
|
||||
context_feedback=context_feedback,
|
||||
screenshot_path=img_to_read,
|
||||
state=state,
|
||||
minimal_list=minimal_list,
|
||||
prompt_template=prompt_template,
|
||||
max_turns=self.max_turns,
|
||||
)
|
||||
|
||||
client = self._ensure_genai_client()
|
||||
model_name, temperature, thinking_level, _fallback_model = self._resolve_model_params()
|
||||
return await self._run_native(
|
||||
client=client,
|
||||
return await self._run_universal(
|
||||
query=query,
|
||||
context_feedback=context_feedback,
|
||||
screenshot_path=img_to_read,
|
||||
state=state,
|
||||
minimal_list=minimal_list,
|
||||
img_to_read=img_to_read,
|
||||
enable_caching=self._resolve_caching(tier, enable_caching),
|
||||
prompt_template=prompt_template,
|
||||
model_name=model_name,
|
||||
temperature=temperature,
|
||||
thinking_level=thinking_level,
|
||||
max_turns=self.max_turns,
|
||||
)
|
||||
|
||||
|
||||
@@ -1,878 +0,0 @@
|
||||
# Copyright 2026 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Native Gemini execution loop of ``Explorer.run``.
|
||||
|
||||
Split out of ``artemis.agents.explorer.explorer``: the native-SDK reasoning
|
||||
loop and its named phases (model invocation, submit_answer interception and
|
||||
validation, tool dispatching, failure mapping, resource cleanup), packaged as
|
||||
a mixin consumed by ``Explorer``. Every google-genai call goes through the
|
||||
SDK's ``client.aio`` surface so uploads, cache management and generation
|
||||
never block the event loop. This module also owns
|
||||
:func:`_generate_content_with_reliability`, the shared reliability wrapper
|
||||
for native google-genai model calls.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from collections.abc import Awaitable, Callable
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
from typing import TYPE_CHECKING, Any
|
||||
|
||||
from google.genai import types
|
||||
|
||||
from artemis.agents.explorer.geometry import is_valid_norm_point
|
||||
from artemis.agents.explorer.tiers import SUBMIT_TOOL
|
||||
from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
|
||||
from artemis.data_engine.trace import TraceSpan
|
||||
from artemis.llm.reliability import (
|
||||
LLMExhaustedError,
|
||||
LLMPermanentError,
|
||||
classify_failure,
|
||||
retry_policy_for,
|
||||
)
|
||||
from artemis.services.llm import _record_llm_event, _record_llm_retry
|
||||
from artemis.utils.logger import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
#: Loads a local image as a ``Part`` (inline bytes or a File API reference).
|
||||
ImagePartGetter = Callable[[str], Awaitable[types.Part]]
|
||||
|
||||
|
||||
async def _generate_content_with_reliability(operation, *, label: str = "Explorer model call"):
|
||||
"""Run one native google-genai model call under the shared reliability layer.
|
||||
|
||||
``operation`` must return a fresh awaitable on each call. Non-retryable
|
||||
failures raise ``LLMPermanentError``; exhausted retries raise
|
||||
``LLMExhaustedError`` according to the shared category policy.
|
||||
"""
|
||||
attempt = 0
|
||||
while True:
|
||||
try:
|
||||
return await operation()
|
||||
except Exception as call_err:
|
||||
attempt += 1
|
||||
failure = classify_failure(call_err)
|
||||
if not failure.retryable:
|
||||
logger.error(f"{label} permanently failed [{failure.category.value}]: {call_err}")
|
||||
_record_llm_event(
|
||||
"llm_gave_up",
|
||||
{
|
||||
"source": "explorer",
|
||||
"error": str(call_err)[:1000],
|
||||
"category": failure.category.value,
|
||||
"retryable": False,
|
||||
},
|
||||
status="failed",
|
||||
)
|
||||
raise LLMPermanentError(
|
||||
f"{label} failed [{failure.category.value}]: {call_err}",
|
||||
failure=failure,
|
||||
cause=call_err,
|
||||
) from call_err
|
||||
policy = retry_policy_for(failure.category)
|
||||
if attempt >= policy.max_attempts:
|
||||
logger.error(
|
||||
f"{label} exhausted {attempt} attempt(s) [{failure.category.value}]: {call_err}"
|
||||
)
|
||||
_record_llm_event(
|
||||
"llm_gave_up",
|
||||
{
|
||||
"source": "explorer",
|
||||
"error": str(call_err)[:1000],
|
||||
"category": failure.category.value,
|
||||
"retryable": True,
|
||||
"attempts": attempt,
|
||||
},
|
||||
status="failed",
|
||||
)
|
||||
raise LLMExhaustedError(
|
||||
f"{label} exhausted {attempt} attempt(s)"
|
||||
f" [{failure.category.value}]: {call_err}",
|
||||
failure=failure,
|
||||
cause=call_err,
|
||||
) from call_err
|
||||
delay = policy.delay_for(attempt)
|
||||
logger.warning(
|
||||
f"{label} failed [{failure.category.value}] on attempt"
|
||||
f" {attempt}/{policy.max_attempts}: {call_err}."
|
||||
f" Retrying in {delay:.2f}s..."
|
||||
)
|
||||
_record_llm_retry(
|
||||
str(call_err),
|
||||
delay,
|
||||
attempt=attempt,
|
||||
max_retries=policy.max_attempts,
|
||||
source="explorer",
|
||||
)
|
||||
await asyncio.sleep(delay)
|
||||
|
||||
|
||||
def _bare_tool_name(name: str) -> str:
|
||||
"""Strips a namespace prefix (``ns:tool``) some models add to tool names."""
|
||||
return name.split(":")[-1] if ":" in name else name
|
||||
|
||||
|
||||
class NativeRunnerMixin:
|
||||
"""Native Gemini SDK reasoning loop of :class:`Explorer`."""
|
||||
|
||||
# This mixin is composed into ``Explorer``. Declaring the attributes and
|
||||
# sibling-mixin methods it consumes makes that host contract visible to
|
||||
# static analysis without adding runtime shims or changing MRO behavior.
|
||||
if TYPE_CHECKING:
|
||||
from artemis.context import ArtemisContext
|
||||
|
||||
ctx: ArtemisContext
|
||||
denylisted_tools: set[str]
|
||||
turn_latencies: list[float]
|
||||
turn_cached_tokens: list[int]
|
||||
trace_history: list[dict[str, Any]]
|
||||
|
||||
def get_exposed_tools(
|
||||
self, only_submit: bool = False
|
||||
) -> list[types.FunctionDeclaration]: ...
|
||||
|
||||
def _prune_historical_images(self, contents: list, keep_last: int = 1) -> None: ...
|
||||
|
||||
def _enrich_candidates(self, candidates: list[Any]) -> list[Any]: ...
|
||||
|
||||
async def exec_ask_perception_tool(
|
||||
self,
|
||||
search_query: str,
|
||||
nx: int,
|
||||
ny: int,
|
||||
detect_queries: list[str],
|
||||
) -> dict[str, Any]: ...
|
||||
|
||||
async def exec_detect_objects(
|
||||
self, queries: list[str], target_image_id: str = "img_0"
|
||||
) -> dict[str, Any]: ...
|
||||
|
||||
async def exec_ask_image_processor(
|
||||
self, instruction: str, target_image_id: str = "img_0"
|
||||
) -> dict[str, Any]: ...
|
||||
|
||||
async def exec_get_ocr_list(self) -> dict[str, Any]: ...
|
||||
|
||||
async def exec_inspect_region(
|
||||
self,
|
||||
x_min: int,
|
||||
y_min: int,
|
||||
x_max: int,
|
||||
y_max: int,
|
||||
zoom_factor: float = 2.0,
|
||||
) -> dict[str, Any]: ...
|
||||
|
||||
def _make_image_part_getter(
|
||||
self, client, use_file_api: bool, uploaded_files: list
|
||||
) -> ImagePartGetter:
|
||||
"""Builds the image-part loader used for screenshots and tool images."""
|
||||
|
||||
async def get_image_part(file_path: str) -> types.Part:
|
||||
if use_file_api:
|
||||
file_ref = await client.aio.files.upload(file=file_path)
|
||||
uploaded_files.append(file_ref)
|
||||
return types.Part(
|
||||
file_data=types.FileData(
|
||||
file_uri=file_ref.uri,
|
||||
mime_type=file_ref.mime_type or "image/jpeg",
|
||||
)
|
||||
)
|
||||
with open(file_path, "rb") as f:
|
||||
img_bytes = f.read()
|
||||
return types.Part.from_bytes(data=img_bytes, mime_type="image/jpeg")
|
||||
|
||||
return get_image_part
|
||||
|
||||
def _build_initial_contents(
|
||||
self, query: str, context_feedback: str, minimal_list: str, initial_part: types.Part
|
||||
) -> list[types.Content]:
|
||||
"""Builds the initial user content with the operator request and screenshot."""
|
||||
return [
|
||||
types.Content(
|
||||
role="user",
|
||||
parts=[
|
||||
types.Part.from_text(
|
||||
text=f"""Operator Request:
|
||||
- Query: {query}
|
||||
- Context Feedback: {context_feedback}
|
||||
|
||||
Initial marked UI elements list (corresponding to numbers ①, ②... in the image):
|
||||
{minimal_list}
|
||||
"""
|
||||
),
|
||||
initial_part,
|
||||
],
|
||||
)
|
||||
]
|
||||
|
||||
async def _create_cache_resource(
|
||||
self, client, model_name: str, prompt_template: str, contents: list
|
||||
):
|
||||
"""Creates the explicit cache resource; returns None on failure."""
|
||||
try:
|
||||
logger.info("Creating cache resource for Explorer...")
|
||||
cached_content = await client.aio.caches.create(
|
||||
model=model_name,
|
||||
config=types.CreateCachedContentConfig(
|
||||
contents=[contents[0]],
|
||||
system_instruction=prompt_template,
|
||||
tools=[types.Tool(function_declarations=self.get_exposed_tools())],
|
||||
tool_config=types.ToolConfig(
|
||||
function_calling_config=types.FunctionCallingConfig(
|
||||
mode=types.FunctionCallingConfigMode.ANY
|
||||
),
|
||||
include_server_side_tool_invocations=True,
|
||||
),
|
||||
ttl="300s",
|
||||
),
|
||||
)
|
||||
logger.info(f"Cache resource created successfully: {cached_content.name}")
|
||||
return cached_content
|
||||
except Exception as cache_err:
|
||||
logger.error(f"Failed to create cache resource: {cache_err}")
|
||||
return None
|
||||
|
||||
async def _run_native(
|
||||
self,
|
||||
*,
|
||||
client,
|
||||
query: str,
|
||||
context_feedback: str,
|
||||
minimal_list: str,
|
||||
img_to_read: str,
|
||||
enable_caching: bool,
|
||||
prompt_template: str,
|
||||
model_name: str,
|
||||
temperature,
|
||||
thinking_level,
|
||||
max_turns: int,
|
||||
) -> str:
|
||||
"""Runs the native Gemini path: cache setup, reasoning loop, cleanup."""
|
||||
use_file_api = os.getenv("ARTEMIS_USE_FILE_API", "false").lower() == "true"
|
||||
uploaded_files: list = []
|
||||
get_image_part = self._make_image_part_getter(client, use_file_api, uploaded_files)
|
||||
|
||||
cached_content = None
|
||||
try:
|
||||
initial_part = await get_image_part(img_to_read)
|
||||
contents = self._build_initial_contents(
|
||||
query, context_feedback, minimal_list, initial_part
|
||||
)
|
||||
|
||||
if enable_caching:
|
||||
cached_content = await self._create_cache_resource(
|
||||
client, model_name, prompt_template, contents
|
||||
)
|
||||
|
||||
agent_outcome = await self._native_loop(
|
||||
client=client,
|
||||
contents=contents,
|
||||
get_image_part=get_image_part,
|
||||
cached_content=cached_content,
|
||||
prompt_template=prompt_template,
|
||||
model_name=model_name,
|
||||
temperature=temperature,
|
||||
thinking_level=thinking_level,
|
||||
max_turns=max_turns,
|
||||
)
|
||||
|
||||
except Exception as e:
|
||||
agent_outcome = self._format_native_failure(e, model_name)
|
||||
finally:
|
||||
await self._cleanup_native_resources(client, cached_content, uploaded_files)
|
||||
|
||||
return agent_outcome
|
||||
|
||||
async def _native_loop(
|
||||
self,
|
||||
*,
|
||||
client,
|
||||
contents: list,
|
||||
get_image_part: ImagePartGetter,
|
||||
cached_content,
|
||||
prompt_template: str,
|
||||
model_name: str,
|
||||
temperature,
|
||||
thinking_level,
|
||||
max_turns: int,
|
||||
) -> str:
|
||||
"""Main native reasoning loop: one model turn per iteration."""
|
||||
turn = 0
|
||||
agent_outcome = ""
|
||||
self.turn_latencies = []
|
||||
self.turn_cached_tokens = []
|
||||
self.trace_history = []
|
||||
|
||||
while turn < max_turns:
|
||||
turn += 1
|
||||
logger.info(f"Turn {turn}/{max_turns}: Invoking Native Gemini SDK for Explorer...")
|
||||
|
||||
self._prune_historical_images(contents, keep_last=1)
|
||||
|
||||
is_final_turn = turn == max_turns
|
||||
if is_final_turn:
|
||||
warning_msg = (
|
||||
"\n[WARNING] This is your final iteration. You MUST"
|
||||
" call 'submit_answer' to submit your final result. No"
|
||||
" other tools are available."
|
||||
)
|
||||
contents.append(
|
||||
types.Content(
|
||||
role="user",
|
||||
parts=[types.Part.from_text(text=warning_msg)],
|
||||
)
|
||||
)
|
||||
|
||||
response, thinking_parts, text_parts = await self._invoke_native_model(
|
||||
client=client,
|
||||
contents=contents,
|
||||
cached_content=cached_content,
|
||||
is_final_turn=is_final_turn,
|
||||
prompt_template=prompt_template,
|
||||
model_name=model_name,
|
||||
temperature=temperature,
|
||||
thinking_level=thinking_level,
|
||||
)
|
||||
|
||||
turn_record = {
|
||||
"iteration": turn,
|
||||
"thoughts": ("\n".join(thinking_parts) if thinking_parts else ""),
|
||||
"tool_calls": [],
|
||||
}
|
||||
|
||||
function_calls = response.function_calls
|
||||
if not function_calls:
|
||||
self._handle_missing_tool_calls(
|
||||
contents, response, turn_record, text_parts, max_turns, turn
|
||||
)
|
||||
continue
|
||||
|
||||
# Record the model's function call request in history
|
||||
contents.append(self._model_content(response, function_calls))
|
||||
|
||||
tool_response_parts: list[types.Part] = []
|
||||
|
||||
# Intercept and validate submit_answer to manage the task lifecycle
|
||||
submit_outcome = self._process_submit_answer(
|
||||
function_calls, turn_record, tool_response_parts
|
||||
)
|
||||
if submit_outcome is not None:
|
||||
agent_outcome = submit_outcome
|
||||
break
|
||||
|
||||
await self._dispatch_tool_calls(
|
||||
function_calls, get_image_part, turn_record, tool_response_parts
|
||||
)
|
||||
|
||||
if tool_response_parts:
|
||||
# Native Gemini SDK allows the 'user' role to return mixed parts
|
||||
contents.append(types.Content(role="user", parts=tool_response_parts))
|
||||
|
||||
self.trace_history.append(turn_record)
|
||||
|
||||
if turn >= max_turns and not agent_outcome:
|
||||
agent_outcome = (
|
||||
"Error: Explorer reached maximum iterations without a conclusive answer."
|
||||
)
|
||||
|
||||
return agent_outcome
|
||||
|
||||
@staticmethod
|
||||
def _model_content(response, function_calls: list) -> types.Content:
|
||||
"""The model turn to record: the SDK's content when present, else a rebuild."""
|
||||
candidates = getattr(response, "candidates", None)
|
||||
if candidates:
|
||||
content = getattr(candidates[0], "content", None)
|
||||
if content:
|
||||
return content
|
||||
return types.Content(
|
||||
role="model",
|
||||
parts=[types.Part(function_call=fc) for fc in function_calls],
|
||||
)
|
||||
|
||||
def _build_generate_config(
|
||||
self,
|
||||
*,
|
||||
cached_content,
|
||||
is_final_turn: bool,
|
||||
prompt_template: str,
|
||||
temperature,
|
||||
thinking_level,
|
||||
) -> types.GenerateContentConfig:
|
||||
"""Builds the per-turn generation config (cached vs full tool config)."""
|
||||
thinking_config = (
|
||||
types.ThinkingConfig(thinking_level=thinking_level) if thinking_level else None
|
||||
)
|
||||
if cached_content and not is_final_turn:
|
||||
return types.GenerateContentConfig(
|
||||
cached_content=cached_content.name,
|
||||
temperature=temperature,
|
||||
safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
|
||||
thinking_config=thinking_config,
|
||||
)
|
||||
return types.GenerateContentConfig(
|
||||
system_instruction=prompt_template,
|
||||
temperature=temperature,
|
||||
safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
|
||||
tools=[
|
||||
types.Tool(function_declarations=self.get_exposed_tools(only_submit=is_final_turn))
|
||||
],
|
||||
tool_config=types.ToolConfig(
|
||||
function_calling_config=types.FunctionCallingConfig(
|
||||
mode=types.FunctionCallingConfigMode.ANY
|
||||
),
|
||||
include_server_side_tool_invocations=True,
|
||||
),
|
||||
thinking_config=thinking_config,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _extract_response_texts(response) -> tuple[list[str], list[str]]:
|
||||
"""Extracts model thoughts and plain text parts from a response."""
|
||||
thinking_parts = []
|
||||
text_parts = []
|
||||
candidates = getattr(response, "candidates", None)
|
||||
if not candidates and isinstance(response, dict):
|
||||
candidates = response.get("candidates")
|
||||
|
||||
if candidates and len(candidates) > 0:
|
||||
candidate = candidates[0]
|
||||
content = getattr(candidate, "content", None)
|
||||
if not content and isinstance(candidate, dict):
|
||||
content = candidate.get("content")
|
||||
|
||||
parts = getattr(content, "parts", None)
|
||||
if not parts and isinstance(content, dict):
|
||||
parts = content.get("parts")
|
||||
|
||||
if parts:
|
||||
for part in parts:
|
||||
is_thought = False
|
||||
part_text = None
|
||||
if isinstance(part, dict):
|
||||
is_thought = part.get("thought", False)
|
||||
part_text = part.get("text", None)
|
||||
else:
|
||||
is_thought = getattr(part, "thought", False)
|
||||
part_text = getattr(part, "text", None)
|
||||
|
||||
if is_thought and part_text:
|
||||
thinking_parts.append(part_text)
|
||||
elif part_text:
|
||||
text_parts.append(part_text)
|
||||
return thinking_parts, text_parts
|
||||
|
||||
async def _invoke_native_model(
|
||||
self,
|
||||
*,
|
||||
client,
|
||||
contents: list,
|
||||
cached_content,
|
||||
is_final_turn: bool,
|
||||
prompt_template: str,
|
||||
model_name: str,
|
||||
temperature,
|
||||
thinking_level,
|
||||
):
|
||||
"""Invokes the model for one turn under a trace span; returns response and texts."""
|
||||
ctx = self.ctx
|
||||
start_turn = time.perf_counter()
|
||||
with TraceSpan(name="gemini_explorer_call", ctx=ctx) as span:
|
||||
generate_config = self._build_generate_config(
|
||||
cached_content=cached_content,
|
||||
is_final_turn=is_final_turn,
|
||||
prompt_template=prompt_template,
|
||||
temperature=temperature,
|
||||
thinking_level=thinking_level,
|
||||
)
|
||||
|
||||
response = await _generate_content_with_reliability(
|
||||
lambda: asyncio.wait_for(
|
||||
client.aio.models.generate_content(
|
||||
model=model_name,
|
||||
contents=contents,
|
||||
config=generate_config,
|
||||
),
|
||||
timeout=180,
|
||||
),
|
||||
)
|
||||
end_turn = time.perf_counter()
|
||||
turn_dur = end_turn - start_turn
|
||||
self.turn_latencies.append(turn_dur)
|
||||
|
||||
cached_tokens = 0
|
||||
if getattr(response, "usage_metadata", None) and getattr(
|
||||
response.usage_metadata,
|
||||
"cached_content_token_count",
|
||||
None,
|
||||
):
|
||||
cached_tokens = response.usage_metadata.cached_content_token_count
|
||||
self.turn_cached_tokens.append(cached_tokens)
|
||||
span.result = (
|
||||
f"Function calls: {len(response.function_calls)}"
|
||||
if response.function_calls
|
||||
else "Final answer"
|
||||
)
|
||||
# Extract and log model thoughts and text if present
|
||||
thinking_parts, text_parts = self._extract_response_texts(response)
|
||||
if thinking_parts:
|
||||
span.payload["explorer_thought"] = "\n".join(thinking_parts)
|
||||
if text_parts:
|
||||
span.payload["explorer_text"] = "\n".join(text_parts)
|
||||
|
||||
return response, thinking_parts, text_parts
|
||||
|
||||
def _handle_missing_tool_calls(
|
||||
self,
|
||||
contents: list,
|
||||
response,
|
||||
turn_record: dict,
|
||||
text_parts: list[str],
|
||||
max_turns: int,
|
||||
turn: int,
|
||||
) -> None:
|
||||
"""Handles a turn where the model produced plain text instead of tool calls.
|
||||
|
||||
Responses without candidates (safety blocks, empty replies) are
|
||||
recorded as a synthetic model text part so the conversation history
|
||||
stays well-formed for the retry.
|
||||
"""
|
||||
logger.warning(f"Explorer turn {turn}: Model hallucinated plain text. Forcing retry.")
|
||||
model_text = "\n".join(text_parts) if text_parts else ""
|
||||
turn_record["tool_calls"].append(
|
||||
{
|
||||
"name": "hallucinated_plain_text",
|
||||
"args": {"text": model_text},
|
||||
"response": {
|
||||
"error": (
|
||||
"Forced retry because model failed to call submit_answer or any other tool"
|
||||
)
|
||||
},
|
||||
}
|
||||
)
|
||||
self.trace_history.append(turn_record)
|
||||
|
||||
candidates = getattr(response, "candidates", None)
|
||||
model_content = getattr(candidates[0], "content", None) if candidates else None
|
||||
if not model_content:
|
||||
model_content = types.Content(
|
||||
role="model",
|
||||
parts=[types.Part.from_text(text=model_text or "(no response)")],
|
||||
)
|
||||
contents.append(model_content)
|
||||
contents.append(
|
||||
types.Content(
|
||||
role="user",
|
||||
parts=[
|
||||
types.Part.from_text(
|
||||
text=(
|
||||
"You have not called 'submit_answer'"
|
||||
" to submit your final answer yet. If"
|
||||
" you are not certain about the final"
|
||||
" answer, you can continue exploring"
|
||||
f" {max_turns - turn} more"
|
||||
" time(s)."
|
||||
)
|
||||
)
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _validate_submit_args(function_calls: list, args: dict, candidates: list) -> list[str]:
|
||||
"""Validates a submit_answer call; returns the list of validation errors."""
|
||||
errors = []
|
||||
|
||||
if len(function_calls) > 1:
|
||||
errors.append(
|
||||
"The tool 'submit_answer' MUST be called"
|
||||
" individually in your final turn without any other"
|
||||
" parallel tool calls. Please remove other tool"
|
||||
" calls and re-submit."
|
||||
)
|
||||
|
||||
fallback_message = args.get("fallback_message", "")
|
||||
if not candidates and not fallback_message:
|
||||
errors.append(
|
||||
"The candidates list is empty. You must provide at"
|
||||
" least one candidate matching the query, or"
|
||||
" explain observations via fallback_message."
|
||||
)
|
||||
|
||||
for i, cand in enumerate(candidates):
|
||||
label = cand.get("label")
|
||||
coords = cand.get("coords")
|
||||
if not label:
|
||||
errors.append(f"Candidate at index {i} is missing a 'label'.")
|
||||
if not coords or not isinstance(coords, list) or len(coords) != 2:
|
||||
errors.append(
|
||||
f"Candidate '{label or i}' must have 'coords'"
|
||||
" as an array of exactly 2 integers `[nx,"
|
||||
" ny]`."
|
||||
)
|
||||
elif not is_valid_norm_point(coords):
|
||||
try:
|
||||
nx, ny = int(coords[0]), int(coords[1])
|
||||
except (ValueError, TypeError):
|
||||
errors.append(f"Candidate '{label or i}' coordinates are invalid integers.")
|
||||
else:
|
||||
errors.append(
|
||||
f"Candidate '{label or i}' coordinates"
|
||||
f" `[{nx}, {ny}]` must strictly be in"
|
||||
" the `[0-1000]` normalized scale"
|
||||
" range inclusive."
|
||||
)
|
||||
|
||||
return errors
|
||||
|
||||
def _process_submit_answer(
|
||||
self, function_calls: list, turn_record: dict, tool_response_parts: list
|
||||
) -> str | None:
|
||||
"""Intercepts submit_answer; returns the final outcome JSON when valid.
|
||||
|
||||
On validation failure the error is fed back as a tool response and
|
||||
None is returned so the ReAct loop can continue and self-correct.
|
||||
"""
|
||||
submit_call = next(
|
||||
(fc for fc in function_calls if _bare_tool_name(fc.name) == SUBMIT_TOOL),
|
||||
None,
|
||||
)
|
||||
if not submit_call:
|
||||
return None
|
||||
|
||||
args = submit_call.args or {}
|
||||
candidates = args.get("candidates", [])
|
||||
errors = self._validate_submit_args(function_calls, args, candidates)
|
||||
|
||||
if errors:
|
||||
logger.warning(f"Explorer submit_answer validation failed: {errors}")
|
||||
turn_record["tool_calls"].append(
|
||||
{
|
||||
"name": SUBMIT_TOOL,
|
||||
"args": args,
|
||||
"response": {"error": ("Validation Failed:\n" + "\n".join(errors))},
|
||||
}
|
||||
)
|
||||
tool_response_parts.append(
|
||||
types.Part.from_function_response(
|
||||
name=SUBMIT_TOOL,
|
||||
response={
|
||||
"error": (
|
||||
"Validation Failed:\n"
|
||||
+ "\n".join(errors)
|
||||
+ "\nPlease correct these formatting"
|
||||
" errors and re-submit using"
|
||||
" submit_answer."
|
||||
)
|
||||
},
|
||||
)
|
||||
)
|
||||
# Allow the ReAct loop to continue so the model can self-correct
|
||||
return None
|
||||
|
||||
logger.info("Explorer submit_answer validation passed successfully!")
|
||||
# Candidates submitted by a registered label inherit that element's
|
||||
# bounds; the rest of the model's arguments are passed through as is.
|
||||
outcome_args = dict(args)
|
||||
outcome_args["candidates"] = self._enrich_candidates(candidates)
|
||||
agent_outcome = json.dumps(outcome_args, ensure_ascii=False)
|
||||
turn_record["tool_calls"].append(
|
||||
{"name": SUBMIT_TOOL, "args": args, "response": {"result": "success"}}
|
||||
)
|
||||
self.trace_history.append(turn_record)
|
||||
return agent_outcome
|
||||
|
||||
@staticmethod
|
||||
async def _append_tool_response_with_images(
|
||||
name: str,
|
||||
res: dict,
|
||||
get_image_part: ImagePartGetter,
|
||||
tool_response_parts: list,
|
||||
*,
|
||||
result_key: str = "result",
|
||||
) -> None:
|
||||
"""Appends a tool result part plus every annotated image it produced.
|
||||
|
||||
Image loading failures are logged and skipped: the textual result is
|
||||
still delivered, so one unreadable annotation cannot sink the turn.
|
||||
"""
|
||||
tool_response_parts.append(
|
||||
types.Part.from_function_response(name=name, response={result_key: res.get("text")})
|
||||
)
|
||||
image_paths = [p for p in [res.get("image_path"), *(res.get("image_paths") or [])] if p]
|
||||
for img_p in image_paths:
|
||||
try:
|
||||
tool_response_parts.append(await get_image_part(img_p))
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to load image response part for {img_p}: {e}")
|
||||
|
||||
async def _execute_native_tool(
|
||||
self,
|
||||
name: str,
|
||||
args: dict,
|
||||
get_image_part: ImagePartGetter,
|
||||
tool_call_trace: dict,
|
||||
tool_response_parts: list,
|
||||
) -> None:
|
||||
"""Executes one non-submit tool call and appends its response parts."""
|
||||
if name == "ask_perception_tool":
|
||||
search_query = args.get("search_query")
|
||||
nx = args.get("nx")
|
||||
ny = args.get("ny")
|
||||
detect_queries = args.get("detect_queries")
|
||||
res = await self.exec_ask_perception_tool(
|
||||
search_query=search_query if isinstance(search_query, str) else "",
|
||||
nx=nx if isinstance(nx, int) else -1,
|
||||
ny=ny if isinstance(ny, int) else -1,
|
||||
detect_queries=detect_queries if isinstance(detect_queries, list) else [],
|
||||
)
|
||||
tool_call_trace["response"] = res
|
||||
await self._append_tool_response_with_images(
|
||||
name, res, get_image_part, tool_response_parts, result_key="text"
|
||||
)
|
||||
elif name == "detect_objects":
|
||||
res = await self.exec_detect_objects(
|
||||
queries=args.get("queries") or [],
|
||||
target_image_id=args.get("target_image_id", "img_0"),
|
||||
)
|
||||
tool_call_trace["response"] = res
|
||||
await self._append_tool_response_with_images(
|
||||
name, res, get_image_part, tool_response_parts
|
||||
)
|
||||
elif name == "ask_image_processor":
|
||||
res = await self.exec_ask_image_processor(
|
||||
instruction=args.get("instruction") or "",
|
||||
target_image_id=args.get("target_image_id", "img_0"),
|
||||
)
|
||||
tool_call_trace["response"] = res
|
||||
await self._append_tool_response_with_images(
|
||||
name, res, get_image_part, tool_response_parts
|
||||
)
|
||||
elif name == "get_ocr_list":
|
||||
res = await self.exec_get_ocr_list()
|
||||
tool_call_trace["response"] = res
|
||||
await self._append_tool_response_with_images(
|
||||
name, res, get_image_part, tool_response_parts
|
||||
)
|
||||
elif name == "inspect_region":
|
||||
res = await self.exec_inspect_region(
|
||||
x_min=args["x_min"],
|
||||
y_min=args["y_min"],
|
||||
x_max=args["x_max"],
|
||||
y_max=args["y_max"],
|
||||
zoom_factor=args.get("zoom_factor", 2.0),
|
||||
)
|
||||
tool_call_trace["response"] = res
|
||||
await self._append_tool_response_with_images(
|
||||
name, res, get_image_part, tool_response_parts
|
||||
)
|
||||
else:
|
||||
tool_call_trace["response"] = {"error": f"Tool {name} not found"}
|
||||
tool_response_parts.append(
|
||||
types.Part.from_function_response(
|
||||
name=name,
|
||||
response={"error": f"Tool {name} not found"},
|
||||
)
|
||||
)
|
||||
|
||||
async def _dispatch_tool_calls(
|
||||
self,
|
||||
function_calls: list,
|
||||
get_image_part: ImagePartGetter,
|
||||
turn_record: dict,
|
||||
tool_response_parts: list,
|
||||
) -> None:
|
||||
"""Dispatches all non-submit tool calls sequentially with tracing.
|
||||
|
||||
Denylisted names are refused here as well, even though they are
|
||||
absent from the declarations, because models occasionally call tools
|
||||
they were never offered.
|
||||
"""
|
||||
for fc in function_calls:
|
||||
name = _bare_tool_name(fc.name)
|
||||
if name == SUBMIT_TOOL:
|
||||
continue
|
||||
args = fc.args or {}
|
||||
tool_call_trace = {"name": name, "args": args}
|
||||
if name in self.denylisted_tools:
|
||||
logger.warning(
|
||||
f"Explorer attempted to call denylisted tool '{name}'. Blocking execution."
|
||||
)
|
||||
error = f"Tool '{name}' is denylisted and unavailable."
|
||||
tool_call_trace["response"] = {"error": error}
|
||||
turn_record["tool_calls"].append(tool_call_trace)
|
||||
tool_response_parts.append(
|
||||
types.Part.from_function_response(name=fc.name, response={"error": error})
|
||||
)
|
||||
continue
|
||||
|
||||
logger.info(f"Explorer executing tool '{name}' sequentially...")
|
||||
|
||||
try:
|
||||
await self._execute_native_tool(
|
||||
name, args, get_image_part, tool_call_trace, tool_response_parts
|
||||
)
|
||||
except Exception as e:
|
||||
logger.error(f"Explorer tool {name} execution failed: {e}")
|
||||
tool_call_trace["response"] = {"error": str(e)}
|
||||
tool_response_parts.append(
|
||||
types.Part.from_function_response(name=name, response={"error": str(e)})
|
||||
)
|
||||
finally:
|
||||
turn_record["tool_calls"].append(tool_call_trace)
|
||||
|
||||
def _format_native_failure(self, e: Exception, model_name: str) -> str:
|
||||
"""Maps a loop failure to the user-facing fallback outcome JSON."""
|
||||
logger.error(f"Explorer execution loop failed: {e}")
|
||||
|
||||
err_msg = str(e)
|
||||
if "preempted" in err_msg.lower():
|
||||
clean_msg = (
|
||||
"Model request was preempted by the server due to high"
|
||||
" demand. Please try again later."
|
||||
)
|
||||
elif "overloaded" in err_msg.lower() or "503" in err_msg:
|
||||
clean_msg = "Model service is currently overloaded. Please try again later."
|
||||
elif "quota" in err_msg.lower() or "429" in err_msg:
|
||||
clean_msg = "Model API quota exceeded or rate limited."
|
||||
elif "thinkingconfig" in err_msg.lower():
|
||||
clean_msg = (
|
||||
"Model configuration error: ThinkingConfig is not"
|
||||
f" supported for model {model_name}."
|
||||
)
|
||||
else:
|
||||
clean_msg = f"Explorer agent execution failed: {err_msg}"
|
||||
|
||||
return json.dumps(
|
||||
{"candidates": [], "fallback_message": clean_msg},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
|
||||
async def _cleanup_native_resources(self, client, cached_content, uploaded_files: list) -> None:
|
||||
"""Deletes the cache resource and File API uploads created for this run."""
|
||||
if cached_content:
|
||||
try:
|
||||
logger.info(f"Deleting cache resource: {cached_content.name}")
|
||||
await client.aio.caches.delete(name=cached_content.name)
|
||||
except Exception as cleanup_cache_err:
|
||||
logger.warning(f"Failed to delete cache resource: {cleanup_cache_err}")
|
||||
for file_ref in uploaded_files:
|
||||
try:
|
||||
await client.aio.files.delete(name=file_ref.name)
|
||||
except Exception as cleanup_err:
|
||||
logger.warning(f"Failed to delete uploaded file {file_ref.name}: {cleanup_err}")
|
||||
logger.info(f"Explorer turn latencies: {self.turn_latencies}")
|
||||
@@ -14,14 +14,9 @@
|
||||
|
||||
"""Static tool declarations for the Explorer agent.
|
||||
|
||||
Split out of ``artemis.agents.explorer.explorer`` as pure data:
|
||||
``UNIVERSAL_EXPLORER_TOOLS`` (OpenAI-style dict schemas for the universal
|
||||
LangChain path) and ``NATIVE_EXPLORER_TOOL_DECLARATIONS`` (google-genai
|
||||
``FunctionDeclaration`` objects for the native Gemini path).
|
||||
OpenAI function schemas consumed by the Responses model adapter.
|
||||
"""
|
||||
|
||||
from google.genai import types
|
||||
|
||||
UNIVERSAL_EXPLORER_TOOLS = [
|
||||
{
|
||||
"type": "function",
|
||||
@@ -183,212 +178,3 @@ UNIVERSAL_EXPLORER_TOOLS = [
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
NATIVE_EXPLORER_TOOL_DECLARATIONS = [
|
||||
types.FunctionDeclaration(
|
||||
name="ask_perception_tool",
|
||||
description=(
|
||||
"[Perception] Concurrently executes and awaits three sub-tasks"
|
||||
" in parallel, including UI-tree text search, coordinate audit,"
|
||||
" and visual object detection on the screen.\n\nOutput"
|
||||
" Format:\nReturns:'text' (str): A single string consolidating"
|
||||
" results with 0-1000 normalized coordinates, including UI"
|
||||
" search matches labeled [X],"
|
||||
" coordinate search matches labeled [X], and visually detected"
|
||||
" object targets labeled [D].\n(Annotated screenshots are also"
|
||||
" added to the context containing corresponding colored label"
|
||||
" dots)."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"search_query": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=("Required. Text query to search for in UI elements."),
|
||||
),
|
||||
"nx": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description=(
|
||||
"Required. Normalized X coordinate (0-1000 range)"
|
||||
" to audit for overlapping elements."
|
||||
),
|
||||
),
|
||||
"ny": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description=(
|
||||
"Required. Normalized Y coordinate (0-1000 range)"
|
||||
" to audit for overlapping elements."
|
||||
),
|
||||
),
|
||||
"detect_queries": types.Schema(
|
||||
type=types.Type.ARRAY,
|
||||
items=types.Schema(type=types.Type.STRING),
|
||||
description=(
|
||||
"Required. List of query strings to visually detect on the screen."
|
||||
),
|
||||
),
|
||||
},
|
||||
required=["search_query", "nx", "ny", "detect_queries"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="detect_objects",
|
||||
description=(
|
||||
"[Perception] Locates elements, shapes and more using a"
|
||||
" VLM. Returns their normalized coordinates (in a [0, 1000]"
|
||||
" scale) labeled with [D1], [D2] etc. in the text output, along"
|
||||
" with an annotated image containing corresponding visual label"
|
||||
" dots."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"target_image_id": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="The ID of the image to process.",
|
||||
),
|
||||
"queries": types.Schema(
|
||||
type=types.Type.ARRAY,
|
||||
items=types.Schema(type=types.Type.STRING),
|
||||
description=("A list of single query strings to detect on the screen."),
|
||||
),
|
||||
},
|
||||
required=["queries", "target_image_id"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="get_ocr_list",
|
||||
description=(
|
||||
"[Perception] Retrieves all text elements detected on the"
|
||||
" original, unprocessed screen via OCR. Returns their"
|
||||
" normalized coordinates (in a [0, 1000] scale) labeled with"
|
||||
" [O1], [O2] etc. in the text output, along with an annotated"
|
||||
" image containing corresponding visual label dots."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={},
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="ask_image_processor",
|
||||
description=(
|
||||
"[Transformation] A scripting tool for pixel-level image"
|
||||
" processing. CAUTION: This tool is slow and expensive. Returns"
|
||||
" a summary of the operations performed and new image IDs,"
|
||||
" along with new labels and screen coordinates."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"target_image_id": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=(
|
||||
"The ID of the image in the Image Pool to start"
|
||||
" from (e.g. 'img_0', 'img_1')."
|
||||
),
|
||||
),
|
||||
"instruction": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=(
|
||||
"1-2 precise commands describing a simple task."
|
||||
" Include all necessary coordinates and other"
|
||||
" parameters.\n"
|
||||
),
|
||||
),
|
||||
},
|
||||
required=["instruction", "target_image_id"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="submit_answer",
|
||||
description=(
|
||||
"[Finalization] Submits the final list of ranked candidate UI"
|
||||
" elements matching the search query, or a fallback message if"
|
||||
" no elements were found, to complete the search task. Fails"
|
||||
" validation if: (1) candidates is empty and no"
|
||||
" fallback_message is provided, (2) coordinates are not exactly"
|
||||
" 2 integers [nx, ny], or (3) coordinates lie outside"
|
||||
" normalized [0, 1000] range."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"candidates": types.Schema(
|
||||
type=types.Type.ARRAY,
|
||||
items=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"label": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=(
|
||||
"Reference label (e.g. X1, O2, D3, or a number like 3)"
|
||||
),
|
||||
),
|
||||
"coords": types.Schema(
|
||||
type=types.Type.ARRAY,
|
||||
items=types.Schema(type=types.Type.INTEGER),
|
||||
description=("[nx, ny] coordinates in normalized 0-1000 scale"),
|
||||
),
|
||||
"description": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=("Brief description of this candidate element"),
|
||||
),
|
||||
},
|
||||
required=["label", "coords"],
|
||||
),
|
||||
description=(
|
||||
"A ranked list of candidate UI elements that match"
|
||||
" the query, sorted from highest to lowest"
|
||||
" confidence (maximum 10 candidates). Leave empty"
|
||||
" if no matching elements were found on the screen."
|
||||
),
|
||||
),
|
||||
"fallback_message": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=(
|
||||
"Optional fallback explanation if target elements"
|
||||
" were not found or if there are special"
|
||||
" observations."
|
||||
),
|
||||
),
|
||||
},
|
||||
required=["candidates"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="inspect_region",
|
||||
description=(
|
||||
"[Perception] Crops and resizes a bounding box region of the"
|
||||
" original screenshot. Returns the cropped and resized image"
|
||||
" directly, without any image ID or coordinate transform."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"x_min": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Left X coordinate (0-1000).",
|
||||
),
|
||||
"y_min": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Top Y coordinate (0-1000).",
|
||||
),
|
||||
"x_max": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Right X coordinate (0-1000).",
|
||||
),
|
||||
"y_max": types.Schema(
|
||||
type=types.Type.INTEGER,
|
||||
description="Bottom Y coordinate (0-1000).",
|
||||
),
|
||||
"zoom_factor": types.Schema(
|
||||
type=types.Type.NUMBER,
|
||||
description=("Zoom factor to upscale the cropped region (1.0 to 4.0)."),
|
||||
),
|
||||
},
|
||||
required=["x_min", "y_min", "x_max", "y_max", "zoom_factor"],
|
||||
),
|
||||
),
|
||||
]
|
||||
@@ -30,6 +30,8 @@ from langchain_core.messages import (
|
||||
ToolMessage,
|
||||
)
|
||||
|
||||
from artemis.llm.structured import content_to_text
|
||||
|
||||
from artemis.agents.explorer.geometry import is_valid_norm_point
|
||||
from artemis.agents.explorer.tiers import SUBMIT_TOOL
|
||||
from artemis.agents.explorer.tool_declarations import UNIVERSAL_EXPLORER_TOOLS
|
||||
@@ -128,7 +130,8 @@ class UniversalRunnerMixin:
|
||||
return json.dumps(
|
||||
{
|
||||
"candidates": [],
|
||||
"fallback_message": str(response.content) or "No candidates found.",
|
||||
"fallback_message": content_to_text(response.content)
|
||||
or "No candidates found.",
|
||||
},
|
||||
ensure_ascii=False,
|
||||
)
|
||||
|
||||
@@ -1,3 +1,11 @@
|
||||
# 输出语言要求
|
||||
所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。
|
||||
保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
|
||||
遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。
|
||||
|
||||
每次调用设备操作工具前,用一小段简体中文说明当前观察、即将执行的动作和目的;操作后仅在获得结果或新观察时说明实际变化。不要只在最终答复中使用中文。工具参数中的自由描述文本也使用简体中文,精确引用的界面文字仍保留原文。
|
||||
|
||||
|
||||
# ROLE & OBJECTIVE
|
||||
You are an autonomous and highly efficient Android Device Execution Agent. Your goal is to accomplish the user's objective on the device.
|
||||
|
||||
|
||||
@@ -1,11 +1,17 @@
|
||||
# 输出语言要求
|
||||
所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。
|
||||
保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
|
||||
遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。
|
||||
|
||||
|
||||
# ROLE & OBJECTIVE
|
||||
You are the Step Summarizer for an Android UI automation agent. Your task is to synthesize the Before/After screenshots and physical action into a concise, high-information-density, strictly objective first-person historical memory.
|
||||
|
||||
---
|
||||
|
||||
# PERSPECTIVE & FORMAT CONSTRAINTS
|
||||
1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **"I"** (e.g., "In Step {{ step_number }}, I tapped...", "I swiped up on... and observed..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
|
||||
2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 sentences, 60–100 words).
|
||||
1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **“我”** (e.g., "In Step {{ step_number }}, I tapped...", "I swiped up on... and observed..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
|
||||
2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 句,约 100–200 个汉字;保留关键原文证据,不为凑字数添加内容).
|
||||
3. **No Lists or Formatting**: Do NOT use bullet points, numbered lists, markdown headers, bold labels, or line breaks in your output. Your output must NEVER contain `---` separators or section-marker lines — those belong to the input, not the summary.
|
||||
|
||||
---
|
||||
@@ -13,7 +19,7 @@ You are the Step Summarizer for an Android UI automation agent. Your task is to
|
||||
# CORE OBSERVATION RULES
|
||||
1. **Zero Subjective Validation (STRICT)**:
|
||||
- Absolutely avoid declaring semantic goal achievement, subjective success, or final failure. In historical compression, subjective assumptions cause future turns to falsely believe a subgoal was definitively accomplished or permanently blocked.
|
||||
- **STRICTLY BANNED WORDS**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
|
||||
- **STRICTLY BANNED WORDS (including Chinese equivalents such as “成功完成”、“已达成”、“失败” and “不可能”)**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
|
||||
2. **Action & Visual Delta Structure**:
|
||||
- **Target & Action**: Identify what control/button was targeted (referencing the red visual indicator on the BEFORE screen) and the physical action performed.
|
||||
- **Objective Transition**: Describe the physical screen change (e.g., page transitioned, modal dialog opened, list scrolled revealing new items, checkbox/toggle toggled, keyboard appeared).
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
# 输出语言要求
|
||||
所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。
|
||||
保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
|
||||
遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。
|
||||
|
||||
|
||||
# ROLE & OBJECTIVE
|
||||
You are the Step Summarizer for an Android UI automation agent. Exactly ONE screenshot is available for this step — the decision frame captured when the action was issued. There is NO after-action screenshot. Your task is to describe, strictly objectively and in first person, what THIS screen showed and where the physical action landed.
|
||||
|
||||
@@ -11,8 +17,8 @@ You are the Step Summarizer for an Android UI automation agent. Exactly ONE scre
|
||||
---
|
||||
|
||||
# PERSPECTIVE & FORMAT CONSTRAINTS
|
||||
1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **"I"** (e.g., "In Step {{ step_number }}, I tapped... on a screen showing..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
|
||||
2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 sentences, 60–100 words).
|
||||
1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **“我”** (e.g., "In Step {{ step_number }}, I tapped... on a screen showing..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
|
||||
2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 句,约 100–200 个汉字;保留关键原文证据,不为凑字数添加内容).
|
||||
3. **No Lists or Formatting**: Do NOT use bullet points, numbered lists, markdown headers, bold labels, or line breaks. Your output must NEVER contain `---` separators or section-marker lines — those belong to the input, not the summary.
|
||||
|
||||
---
|
||||
@@ -20,7 +26,7 @@ You are the Step Summarizer for an Android UI automation agent. Exactly ONE scre
|
||||
# CORE OBSERVATION RULES
|
||||
1. **Zero Subjective Validation (STRICT)**:
|
||||
- Absolutely avoid declaring semantic goal achievement, subjective success, or final failure. In historical compression, subjective assumptions cause future turns to falsely believe a subgoal was definitively accomplished or permanently blocked.
|
||||
- **STRICTLY BANNED WORDS**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
|
||||
- **STRICTLY BANNED WORDS (including Chinese equivalents such as “成功完成”、“已达成”、“失败” and “不可能”)**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
|
||||
2. **Preserve Critical Data**: Faithfully transcribe visible text, toast alerts, error banners, verification codes, prices, account names, or tracking numbers that appear on this screen.
|
||||
|
||||
---
|
||||
|
||||
@@ -14,8 +14,7 @@
|
||||
|
||||
"""Universal Multi-Model FlashRunner for Artemis.
|
||||
|
||||
Executes autonomous reactive mobile workflows across Google Gemini,
|
||||
OpenAI GPT-4o/o3, Anthropic Claude 3.5/3.7, and OpenRouter endpoints.
|
||||
Executes autonomous reactive mobile workflows through OpenAI Responses.
|
||||
|
||||
Conversation shape (shared with the Pro operator, history redesign §3.2):
|
||||
the prompt is built every turn from a session :class:`TranscriptLedger` —
|
||||
@@ -38,6 +37,7 @@ active window until its turn is chunked, whose user lines then carry it on.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from artemis.config.settings import settings
|
||||
import base64
|
||||
from dataclasses import dataclass, field
|
||||
import json
|
||||
@@ -83,7 +83,7 @@ from artemis.memory.transcript import PRO_UI_LIST_MARKER, TranscriptLedger, mark
|
||||
from artemis.services.llm import (
|
||||
RobustChatModelWrapper,
|
||||
acomplete,
|
||||
get_google_llm,
|
||||
get_openai_llm,
|
||||
get_llm,
|
||||
invoke_llm_with_timeout_message,
|
||||
)
|
||||
@@ -284,7 +284,9 @@ class FlashRunner:
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to get operator LLM from config, using default: {e}")
|
||||
|
||||
return RobustChatModelWrapper(get_google_llm(model_name="gemini-2.5-flash"), self.ctx)
|
||||
return RobustChatModelWrapper(
|
||||
get_openai_llm(model_name=settings.OPENAI_MODEL), self.ctx
|
||||
)
|
||||
|
||||
def _render_system_prompt(self, tools_declaration: list) -> str:
|
||||
"""Renders the system prompt from the flash_runner.md template.
|
||||
|
||||
@@ -26,6 +26,7 @@ frame, the ``flash_summarizer.md`` neutral-wording contract, and versioned
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from artemis.config.settings import settings
|
||||
import base64
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
@@ -36,7 +37,7 @@ from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
|
||||
|
||||
from artemis.context import ArtemisContext
|
||||
from artemis.memory.step_memory import JobKey, StepMemoryService
|
||||
from artemis.services.llm import RobustChatModelWrapper, get_google_llm, get_llm
|
||||
from artemis.services.llm import RobustChatModelWrapper, get_openai_llm, get_llm
|
||||
from artemis.services.token_meter import record_llm_usage
|
||||
from artemis.utils.logger import get_logger
|
||||
from artemis.utils.task_tree import format_actions_clean
|
||||
@@ -162,15 +163,15 @@ class VisualStepSummarizer(StepMemoryService):
|
||||
)
|
||||
|
||||
# Initialize lightweight VLM: prioritize explicit model_name
|
||||
target_model = model_name or "gemini-2.5-flash-lite"
|
||||
target_model = model_name or settings.OPENAI_MODEL
|
||||
self._model_name = target_model
|
||||
try:
|
||||
if model_name:
|
||||
self._llm = get_google_llm(model_name=target_model, temperature=0.0)
|
||||
self._llm = get_openai_llm(model_name=target_model, temperature=0.0)
|
||||
else:
|
||||
self._llm = get_llm(ctx, name="summarizer", is_utils=True)
|
||||
except Exception:
|
||||
self._llm = get_google_llm(model_name=target_model, temperature=0.0)
|
||||
self._llm = get_openai_llm(model_name=target_model, temperature=0.0)
|
||||
try:
|
||||
configured = getattr(self._llm, "model", None) or getattr(self._llm, "model_name", None)
|
||||
if isinstance(configured, str) and configured:
|
||||
@@ -341,7 +342,7 @@ class VisualStepSummarizer(StepMemoryService):
|
||||
"""Meter one raw-model lens call as an ``llm_usage`` trace, best-effort.
|
||||
|
||||
Gateway-wrapped models already meter at the wrapper exit; only the raw
|
||||
``get_google_llm`` bypass needs explicit metering here. Lens prompts
|
||||
``get_openai_llm`` bypass needs explicit metering here. Lens prompts
|
||||
are tiny and must not overwrite the session's ``last_prompt_tokens``
|
||||
(the compaction thresholds' live context base), hence
|
||||
``update_last_prompt=False``.
|
||||
|
||||
@@ -25,6 +25,8 @@ from pathlib import Path
|
||||
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
|
||||
from langchain_core.tools import BaseTool
|
||||
|
||||
from artemis.llm.structured import content_to_text
|
||||
|
||||
from artemis.core.tool_failure import is_tool_failure
|
||||
from artemis.context import ArtemisContext
|
||||
from artemis.data_engine.trace import TraceSpan, trace
|
||||
@@ -139,7 +141,7 @@ class HistoryAnalyzer:
|
||||
return (
|
||||
response.content.strip()
|
||||
if isinstance(response.content, str)
|
||||
else str(response.content)
|
||||
else content_to_text(response.content)
|
||||
)
|
||||
|
||||
# Process tool calls
|
||||
|
||||
@@ -51,6 +51,8 @@ class SpawnLogReaderArgs(BaseModel):
|
||||
|
||||
from langgraph.prebuilt import InjectedState
|
||||
|
||||
from artemis.llm.structured import content_to_text
|
||||
|
||||
from artemis.core.tool_failure import ToolFailure, is_tool_failure
|
||||
from artemis.context import ArtemisContext
|
||||
from artemis.data_engine.trace import (
|
||||
@@ -224,7 +226,7 @@ class LogAnalyzerNode:
|
||||
response = await invoke_llm_with_timeout_message(acomplete(llm, current_messages))
|
||||
|
||||
if response.content:
|
||||
outcome = response.content
|
||||
outcome = content_to_text(response.content)
|
||||
|
||||
if not response.tool_calls:
|
||||
break
|
||||
|
||||
@@ -16,6 +16,8 @@ from pathlib import Path
|
||||
from typing import Annotated
|
||||
from langchain_core.messages import HumanMessage, SystemMessage, ToolMessage
|
||||
from langchain_core.tools import tool
|
||||
from artemis.llm.structured import content_to_text
|
||||
|
||||
from artemis.core.tool_failure import ToolFailure, is_tool_failure
|
||||
from artemis.context import ArtemisContext
|
||||
from artemis.services.llm import get_llm, invoke_llm_with_timeout_message
|
||||
@@ -131,7 +133,7 @@ class TaskOutputAnalyzerNode:
|
||||
response = await invoke_llm_with_timeout_message(llm.ainvoke(current_messages))
|
||||
|
||||
if response.content:
|
||||
outcome = response.content
|
||||
outcome = content_to_text(response.content)
|
||||
|
||||
if not response.tool_calls:
|
||||
break
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
{
|
||||
"main_template": "# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n[% if \"wait_for_delay\" in available_tools %]- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n[% endif %]- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n 1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.\n 2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading state. This boundary probe must remain unidirectional; never reverse direction merely to prove completion. If the task declares an earlier exit condition (for example, find one verified match or process N items), that explicit condition may end the traversal without reaching the list boundary.\n 3. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.\n 4. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed with confirmed anchor continuity and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for \"double-checking\", even if older execution history has been pruned. Resume from the last recorded anchor and direction. Only the single bounded recovery above, or a refreshed/mutated list, permits limited re-verification; otherwise continue the single pass over new content.\n- **Dynamic Monitoring & Reactive Loop Protocol (Monitor-Trigger-Action)**: When executing an active milestone matching `[Loop] <Goal> into note <TargetNote> (Exit: <ExitCondition>; Interval: <Interval>)` or `[Loop:continuous] <Goal> into note <TargetNote> (Interval: <Interval>)` (the milestone grammar itself is defined under Task Plan Grammar below), run this 5-step cyclic state machine:\n - **Step 1 (Baseline Sync)**: On the initial cycle, observe all visible entries/metrics in the primary monitoring view and save the full snaLine truncated
|
||||
"main_template": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。\n\n每次调用设备操作工具前,用一小段简体中文说明当前观察、即将执行的动作和目的;操作后仅在获得结果或新观察时说明实际变化。不要只在最终答复中使用中文。工具参数中的自由描述文本也使用简体中文,精确引用的界面文字仍保留原文。\n\n# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n[% if \"wait_for_delay\" in available_tools %]- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n[% endif %]- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n 1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.\n 2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading state. This boundary probe must remain unidirectional; never reverse direction merely to prove completion. If the task declares an earlier exit condition (for example, find one verified match or process N items), that explicit condition may end the traversal without reaching the list boundary.\n 3. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtLine truncated
|
||||
}
|
||||
@@ -29,7 +29,7 @@ from artemis.data_engine.trace import (
|
||||
from artemis.graph.state import State
|
||||
from artemis.graph.visibility import strict_state
|
||||
from artemis.mcp.action_specs import OPERATOR_SHELL_ORDER, operator_shell_tool
|
||||
from artemis.llm.google import usage_from_message
|
||||
from artemis.llm.usage import usage_from_message
|
||||
from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
|
||||
from artemis.tools.command_tool import (
|
||||
analyze_task_output_wrapper,
|
||||
|
||||
@@ -1,3 +1,9 @@
|
||||
# 输出语言要求
|
||||
所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。
|
||||
保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
|
||||
遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。
|
||||
|
||||
|
||||
# Context & Inputs
|
||||
|
||||
## User's Initial Goal
|
||||
|
||||
@@ -20,6 +20,8 @@ from pathlib import Path
|
||||
|
||||
from jinja2 import Template
|
||||
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
|
||||
from artemis.llm.structured import content_to_text
|
||||
|
||||
from artemis.core.tool_failure import ToolFailure, is_tool_failure
|
||||
from artemis.config import OutputConfig
|
||||
from artemis.context import ArtemisContext
|
||||
@@ -265,7 +267,7 @@ async def outputter(
|
||||
messages.append(response)
|
||||
|
||||
if not response.tool_calls:
|
||||
raw_answer = response.content
|
||||
raw_answer = content_to_text(response.content)
|
||||
break
|
||||
|
||||
for tc in response.tool_calls:
|
||||
|
||||
@@ -1,13 +1,13 @@
|
||||
{
|
||||
"blocks": {
|
||||
"role": "## Role\nYou are a Senior Test Engineer leading a team of mobile execution agents. Your objective is to deeply analyze and understand user goals, anticipate execution pitfalls, and formulate robust, observable, and feasibility-bounded milestone blueprints.",
|
||||
"role": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。\n\n## Role\nYou are a Senior Test Engineer leading a team of mobile execution agents. Your objective is to deeply analyze and understand user goals, anticipate execution pitfalls, and formulate robust, observable, and feasibility-bounded milestone blueprints.",
|
||||
"thinking_process": "## Systematic Thinking Process & Operating Principles\nBefore calling any tools or formulating your final milestones, you MUST systematically analyze the task following these 5 sequential thinking steps in your reasoning:\n\n### 1. Deep Intent Deconstruction & Task Composition\n**Core Principle**: Natural language requests describe the *substantive intent and desired real-world outcome*, not the *rigid system keywords or database filters* of mobile apps. Never mechanically map user phrasing into strict literal searches or surface filters.\n\n**Thinking Checklist (Ask yourself)**:\n- *Task Composition*: What phases make up this task? Is it a sequence of one-off linear steps (e.g., open app, configure settings), an iterative loop over multiple items (e.g., inspecting a list of search results), or a periodic polling loop (e.g., refreshing to monitor new messages, waiting for battery level)?\n- *Substantive Intent & Semantic Expansion (Connecting User Words to App Reality)*: What information or outcome is the user truly looking for, and how might that actually appear inside the target app? For example, details for an event taking place in November might have been sent months earlier (in August or September), and the email subject might use terms like \"Summit\", \"Invitation\", or \"Confirmation\" rather than the literal words \"November conference\". Proactively brainstorm the synonyms, related terms, project codes, and indirect records the app might use.\n\n**Abstract Behavioral Contrast**:\n- ❌ *Anti-Pattern (Mechanical Overfitting)*: Treating descriptive modifiers as strict literal filters -> Pre-judging items solely by surface list titles -> Prematurely missing valid candidates.\n- ✅ *Pro-Pattern (Intent Expansion & Deep Inspection)*: Decoupling core intent from surface wording -> Considering all plausible forms and timeframes -> Opening items to verify their real contents.\n\n### 2. Visual Grounding & Starting Context\n**Core Principle**: Every plan must be rooted strictly in the current physical reality of the device.\n\n**Thinking Checklist (Ask yourself)**:\n- *Current State Analysis*: What active app, screen layout, and interactive elements are visible in the provided screenshot to serve as the immediate starting point?\n\n### 3. Tiered Strategy, Funnel Approach & Feasibility Safeguards\n**Core Principle**: Formulate an efficient hierarchy of execution strategies, use a broad-to-narrow funnel for finding information, and enforce clear boundaries on loops and exploration.\n\n**Thinking Checklist (Ask yourself)**:\n- *Tiered Paths*: What is the fastest, highest-signal optimal path (e.g., targeted in-app search, direct shortcuts)? If that fails or yields no results, what is the pragmatic fallback path (e.g., browsing categories, chronological scanning)?\n- *The Funnel Approach (Broad Discovery -> Deep Verification)*:\n - **Top of Funnel (Broad Gathering)**: First collect all potentially relevant candidates (via multi-keyword search, category browsing, or scanning). Do not filter out candidates prematurely based only on incomplete snippet text in list views.\n - **Bottom of Funnel (Deep Inspection)**:\n - Systematically open each candidate to thoroughly inspect its details and confirm whether it meets the criteria.\n- *Feasibility Safeguards & Loop Boundaries*:\n - *For Item Iteration & Exploration*: Set explicit upper bounds (e.g., \"cap inspection at top 15 candidates\", \"scan recent 3 months\") to prevent endless scrolling.\n - *For Periodic Monitoring / Polling*: Specify the check action (e.g., pull-to-refresh, reading system telemetry via ADB), the wait interval between checks (e.g., check every 5 minutes to conserve resources), and the exit criteria (e.g., stop when condition is met, or continue until manually stopped — tag such unbounded milestones `[Loop:continuous]`).\n\n### 4. Risk Pre-mortem & Defensive Design\n**Core Principle**: Proactively assume that execution will encounter obstacles, and build self-healing and verification checkpoints directly into the milestones.\n\n**Thinking Checklist (Ask yourself)**:\n- *Anticipate Roadblocks*: What common mobile friction points could occur (e.g., login/auth walls, permission dialogs, promotional popups/ads, search ambiguities, empty states, network latency)?\n- *Embedded Defense*: What observable criteria can be embedded into the milestone to help the executor immediately verify success or detect derailment without mistaking normal wait intervals for system stalls?\n\n### 5. Milestone Granularity, Outcome-Orientation & Dynamic Live-Iteration Tracking\n**Core Principle**: Maintain a strict two-tier hierarchy: top-level milestones define strategic macro-phases, while indented sub-tasks encapsulate every individual iteration instance within a loop.\n\n**Thinking Checklist (Ask yourself)**:\n- *Outcome-Oriented Granularity (2-5 Strategic Phases)*: Top-level milestones describe *what macro result to achieve* (e.g., \Line truncated
|
||||
"capabilities_operator": "## Execution Capabilities\n\nThe execution agent primarily interacts with the mobile device and analyzes screen recordings and logs. It can perform the following types of actions:\n- **UI Interaction**: Tapping, swiping, and entering text on the screen.\n- **App Management**: Launching and stopping specific applications.\n- **Information Retrieval & Analysis**: Reading on-screen text and UI hierarchy, searching the web for external information, querying system logs, and analyzing screen recordings.\n- **Terminal & Background Operations**: Executing arbitrary ADB shell commands on the device for exploration. It is capable of running long-running ADB commands asynchronously in the background as tasks.\n\n### Capability Boundaries & Planning Rules\n- **Cross-App Operations**: When a task requires interacting with multiple applications, split the operations into distinct milestones for each app. Avoid mixing actions for different apps in a single milestone.",
|
||||
"output_protocol_initial": "## Task Plan & Scratchpad Protocol\n\n1. **Articulate Thinking**: Begin by reasoning through the 5-step **Systematic Thinking Process** to establish your strategy, funnel approach, bounds, and pre-mortem checks.\n2. **Designate Notes in Milestones & Pre-generate Answer Cards**: If the task involves collecting findings, logging periodic monitoring events, recording candidates, or analytical summaries, designate clear, descriptive note names directly in the milestone descriptions (e.g., `candidates_list`, `monitoring_log`, `schedule_summary`). You can and should invoke the `save_note` tool to pre-generate these structured template files (answer card / scratchpad skeletons with Markdown headers, sections, or tables) before submitting `task_plan`. This provides downstream executors with a clear, ready-to-fill schema.\n3. **Submit Plan**: Submit your initial plan by invoking the `save_note` tool with `key='task_plan'`. Avoid writing the plan only in your text response, as it will only be recognized by the system if submitted via the tool.\n\n### Plan Formatting Rules\n- Format the plan strictly as a Markdown task list starting with `- [ ]` for uncompleted items.\n{{ plan_grammar }}\n- For linear macro-milestones: `- [ ] <Phase Description>`\n- For bounded repetitive, iterative, or periodic polling milestones, tag the item with `[Loop]` and declare its Exit Boundary and Interval (e.g., `- [ ] [Loop] <Overall Loop / Monitoring Goal> (Exit: <Condition>; Interval: <e.g., every 5 minutes>)`).\n- For unbounded continuous monitoring that only the user can stop, tag the item with `[Loop:continuous]` instead (e.g., `- [ ] [Loop:continuous] <Monitoring Goal> (Interval: <e.g., every 5 minutes>)`); never declare a self-serve exit condition for it.\n- **Strict Two-Tier Hierarchy & Loop Subtask Contract**:\n 1. **Subtask Nesting Rule**: ALL individual iteration instances, polling checks, candidate inspections, or baseline setups MUST be formatted as 2-space indented sub-tasks (` - [ ]`) directly nested under their parent `[Loop]` milestone.\n 2. **Mandatory First-Instance Initialization**: Every `[Loop]` milestone MUST be initialized with its first concrete child sub-task (e.g., ` - [ ] Inspect Candidate #1: ...` or ` - [ ] Polling Check #1 (Baseline & Initial Sync): ...`).\n 3. **No Flattened Cycles**: NEVER elevate individual checks, recurring polling cycles, or candidate inspections to top-level milestones outside the `[Loop]`. Top-level items are reserved exclusively for strategic macro-phases.\n- Ensure the content of the `task_plan` note contains only the task list, excluding introductions, explanations, conversational text, or headers.",
|
||||
"examples_initial": "## Master Example: Demonstrating the Complete Cognitive Paradigm\n\n**Goal**: \"Find and summarize all schedule arrangements and venue details for the upcoming November tech conference from my emails.\"\n\n### Naive / Incorrect Approaches:\n1. *Flawed Semantics & Too Granular*:\n- [ ] Tap search icon in Gmail\n- [ ] Type \"November\"\n- [ ] Open first email and tap back\n*(Fails: Micromanages atomic UI taps; assumes emails for November arrangements must have \"November\" in the title or be received in November)*\n2. *Vague & Unbounded Trap*:\n- [ ] Open email and check all emails for conference details\n*(Fails: Unbounded scan trap; causes infinite scrolling and step budget exhaustion on mobile)*\n\n---\n\n### Correct Approach (Full Cognitive Thinking & Robust Plan):\n\n#### Step-by-Step Thinking:\n1. **Intent & Semantic Expansion**:\n - *Core Objective*: Extract confirmed schedule dates, sessions, and venue info for the November tech conference.\n - *Semantic Insight*: Conference communications are often sent months in advance (e.g., ticket confirmations or registration emails in August/September). The subject line may say \"Global Developer Summit\", \"Tech Expo Agenda\", or \"Registration Confirmed\" rather than literally \"November conference\".\n - *Task Composition*: Linear setup (open email app and search) -> Iterative candidate inspection loop (examine each promising email) -> Final consolidation.\n2. **Visual Grounding**:\n - The device screenshot shows the Home screen with the Gmail app icon visible. The plan must start by opening the email client.\n3. **Tiered Strategy, Funnel Approach & Safeguards**:\n - *Tier 1 (Optimal)*: Use in-app search with rich semantic keywords (\"conference\", \"summit\", \"schedule\", \"agenda\", \"venue\", \"November\").\n - *Tier 2 (Fallback)*: If search is unavailable or yields zero results, fallback to chronological scanning of recent 3 months.\n - *Funnel (Broad Gathering -> Deep Inspection)*: Gather candidate emails from search results; do not discard candidates based solely on list previews. Open each candidate thread to inspect full body details.\n - *Feasibility Boundary*: Cap the candidate email inspection pool at top 15 emails to prevent infinite scrolling.\n4. **Risk Pre-mortem**:\n - *Risks*: Promotional marketing spam; conflicting drafts with preliminary dates; cancelled sessions.\n - *Defensive Check*: Instruct downstream executor to filter out promotional newsletters, verify session timestamps, and confirm final venue address.\n5. **Milestone Granularity & Scratchpad Blueprint**:\n - Focus milestones on business outcomes (search, inspect candidates, consolidate), avoiding micromanaged click-by-click steps.\n - Use indented sub-tasks to guide the loop of inspecting each candidate email, initializing with Item #1 and adding subsequent items as they are identified. This prevents the agent from losing state or re-inspecting the same items.\n - Designate note `november_conference_summary` as the destination for recording findings and pre-generate the structured template note.\n\n#### Tool Invocation:\n1. Pre-generate designated answer card note template:\n`save_note(key='november_conference_summary', content='# November Tech Conference Summary\\n\\n## Key Details\\n- Conference Name: \\n- Dates: \\n- Venue / Address: \\n\\n## Daily Schedule & Sessions\\n(To be populated during email inspection)\\n')`\n\n2. Submit milestone plan:\n`save_note(key='task_plan', content='- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\\n - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\\n- [ ] Review and consolidate all extracted information in `november_conference_summary` into a clear, finalized conference itinerary')`\n\n#### Content of `task_plan` Note:\n- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\n - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\n- [ ] ReviLine truncated
|
||||
"check_generation": "## Declaring Verification Standards (check lines)\nYou may attach check lines to the plan. {% if midway_checks %}An independent Checker judges each line in the background when its milestone completes, and a failed verify reopens that milestone for repair{% if final_check %}; a final review at task exit then audits the whole run against the user's goal{% endif %}.{% else %}A final review at task exit judges every line once, from the recorded evidence around each milestone's completion; there is no midway repair loop.{% endif %}\n\n- **User-declared expectations become `- assert:` lines.** Whenever the user's goal states an explicit expectation (\"should show X\", \"must not crash\", \"the toast Y should appear\"), declare it as an assert line preserving the user's wording verbatim. Expectations about the whole run or the final state use the `@end` suffix (task-level lines at the end of the plan).\n- **`- verify:` lines only for milestones that genuinely need acceptance checks** — completion points of irreversible operations, or milestones that directly contribute to the final goal. Do NOT attach a verify line to every subgoal: restraint is an explicit requirement (checking cost scales with the number of check lines).\n- **Do not fabricate assertions** for subgoals that have nothing verifiable. Rewrite transient expectations as persistently probeable state wherever possible (e.g., \"alarm exists in the alarm list\" instead of \"a toast flashed\") — evidence for transient moments only exists if the execution history happened to capture it.\n- **Ordering requirements are post-hoc.** If the user demands \"A must be confirmed before B runs\", still generate the assertion preserving their semantics, but annotate in the plan that ordering is verified retroactively from history — the system performs post-hoc audits and CANNOT block B from executing. Never silently promise enforcement that does not exist.",
|
||||
"check_audit": "- **Check Standard Protection**: Check lines (`- verify:` / `- assert:`) declared in the plan are the task's verification standards. FLAG any proposed plan that deletes, weakens, or rewords existing check lines — softening a declared standard is never an acceptable adaptation. Adding new check lines is allowed.",
|
||||
"role_validator": "## Role\nYou are the Task Plan Reviewer, a fast advisory second opinion. The Operator has modified the top-level task plan and the change is already applied. Your job is to spot a concrete concern — goal drift, a broken loop contract, a weakened check standard, a logical dead end — and, if there is one, state it with its reason so the Operator can decide. You never block or roll back anything.",
|
||||
"role_validator": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。\n\n## Role\nYou are the Task Plan Reviewer, a fast advisory second opinion. The Operator has modified the top-level task plan and the change is already applied. Your job is to spot a concrete concern — goal drift, a broken loop contract, a weakened check standard, a logical dead end — and, if there is one, state it with its reason so the Operator can decide. You never block or roll back anything.",
|
||||
"validator_guidelines": "## Validation Principles\n- **Goal Alignment**: Ensure the new plan ultimately accomplishes the `Initial Goal` requested by the user. If the Operator has abandoned the core objective or fundamentally altered it, flag the plan.\n- **Logical Coherence**: Verify that the proposed milestones form a logical, achievable sequence to reach the goal. Support valid ongoing iteration/polling subgoals and trigger branches without misconstruing periodic checks as infinite loops. Flag the plan if it introduces contradictory steps, unhandled deadlocks, or obvious logical fallacies.\n{{ plan_grammar }}\n- **Loop & Monitoring Protection**: If the existing plan contains an active `[Loop]` or `[Loop:continuous]` milestone, you MUST FLAG any proposed plan that deletes the milestone or removes its loop tag, as well as any plan that marks a `[Loop:continuous]` continuous monitoring milestone as completed (`[x]`) without an explicit external user stop signal.\n- **Adaptive Acceptance**: The Operator operates in a dynamic environment and may encounter unexpected UI states or dynamic event triggers. You should approve alternative strategies, extra intermediate steps, loop status tracking, or necessary workarounds as long as they serve the primary goal effectively.\n- **Advisory Verdict & Constructive Feedback**: Your verdict is advisory — the change stays applied whether or not you approve it. Approve unless you have a concrete, evidence-backed concern; a vague unease is not a concern. If you flag the plan, give one or two sentences naming the flaw or goal deviation and the correction you suggest, concrete enough for the Operator to weigh it against its own observations."
|
||||
},
|
||||
"modes": {
|
||||
|
||||
@@ -23,7 +23,6 @@ import base64
|
||||
import json
|
||||
from typing import Any, Literal
|
||||
|
||||
from google.genai import types as genai_types
|
||||
from langchain_core.messages import ToolMessage
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
@@ -99,44 +98,6 @@ def prune_intermediate_screenshots(messages: list[Any]) -> None:
|
||||
if not messages:
|
||||
return
|
||||
|
||||
# Check for google.genai.types.Content objects
|
||||
if hasattr(messages[0], "parts"):
|
||||
user_indices_with_images = []
|
||||
for idx in range(1, len(messages)):
|
||||
content = messages[idx]
|
||||
if getattr(content, "role", None) == "user" and getattr(content, "parts", None):
|
||||
for part in content.parts:
|
||||
if getattr(part, "inline_data", None) is not None:
|
||||
user_indices_with_images.append(idx)
|
||||
break
|
||||
|
||||
if len(user_indices_with_images) > 1:
|
||||
indices_to_prune = user_indices_with_images[:-1]
|
||||
for idx in indices_to_prune:
|
||||
content = messages[idx]
|
||||
new_parts = []
|
||||
for part in content.parts:
|
||||
if getattr(part, "inline_data", None) is not None:
|
||||
try:
|
||||
new_parts.append(
|
||||
genai_types.Part.from_text(
|
||||
text=(
|
||||
"[Screenshot of intermediate step omitted for performance]"
|
||||
)
|
||||
)
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.debug(
|
||||
"Could not build placeholder text part for pruned screenshot;"
|
||||
" dropping it: %s",
|
||||
exc,
|
||||
exc_info=True,
|
||||
)
|
||||
else:
|
||||
new_parts.append(part)
|
||||
content.parts = new_parts
|
||||
return
|
||||
|
||||
# LangChain BaseMessage format
|
||||
last_img_msg_idx = -1
|
||||
for idx in range(len(messages) - 1, -1, -1):
|
||||
|
||||
@@ -12,4 +12,4 @@
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Video analyzer utility for analyzing video content with Gemini models."""
|
||||
"""Video analyzer utility for analyzing video content with OpenAI Responses models."""
|
||||
+16
-258
@@ -12,7 +12,7 @@
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Native Gemini execution path for audio-only segment analysis.
|
||||
"""Responses execution path for audio-only segment analysis.
|
||||
|
||||
Extracted from ``video_analyzer.py`` as a pure structural split. Patchable
|
||||
collaborators are looked up late through the facade module so
|
||||
@@ -23,24 +23,12 @@ keep working.
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
|
||||
from google.genai import types
|
||||
|
||||
from artemis.agents.video_analyzer import video_analyzer as _va
|
||||
from artemis.agents.video_analyzer.chunk_conversation import (
|
||||
_MISSING_SUMMARY_ERROR,
|
||||
_NO_TOOL_CALL_ERROR,
|
||||
_SUBMIT_NOW_WARNING,
|
||||
_WRONG_TOOL_ERROR,
|
||||
SUB_AGENT_MAX_TURNS,
|
||||
_append_rejection,
|
||||
_clean_text,
|
||||
_tool_name,
|
||||
)
|
||||
from artemis.agents.video_analyzer.reliability import (
|
||||
SubAgentAnswerExhausted,
|
||||
classify_video_failure,
|
||||
)
|
||||
from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
|
||||
from artemis.data_engine.trace import TraceSpan
|
||||
from artemis.llm.reliability import retry_policy_for
|
||||
from artemis.utils.logger import get_logger
|
||||
@@ -72,7 +60,7 @@ async def exec_analyze_audio_only(
|
||||
return "Error: start_time must be a numeric recording-relative timestamp."
|
||||
if not str(specific_query or "").strip():
|
||||
return "Error: specific_query must describe the audio evidence to find."
|
||||
max_retries = analyzer.native_max_retries
|
||||
max_retries = analyzer.max_retries
|
||||
current_start = round(start_time, 1) if isinstance(start_time, (int, float)) else start_time
|
||||
current_end = round(end_time, 1) if isinstance(end_time, (int, float)) else end_time
|
||||
|
||||
@@ -134,37 +122,25 @@ async def exec_analyze_audio_only(
|
||||
analyzer, result, media.actual_start, start_time, end_time, specific_query
|
||||
)
|
||||
|
||||
if not analyzer.use_native_gemini:
|
||||
audio_result = await analyzer._exec_analyze_audio_universal(
|
||||
audio_path=media.audio_path,
|
||||
start_time=current_start,
|
||||
end_time=current_end,
|
||||
actual_start=media.actual_start,
|
||||
prompt_with_context=media.prompt_with_context,
|
||||
specific_query=specific_query,
|
||||
)
|
||||
summary, analysis = analyzer._parse_chunk_result(audio_result)
|
||||
analyzer.blackboard.complete_segment(
|
||||
current_start,
|
||||
current_end,
|
||||
specific_query,
|
||||
lease_owner,
|
||||
summary,
|
||||
analysis,
|
||||
modality="audio",
|
||||
)
|
||||
return audio_result
|
||||
|
||||
return await _run_native_audio_conversation(
|
||||
analyzer,
|
||||
media,
|
||||
audio_result = await analyzer._exec_analyze_audio_universal(
|
||||
audio_path=media.audio_path,
|
||||
start_time=current_start,
|
||||
end_time=current_end,
|
||||
actual_start=media.actual_start,
|
||||
prompt_with_context=media.prompt_with_context,
|
||||
specific_query=specific_query,
|
||||
)
|
||||
summary, analysis = analyzer._parse_chunk_result(audio_result)
|
||||
analyzer.blackboard.complete_segment(
|
||||
current_start,
|
||||
current_end,
|
||||
start_time,
|
||||
end_time,
|
||||
specific_query,
|
||||
lease_owner,
|
||||
summary,
|
||||
analysis,
|
||||
modality="audio",
|
||||
)
|
||||
return audio_result
|
||||
|
||||
except Exception as e:
|
||||
# Same contract as the video sub-agent loop: classification
|
||||
@@ -232,49 +208,6 @@ async def _finish_failed_audio(
|
||||
):
|
||||
"""Records terminal audio failure, attempting the universal fallback first."""
|
||||
failure = classify_video_failure(error)
|
||||
if (
|
||||
analyzer.use_native_gemini
|
||||
and failure.should_fallback
|
||||
and media.audio_path is not None
|
||||
and media.actual_start is not None
|
||||
and media.prompt_with_context is not None
|
||||
):
|
||||
try:
|
||||
logger.warning(
|
||||
f"Native audio analysis failed [{failure.category.value}]; using universal fallback"
|
||||
)
|
||||
_va._record_llm_event(
|
||||
"llm_fallback",
|
||||
{
|
||||
"reason": "native_audio_analysis_failed",
|
||||
"category": failure.category.value,
|
||||
"error": str(error)[:500],
|
||||
},
|
||||
)
|
||||
fallback_result = await analyzer._exec_analyze_audio_universal(
|
||||
audio_path=media.audio_path,
|
||||
start_time=current_start,
|
||||
end_time=current_end,
|
||||
actual_start=media.actual_start,
|
||||
prompt_with_context=media.prompt_with_context,
|
||||
specific_query=specific_query,
|
||||
force_fallback=True,
|
||||
)
|
||||
summary, analysis = analyzer._parse_chunk_result(fallback_result)
|
||||
analyzer.blackboard.complete_segment(
|
||||
current_start,
|
||||
current_end,
|
||||
specific_query,
|
||||
lease_owner,
|
||||
summary,
|
||||
analysis,
|
||||
modality="audio",
|
||||
)
|
||||
return fallback_result
|
||||
except Exception as fallback_error:
|
||||
error = fallback_error
|
||||
failure = classify_video_failure(error)
|
||||
retryable = None
|
||||
analyzer.blackboard.fail_segment(
|
||||
current_start,
|
||||
current_end,
|
||||
@@ -366,178 +299,3 @@ def _build_audio_prompt(
|
||||
)
|
||||
prompt_with_context += f"{specific_query}"
|
||||
return prompt_with_context
|
||||
|
||||
|
||||
async def _run_native_audio_conversation(
|
||||
analyzer,
|
||||
media: _AudioMedia,
|
||||
current_start: float,
|
||||
current_end: float,
|
||||
start_time: float,
|
||||
end_time: float | None,
|
||||
specific_query: str,
|
||||
lease_owner: str,
|
||||
) -> str:
|
||||
"""Uploads the audio and drives the native Gemini audio agent to a committed answer."""
|
||||
file = None
|
||||
try:
|
||||
with TraceSpan(name="upload_audio_to_gemini") as span:
|
||||
file = await analyzer.upload_and_poll_file(media.audio_path)
|
||||
span.result = f"Uploaded {file.name}"
|
||||
logger.info(f"Invoking Gemini for audio-only task with model {analyzer.model_name}...")
|
||||
|
||||
sub_agent_contents = [
|
||||
types.Content(
|
||||
role="user",
|
||||
parts=[
|
||||
types.Part.from_uri(file_uri=file.uri, mime_type=file.mime_type)
|
||||
if hasattr(file, "uri")
|
||||
else file,
|
||||
types.Part.from_text(text=media.prompt_with_context),
|
||||
],
|
||||
)
|
||||
]
|
||||
|
||||
(
|
||||
final_summary,
|
||||
final_analysis,
|
||||
final_confidence_score,
|
||||
) = await _drive_audio_agent_loop(analyzer, sub_agent_contents)
|
||||
|
||||
end_val = end_time if end_time is not None else "unknown"
|
||||
analyzer._record_blackboard_entry(
|
||||
{
|
||||
"start": start_time,
|
||||
"end": end_val,
|
||||
"target": specific_query,
|
||||
"summary": final_summary,
|
||||
"confidence_score": final_confidence_score,
|
||||
},
|
||||
modality="audio",
|
||||
)
|
||||
|
||||
end_str = f"{end_time:.1f}s" if end_time is not None else "unknown"
|
||||
audio_result = (
|
||||
f"[from {start_time:.1f}s to {end_str}] Summary:"
|
||||
f" {final_summary} Analysis: {final_analysis}"
|
||||
)
|
||||
analyzer.blackboard.complete_segment(
|
||||
current_start,
|
||||
current_end,
|
||||
specific_query,
|
||||
lease_owner,
|
||||
final_summary,
|
||||
final_analysis,
|
||||
modality="audio",
|
||||
)
|
||||
return audio_result
|
||||
finally:
|
||||
if file:
|
||||
logger.info(f"Cleaning up cloud file {file.name}...")
|
||||
try:
|
||||
await asyncio.wait_for(
|
||||
analyzer.client.aio.files.delete(name=file.name),
|
||||
timeout=30,
|
||||
)
|
||||
analyzer.cloud_files_to_cleanup.discard(file.name)
|
||||
except Exception as ce:
|
||||
logger.error(f"Failed to delete cloud file {file.name}: {ce}")
|
||||
|
||||
|
||||
async def _drive_audio_agent_loop(analyzer, sub_agent_contents: list) -> tuple[str, str, float]:
|
||||
"""Return a validated (summary, analysis, confidence_score).
|
||||
|
||||
Raise ``SubAgentAnswerExhausted`` if no answer is accepted within the turn limit.
|
||||
"""
|
||||
rejection = _NO_TOOL_CALL_ERROR
|
||||
|
||||
for turn in range(1, SUB_AGENT_MAX_TURNS + 1):
|
||||
if turn == SUB_AGENT_MAX_TURNS - 1:
|
||||
sub_agent_contents.append(
|
||||
types.Content(
|
||||
role="user",
|
||||
parts=[types.Part.from_text(text=_SUBMIT_NOW_WARNING)],
|
||||
)
|
||||
)
|
||||
|
||||
with TraceSpan(name="gemini_generate_audio_content") as span:
|
||||
response = await asyncio.wait_for(
|
||||
analyzer.client.aio.models.generate_content(
|
||||
model=analyzer.model_name,
|
||||
contents=sub_agent_contents,
|
||||
config=types.GenerateContentConfig(
|
||||
system_instruction=analyzer.audio_system_prompt,
|
||||
tools=[types.Tool(function_declarations=[analyzer.submit_answer_tool])],
|
||||
safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
|
||||
),
|
||||
),
|
||||
timeout=analyzer.model_call_timeout_seconds,
|
||||
)
|
||||
|
||||
full_text = response.text or ""
|
||||
function_calls = response.function_calls or []
|
||||
|
||||
if function_calls:
|
||||
answer, rejection = _handle_audio_function_calls(
|
||||
sub_agent_contents, response, function_calls, full_text
|
||||
)
|
||||
if answer is not None:
|
||||
return answer
|
||||
else:
|
||||
rejection = _NO_TOOL_CALL_ERROR
|
||||
sub_agent_contents.append(
|
||||
types.Content(
|
||||
role="model",
|
||||
parts=[types.Part.from_text(text=full_text)],
|
||||
)
|
||||
)
|
||||
|
||||
raise SubAgentAnswerExhausted(SUB_AGENT_MAX_TURNS, rejection)
|
||||
|
||||
|
||||
def _validate_audio_answer(summary: str, confidence_score) -> str | None:
|
||||
"""Return a validation error for the audio answer, or None if valid."""
|
||||
if not summary:
|
||||
return _MISSING_SUMMARY_ERROR
|
||||
if (
|
||||
not isinstance(confidence_score, (int, float))
|
||||
or isinstance(confidence_score, bool)
|
||||
or not (0.0 <= float(confidence_score) <= 1.0)
|
||||
):
|
||||
return _BAD_CONFIDENCE_ERROR
|
||||
return None
|
||||
|
||||
|
||||
def _handle_audio_function_calls(
|
||||
sub_agent_contents: list,
|
||||
response,
|
||||
function_calls: list,
|
||||
full_text: str,
|
||||
) -> tuple[tuple[str, str, float] | None, str | None]:
|
||||
"""Return (answer, rejection), appending any rejection to the conversation."""
|
||||
original_parts = response.candidates[0].content.parts
|
||||
for fc in function_calls:
|
||||
if _tool_name(fc.name) != "submit_answer":
|
||||
continue
|
||||
args = dict(fc.args or {})
|
||||
summary = _clean_text(args.get("summary"))
|
||||
analysis = _clean_text(args.get("analysis"))
|
||||
|
||||
rejection = _validate_audio_answer(summary, args.get("confidence_score"))
|
||||
if rejection is not None:
|
||||
_append_rejection(
|
||||
sub_agent_contents, original_parts, function_calls, full_text, fc.name, rejection
|
||||
)
|
||||
return None, rejection
|
||||
|
||||
return (summary, analysis or "No analysis provided.", float(args["confidence_score"])), None
|
||||
|
||||
_append_rejection(
|
||||
sub_agent_contents,
|
||||
original_parts,
|
||||
function_calls,
|
||||
full_text,
|
||||
function_calls[0].name,
|
||||
_WRONG_TOOL_ERROR,
|
||||
)
|
||||
return None, _WRONG_TOOL_ERROR
|
||||
@@ -1,455 +0,0 @@
|
||||
# Copyright 2026 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Analyze video chunks through the Gemini Interactions API.
|
||||
|
||||
``processing: "agentic"`` lets the model search and revisit an uploaded clip.
|
||||
Validation errors are returned as function results, with video context retained
|
||||
through ``previous_interaction_id``. Only accepted answers reach the blackboard.
|
||||
|
||||
The stream consumer reconstructs function arguments from step deltas. Completion
|
||||
events may omit steps, so it fetches the interaction when reconstruction fails.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import asyncio
|
||||
import copy
|
||||
from dataclasses import dataclass, field
|
||||
import json
|
||||
from types import SimpleNamespace
|
||||
from typing import Any
|
||||
|
||||
from artemis.agents.video_analyzer import gemini_files
|
||||
from artemis.agents.video_analyzer import video_analyzer as _va
|
||||
from artemis.agents.video_analyzer.chunk_conversation import (
|
||||
_NO_TOOL_CALL_ERROR,
|
||||
_SUBMIT_NOW_WARNING,
|
||||
_WRONG_TOOL_ERROR,
|
||||
SUB_AGENT_MAX_TURNS,
|
||||
SubAgentAnswerExhausted,
|
||||
_ChunkMedia,
|
||||
_clean_text,
|
||||
_persist_timeline_events,
|
||||
_tool_name,
|
||||
_validate_answer,
|
||||
)
|
||||
from artemis.agents.video_analyzer.universal_tools import UNIVERSAL_SUBMIT_ANSWER_TOOL
|
||||
from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan
|
||||
from artemis.llm.google import normalize_usage
|
||||
from artemis.utils.logger import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
STREAM_START_TIMEOUT_SECONDS = 60.0
|
||||
INTERACTION_READ_TIMEOUT_SECONDS = 30.0
|
||||
|
||||
# Interaction statuses after which no follow-up turn can be sent
|
||||
# (``google.genai`` ``InteractionStatus``).
|
||||
_TERMINAL_FAILURE_STATUSES = frozenset({"failed", "cancelled", "incomplete", "budget_exceeded"})
|
||||
|
||||
|
||||
def build_interactions_submit_tool() -> dict[str, Any]:
|
||||
"""The ``submit_answer`` declaration in Interactions API (flat JSON schema) form.
|
||||
|
||||
Derived from the universal (OpenAI-style) declaration so the schema is
|
||||
written once; ``timeline_events`` is required to match the native path.
|
||||
"""
|
||||
function = UNIVERSAL_SUBMIT_ANSWER_TOOL["function"]
|
||||
parameters = copy.deepcopy(function["parameters"])
|
||||
required = list(parameters.get("required", []))
|
||||
if "timeline_events" not in required:
|
||||
required.append("timeline_events")
|
||||
parameters["required"] = required
|
||||
return {
|
||||
"type": "function",
|
||||
"name": function["name"],
|
||||
"description": function["description"],
|
||||
"parameters": parameters,
|
||||
}
|
||||
|
||||
|
||||
def _steps(interaction: Any) -> list:
|
||||
return list(getattr(interaction, "steps", None) or [])
|
||||
|
||||
|
||||
def _find_function_call(interaction: Any):
|
||||
"""The last ``function_call`` step of an interaction, if any."""
|
||||
for step in reversed(_steps(interaction)):
|
||||
if getattr(step, "type", None) == "function_call":
|
||||
return step
|
||||
return None
|
||||
|
||||
|
||||
def _raise_if_failed(interaction: Any) -> None:
|
||||
"""Raises when an interaction ended in a terminal failure state.
|
||||
|
||||
A follow-up with ``previous_interaction_id`` onto a failed interaction is
|
||||
rejected by the API, so the failure surfaces here as an exception and
|
||||
flows through the shared retry / classification layer instead.
|
||||
"""
|
||||
status = str(getattr(interaction, "status", None) or "").lower()
|
||||
errors = list(getattr(interaction, "errors", None) or [])
|
||||
if status not in _TERMINAL_FAILURE_STATUSES and not errors:
|
||||
return
|
||||
details = "; ".join(
|
||||
f"[{getattr(error, 'code', None)}] {getattr(error, 'message', error)}" for error in errors
|
||||
)
|
||||
raise RuntimeError(
|
||||
f"Gemini Interactions interaction {getattr(interaction, 'id', None)} ended with"
|
||||
f" status {status or 'unknown'}" + (f": {details}" if details else "")
|
||||
)
|
||||
|
||||
|
||||
async def run_agentic_chunk_conversation(
|
||||
analyzer,
|
||||
media: _ChunkMedia,
|
||||
current_start: float,
|
||||
current_end: float,
|
||||
start_time: float,
|
||||
end_time: float | None,
|
||||
specific_query: str,
|
||||
lease_owner: str,
|
||||
) -> str:
|
||||
"""Uploads the chunk and drives the agentic sub-agent to a committed answer."""
|
||||
file = None
|
||||
try:
|
||||
with TraceSpan(name="upload_video_to_gemini") as span:
|
||||
file = await analyzer.upload_and_poll_file(media.compressed_path)
|
||||
span.result = f"Uploaded {file.name}"
|
||||
logger.info(
|
||||
f"Invoking Gemini agentic video understanding for sub-agent task with model"
|
||||
f" {analyzer.model_name} (streaming)..."
|
||||
)
|
||||
trace_id = CURRENT_TRACE_ID.get()
|
||||
|
||||
if analyzer.ctx.data_engine and trace_id:
|
||||
analyzer.ctx.data_engine.record_trace(
|
||||
type="llm_call",
|
||||
name="video_sub_agent",
|
||||
payload={
|
||||
"contents": [
|
||||
f"file://{file.name}",
|
||||
media.prompt_with_context,
|
||||
],
|
||||
"system_instruction": analyzer.sub_system_prompt,
|
||||
"video_processing": "agentic",
|
||||
},
|
||||
parent_trace_id=trace_id,
|
||||
)
|
||||
|
||||
final_summary, final_analysis = await _drive_agentic_loop(
|
||||
analyzer,
|
||||
file,
|
||||
trace_id,
|
||||
media,
|
||||
start_time,
|
||||
end_time,
|
||||
specific_query,
|
||||
)
|
||||
|
||||
end_str = f"{current_end:.1f}s" if current_end is not None else "unknown"
|
||||
chunk_result = (
|
||||
f"[from {current_start:.1f}s to {end_str}] Summary:"
|
||||
f" {final_summary} Analysis: {final_analysis}"
|
||||
)
|
||||
analyzer.blackboard.complete_segment(
|
||||
current_start,
|
||||
current_end,
|
||||
specific_query,
|
||||
lease_owner,
|
||||
final_summary,
|
||||
final_analysis,
|
||||
)
|
||||
return chunk_result
|
||||
finally:
|
||||
if file:
|
||||
await gemini_files.delete_cloud_file(
|
||||
analyzer.client, file.name, analyzer.cloud_files_to_cleanup
|
||||
)
|
||||
|
||||
|
||||
def _initial_request(analyzer, file, prompt: str, tool: dict[str, Any]) -> dict[str, Any]:
|
||||
return {
|
||||
"model": analyzer.model_name,
|
||||
"input": [
|
||||
{
|
||||
"type": "video",
|
||||
"uri": file.uri,
|
||||
"mime_type": file.mime_type,
|
||||
"processing": "agentic",
|
||||
},
|
||||
{"type": "text", "text": prompt},
|
||||
],
|
||||
"system_instruction": analyzer.sub_system_prompt,
|
||||
"tools": [tool],
|
||||
"stream": True,
|
||||
}
|
||||
|
||||
|
||||
def _follow_up_request(
|
||||
analyzer, previous_id: str, steps: list[dict[str, Any]], tool: dict[str, Any]
|
||||
) -> dict[str, Any]:
|
||||
"""A stateful follow-up turn: the server keeps the video context.
|
||||
|
||||
Only the conversation state is inherited through
|
||||
``previous_interaction_id``; ``system_instruction`` and ``tools`` are
|
||||
interaction-scoped and must be re-sent on every turn.
|
||||
"""
|
||||
return {
|
||||
"model": analyzer.model_name,
|
||||
"previous_interaction_id": previous_id,
|
||||
"input": steps,
|
||||
"system_instruction": analyzer.sub_system_prompt,
|
||||
"tools": [tool],
|
||||
"stream": True,
|
||||
}
|
||||
|
||||
|
||||
def _function_result(call_id: str, error: str) -> dict[str, Any]:
|
||||
return {
|
||||
"type": "function_result",
|
||||
"call_id": call_id,
|
||||
"name": "submit_answer",
|
||||
"result": {"error": error},
|
||||
}
|
||||
|
||||
|
||||
async def _drive_agentic_loop(
|
||||
analyzer,
|
||||
file,
|
||||
trace_id,
|
||||
media: _ChunkMedia,
|
||||
start_time: float,
|
||||
end_time: float | None,
|
||||
specific_query: str,
|
||||
) -> tuple[str, str]:
|
||||
"""Return a validated answer and persist its timeline events.
|
||||
|
||||
Raise ``SubAgentAnswerExhausted`` if no answer is accepted within the turn limit.
|
||||
"""
|
||||
tool = build_interactions_submit_tool()
|
||||
request = _initial_request(analyzer, file, media.prompt_with_context, tool)
|
||||
rejection = _NO_TOOL_CALL_ERROR
|
||||
|
||||
for turn in range(1, SUB_AGENT_MAX_TURNS + 1):
|
||||
interaction = await _stream_agentic_turn(analyzer, request, trace_id)
|
||||
interaction_id = getattr(interaction, "id", None) or ""
|
||||
call = _find_function_call(interaction)
|
||||
|
||||
if call is None:
|
||||
rejection = _NO_TOOL_CALL_ERROR
|
||||
steps = [{"type": "text", "text": _SUBMIT_NOW_WARNING}]
|
||||
elif _tool_name(getattr(call, "name", None)) != "submit_answer":
|
||||
rejection = _WRONG_TOOL_ERROR
|
||||
steps = [_function_result(call.id, rejection)]
|
||||
else:
|
||||
args = dict(getattr(call, "arguments", None) or {})
|
||||
timeline_events = args.get("timeline_events", [])
|
||||
if not isinstance(timeline_events, list):
|
||||
timeline_events = []
|
||||
summary = _clean_text(args.get("summary"))
|
||||
analysis = _clean_text(args.get("analysis"))
|
||||
|
||||
rejection = _validate_answer(summary, timeline_events, start_time, end_time)
|
||||
if rejection is None:
|
||||
await _persist_timeline_events(
|
||||
analyzer, timeline_events, media, start_time, end_time, specific_query
|
||||
)
|
||||
return summary, analysis or "No analysis provided."
|
||||
steps = [_function_result(call.id, rejection)]
|
||||
|
||||
if turn < SUB_AGENT_MAX_TURNS:
|
||||
request = _follow_up_request(analyzer, interaction_id, steps, tool)
|
||||
|
||||
raise SubAgentAnswerExhausted(SUB_AGENT_MAX_TURNS, rejection)
|
||||
|
||||
|
||||
async def _stream_agentic_turn(analyzer, request: dict[str, Any], trace_id):
|
||||
"""Runs one traced streaming Interactions turn and returns the completed interaction."""
|
||||
with TraceSpan(name="gemini_agentic_video_call") as span:
|
||||
span.payload = {
|
||||
"model": request.get("model"),
|
||||
"video_processing": "agentic",
|
||||
"previous_interaction_id": request.get("previous_interaction_id"),
|
||||
}
|
||||
stream = await asyncio.wait_for(
|
||||
analyzer.client.aio.interactions.create(**request),
|
||||
timeout=STREAM_START_TIMEOUT_SECONDS,
|
||||
)
|
||||
interaction = await asyncio.wait_for(
|
||||
_consume_agentic_stream(analyzer, span, stream, trace_id),
|
||||
timeout=analyzer.agentic_call_timeout_seconds,
|
||||
)
|
||||
call = _find_function_call(interaction)
|
||||
span.result = (
|
||||
f"Function call: {_tool_name(getattr(call, 'name', None))}"
|
||||
if call is not None
|
||||
else f"Status: {getattr(interaction, 'status', 'unknown')}"
|
||||
)
|
||||
return interaction
|
||||
|
||||
|
||||
@dataclass
|
||||
class _PendingStep:
|
||||
"""A step under construction while its start/delta/stop events stream in."""
|
||||
|
||||
kind: str
|
||||
call_id: str | None = None
|
||||
name: str | None = None
|
||||
seed_arguments: dict[str, Any] = field(default_factory=dict)
|
||||
arguments_json: list[str] = field(default_factory=list)
|
||||
text: list[str] = field(default_factory=list)
|
||||
|
||||
def finish(self):
|
||||
"""Materializes the step in the shape the SDK's typed steps expose.
|
||||
|
||||
Returns ``None`` for a function call whose streamed arguments are not
|
||||
valid JSON: a half-built call must not stand in for the real one, so
|
||||
the step is dropped and the interaction is re-read by id instead.
|
||||
"""
|
||||
if self.kind == "function_call":
|
||||
arguments: dict[str, Any] = dict(self.seed_arguments)
|
||||
raw = "".join(self.arguments_json)
|
||||
if raw.strip():
|
||||
try:
|
||||
parsed = json.loads(raw)
|
||||
except ValueError:
|
||||
logger.warning(
|
||||
f"Could not parse streamed arguments for {self.name}: {raw[:200]!r};"
|
||||
" dropping the step"
|
||||
)
|
||||
return None
|
||||
if isinstance(parsed, dict):
|
||||
arguments = parsed
|
||||
return SimpleNamespace(
|
||||
type="function_call", id=self.call_id, name=self.name, arguments=arguments
|
||||
)
|
||||
if self.kind == "model_output":
|
||||
return SimpleNamespace(
|
||||
type="model_output",
|
||||
content=[SimpleNamespace(type="text", text="".join(self.text))],
|
||||
)
|
||||
return SimpleNamespace(type=self.kind, id=self.call_id)
|
||||
|
||||
|
||||
async def _consume_agentic_stream(analyzer, span, stream, trace_id):
|
||||
"""Consumes Interactions SSE events, rebuilding steps and mirroring progress.
|
||||
|
||||
Returns an interaction-shaped object (``id`` / ``status`` / ``steps`` /
|
||||
``usage``). The interaction is re-read by id when the stream ended
|
||||
without a completion event, or when it stopped at ``requires_action``
|
||||
without a parsable function call.
|
||||
"""
|
||||
data_engine = analyzer.ctx.data_engine if trace_id else None
|
||||
interaction_id: str | None = None
|
||||
status: str | None = None
|
||||
completed_steps: list | None = None
|
||||
usage: dict[str, int] | None = None
|
||||
pending: dict[int, _PendingStep] = {}
|
||||
steps: list = []
|
||||
processing_calls = 0
|
||||
|
||||
def _emit(text: str, *, thinking: bool) -> None:
|
||||
if data_engine and text:
|
||||
data_engine.stream_output(trace_id, text, is_thinking=thinking)
|
||||
|
||||
async for event in stream:
|
||||
data = getattr(event, "data", event)
|
||||
event_type = getattr(data, "event_type", None)
|
||||
|
||||
if event_type == "interaction.created":
|
||||
created = getattr(data, "interaction", None)
|
||||
interaction_id = getattr(created, "id", None) or interaction_id
|
||||
elif event_type == "step.start":
|
||||
step = getattr(data, "step", None)
|
||||
index = getattr(data, "index", None)
|
||||
kind = str(getattr(step, "type", None) or "")
|
||||
pending[index] = _PendingStep(
|
||||
kind=kind,
|
||||
call_id=getattr(step, "id", None) or getattr(step, "call_id", None),
|
||||
name=getattr(step, "name", None),
|
||||
seed_arguments=dict(getattr(step, "arguments", None) or {}),
|
||||
)
|
||||
if kind == "processing_call":
|
||||
processing_calls += 1
|
||||
_emit(
|
||||
f"Inspecting the recording (video lookup {processing_calls})\n",
|
||||
thinking=True,
|
||||
)
|
||||
elif event_type == "step.delta":
|
||||
delta = getattr(data, "delta", None)
|
||||
delta_type = getattr(delta, "type", None)
|
||||
current = pending.get(getattr(data, "index", None))
|
||||
if delta_type == "arguments_delta":
|
||||
fragment = getattr(delta, "arguments", None)
|
||||
if current is not None and isinstance(fragment, str):
|
||||
current.arguments_json.append(fragment)
|
||||
elif delta_type == "text":
|
||||
text = getattr(delta, "text", None)
|
||||
if current is not None and isinstance(text, str):
|
||||
current.text.append(text)
|
||||
_emit(text or "", thinking=False)
|
||||
elif delta_type == "thought_summary":
|
||||
content = getattr(delta, "content", None)
|
||||
_emit(getattr(content, "text", None) or "", thinking=True)
|
||||
elif event_type == "step.stop":
|
||||
current = pending.pop(getattr(data, "index", None), None)
|
||||
finished = current.finish() if current is not None else None
|
||||
if finished is not None:
|
||||
steps.append(finished)
|
||||
usage = normalize_usage(getattr(data, "usage", None)) or usage
|
||||
elif event_type == "interaction.completed":
|
||||
completed = getattr(data, "interaction", None)
|
||||
interaction_id = getattr(completed, "id", None) or interaction_id
|
||||
status = getattr(completed, "status", None) or status
|
||||
completed_steps = getattr(completed, "steps", None) or None
|
||||
usage = normalize_usage(getattr(completed, "usage", None)) or usage
|
||||
_raise_if_failed(completed)
|
||||
elif event_type == "interaction.status_update":
|
||||
status = getattr(data, "status", None) or status
|
||||
elif event_type == "error":
|
||||
error = getattr(data, "error", None)
|
||||
raise RuntimeError(
|
||||
"Gemini Interactions stream error"
|
||||
f" [{getattr(error, 'code', None)}]: {getattr(error, 'message', None)}"
|
||||
)
|
||||
|
||||
for current in pending.values():
|
||||
finished = current.finish()
|
||||
if finished is not None:
|
||||
steps.append(finished)
|
||||
span.payload["processing_calls"] = processing_calls
|
||||
if completed_steps:
|
||||
steps = list(completed_steps)
|
||||
|
||||
needs_reread = status is None or (
|
||||
status == "requires_action" and _find_function_call(SimpleNamespace(steps=steps)) is None
|
||||
)
|
||||
if needs_reread:
|
||||
if not interaction_id:
|
||||
raise RuntimeError("Gemini Interactions stream ended without an interaction")
|
||||
logger.info(f"Re-reading interaction {interaction_id} (status={status})")
|
||||
fetched = await asyncio.wait_for(
|
||||
analyzer.client.aio.interactions.get(interaction_id),
|
||||
timeout=INTERACTION_READ_TIMEOUT_SECONDS,
|
||||
)
|
||||
_raise_if_failed(fetched)
|
||||
status = getattr(fetched, "status", None) or status
|
||||
steps = _steps(fetched) or steps
|
||||
usage = normalize_usage(getattr(fetched, "usage", None)) or usage
|
||||
|
||||
if usage:
|
||||
span.payload["usage_metadata"] = usage
|
||||
return SimpleNamespace(id=interaction_id, status=status, steps=steps, usage=usage)
|
||||
@@ -1,491 +0,0 @@
|
||||
# Copyright 2026 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Native Gemini sub-agent conversation and submit_answer handling for chunks.
|
||||
|
||||
Extracted from ``video_analyzer.py`` as a pure structural split. Patchable
|
||||
collaborators are looked up late through the facade module so
|
||||
``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")`` targets
|
||||
keep working.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import glob
|
||||
from pathlib import Path
|
||||
import re
|
||||
|
||||
from google.genai import types
|
||||
|
||||
from artemis.agents.video_analyzer import gemini_files
|
||||
from artemis.agents.video_analyzer import video_analyzer as _va
|
||||
from artemis.agents.video_analyzer.reliability import SubAgentAnswerExhausted
|
||||
from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
|
||||
from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan
|
||||
from artemis.llm.google import normalize_usage
|
||||
from artemis.utils.logger import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
#: Turns the chunk sub-agent gets to produce an accepted ``submit_answer``
|
||||
#: (shared by the static and agentic paths).
|
||||
SUB_AGENT_MAX_TURNS = 2
|
||||
|
||||
_SUBMIT_NOW_WARNING = "[WARNING] You must call the submit_answer tool now, time is running out."
|
||||
_WRONG_TOOL_ERROR = "Tool not recognized. Please use submit_answer."
|
||||
_NO_TOOL_CALL_ERROR = "The model replied with text only and never called submit_answer."
|
||||
_MISSING_SUMMARY_ERROR = (
|
||||
"submit_answer was called without a 'summary'. Call it again with a non-empty"
|
||||
" 'summary' (and 'analysis') describing what you observed."
|
||||
)
|
||||
|
||||
|
||||
def _tool_name(name: str | None) -> str:
|
||||
name = name or ""
|
||||
return name.split(":")[-1] if ":" in name else name
|
||||
|
||||
|
||||
def _clean_text(value) -> str:
|
||||
return str(value or "").strip().replace("\n", " ")
|
||||
|
||||
|
||||
def _validate_answer(
|
||||
summary: str, timeline_events: list, start_time: float, end_time: float | None
|
||||
) -> str | None:
|
||||
"""Return a validation error for submit_answer, or None if valid."""
|
||||
if not summary:
|
||||
return _MISSING_SUMMARY_ERROR
|
||||
return _validate_timeline_events(timeline_events, start_time, end_time)
|
||||
|
||||
|
||||
class _ChunkMedia:
|
||||
"""Mutable holder for per-attempt media artifacts shared with failure handling."""
|
||||
|
||||
__slots__ = ("path", "compressed_path", "actual_start", "prompt_with_context")
|
||||
|
||||
def __init__(self):
|
||||
self.path: Path | None = None
|
||||
self.compressed_path: Path | None = None
|
||||
self.actual_start: float | None = None
|
||||
self.prompt_with_context: str | None = None
|
||||
|
||||
|
||||
async def _run_native_chunk_conversation(
|
||||
analyzer,
|
||||
media: _ChunkMedia,
|
||||
current_start: float,
|
||||
current_end: float,
|
||||
start_time: float,
|
||||
end_time: float | None,
|
||||
specific_query: str,
|
||||
lease_owner: str,
|
||||
) -> str:
|
||||
"""Uploads the chunk and drives the native Gemini sub-agent to a committed answer."""
|
||||
file = None
|
||||
try:
|
||||
with TraceSpan(name="upload_video_to_gemini") as span:
|
||||
file = await analyzer.upload_and_poll_file(media.compressed_path)
|
||||
span.result = f"Uploaded {file.name}"
|
||||
logger.info(
|
||||
f"Invoking Gemini for sub-agent task with model {analyzer.model_name} (streaming)..."
|
||||
)
|
||||
trace_id = CURRENT_TRACE_ID.get()
|
||||
|
||||
if analyzer.ctx.data_engine and trace_id:
|
||||
analyzer.ctx.data_engine.record_trace(
|
||||
type="llm_call",
|
||||
name="video_sub_agent",
|
||||
payload={
|
||||
"contents": [
|
||||
f"file://{file.name}",
|
||||
media.prompt_with_context,
|
||||
],
|
||||
"system_instruction": analyzer.sub_system_prompt,
|
||||
},
|
||||
parent_trace_id=trace_id,
|
||||
)
|
||||
|
||||
sub_agent_contents = [
|
||||
types.Content(
|
||||
role="user",
|
||||
parts=[
|
||||
types.Part.from_uri(file_uri=file.uri, mime_type=file.mime_type)
|
||||
if hasattr(file, "uri")
|
||||
else file,
|
||||
types.Part.from_text(text=media.prompt_with_context),
|
||||
],
|
||||
)
|
||||
]
|
||||
|
||||
final_summary, final_analysis = await _drive_sub_agent_loop(
|
||||
analyzer,
|
||||
sub_agent_contents,
|
||||
trace_id,
|
||||
media,
|
||||
start_time,
|
||||
end_time,
|
||||
specific_query,
|
||||
)
|
||||
|
||||
end_str = f"{current_end:.1f}s" if current_end is not None else "unknown"
|
||||
chunk_result = (
|
||||
f"[from {current_start:.1f}s to {end_str}] Summary:"
|
||||
f" {final_summary} Analysis: {final_analysis}"
|
||||
)
|
||||
analyzer.blackboard.complete_segment(
|
||||
current_start,
|
||||
current_end,
|
||||
specific_query,
|
||||
lease_owner,
|
||||
final_summary,
|
||||
final_analysis,
|
||||
)
|
||||
return chunk_result
|
||||
finally:
|
||||
if file:
|
||||
await gemini_files.delete_cloud_file(
|
||||
analyzer.client, file.name, analyzer.cloud_files_to_cleanup
|
||||
)
|
||||
|
||||
|
||||
async def _drive_sub_agent_loop(
|
||||
analyzer,
|
||||
sub_agent_contents: list,
|
||||
trace_id,
|
||||
media: _ChunkMedia,
|
||||
start_time: float,
|
||||
end_time: float | None,
|
||||
specific_query: str,
|
||||
) -> tuple[str, str]:
|
||||
"""Return a validated answer and persist its timeline events.
|
||||
|
||||
Raise ``SubAgentAnswerExhausted`` if no answer is accepted within the turn limit.
|
||||
"""
|
||||
rejection = _NO_TOOL_CALL_ERROR
|
||||
|
||||
for turn in range(1, SUB_AGENT_MAX_TURNS + 1):
|
||||
if turn == SUB_AGENT_MAX_TURNS - 1:
|
||||
sub_agent_contents.append(
|
||||
types.Content(
|
||||
role="user",
|
||||
parts=[types.Part.from_text(text=_SUBMIT_NOW_WARNING)],
|
||||
)
|
||||
)
|
||||
|
||||
full_text, function_calls, original_parts = await _stream_sub_agent_turn(
|
||||
analyzer, sub_agent_contents, trace_id
|
||||
)
|
||||
|
||||
if function_calls:
|
||||
answer, rejection = await _handle_sub_agent_calls(
|
||||
analyzer,
|
||||
sub_agent_contents,
|
||||
function_calls,
|
||||
original_parts,
|
||||
full_text,
|
||||
media,
|
||||
start_time,
|
||||
end_time,
|
||||
specific_query,
|
||||
)
|
||||
if answer is not None:
|
||||
return answer
|
||||
else:
|
||||
rejection = _NO_TOOL_CALL_ERROR
|
||||
sub_agent_contents.append(
|
||||
types.Content(
|
||||
role="model",
|
||||
parts=[types.Part.from_text(text=full_text)],
|
||||
)
|
||||
)
|
||||
|
||||
raise SubAgentAnswerExhausted(SUB_AGENT_MAX_TURNS, rejection)
|
||||
|
||||
|
||||
async def _stream_sub_agent_turn(
|
||||
analyzer, sub_agent_contents: list, trace_id
|
||||
) -> tuple[str, list, list]:
|
||||
"""Starts one streamed Gemini turn and consumes it fully."""
|
||||
with TraceSpan(name="gemini_stream_content_main") as span:
|
||||
stream = await asyncio.wait_for(
|
||||
analyzer.client.aio.models.generate_content_stream(
|
||||
model=analyzer.model_name,
|
||||
contents=sub_agent_contents,
|
||||
config=types.GenerateContentConfig(
|
||||
system_instruction=analyzer.sub_system_prompt,
|
||||
tools=[types.Tool(function_declarations=[analyzer.submit_answer_tool])],
|
||||
safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
|
||||
),
|
||||
),
|
||||
timeout=60,
|
||||
)
|
||||
|
||||
full_text, function_calls, original_parts = await asyncio.wait_for(
|
||||
_read_sub_agent_stream(analyzer, span, stream, trace_id),
|
||||
timeout=analyzer.model_call_timeout_seconds,
|
||||
)
|
||||
return full_text, function_calls, original_parts
|
||||
|
||||
|
||||
async def _read_sub_agent_stream(analyzer, span, target_stream, current_trace_id):
|
||||
"""Consumes a streamed response, mirroring text/thoughts into the trace."""
|
||||
text = ""
|
||||
function_calls = []
|
||||
accumulated_parts = []
|
||||
async for chunk in target_stream:
|
||||
usage = normalize_usage(getattr(chunk, "usage_metadata", None))
|
||||
if usage:
|
||||
span.payload["usage_metadata"] = usage
|
||||
|
||||
chunk_text = chunk.text or ""
|
||||
if chunk_text:
|
||||
text += chunk_text
|
||||
if analyzer.ctx.data_engine and current_trace_id:
|
||||
analyzer.ctx.data_engine.stream_output(current_trace_id, chunk_text)
|
||||
if chunk.function_calls:
|
||||
function_calls.extend(chunk.function_calls)
|
||||
|
||||
candidates = getattr(chunk, "candidates", None)
|
||||
if candidates and isinstance(candidates, list) and len(candidates) > 0:
|
||||
content = getattr(candidates[0], "content", None)
|
||||
parts = getattr(content, "parts", None) if content else None
|
||||
if parts:
|
||||
for part in parts:
|
||||
accumulated_parts.append(part)
|
||||
if getattr(part, "thought", False) and part.text:
|
||||
if analyzer.ctx.data_engine and current_trace_id:
|
||||
analyzer.ctx.data_engine.stream_output(
|
||||
current_trace_id,
|
||||
part.text,
|
||||
is_thinking=True,
|
||||
)
|
||||
else:
|
||||
if chunk.function_calls:
|
||||
for fc in chunk.function_calls:
|
||||
accumulated_parts.append(types.Part(function_call=fc))
|
||||
|
||||
return text, function_calls, accumulated_parts
|
||||
|
||||
|
||||
async def _handle_sub_agent_calls(
|
||||
analyzer,
|
||||
sub_agent_contents: list,
|
||||
function_calls: list,
|
||||
original_parts: list,
|
||||
full_text: str,
|
||||
media: _ChunkMedia,
|
||||
start_time: float,
|
||||
end_time: float | None,
|
||||
specific_query: str,
|
||||
) -> tuple[tuple[str, str] | None, str | None]:
|
||||
"""Return ``(answer, rejection)`` after processing one turn's tool calls.
|
||||
|
||||
Persist events for accepted answers; append rejected calls to the conversation
|
||||
with an error for the model to correct.
|
||||
"""
|
||||
for fc in function_calls:
|
||||
if _tool_name(fc.name) != "submit_answer":
|
||||
continue
|
||||
args = dict(fc.args or {})
|
||||
timeline_events = args.get("timeline_events", [])
|
||||
if not isinstance(timeline_events, list):
|
||||
timeline_events = []
|
||||
summary = _clean_text(args.get("summary"))
|
||||
analysis = _clean_text(args.get("analysis"))
|
||||
|
||||
rejection = _validate_answer(summary, timeline_events, start_time, end_time)
|
||||
if rejection is not None:
|
||||
_append_rejection(
|
||||
sub_agent_contents, original_parts, function_calls, full_text, fc.name, rejection
|
||||
)
|
||||
return None, rejection
|
||||
|
||||
await _persist_timeline_events(
|
||||
analyzer, timeline_events, media, start_time, end_time, specific_query
|
||||
)
|
||||
return (summary, analysis or "No analysis provided."), None
|
||||
|
||||
_append_rejection(
|
||||
sub_agent_contents,
|
||||
original_parts,
|
||||
function_calls,
|
||||
full_text,
|
||||
function_calls[0].name,
|
||||
_WRONG_TOOL_ERROR,
|
||||
)
|
||||
return None, _WRONG_TOOL_ERROR
|
||||
|
||||
|
||||
def _validate_timeline_events(
|
||||
timeline_events: list, start_time: float, end_time: float | None
|
||||
) -> str | None:
|
||||
"""Returns the first validation error message for the submitted events, if any."""
|
||||
for event in timeline_events:
|
||||
if (
|
||||
"confidence_score" not in event
|
||||
or not isinstance(
|
||||
event["confidence_score"],
|
||||
(int, float),
|
||||
)
|
||||
or isinstance(event["confidence_score"], bool)
|
||||
or not (0.0 <= float(event["confidence_score"]) <= 1.0)
|
||||
):
|
||||
return (
|
||||
"Error: 'confidence_score' is"
|
||||
" missing or out of bounds in"
|
||||
" one of the timeline_events."
|
||||
" It must be a float between"
|
||||
" 0.0 and 1.0 inclusive. Fix"
|
||||
" the payload and call"
|
||||
" submit_answer again."
|
||||
)
|
||||
|
||||
if (
|
||||
"verification_timestamp_secs" not in event
|
||||
or not isinstance(
|
||||
event["verification_timestamp_secs"],
|
||||
(int, float),
|
||||
)
|
||||
or isinstance(event["verification_timestamp_secs"], bool)
|
||||
):
|
||||
return (
|
||||
"Error:"
|
||||
" 'verification_timestamp_secs'"
|
||||
" is missing or invalid in one"
|
||||
" of the timeline_events. Fix"
|
||||
" the payload and call"
|
||||
" submit_answer again."
|
||||
)
|
||||
|
||||
verification_ts = float(
|
||||
event.get(
|
||||
"verification_timestamp_secs",
|
||||
start_time,
|
||||
)
|
||||
)
|
||||
confidence = float(event.get("confidence_score", 0.0)) # noqa: F841
|
||||
|
||||
if verification_ts < start_time or (end_time is not None and verification_ts > end_time):
|
||||
return (
|
||||
"Error:"
|
||||
" 'verification_timestamp_secs'"
|
||||
f" {verification_ts} must fall"
|
||||
" within the start_time"
|
||||
f" ({start_time}) and end_time"
|
||||
f" ({end_time}) boundaries. Fix"
|
||||
" the payload and call"
|
||||
" submit_answer again."
|
||||
)
|
||||
return None
|
||||
|
||||
|
||||
def _append_rejection(
|
||||
sub_agent_contents: list,
|
||||
original_parts: list,
|
||||
function_calls: list,
|
||||
full_text: str,
|
||||
response_name: str,
|
||||
error_msg: str,
|
||||
) -> None:
|
||||
"""Appends the model turn plus an error function response to the conversation."""
|
||||
sub_agent_contents.append(
|
||||
types.Content(
|
||||
role="model",
|
||||
parts=original_parts
|
||||
if original_parts
|
||||
else [types.Part(function_call=fc) for fc in function_calls]
|
||||
+ ([types.Part.from_text(text=full_text)] if full_text else []),
|
||||
)
|
||||
)
|
||||
sub_agent_contents.append(
|
||||
types.Content(
|
||||
role="user",
|
||||
parts=[
|
||||
types.Part.from_function_response(
|
||||
name=response_name,
|
||||
response={"error": (error_msg)},
|
||||
)
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
async def _persist_timeline_events(
|
||||
analyzer,
|
||||
timeline_events: list,
|
||||
media: _ChunkMedia,
|
||||
start_time: float,
|
||||
end_time: float | None,
|
||||
specific_query: str,
|
||||
) -> None:
|
||||
"""Captures verification screenshots and records validated events on the blackboard."""
|
||||
for event in timeline_events:
|
||||
verification_ts = float(
|
||||
event.get(
|
||||
"verification_timestamp_secs",
|
||||
start_time,
|
||||
)
|
||||
)
|
||||
confidence = float(event.get("confidence_score", 0.0))
|
||||
relative_offset = max(0.0, verification_ts - media.actual_start)
|
||||
|
||||
out_dir = _va.get_temp_dir("video_analyzer")
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
existing_imgs = glob.glob(f"{out_dir}/img_*.jpg")
|
||||
max_idx = -1
|
||||
for img in existing_imgs:
|
||||
match = re.search(r"img_(\d+)\.jpg$", img)
|
||||
if match:
|
||||
max_idx = max(max_idx, int(match.group(1)))
|
||||
next_idx = max_idx + 1
|
||||
screenshot_output_path = out_dir / f"img_{next_idx}.jpg"
|
||||
while screenshot_output_path.exists():
|
||||
next_idx += 1
|
||||
screenshot_output_path = out_dir / f"img_{next_idx}.jpg"
|
||||
screenshot_output_path.touch()
|
||||
analyzer.local_files_to_cleanup.add(screenshot_output_path)
|
||||
ffmpeg_cmd = [
|
||||
"ffmpeg",
|
||||
"-ss",
|
||||
str(relative_offset),
|
||||
"-i",
|
||||
str(media.path),
|
||||
"-vframes",
|
||||
"1",
|
||||
"-q:v",
|
||||
"2",
|
||||
"-y",
|
||||
str(screenshot_output_path),
|
||||
]
|
||||
try:
|
||||
proc = await asyncio.create_subprocess_exec(
|
||||
*ffmpeg_cmd,
|
||||
stdout=asyncio.subprocess.PIPE,
|
||||
stderr=asyncio.subprocess.PIPE,
|
||||
)
|
||||
await proc.communicate()
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to extract frame: {e}")
|
||||
|
||||
screenshot_path_str = str(screenshot_output_path)
|
||||
entry = {
|
||||
"start": event.get("start_time", start_time),
|
||||
"end": event.get("end_time", end_time),
|
||||
"target": specific_query,
|
||||
"summary": event.get("transcription", ""),
|
||||
"confidence_score": confidence,
|
||||
}
|
||||
if Path(screenshot_output_path).exists():
|
||||
entry["screenshot"] = str(screenshot_path_str)
|
||||
|
||||
analyzer._record_blackboard_entry(entry)
|
||||
+29
-126
@@ -12,29 +12,23 @@
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Native Gemini execution path for a single video chunk sub-agent.
|
||||
"""Responses execution path for a single video chunk sub-agent.
|
||||
|
||||
Extracted from ``video_analyzer.py`` as a pure structural split. Patchable
|
||||
collaborators (``get_controller``, ``compress_video_for_api``, semaphores,
|
||||
LLM event recorders, ...) are looked up late through the facade module so
|
||||
``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")`` targets
|
||||
keep working. The sub-agent conversation itself lives in
|
||||
``chunk_conversation``.
|
||||
``universal_engine``.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
|
||||
from artemis.agents.video_analyzer import video_analyzer as _va
|
||||
from artemis.agents.video_analyzer.chunk_agentic import run_agentic_chunk_conversation
|
||||
from artemis.agents.video_analyzer.chunk_conversation import (
|
||||
_ChunkMedia,
|
||||
_run_native_chunk_conversation,
|
||||
)
|
||||
from artemis.agents.video_analyzer.reliability import (
|
||||
AgenticVideoDegraded,
|
||||
classify_video_failure,
|
||||
is_agentic_rejection,
|
||||
)
|
||||
from artemis.llm.reliability import retry_policy_for
|
||||
from artemis.utils.logger import get_logger
|
||||
@@ -42,11 +36,23 @@ from artemis.utils.logger import get_logger
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
class _ChunkMedia:
|
||||
"""Mutable holder for per-attempt media artifacts shared with failure handling."""
|
||||
|
||||
__slots__ = ("path", "compressed_path", "actual_start", "prompt_with_context")
|
||||
|
||||
def __init__(self):
|
||||
self.path: Path | None = None
|
||||
self.compressed_path: Path | None = None
|
||||
self.actual_start: float | None = None
|
||||
self.prompt_with_context: str | None = None
|
||||
|
||||
|
||||
async def exec_single_chunk(
|
||||
analyzer, start_time: float, end_time: float | None, specific_query: str
|
||||
) -> str:
|
||||
"""Analyzes one closed chunk interval, with claim/lease and retry handling."""
|
||||
max_retries = analyzer.native_max_retries
|
||||
max_retries = analyzer.max_retries
|
||||
current_start = round(start_time, 1) if isinstance(start_time, (int, float)) else start_time
|
||||
current_end = round(end_time, 1) if isinstance(end_time, (int, float)) else end_time
|
||||
|
||||
@@ -85,7 +91,6 @@ async def exec_single_chunk(
|
||||
while attempt <= max_retries:
|
||||
# The mode this attempt was launched in: a parallel chunk may degrade
|
||||
# the run while this one is still in flight.
|
||||
attempt_mode = analyzer.video_processing
|
||||
try:
|
||||
logger.info(
|
||||
f"Attempt {attempt + 1}/{max_retries + 1} for sub-agent:"
|
||||
@@ -108,49 +113,25 @@ async def exec_single_chunk(
|
||||
)
|
||||
_ensure_fits_static_ceiling(analyzer, current_start, current_end)
|
||||
|
||||
if not analyzer.use_native_gemini:
|
||||
chunk_result = await analyzer._exec_single_chunk_universal(
|
||||
compressed_path=media.compressed_path,
|
||||
raw_path=media.path,
|
||||
start_time=current_start,
|
||||
end_time=current_end,
|
||||
actual_start=media.actual_start,
|
||||
prompt_with_context=media.prompt_with_context,
|
||||
specific_query=specific_query,
|
||||
)
|
||||
summary, analysis = analyzer._parse_chunk_result(chunk_result)
|
||||
analyzer.blackboard.complete_segment(
|
||||
current_start,
|
||||
current_end,
|
||||
specific_query,
|
||||
lease_owner,
|
||||
summary,
|
||||
analysis,
|
||||
)
|
||||
return chunk_result
|
||||
|
||||
if analyzer.video_processing == "agentic":
|
||||
return await run_agentic_chunk_conversation(
|
||||
analyzer,
|
||||
media,
|
||||
current_start,
|
||||
current_end,
|
||||
start_time,
|
||||
end_time,
|
||||
specific_query,
|
||||
lease_owner,
|
||||
)
|
||||
|
||||
return await _run_native_chunk_conversation(
|
||||
analyzer,
|
||||
media,
|
||||
chunk_result = await analyzer._exec_single_chunk_universal(
|
||||
compressed_path=media.compressed_path,
|
||||
raw_path=media.path,
|
||||
start_time=current_start,
|
||||
end_time=current_end,
|
||||
actual_start=media.actual_start,
|
||||
prompt_with_context=media.prompt_with_context,
|
||||
specific_query=specific_query,
|
||||
)
|
||||
summary, analysis = analyzer._parse_chunk_result(chunk_result)
|
||||
analyzer.blackboard.complete_segment(
|
||||
current_start,
|
||||
current_end,
|
||||
start_time,
|
||||
end_time,
|
||||
specific_query,
|
||||
lease_owner,
|
||||
summary,
|
||||
analysis,
|
||||
)
|
||||
return chunk_result
|
||||
|
||||
except Exception as e:
|
||||
# Retry decisions are owned by the shared classification layer.
|
||||
@@ -159,22 +140,6 @@ async def exec_single_chunk(
|
||||
# stream output from the failed attempt is discarded, never
|
||||
# surfaced as a complete answer.
|
||||
failure = classify_video_failure(e)
|
||||
if attempt_mode == "agentic" and is_agentic_rejection(failure, e):
|
||||
# Switch to static and let the coordinator resize the interval.
|
||||
_degrade_agentic_to_static(analyzer, e, failure)
|
||||
return await _finish_failed_chunk(
|
||||
analyzer,
|
||||
media,
|
||||
current_start,
|
||||
current_end,
|
||||
specific_query,
|
||||
lease_owner,
|
||||
AgenticVideoDegraded(
|
||||
f"Agentic video processing rejected ({e}); re-plan"
|
||||
f" {current_start:.1f}s-{current_end:.1f}s at the static chunk size"
|
||||
),
|
||||
retryable=False,
|
||||
)
|
||||
attempts_so_far = attempt + 1
|
||||
if not failure.retryable:
|
||||
logger.warning(
|
||||
@@ -229,26 +194,6 @@ async def exec_single_chunk(
|
||||
)
|
||||
|
||||
|
||||
def _degrade_agentic_to_static(analyzer, error: Exception, failure) -> None:
|
||||
"""Switches the rest of this run to static video understanding (idempotent)."""
|
||||
if analyzer.video_processing != "agentic":
|
||||
return
|
||||
logger.warning(
|
||||
f"Agentic video processing rejected ({error}); the rest of this run"
|
||||
" analyzes video in static mode"
|
||||
)
|
||||
_va._record_llm_event(
|
||||
"llm_fallback",
|
||||
{
|
||||
"reason": "agentic_video_rejected",
|
||||
"category": failure.category.value,
|
||||
"error": str(error)[:500],
|
||||
},
|
||||
)
|
||||
analyzer.video_processing = "static"
|
||||
analyzer.chunk_size_seconds = analyzer.static_chunk_size_seconds
|
||||
|
||||
|
||||
def _ensure_fits_static_ceiling(analyzer, current_start: float, current_end: float) -> None:
|
||||
"""Refuses to analyze a clip sized for the agentic ceiling once the run is static.
|
||||
|
||||
@@ -280,48 +225,6 @@ async def _finish_failed_chunk(
|
||||
):
|
||||
"""Records terminal failure, attempting the universal fallback first when eligible."""
|
||||
failure = classify_video_failure(error)
|
||||
if (
|
||||
analyzer.use_native_gemini
|
||||
and failure.should_fallback
|
||||
and media.path is not None
|
||||
and media.compressed_path is not None
|
||||
and media.actual_start is not None
|
||||
and media.prompt_with_context is not None
|
||||
):
|
||||
try:
|
||||
logger.warning("Native video chunk exhausted retries; using universal fallback")
|
||||
_va._record_llm_event(
|
||||
"llm_fallback",
|
||||
{
|
||||
"reason": "native_video_chunk_failed",
|
||||
"category": failure.category.value,
|
||||
"error": str(error)[:500],
|
||||
},
|
||||
)
|
||||
fallback_result = await analyzer._exec_single_chunk_universal(
|
||||
compressed_path=media.compressed_path,
|
||||
raw_path=media.path,
|
||||
start_time=current_start,
|
||||
end_time=current_end,
|
||||
actual_start=media.actual_start,
|
||||
prompt_with_context=media.prompt_with_context,
|
||||
specific_query=specific_query,
|
||||
force_fallback=True,
|
||||
)
|
||||
summary, analysis = analyzer._parse_chunk_result(fallback_result)
|
||||
analyzer.blackboard.complete_segment(
|
||||
current_start,
|
||||
current_end,
|
||||
specific_query,
|
||||
lease_owner,
|
||||
summary,
|
||||
analysis,
|
||||
)
|
||||
return fallback_result
|
||||
except Exception as fallback_error:
|
||||
error = fallback_error
|
||||
failure = classify_video_failure(error)
|
||||
retryable = None
|
||||
analyzer.blackboard.fail_segment(
|
||||
current_start,
|
||||
current_end,
|
||||
@@ -1,162 +0,0 @@
|
||||
# Copyright 2026 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Gemini File API helpers: upload/poll lifecycle and abandoned-file cleanup."""
|
||||
|
||||
import asyncio
|
||||
from datetime import datetime
|
||||
from pathlib import Path
|
||||
import time
|
||||
|
||||
from artemis.utils.logger import get_logger
|
||||
|
||||
try:
|
||||
from datetime import UTC
|
||||
except ImportError:
|
||||
UTC = UTC
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
_LAST_CLEANUP_TIME = 0.0
|
||||
_CLEANUP_LOCK = asyncio.Lock()
|
||||
|
||||
|
||||
async def cleanup_abandoned_gemini_files(client) -> None:
|
||||
"""Scan and delete remaining cloud video files whose creation time has elapsed standard TTL."""
|
||||
global _LAST_CLEANUP_TIME
|
||||
if _CLEANUP_LOCK.locked():
|
||||
return
|
||||
|
||||
async with _CLEANUP_LOCK:
|
||||
now_ts = time.time()
|
||||
if now_ts - _LAST_CLEANUP_TIME < 3600:
|
||||
return
|
||||
_LAST_CLEANUP_TIME = now_ts
|
||||
|
||||
try:
|
||||
pager = await asyncio.wait_for(client.aio.files.list(), timeout=30)
|
||||
files = []
|
||||
async for f in pager:
|
||||
files.append(f)
|
||||
now = datetime.now(UTC)
|
||||
files_to_delete = []
|
||||
|
||||
for f in files:
|
||||
display_name = getattr(f, "display_name", "") or ""
|
||||
|
||||
if (
|
||||
display_name.startswith("compressed_")
|
||||
or display_name.startswith("audio_")
|
||||
or "artemis" in display_name
|
||||
):
|
||||
create_time = getattr(f, "created_at", None) or getattr(f, "create_time", None)
|
||||
if not create_time:
|
||||
continue
|
||||
|
||||
if isinstance(create_time, str):
|
||||
try:
|
||||
parsed_time = datetime.fromisoformat(create_time.replace("Z", "+00:00"))
|
||||
except Exception:
|
||||
continue
|
||||
elif isinstance(create_time, datetime):
|
||||
parsed_time = create_time
|
||||
else:
|
||||
continue
|
||||
|
||||
if parsed_time.tzinfo is None:
|
||||
parsed_time = parsed_time.replace(tzinfo=UTC)
|
||||
|
||||
age_seconds = (now - parsed_time).total_seconds()
|
||||
|
||||
if age_seconds > 7200:
|
||||
files_to_delete.append(f)
|
||||
logger.info(
|
||||
f"Marked expired cloud asset for deletion: {f.name}"
|
||||
f" ({display_name}), Age: {age_seconds / 3600:.1f}h"
|
||||
)
|
||||
|
||||
if files_to_delete:
|
||||
logger.info(f"Purging {len(files_to_delete)} expired cloud assets in parallel...")
|
||||
tasks = [
|
||||
asyncio.wait_for(client.aio.files.delete(name=f.name), timeout=30)
|
||||
for f in files_to_delete
|
||||
]
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
except Exception as e:
|
||||
logger.warning(f"Routine cloud maintenance skipped: {e}")
|
||||
|
||||
|
||||
async def delete_cloud_file(
|
||||
client, file_name: str, cloud_files_to_cleanup: set, *, timeout: float = 30.0
|
||||
) -> None:
|
||||
"""Best-effort deletion of an uploaded file; failures are logged, never raised."""
|
||||
logger.info(f"Cleaning up cloud file {file_name}...")
|
||||
try:
|
||||
await asyncio.wait_for(client.aio.files.delete(name=file_name), timeout=timeout)
|
||||
cloud_files_to_cleanup.discard(file_name)
|
||||
except Exception as ce:
|
||||
logger.error(f"Failed to delete cloud file {file_name}: {ce}")
|
||||
|
||||
|
||||
async def upload_and_poll_file(client, compressed_path: Path, cloud_files_to_cleanup: set) -> any:
|
||||
"""Upload a media file to the Gemini File API and poll until it is ACTIVE."""
|
||||
logger.info(f"Uploading {compressed_path} to Gemini File API...")
|
||||
file_size_mb = compressed_path.stat().st_size / (1024 * 1024)
|
||||
upload_timeout = max(30.0, min(120.0, file_size_mb * 2.0))
|
||||
|
||||
file = await asyncio.wait_for(
|
||||
client.aio.files.upload(file=compressed_path),
|
||||
timeout=upload_timeout,
|
||||
)
|
||||
cloud_files_to_cleanup.add(file.name)
|
||||
|
||||
max_wait = max(60, min(180, 60 + int(file_size_mb * 2)))
|
||||
|
||||
wait_interval = 0.5
|
||||
start_wait = time.time()
|
||||
retry_count = 0
|
||||
max_retries = 3
|
||||
|
||||
while True:
|
||||
try:
|
||||
f_state = await asyncio.wait_for(client.aio.files.get(name=file.name), timeout=20)
|
||||
retry_count = 0
|
||||
except Exception as poll_error:
|
||||
retry_count += 1
|
||||
logger.warning(
|
||||
f"Temporary issue polling file {file.name} (attempt"
|
||||
f" {retry_count}/{max_retries}): {poll_error}"
|
||||
)
|
||||
if retry_count > max_retries:
|
||||
raise RuntimeError(
|
||||
f"Failed to poll Gemini File API after {max_retries} attempts: {poll_error}"
|
||||
)
|
||||
await asyncio.sleep(2.0)
|
||||
continue
|
||||
|
||||
if f_state.state.name == "ACTIVE":
|
||||
break
|
||||
elif f_state.state.name == "FAILED":
|
||||
raise RuntimeError(f"Gemini File API processing failed for {file.name}")
|
||||
|
||||
if time.time() - start_wait > max_wait:
|
||||
raise TimeoutError(
|
||||
f"Gemini File API processing timeout for {file.name}"
|
||||
f" (Waited {max_wait}s for size {file_size_mb:.1f}MB)"
|
||||
)
|
||||
|
||||
logger.info(f"File {file.name} is {f_state.state.name}, waiting {wait_interval}s...")
|
||||
await asyncio.sleep(wait_interval)
|
||||
wait_interval = min(3.0, wait_interval * 1.5)
|
||||
return file
|
||||
@@ -1,541 +0,0 @@
|
||||
# Copyright 2026 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Native Gemini coordinator loop for the video analyzer main agent.
|
||||
|
||||
Extracted from ``video_analyzer.py`` as a pure structural split. Patchable
|
||||
collaborators are looked up late through the facade module so
|
||||
``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")`` targets
|
||||
keep working.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
import os
|
||||
from pathlib import Path
|
||||
import shutil
|
||||
import tempfile
|
||||
|
||||
from google.genai import types
|
||||
|
||||
from artemis.agents.video_analyzer import video_analyzer as _va
|
||||
from artemis.agents.video_analyzer.reliability import classify_video_failure
|
||||
from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
|
||||
from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan
|
||||
from artemis.llm.google import normalize_usage
|
||||
from artemis.utils.logger import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
class _DummyResponse:
|
||||
"""Aggregated view over a consumed stream, shaped like a full response."""
|
||||
|
||||
def __init__(self, text, function_calls, parts):
|
||||
self.text = text
|
||||
self.function_calls = function_calls
|
||||
self.candidates = [
|
||||
types.Candidate(
|
||||
content=types.Content(
|
||||
role="model",
|
||||
parts=parts if parts else ([types.Part.from_text(text=text)] if text else []),
|
||||
)
|
||||
)
|
||||
]
|
||||
|
||||
|
||||
async def run_native(
|
||||
analyzer,
|
||||
ctx,
|
||||
time_description: str,
|
||||
purpose: str,
|
||||
system_prompt: str,
|
||||
temperature: float,
|
||||
thinking_level,
|
||||
) -> tuple[str, str]:
|
||||
"""Drives the native Gemini coordinator loop to a final answer."""
|
||||
tools_declaration = analyzer.tools_declaration
|
||||
|
||||
contents = [
|
||||
types.Content(
|
||||
role="user",
|
||||
parts=[
|
||||
types.Part.from_text(
|
||||
text=(f"Time description: {time_description}\nPurpose: {purpose}")
|
||||
)
|
||||
],
|
||||
)
|
||||
]
|
||||
|
||||
agent_outcome = ""
|
||||
status = "success"
|
||||
max_iterations = 10
|
||||
iterations = 0
|
||||
last_ledger_index = 0
|
||||
|
||||
try:
|
||||
while iterations < max_iterations:
|
||||
iterations += 1
|
||||
logger.info(f"Iteration {iterations}: Invoking Native Gemini SDK...")
|
||||
|
||||
analyzer._clean_blackboard()
|
||||
|
||||
last_ledger_index = _append_ledger_diff(analyzer, contents, last_ledger_index)
|
||||
|
||||
is_final_turn = iterations == max_iterations
|
||||
_append_turn_guidance(contents, iterations, max_iterations, is_final_turn)
|
||||
|
||||
response = await _stream_main_turn(
|
||||
analyzer,
|
||||
ctx,
|
||||
contents,
|
||||
system_prompt,
|
||||
temperature,
|
||||
thinking_level,
|
||||
tools_declaration,
|
||||
is_final_turn,
|
||||
)
|
||||
|
||||
function_calls = response.function_calls
|
||||
if not function_calls:
|
||||
agent_outcome = response.text
|
||||
break
|
||||
|
||||
if response.candidates and response.candidates[0].content:
|
||||
contents.append(response.candidates[0].content)
|
||||
else:
|
||||
contents.append(
|
||||
types.Content(
|
||||
role="model",
|
||||
parts=[types.Part(function_call=fc) for fc in function_calls],
|
||||
)
|
||||
)
|
||||
|
||||
tool_response_parts = await _execute_native_tool_calls(analyzer, function_calls)
|
||||
|
||||
if tool_response_parts:
|
||||
contents.append(types.Content(role="user", parts=tool_response_parts))
|
||||
|
||||
if iterations >= max_iterations and not agent_outcome:
|
||||
agent_outcome = (
|
||||
"Error: Video analyzer agent reached maximum iterations"
|
||||
" without settling on a final answer."
|
||||
)
|
||||
status = "error"
|
||||
|
||||
except Exception as e:
|
||||
logger.error(f"Video analyzer failed: {e}")
|
||||
failure = classify_video_failure(e)
|
||||
if failure.should_fallback:
|
||||
logger.warning(
|
||||
f"Native coordinator failed with {failure.category.value}; "
|
||||
"continuing through universal fallback"
|
||||
)
|
||||
agent_outcome, status = await analyzer._run_universal(
|
||||
time_description,
|
||||
purpose,
|
||||
system_prompt,
|
||||
force_fallback=True,
|
||||
)
|
||||
else:
|
||||
agent_outcome = f"Video analysis failed: {e}"
|
||||
status = "error"
|
||||
finally:
|
||||
analyzer._record_reliability_metrics()
|
||||
await _cleanup_native_run(analyzer)
|
||||
|
||||
return agent_outcome, status
|
||||
|
||||
|
||||
def _append_ledger_diff(analyzer, contents: list, last_ledger_index: int) -> int:
|
||||
"""Appends new blackboard entries (and proofs) since the last coordinator turn."""
|
||||
new_entries = analyzer.blackboard_entries[last_ledger_index:]
|
||||
if analyzer.enable_ledger and new_entries:
|
||||
lines = []
|
||||
image_parts = []
|
||||
for e in new_entries:
|
||||
line = f"[{e['start']}s - {e['end']}s] SUMMARY: {e['summary']}"
|
||||
screenshot_file = e.get("screenshot") or e.get("screenshot_path")
|
||||
if screenshot_file and Path(screenshot_file).exists():
|
||||
line += f" PROOF: {Path(screenshot_file).name}"
|
||||
image_parts.append(
|
||||
types.Part.from_text(text=f"PROOF: {Path(screenshot_file).name}")
|
||||
)
|
||||
image_parts.append(
|
||||
types.Part.from_bytes(
|
||||
data=Path(screenshot_file).read_bytes(),
|
||||
mime_type="image/jpeg",
|
||||
)
|
||||
)
|
||||
lines.append(line)
|
||||
diff_text = "NEW BLACKBOARD ENTRIES ADDED: " + " | ".join(lines)
|
||||
contents.append(
|
||||
types.Content(
|
||||
role="user",
|
||||
parts=[types.Part.from_text(text=diff_text)],
|
||||
)
|
||||
)
|
||||
if image_parts:
|
||||
contents.append(types.Content(role="user", parts=image_parts))
|
||||
last_ledger_index = len(analyzer.blackboard_entries)
|
||||
return last_ledger_index
|
||||
|
||||
|
||||
def _append_turn_guidance(
|
||||
contents: list, iterations: int, max_iterations: int, is_final_turn: bool
|
||||
) -> None:
|
||||
"""Appends per-turn steering messages (final-turn notice or progress nudge)."""
|
||||
if is_final_turn:
|
||||
contents.append(
|
||||
types.Content(
|
||||
role="user",
|
||||
parts=[
|
||||
types.Part.from_text(
|
||||
text=(
|
||||
"This is your final iteration; all"
|
||||
" tools are stripped, and you must"
|
||||
" provide your final answer directly."
|
||||
)
|
||||
)
|
||||
],
|
||||
)
|
||||
)
|
||||
else:
|
||||
if iterations > 1:
|
||||
contents.append(
|
||||
types.Content(
|
||||
role="user",
|
||||
parts=[
|
||||
types.Part.from_text(
|
||||
text=(
|
||||
"You have not completed the video"
|
||||
" analysis yet (iteration"
|
||||
f" {iterations} of"
|
||||
f" {max_iterations})."
|
||||
)
|
||||
)
|
||||
],
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def _record_main_llm_trace(
|
||||
ctx, trace_id, contents: list, system_prompt: str, tools_declaration, is_final_turn: bool
|
||||
) -> None:
|
||||
"""Records the sanitized coordinator LLM call into the trace store."""
|
||||
serialized_contents = []
|
||||
for c in contents:
|
||||
if hasattr(c, "model_dump"):
|
||||
dumped = c.model_dump()
|
||||
if isinstance(dumped, dict) and "parts" in dumped and dumped["parts"]:
|
||||
for p in dumped["parts"]:
|
||||
if (
|
||||
isinstance(p, dict)
|
||||
and "inline_data" in p
|
||||
and p["inline_data"]
|
||||
and "data" in p["inline_data"]
|
||||
):
|
||||
p["inline_data"]["data"] = "<image_bytes_sanitized>"
|
||||
serialized_contents.append(dumped)
|
||||
else:
|
||||
serialized_contents.append(str(c))
|
||||
|
||||
ctx.data_engine.record_trace(
|
||||
type="llm_call",
|
||||
name="video_analyzer_main",
|
||||
payload={
|
||||
"contents": serialized_contents,
|
||||
"system_instruction": system_prompt,
|
||||
"tools": (
|
||||
[t.name for t in tools_declaration]
|
||||
if (tools_declaration and not is_final_turn)
|
||||
else []
|
||||
),
|
||||
},
|
||||
parent_trace_id=trace_id,
|
||||
)
|
||||
|
||||
|
||||
async def _stream_main_turn(
|
||||
analyzer,
|
||||
ctx,
|
||||
contents: list,
|
||||
system_prompt: str,
|
||||
temperature: float,
|
||||
thinking_level,
|
||||
tools_declaration,
|
||||
is_final_turn: bool,
|
||||
):
|
||||
"""Runs one traced, retry-wrapped streaming turn of the main coordinator."""
|
||||
with TraceSpan(name="gemini_main_agent_call") as span:
|
||||
trace_id = CURRENT_TRACE_ID.get()
|
||||
|
||||
if ctx.data_engine and trace_id:
|
||||
_record_main_llm_trace(
|
||||
ctx, trace_id, contents, system_prompt, tools_declaration, is_final_turn
|
||||
)
|
||||
|
||||
async def run_stream():
|
||||
return await _consume_main_stream(
|
||||
analyzer,
|
||||
ctx,
|
||||
span,
|
||||
trace_id,
|
||||
contents,
|
||||
system_prompt,
|
||||
temperature,
|
||||
thinking_level,
|
||||
tools_declaration,
|
||||
is_final_turn,
|
||||
)
|
||||
|
||||
response = await _va._invoke_with_retry(
|
||||
lambda: asyncio.wait_for(run_stream(), timeout=300),
|
||||
"Native Gemini video coordinator",
|
||||
)
|
||||
span.result = (
|
||||
f"Function calls: {len(response.function_calls)}"
|
||||
if response.function_calls
|
||||
else "Final answer"
|
||||
)
|
||||
return response
|
||||
|
||||
|
||||
async def _consume_main_stream(
|
||||
analyzer,
|
||||
ctx,
|
||||
span,
|
||||
trace_id,
|
||||
contents: list,
|
||||
system_prompt: str,
|
||||
temperature: float,
|
||||
thinking_level,
|
||||
tools_declaration,
|
||||
is_final_turn: bool,
|
||||
) -> _DummyResponse:
|
||||
"""Starts the coordinator stream and aggregates it into a response object."""
|
||||
stream = await asyncio.wait_for(
|
||||
analyzer.client.aio.models.generate_content_stream(
|
||||
model=analyzer.model_name,
|
||||
contents=contents,
|
||||
config=types.GenerateContentConfig(
|
||||
system_instruction=system_prompt,
|
||||
temperature=temperature,
|
||||
tools=[]
|
||||
if is_final_turn
|
||||
else [types.Tool(function_declarations=tools_declaration)],
|
||||
safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
|
||||
**(
|
||||
{"thinking_config": types.ThinkingConfig(thinking_level=thinking_level)}
|
||||
if thinking_level
|
||||
else {}
|
||||
),
|
||||
),
|
||||
),
|
||||
timeout=60,
|
||||
)
|
||||
|
||||
text = ""
|
||||
accumulated_parts = []
|
||||
function_calls = []
|
||||
|
||||
async for chunk in stream:
|
||||
usage = normalize_usage(getattr(chunk, "usage_metadata", None))
|
||||
if usage:
|
||||
span.payload["usage_metadata"] = usage
|
||||
|
||||
chunk_text = chunk.text or ""
|
||||
text += chunk_text
|
||||
if ctx.data_engine and trace_id and chunk_text:
|
||||
ctx.data_engine.stream_output(trace_id, chunk_text, is_thinking=False)
|
||||
|
||||
if chunk.function_calls:
|
||||
function_calls.extend(chunk.function_calls)
|
||||
|
||||
candidates = getattr(chunk, "candidates", None)
|
||||
if candidates and isinstance(candidates, list) and len(candidates) > 0:
|
||||
content = getattr(candidates[0], "content", None)
|
||||
parts = getattr(content, "parts", None) if content else None
|
||||
if parts:
|
||||
for part in parts:
|
||||
accumulated_parts.append(part)
|
||||
if getattr(part, "thought", False) and part.text:
|
||||
if ctx.data_engine and trace_id:
|
||||
ctx.data_engine.stream_output(
|
||||
trace_id,
|
||||
part.text,
|
||||
is_thinking=True,
|
||||
)
|
||||
else:
|
||||
if chunk.function_calls:
|
||||
for fc in chunk.function_calls:
|
||||
accumulated_parts.append(types.Part(function_call=fc))
|
||||
|
||||
return _DummyResponse(text, function_calls, accumulated_parts)
|
||||
|
||||
|
||||
async def _execute_native_tool_calls(analyzer, function_calls: list) -> list:
|
||||
"""Executes coordinator tool calls (parallel where safe) and collects responses."""
|
||||
sub_agent_calls = [
|
||||
fc
|
||||
for fc in function_calls
|
||||
if (fc.name.split(":")[-1] if ":" in fc.name else fc.name) == "spawn_sub_agent"
|
||||
]
|
||||
audio_only_calls = [
|
||||
fc
|
||||
for fc in function_calls
|
||||
if (fc.name.split(":")[-1] if ":" in fc.name else fc.name) == "analyze_audio_only"
|
||||
]
|
||||
other_calls = [
|
||||
fc
|
||||
for fc in function_calls
|
||||
if (fc.name.split(":")[-1] if ":" in fc.name else fc.name)
|
||||
not in ["spawn_sub_agent", "analyze_audio_only"]
|
||||
]
|
||||
|
||||
tool_response_parts = []
|
||||
|
||||
if sub_agent_calls:
|
||||
await _run_spawn_calls(analyzer, sub_agent_calls, tool_response_parts)
|
||||
|
||||
if audio_only_calls:
|
||||
await _run_audio_calls(analyzer, audio_only_calls, tool_response_parts)
|
||||
|
||||
await _run_other_calls(analyzer, other_calls, tool_response_parts)
|
||||
|
||||
return tool_response_parts
|
||||
|
||||
|
||||
async def _run_spawn_calls(analyzer, sub_agent_calls: list, tool_response_parts: list) -> None:
|
||||
"""Runs spawn_sub_agent calls in parallel and appends their responses."""
|
||||
logger.info(f"Executing {len(sub_agent_calls)} sub-agent calls in parallel...")
|
||||
|
||||
async def bounded_spawn(fc):
|
||||
# exec_spawn_sub_agent limits each actual model leaf.
|
||||
# Holding the same semaphore around the coordinator
|
||||
# would deadlock when all slots wait for child slots.
|
||||
return await analyzer.exec_spawn_sub_agent(
|
||||
fc.args.get("start_time"),
|
||||
fc.args.get("end_time"),
|
||||
fc.args.get("specific_query"),
|
||||
)
|
||||
|
||||
tasks = [bounded_spawn(fc) for fc in sub_agent_calls]
|
||||
results = await asyncio.gather(*tasks, return_exceptions=True)
|
||||
|
||||
for fc, result in zip(sub_agent_calls, results):
|
||||
if isinstance(result, Exception):
|
||||
error_msg = f"Sub-agent failed. Error: {result}"
|
||||
logger.error(error_msg)
|
||||
tool_response_parts.append(
|
||||
types.Part.from_function_response(name=fc.name, response={"error": error_msg})
|
||||
)
|
||||
else:
|
||||
tool_response_parts.append(
|
||||
types.Part.from_function_response(name=fc.name, response={"result": result})
|
||||
)
|
||||
|
||||
|
||||
async def _run_audio_calls(analyzer, audio_only_calls: list, tool_response_parts: list) -> None:
|
||||
"""Runs analyze_audio_only calls in parallel and appends their responses."""
|
||||
logger.info(f"Executing {len(audio_only_calls)} analyze_audio_only calls in parallel...")
|
||||
|
||||
async def bounded_audio_spawn(fc):
|
||||
async with _va.API_SEMAPHORE:
|
||||
return await analyzer.exec_analyze_audio_only(
|
||||
fc.args.get("start_time"),
|
||||
fc.args.get("end_time"),
|
||||
fc.args.get("specific_query"),
|
||||
)
|
||||
|
||||
tasks = [bounded_audio_spawn(fc) for fc in audio_only_calls]
|
||||
results = await asyncio.gather(*tasks, return_exceptions=True)
|
||||
|
||||
for fc, result in zip(audio_only_calls, results):
|
||||
if isinstance(result, Exception):
|
||||
error_msg = f"Audio analysis failed. Error: {result}"
|
||||
logger.error(error_msg)
|
||||
tool_response_parts.append(
|
||||
types.Part.from_function_response(name=fc.name, response={"error": error_msg})
|
||||
)
|
||||
else:
|
||||
tool_response_parts.append(
|
||||
types.Part.from_function_response(name=fc.name, response={"result": result})
|
||||
)
|
||||
|
||||
|
||||
async def _run_other_calls(analyzer, other_calls: list, tool_response_parts: list) -> None:
|
||||
"""Runs remaining coordinator tool calls sequentially."""
|
||||
for fc in other_calls:
|
||||
name = fc.name.split(":")[-1] if ":" in fc.name else fc.name
|
||||
args = fc.args
|
||||
logger.info(f"Executing tool '{name}' sequentially...")
|
||||
|
||||
if name == "extract_segment_metadata":
|
||||
try:
|
||||
res = await analyzer.exec_extract_segment_metadata(
|
||||
args.get("start_time"), args.get("end_time")
|
||||
)
|
||||
tool_response_parts.append(
|
||||
types.Part.from_function_response(name=name, response={"result": res})
|
||||
)
|
||||
except Exception as e:
|
||||
tool_response_parts.append(
|
||||
types.Part.from_function_response(name=name, response={"error": str(e)})
|
||||
)
|
||||
|
||||
|
||||
async def _cleanup_native_run(analyzer) -> None:
|
||||
"""Releases remaining cloud files and temporary local media after a run."""
|
||||
if analyzer.cloud_files_to_cleanup:
|
||||
logger.info(
|
||||
f"Cleaning up {len(analyzer.cloud_files_to_cleanup)} remaining"
|
||||
" cloud files in parallel..."
|
||||
)
|
||||
tasks = [
|
||||
asyncio.wait_for(analyzer.client.aio.files.delete(name=name), timeout=30)
|
||||
for name in list(analyzer.cloud_files_to_cleanup)
|
||||
]
|
||||
await asyncio.gather(*tasks, return_exceptions=True)
|
||||
analyzer.cloud_files_to_cleanup.clear()
|
||||
|
||||
if os.environ.get("KEEP_VIDEOS") or os.environ.get("ARTEMIS_DEBUG"):
|
||||
logger.info("Skipping cleanup of local video files and directories due to debug mode.")
|
||||
else:
|
||||
for path in list(analyzer.local_files_to_cleanup):
|
||||
try:
|
||||
if path.exists():
|
||||
logger.info(f"Cleaning up local file {path}...")
|
||||
path.unlink()
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to delete local file {path}: {e}")
|
||||
|
||||
for dir_path in list(analyzer.local_dirs_to_cleanup):
|
||||
try:
|
||||
resolved = dir_path.resolve()
|
||||
system_dirs = {
|
||||
Path("/").resolve(),
|
||||
Path("/tmp").resolve(),
|
||||
Path("/var/tmp").resolve(),
|
||||
Path.home().resolve(),
|
||||
Path(tempfile.gettempdir()).resolve(),
|
||||
}
|
||||
if resolved in system_dirs:
|
||||
logger.debug(f"Skipping deletion of root/system directory: {dir_path}")
|
||||
continue
|
||||
|
||||
if dir_path.exists():
|
||||
logger.info(f"Cleaning up local directory {dir_path}...")
|
||||
shutil.rmtree(dir_path, ignore_errors=True)
|
||||
except Exception as e:
|
||||
logger.error(f"Failed to delete local directory {dir_path}: {e}")
|
||||
@@ -35,6 +35,8 @@ from langchain_core.messages import (
|
||||
ToolMessage,
|
||||
)
|
||||
|
||||
from artemis.llm.structured import content_to_text
|
||||
|
||||
from artemis.agents.video_analyzer import video_analyzer as _va
|
||||
from artemis.agents.video_analyzer.reliability import (
|
||||
SubAgentAnswerExhausted,
|
||||
@@ -241,7 +243,7 @@ def _extract_submit_payload(response) -> tuple[str, str, list]:
|
||||
return final_summary, final_analysis, list(args.get("timeline_events") or [])
|
||||
|
||||
raw_content = getattr(response, "content", None)
|
||||
text_content = str(raw_content).strip() if raw_content else ""
|
||||
text_content = content_to_text(raw_content).strip()
|
||||
if text_content:
|
||||
return text_content[:200].replace("\n", " "), text_content, []
|
||||
|
||||
@@ -435,7 +437,7 @@ async def run_universal(
|
||||
messages.append(response)
|
||||
|
||||
if not getattr(response, "tool_calls", None):
|
||||
agent_outcome = str(response.content)
|
||||
agent_outcome = content_to_text(response.content)
|
||||
break
|
||||
|
||||
for tc in response.tool_calls:
|
||||
|
||||
@@ -14,14 +14,11 @@
|
||||
|
||||
"""Tool schema declarations for the video analyzer.
|
||||
|
||||
Holds both the OpenAI-style function schemas used by the universal
|
||||
(LangChain) engine and the builders for the native Gemini
|
||||
``FunctionDeclaration`` equivalents. Pure data/declarations only.
|
||||
OpenAI function schemas for the Responses video analysis engine.
|
||||
"""
|
||||
|
||||
from typing import Any
|
||||
|
||||
from google.genai import types
|
||||
|
||||
UNIVERSAL_SUBMIT_ANSWER_TOOL: dict[str, Any] = {
|
||||
"type": "function",
|
||||
@@ -172,152 +169,3 @@ UNIVERSAL_MAIN_TOOLS: list[dict[str, Any]] = [
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def build_native_tools_declaration() -> list[types.FunctionDeclaration]:
|
||||
"""Builds the native Gemini declarations for the coordinator tools."""
|
||||
return [
|
||||
types.FunctionDeclaration(
|
||||
name="extract_segment_metadata",
|
||||
description=(
|
||||
"Retrieves metadata (such as duration_seconds and"
|
||||
" file_size_mb) for a specific video segment. Does not"
|
||||
" return the video content or perform content analysis."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"start_time": types.Schema(
|
||||
type=types.Type.NUMBER,
|
||||
description="Start time in seconds",
|
||||
),
|
||||
"end_time": types.Schema(
|
||||
type=types.Type.NUMBER,
|
||||
description=(
|
||||
"End time in seconds (optional, defaults to latest available time)"
|
||||
),
|
||||
),
|
||||
},
|
||||
required=["start_time"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="spawn_sub_agent",
|
||||
description=(
|
||||
"Spawns a sub-agent to analyze both visual and audio"
|
||||
" content of a video segment. Parallelizes chunk analysis"
|
||||
" across workers. Returns combined summary and analysis."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"start_time": types.Schema(
|
||||
type=types.Type.NUMBER,
|
||||
description="Start time in seconds",
|
||||
),
|
||||
"end_time": types.Schema(
|
||||
type=types.Type.NUMBER,
|
||||
description=(
|
||||
"End time in seconds. Use the maximum available time if not specified."
|
||||
),
|
||||
),
|
||||
"specific_query": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=("Specific query or intent for the sub-agent"),
|
||||
),
|
||||
},
|
||||
required=["start_time", "specific_query"],
|
||||
),
|
||||
),
|
||||
types.FunctionDeclaration(
|
||||
name="analyze_audio_only",
|
||||
description=(
|
||||
"Extracts and analyzes only the audio track from a video"
|
||||
" segment. More efficient for non-visual tasks."
|
||||
),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"start_time": types.Schema(
|
||||
type=types.Type.NUMBER,
|
||||
description="Start time in seconds",
|
||||
),
|
||||
"end_time": types.Schema(
|
||||
type=types.Type.NUMBER,
|
||||
description="End time in seconds (optional)",
|
||||
),
|
||||
"specific_query": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=("Specific query or intent for the audio content"),
|
||||
),
|
||||
},
|
||||
required=["start_time", "specific_query"],
|
||||
),
|
||||
),
|
||||
]
|
||||
|
||||
|
||||
def build_submit_answer_declaration() -> types.FunctionDeclaration:
|
||||
"""Builds the native Gemini declaration for the sub-agent submit_answer tool."""
|
||||
return types.FunctionDeclaration(
|
||||
name="submit_answer",
|
||||
description=("Submit the final findings for the requested segment analysis."),
|
||||
parameters=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"summary": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=("A short paragraph summarizing of your findings."),
|
||||
),
|
||||
"analysis": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description="A short paragraph of detailed analysis.",
|
||||
),
|
||||
"timeline_events": types.Schema(
|
||||
type=types.Type.ARRAY,
|
||||
description=(
|
||||
"List of detected events matching the query. If no"
|
||||
" relevant events occurred in the segment, return"
|
||||
" an empty array."
|
||||
),
|
||||
items=types.Schema(
|
||||
type=types.Type.OBJECT,
|
||||
properties={
|
||||
"start_time": types.Schema(
|
||||
type=types.Type.NUMBER,
|
||||
description="Event start time (seconds)",
|
||||
),
|
||||
"end_time": types.Schema(
|
||||
type=types.Type.NUMBER,
|
||||
description="Event end time (seconds)",
|
||||
),
|
||||
"transcription": types.Schema(
|
||||
type=types.Type.STRING,
|
||||
description=(
|
||||
"A concise transcription of what happened in the event. "
|
||||
),
|
||||
),
|
||||
"confidence_score": types.Schema(
|
||||
type=types.Type.NUMBER,
|
||||
description=("Confidence score between 0 and 1"),
|
||||
),
|
||||
"verification_timestamp_secs": types.Schema(
|
||||
type=types.Type.NUMBER,
|
||||
description=(
|
||||
"Exact timestamp (seconds) for verification screenshot."
|
||||
),
|
||||
),
|
||||
},
|
||||
required=[
|
||||
"start_time",
|
||||
"end_time",
|
||||
"transcription",
|
||||
"confidence_score",
|
||||
"verification_timestamp_secs",
|
||||
],
|
||||
),
|
||||
),
|
||||
},
|
||||
required=["summary", "analysis", "timeline_events"],
|
||||
),
|
||||
)
|
||||
@@ -14,9 +14,8 @@
|
||||
|
||||
"""VideoAnalyzer facade: entry point and shared runtime state.
|
||||
|
||||
The execution engines live in sibling modules (``chunk_native``,
|
||||
``audio_native``, ``universal_engine``, ``native_coordinator``,
|
||||
``gemini_files``, ``universal_tools``) and resolve their patchable
|
||||
The execution engines live in sibling modules (``chunk_runner``,
|
||||
``audio_runner``, ``universal_engine``, ``universal_tools``) and resolve their patchable
|
||||
collaborators through THIS module's namespace, so existing
|
||||
``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")``
|
||||
targets keep working. Keep the imports below in place even when the facade
|
||||
@@ -36,8 +35,6 @@ import tempfile
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from google import genai
|
||||
from google.genai import types
|
||||
from langchain_core.messages import (
|
||||
BaseMessage,
|
||||
HumanMessage,
|
||||
@@ -49,9 +46,6 @@ from artemis.agents.video_analyzer.conflict_resolution import (
|
||||
ConflictResolutionService,
|
||||
)
|
||||
from artemis.agents.video_analyzer.blackboard import get_video_blackboard
|
||||
from artemis.agents.video_analyzer.gemini_files import (
|
||||
cleanup_abandoned_gemini_files,
|
||||
)
|
||||
from artemis.agents.video_analyzer.reliability import (
|
||||
AgenticVideoDegraded,
|
||||
VideoCircuitBreaker,
|
||||
@@ -60,19 +54,11 @@ from artemis.agents.video_analyzer.reliability import (
|
||||
from artemis.agents.video_analyzer.universal_tools import (
|
||||
UNIVERSAL_MAIN_TOOLS,
|
||||
UNIVERSAL_SUBMIT_ANSWER_TOOL,
|
||||
build_native_tools_declaration,
|
||||
build_submit_answer_declaration,
|
||||
)
|
||||
from artemis.config import get_temp_dir, settings
|
||||
from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
|
||||
from artemis.context import ArtemisContext
|
||||
from artemis.controllers.controller_factory import get_controller
|
||||
from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan, trace
|
||||
from artemis.llm.google import (
|
||||
is_gemini_model,
|
||||
resolve_video_processing,
|
||||
strip_provider_prefix,
|
||||
)
|
||||
from artemis.llm.reliability import retry_policy_for
|
||||
from artemis.services.llm import _record_llm_event, _record_llm_retry, get_llm
|
||||
from artemis.utils.logger import get_logger
|
||||
@@ -159,16 +145,8 @@ class VideoAnalyzer:
|
||||
getattr(agent_config, "enable_video_ledger", True),
|
||||
)
|
||||
self.static_chunk_size_seconds = float(
|
||||
getattr(self.video_config, "chunk_size_seconds", 60.0)
|
||||
getattr(self.video_config, "chunk_size_seconds", settings.ARTEMIS_VIDEO_CHUNK_SECONDS)
|
||||
)
|
||||
self.agentic_chunk_size_seconds = float(
|
||||
getattr(self.video_config, "agentic_chunk_size_seconds", 600.0)
|
||||
)
|
||||
self.agentic_call_timeout_seconds = float(
|
||||
getattr(self.video_config, "agentic_call_timeout_seconds", 300.0)
|
||||
)
|
||||
# Effective chunk ceiling; re-derived once the engine picks a
|
||||
# video-processing mode (agentic sub-agents take long clips).
|
||||
self.chunk_size_seconds = self.static_chunk_size_seconds
|
||||
self.video_processing = "static"
|
||||
self.min_chunk_seconds = float(getattr(self.video_config, "min_chunk_seconds", 4.0))
|
||||
@@ -178,9 +156,15 @@ class VideoAnalyzer:
|
||||
self.max_dense_action_frames = int(
|
||||
getattr(self.video_config, "max_dense_action_frames", 24)
|
||||
)
|
||||
self.native_max_retries = int(getattr(self.video_config, "native_max_retries", 1))
|
||||
self.max_retries = int(
|
||||
getattr(self.video_config, "max_retries", settings.ARTEMIS_VIDEO_MAX_RETRIES)
|
||||
)
|
||||
self.model_call_timeout_seconds = float(
|
||||
getattr(self.video_config, "model_call_timeout_seconds", 120.0)
|
||||
getattr(
|
||||
self.video_config,
|
||||
"model_call_timeout_seconds",
|
||||
settings.ARTEMIS_VIDEO_TIMEOUT_SECONDS,
|
||||
)
|
||||
)
|
||||
breaker = getattr(ctx, "_video_circuit_breaker", None)
|
||||
if breaker is None:
|
||||
@@ -210,7 +194,6 @@ class VideoAnalyzer:
|
||||
audio_prompt_path.read_text(encoding="utf-8") if audio_prompt_path.exists() else ""
|
||||
)
|
||||
|
||||
self._init_tools()
|
||||
self._init_engine()
|
||||
|
||||
def _get_universal_llm(self, *, use_fallback: bool = False):
|
||||
@@ -406,65 +389,13 @@ class VideoAnalyzer:
|
||||
analysis = match.group(2).strip() if match.group(2) else None
|
||||
return summary, analysis
|
||||
|
||||
def _init_tools(self) -> None:
|
||||
"""Initializes native and sub-agent tool declarations."""
|
||||
self.tools_declaration = build_native_tools_declaration()
|
||||
self.submit_answer_tool = build_submit_answer_declaration()
|
||||
|
||||
def _init_engine(self) -> None:
|
||||
"""Initializes model engine and decides whether to use native Gemini or Universal path."""
|
||||
ctx = self.ctx
|
||||
llm_config = getattr(ctx, "llm_config", None)
|
||||
llm_config = getattr(self.ctx, "llm_config", None)
|
||||
utils_cfg = getattr(llm_config, "utils", None) if llm_config else None
|
||||
llm_cfg = getattr(utils_cfg, "video_analyzer", None) if utils_cfg else None
|
||||
self.model_name = strip_provider_prefix(
|
||||
llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.8-flash"
|
||||
)
|
||||
|
||||
has_google_key = bool(
|
||||
settings.GOOGLE_API_KEY and settings.GOOGLE_API_KEY.get_secret_value()
|
||||
)
|
||||
|
||||
# If client is already set on ctx or explicit Gemini configuration
|
||||
self.client = getattr(ctx, "_genai_client", None)
|
||||
if self.client is not None:
|
||||
self.use_native_gemini = True
|
||||
elif has_google_key and is_gemini_model(self.model_name):
|
||||
try:
|
||||
self.client = genai.Client(api_key=settings.GOOGLE_API_KEY.get_secret_value())
|
||||
ctx._genai_client = self.client
|
||||
self.use_native_gemini = True
|
||||
asyncio.create_task(cleanup_abandoned_gemini_files(self.client))
|
||||
except Exception as e:
|
||||
logger.warning(
|
||||
f"Failed to initialize native Gemini client: {e}. Using universal engine."
|
||||
)
|
||||
self.use_native_gemini = False
|
||||
else:
|
||||
self.use_native_gemini = False
|
||||
self._resolve_video_processing()
|
||||
|
||||
def _resolve_video_processing(self) -> None:
|
||||
"""Picks static vs agentic video understanding for the sub-agents.
|
||||
|
||||
Agentic processing (Gemini Interactions API) needs the native client
|
||||
and a supporting model; the configured knob (``auto`` / ``agentic`` /
|
||||
``static``) decides the rest. The chunk ceiling follows the mode
|
||||
because agentic sub-agents are built to take long clips.
|
||||
"""
|
||||
configured = getattr(self.video_config, "processing", "auto")
|
||||
mode = "static"
|
||||
if self.use_native_gemini:
|
||||
mode = resolve_video_processing(configured, self.model_name)
|
||||
self.video_processing = mode
|
||||
self.chunk_size_seconds = (
|
||||
self.agentic_chunk_size_seconds if mode == "agentic" else self.static_chunk_size_seconds
|
||||
)
|
||||
if mode == "agentic":
|
||||
logger.info(
|
||||
f"Video sub-agents use agentic video understanding on {self.model_name}"
|
||||
f" (chunk ceiling {self.chunk_size_seconds:.0f}s)"
|
||||
)
|
||||
self.model_name = getattr(llm_cfg, "model", None) or settings.OPENAI_MODEL
|
||||
self.video_processing = "static"
|
||||
self.chunk_size_seconds = self.static_chunk_size_seconds
|
||||
|
||||
@property
|
||||
def blackboard_ledger(self) -> str:
|
||||
@@ -476,13 +407,6 @@ class VideoAnalyzer:
|
||||
lines.append(f"{e['start']}s - {e['end']}s: {e['summary']}")
|
||||
return " | ".join(lines)
|
||||
|
||||
async def upload_and_poll_file(self, compressed_path: Path) -> any:
|
||||
from artemis.agents.video_analyzer import gemini_files
|
||||
|
||||
return await gemini_files.upload_and_poll_file(
|
||||
self.client, compressed_path, self.cloud_files_to_cleanup
|
||||
)
|
||||
|
||||
def get_overlapping_warnings(self, start_time: float, end_time: float | None) -> list[dict]:
|
||||
warnings = []
|
||||
if not isinstance(start_time, (int, float)) or (
|
||||
@@ -739,17 +663,17 @@ class VideoAnalyzer:
|
||||
async def _exec_single_chunk(
|
||||
self, start_time: float, end_time: float | None, specific_query: str
|
||||
) -> str:
|
||||
from artemis.agents.video_analyzer import chunk_native
|
||||
from artemis.agents.video_analyzer import chunk_runner
|
||||
|
||||
return await chunk_native.exec_single_chunk(self, start_time, end_time, specific_query)
|
||||
return await chunk_runner.exec_single_chunk(self, start_time, end_time, specific_query)
|
||||
|
||||
@trace(type="tool", name="analyze_audio_only")
|
||||
async def exec_analyze_audio_only(
|
||||
self, start_time: float, end_time: float | None, specific_query: str
|
||||
) -> str:
|
||||
from artemis.agents.video_analyzer import audio_native
|
||||
from artemis.agents.video_analyzer import audio_runner
|
||||
|
||||
return await audio_native.exec_analyze_audio_only(
|
||||
return await audio_runner.exec_analyze_audio_only(
|
||||
self, start_time, end_time, specific_query
|
||||
)
|
||||
|
||||
@@ -837,15 +761,12 @@ class VideoAnalyzer:
|
||||
utils_cfg = getattr(llm_config, "utils", None) if llm_config else None
|
||||
llm_cfg = getattr(utils_cfg, "video_analyzer", None) if utils_cfg else None
|
||||
temperature = 0.2
|
||||
thinking_level = None
|
||||
if llm_cfg:
|
||||
self.model_name = strip_provider_prefix(llm_cfg.model)
|
||||
self.model_name = llm_cfg.model
|
||||
if getattr(llm_cfg, "temperature", None) is not None:
|
||||
temperature = llm_cfg.temperature
|
||||
if getattr(llm_cfg, "thinking_level", None) is not None:
|
||||
thinking_level = llm_cfg.thinking_level
|
||||
else:
|
||||
self.model_name = "gemini-3.8-flash"
|
||||
self.model_name = settings.OPENAI_MODEL
|
||||
|
||||
# Track files for cleanup
|
||||
self.local_files_to_cleanup = set()
|
||||
@@ -868,24 +789,11 @@ class VideoAnalyzer:
|
||||
" proceeding.\n"
|
||||
)
|
||||
|
||||
if not self.use_native_gemini:
|
||||
try:
|
||||
return await self._run_universal(
|
||||
time_description=time_description,
|
||||
purpose=purpose,
|
||||
system_prompt=system_prompt,
|
||||
)
|
||||
finally:
|
||||
await self._cleanup_universal_resources()
|
||||
|
||||
from artemis.agents.video_analyzer import native_coordinator
|
||||
|
||||
return await native_coordinator.run_native(
|
||||
self,
|
||||
ctx,
|
||||
time_description,
|
||||
purpose,
|
||||
system_prompt,
|
||||
temperature,
|
||||
thinking_level,
|
||||
)
|
||||
try:
|
||||
return await self._run_universal(
|
||||
time_description=time_description,
|
||||
purpose=purpose,
|
||||
system_prompt=system_prompt,
|
||||
)
|
||||
finally:
|
||||
await self._cleanup_universal_resources()
|
||||
@@ -57,14 +57,10 @@ from artemis.config.constants import (
|
||||
ENV_ADB_HOST,
|
||||
ENV_ADB_PORT,
|
||||
ENV_ADB_SERVER_SOCKET,
|
||||
ENV_ANTHROPIC_API_KEY,
|
||||
ENV_ANTIGRAVITY_APP_DIR,
|
||||
ENV_ANTIGRAVITY_LS_ADDRESS,
|
||||
ENV_DATA_ENGINE_DB_PATH,
|
||||
ENV_EVENTS_OUTPUT_PATH,
|
||||
ENV_GCP_API_KEY,
|
||||
ENV_GEMINI_API_KEY,
|
||||
ENV_GOOGLE_API_KEY,
|
||||
ENV_KEEP_VIDEOS,
|
||||
ENV_ARTEMIS_APP_DIR,
|
||||
ENV_ARTEMIS_CLOUD_GATEWAY_URL,
|
||||
@@ -83,11 +79,9 @@ from artemis.config.constants import (
|
||||
ENV_ARTEMIS_TRACES_DIR,
|
||||
ENV_ARTEMIS_USE_FILE_API,
|
||||
ENV_ARTEMIS_USE_USER_DIR,
|
||||
ENV_OPEN_ROUTER_API_KEY,
|
||||
ENV_OPENAI_API_KEY,
|
||||
ENV_OPENAI_BASE_URL,
|
||||
ENV_RESULTS_OUTPUT_PATH,
|
||||
ENV_XAI_API_KEY,
|
||||
IPC_PORT_FILENAME,
|
||||
LLM_CONFIG_FILENAME,
|
||||
LLM_CONFIG_OVERRIDE_FILENAME,
|
||||
@@ -110,7 +104,6 @@ from artemis.config.llm import (
|
||||
initialize_llm_config,
|
||||
load_llm_config_override,
|
||||
parse_llm_config,
|
||||
validate_vertex_ai_credentials,
|
||||
)
|
||||
from artemis.config.output import (
|
||||
OutputConfig,
|
||||
@@ -206,7 +199,6 @@ __all__ = [
|
||||
"get_default_llm_config",
|
||||
"deep_merge_llm_config",
|
||||
"load_llm_config_override",
|
||||
"validate_vertex_ai_credentials",
|
||||
# Agent & Tool Config
|
||||
"AgentGlobalConfig",
|
||||
"CheckerConfig",
|
||||
@@ -290,14 +282,8 @@ __all__ = [
|
||||
"ENV_ADB_HOST",
|
||||
"ENV_ADB_PORT",
|
||||
"ENV_ADB_SERVER_SOCKET",
|
||||
"ENV_GOOGLE_API_KEY",
|
||||
"ENV_GEMINI_API_KEY",
|
||||
"ENV_GCP_API_KEY",
|
||||
"ENV_OPENAI_API_KEY",
|
||||
"ENV_OPENAI_BASE_URL",
|
||||
"ENV_ANTHROPIC_API_KEY",
|
||||
"ENV_OPEN_ROUTER_API_KEY",
|
||||
"ENV_XAI_API_KEY",
|
||||
"ENV_ARTEMIS_EXPLORER_VERSION",
|
||||
"ENV_ARTEMIS_DEFAULT_PROFILE",
|
||||
"ENV_ARTEMIS_DEFAULT_MODEL",
|
||||
|
||||
+14
-38
@@ -26,7 +26,8 @@ from artemis.config.constants import (
|
||||
ExplorerVersion,
|
||||
)
|
||||
from artemis.config.paths import ROOT_DIR, get_config_path
|
||||
from artemis.llm.google import VideoProcessing
|
||||
from artemis.config.settings import settings
|
||||
from artemis.config.core.interpolator import interpolate_config_value
|
||||
from artemis.utils.file import load_jsonc
|
||||
from artemis.utils.logger import get_logger
|
||||
|
||||
@@ -48,7 +49,7 @@ class VideoAnalyzerConfig(BaseModel):
|
||||
description="Whether to enable video action ledger state tracking.",
|
||||
)
|
||||
chunk_size_seconds: float = Field(
|
||||
default=60.0,
|
||||
default_factory=lambda: settings.ARTEMIS_VIDEO_CHUNK_SECONDS,
|
||||
ge=5.0,
|
||||
le=300.0,
|
||||
description="Maximum duration of an initial video-analysis chunk.",
|
||||
@@ -95,44 +96,20 @@ class VideoAnalyzerConfig(BaseModel):
|
||||
le=120,
|
||||
description="Maximum additional action-proximal keyframes per chunk.",
|
||||
)
|
||||
native_max_retries: int = Field(
|
||||
default=1,
|
||||
max_retries: int = Field(
|
||||
default_factory=lambda: settings.ARTEMIS_VIDEO_MAX_RETRIES,
|
||||
ge=0,
|
||||
le=5,
|
||||
description=(
|
||||
"Retries against the native video provider before switching to the "
|
||||
"Retries against the video model before switching to the "
|
||||
"configured universal fallback. A value of 1 means two total attempts."
|
||||
),
|
||||
)
|
||||
model_call_timeout_seconds: float = Field(
|
||||
default=120.0,
|
||||
default_factory=lambda: settings.ARTEMIS_VIDEO_TIMEOUT_SECONDS,
|
||||
ge=15.0,
|
||||
le=600.0,
|
||||
description="Hard timeout for one native or universal video-model response.",
|
||||
)
|
||||
processing: VideoProcessing = Field(
|
||||
default="auto",
|
||||
description=(
|
||||
"How Gemini sub-agents watch a clip. 'agentic' lets the model search,"
|
||||
" scan and re-watch the clip on demand (Gemini Interactions API);"
|
||||
" 'static' samples it at a fixed frame rate; 'auto' picks agentic on"
|
||||
" Gemini 3.6+ Flash models and static everywhere else."
|
||||
),
|
||||
)
|
||||
agentic_chunk_size_seconds: float = Field(
|
||||
default=600.0,
|
||||
ge=30.0,
|
||||
le=1800.0,
|
||||
description=(
|
||||
"Maximum clip length handed to one agentic sub-agent. Static mode keeps"
|
||||
" using chunk_size_seconds."
|
||||
),
|
||||
)
|
||||
agentic_call_timeout_seconds: float = Field(
|
||||
default=300.0,
|
||||
ge=30.0,
|
||||
le=1200.0,
|
||||
description="Hard timeout for one agentic sub-agent response.",
|
||||
description="Hard timeout for one Responses video-model response.",
|
||||
)
|
||||
model_config = {"extra": "allow"}
|
||||
|
||||
@@ -214,12 +191,12 @@ class CheckerConfig(BaseModel):
|
||||
description="Maximum concurrently running checkpoint attempts.",
|
||||
)
|
||||
checkpoint_timeout: float = Field(
|
||||
default=180.0,
|
||||
default_factory=lambda: settings.ARTEMIS_CHECKPOINT_TIMEOUT_SECONDS,
|
||||
gt=0,
|
||||
description="Per-attempt timeout (seconds) for one checkpoint check.",
|
||||
)
|
||||
settlement_timeout: float = Field(
|
||||
default=120.0,
|
||||
default_factory=lambda: settings.ARTEMIS_SETTLEMENT_TIMEOUT_SECONDS,
|
||||
gt=0,
|
||||
description="Aggregate timeout (seconds) for the exit settlement barrier.",
|
||||
)
|
||||
@@ -343,8 +320,7 @@ class ExplorerConfig(BaseModel):
|
||||
caching: bool | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Gemini explicit context caching for multi-turn Explorer tiers. ``None``"
|
||||
" (default) uses the tier's own default: off for pro, on for ultra."
|
||||
"Legacy compatibility field; Responses prompt caching is managed by the endpoint."
|
||||
),
|
||||
)
|
||||
model_config = {"extra": "allow"}
|
||||
@@ -391,7 +367,7 @@ class StepSummarizerConfig(BaseModel):
|
||||
description="Whether to asynchronously summarize historical steps to replace pruned images.",
|
||||
)
|
||||
model: str = Field(
|
||||
default="gemini-2.5-flash-lite",
|
||||
default_factory=lambda: settings.OPENAI_MODEL,
|
||||
description="Lightweight model used for background step state summarization.",
|
||||
)
|
||||
prune_history_xml: bool = Field(
|
||||
@@ -688,7 +664,7 @@ class MemoryChunkingConfig(BaseModel):
|
||||
),
|
||||
)
|
||||
model: str = Field(
|
||||
default="gemini-3.8-flash",
|
||||
default_factory=lambda: settings.OPENAI_MODEL,
|
||||
description="Model used for the chunk-level StepCapsuleLens (bands ①+②).",
|
||||
)
|
||||
max_chunks: int = Field(
|
||||
@@ -1089,7 +1065,7 @@ def load_agent_config(
|
||||
|
||||
try:
|
||||
with open(resolved_path, encoding="utf-8") as f:
|
||||
data = load_jsonc(f)
|
||||
data = interpolate_config_value(load_jsonc(f))
|
||||
if isinstance(data, dict) and "agent" in data:
|
||||
return AgentGlobalConfig.model_validate(data["agent"])
|
||||
return AgentGlobalConfig.model_validate(data)
|
||||
|
||||
@@ -53,17 +53,11 @@ ENV_ARTEMIS_HIERARCHY_BACKEND = "ARTEMIS_HIERARCHY_BACKEND"
|
||||
ENV_ARTEMIS_HELPER_AUTO_INSTALL = "ARTEMIS_HELPER_AUTO_INSTALL"
|
||||
|
||||
# LLM & Vision OCR API Keys
|
||||
ENV_GOOGLE_API_KEY = "GOOGLE_API_KEY"
|
||||
ENV_GEMINI_API_KEY = "GEMINI_API_KEY"
|
||||
ENV_GCP_API_KEY = "GCP_API_KEY"
|
||||
ENV_OCR_API_KEY = "OCR_API_KEY"
|
||||
ENV_VISION_API_KEY = "VISION_API_KEY"
|
||||
ENV_API_KEY = "API_KEY"
|
||||
ENV_OPENAI_API_KEY = "OPENAI_API_KEY"
|
||||
ENV_OPENAI_BASE_URL = "OPENAI_BASE_URL"
|
||||
ENV_ANTHROPIC_API_KEY = "ANTHROPIC_API_KEY"
|
||||
ENV_OPEN_ROUTER_API_KEY = "OPEN_ROUTER_API_KEY"
|
||||
ENV_XAI_API_KEY = "XAI_API_KEY"
|
||||
|
||||
# Explorer & Agent Defaults
|
||||
ENV_ARTEMIS_EXPLORER_VERSION = "ARTEMIS_EXPLORER_VERSION"
|
||||
@@ -110,7 +104,7 @@ DEFAULT_ACTION_TIMEOUT = 30.0
|
||||
DEFAULT_STREAM_PING_INTERVAL = 15.0
|
||||
|
||||
DEFAULT_PROFILE = "pro"
|
||||
DEFAULT_MODEL = "gemini-2.5-flash"
|
||||
DEFAULT_MODEL = "gpt-5-mini"
|
||||
DEFAULT_EXPLORER_VERSION: Literal["flash", "pro", "ultra"] = "flash"
|
||||
|
||||
|
||||
@@ -118,9 +112,7 @@ DEFAULT_EXPLORER_VERSION: Literal["flash", "pro", "ultra"] = "flash"
|
||||
# Literals and Type Aliases
|
||||
# ==============================================================================
|
||||
|
||||
LLMProvider = Literal[
|
||||
"openai", "google", "openrouter", "xai", "vertexai", "anthropic", "ollama", "vllm", "custom"
|
||||
]
|
||||
LLMProvider = Literal["openai"]
|
||||
ExplorerVersion = Literal["flash", "pro", "ultra"]
|
||||
|
||||
LLMUtilsNode = Literal[
|
||||
|
||||
@@ -74,7 +74,7 @@ class CascadingConfigEngine:
|
||||
# 3. Workspace .env
|
||||
ws_env = self.workspace_root / ".env"
|
||||
if ws_env.exists():
|
||||
load_dotenv(dotenv_path=ws_env, override=True, encoding="utf-8")
|
||||
load_dotenv(dotenv_path=ws_env, override=False, encoding="utf-8")
|
||||
|
||||
def load_raw_config(
|
||||
self,
|
||||
|
||||
+19
-46
@@ -15,12 +15,11 @@
|
||||
"""LLM provider, model hierarchy, fallback chaining, and configuration loaders."""
|
||||
|
||||
import os
|
||||
from artemis.config.core.interpolator import interpolate_config_value
|
||||
from pathlib import Path
|
||||
from typing import Any, Literal
|
||||
|
||||
import google.auth
|
||||
from google.auth.exceptions import DefaultCredentialsError
|
||||
from pydantic import BaseModel, ValidationError
|
||||
from pydantic import Field, BaseModel, ValidationError
|
||||
|
||||
from artemis.config.constants import (
|
||||
LLM_CONFIG_FILENAME,
|
||||
@@ -36,18 +35,6 @@ from artemis.utils.logger import get_logger
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
def validate_vertex_ai_credentials() -> None:
|
||||
"""Validate Google Application Default Credentials for VertexAI provider."""
|
||||
try:
|
||||
_, project = google.auth.default()
|
||||
if not project:
|
||||
raise Exception("VertexAI requires a Google Cloud project to be set.")
|
||||
except DefaultCredentialsError as e:
|
||||
raise Exception(
|
||||
f"VertexAI requires valid Google Application Default Credentials (ADC): {e}"
|
||||
)
|
||||
|
||||
|
||||
class _CyFunctionDetectorMeta(type):
|
||||
def __instancecheck__(self, instance):
|
||||
name = type(instance).__name__
|
||||
@@ -72,13 +59,14 @@ class LLM(BaseModel):
|
||||
"""Base model representing an LLM model provider and runtime parameters."""
|
||||
|
||||
model_config = {"ignored_types": (CyFunctionDetector,)}
|
||||
provider: LLMProvider
|
||||
model: str
|
||||
provider: LLMProvider = "openai"
|
||||
model: str = Field(default_factory=lambda: settings.OPENAI_MODEL)
|
||||
api_base: str | None = None
|
||||
timeout: float = Field(default_factory=lambda: settings.OPENAI_TIMEOUT_SECONDS, gt=0)
|
||||
max_retries: int = Field(default_factory=lambda: settings.OPENAI_MAX_RETRIES, ge=0)
|
||||
max_tokens: int | None = Field(default=None, gt=0)
|
||||
temperature: float | None = None
|
||||
thinking_budget: int | None = None
|
||||
thinking_level: Literal["minimal", "low", "medium", "high"] | None = None
|
||||
reasoning_effort: Literal["none", "low", "medium", "high"] | None = None
|
||||
include_thoughts: bool | None = None
|
||||
enable_grounding: bool | None = None
|
||||
|
||||
def validate_provider(self, name: str) -> None:
|
||||
@@ -86,20 +74,6 @@ class LLM(BaseModel):
|
||||
if self.provider == "openai":
|
||||
if not settings.OPENAI_API_KEY:
|
||||
raise Exception(f"{name} requires OPENAI_API_KEY in .env")
|
||||
elif self.provider == "google":
|
||||
if not settings.GOOGLE_API_KEY:
|
||||
raise Exception(f"{name} requires GOOGLE_API_KEY in .env")
|
||||
elif self.provider == "vertexai":
|
||||
validate_vertex_ai_credentials()
|
||||
elif self.provider == "anthropic":
|
||||
if not (settings.ANTHROPIC_API_KEY or os.environ.get("ANTHROPIC_API_KEY")):
|
||||
raise Exception(f"{name} requires ANTHROPIC_API_KEY in .env")
|
||||
elif self.provider == "openrouter":
|
||||
if not settings.OPEN_ROUTER_API_KEY:
|
||||
raise Exception(f"{name} requires OPEN_ROUTER_API_KEY in .env")
|
||||
elif self.provider == "xai":
|
||||
if not settings.XAI_API_KEY:
|
||||
raise Exception(f"{name} requires XAI_API_KEY in .env")
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f"{self.provider}/{self.model}"
|
||||
@@ -110,7 +84,6 @@ class LLMWithFallback(LLM):
|
||||
|
||||
fallback: LLM
|
||||
fix_model: str | None = None
|
||||
timeout: float | None = None
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f"{self.provider}/{self.model} (fallback: {self.fallback})"
|
||||
@@ -118,14 +91,14 @@ class LLMWithFallback(LLM):
|
||||
|
||||
def lightweight_judge_default() -> "LLMWithFallback":
|
||||
"""Factory default for the lightweight judge nodes (pixel safety net and
|
||||
planner validation): a flash-lite model at temperature 0."""
|
||||
planner validation): the configured Responses model at temperature 0."""
|
||||
return LLMWithFallback(
|
||||
provider="google",
|
||||
model="gemini-3.5-flash-lite",
|
||||
provider="openai",
|
||||
model=settings.OPENAI_MODEL,
|
||||
temperature=0.0,
|
||||
fallback=LLM(
|
||||
provider="google",
|
||||
model="gemini-3.1-flash-lite",
|
||||
provider="openai",
|
||||
model=settings.OPENAI_MODEL,
|
||||
temperature=0.0,
|
||||
),
|
||||
)
|
||||
@@ -236,11 +209,11 @@ def _expand_default_into_nodes(config_dict: dict) -> dict:
|
||||
default_model_cfg = config_dict.get(
|
||||
"default",
|
||||
{
|
||||
"provider": "google",
|
||||
"model": "gemini-3.8-flash",
|
||||
"provider": "openai",
|
||||
"model": settings.OPENAI_MODEL,
|
||||
"fallback": {
|
||||
"provider": "google",
|
||||
"model": "gemini-3.7-flash",
|
||||
"provider": "openai",
|
||||
"model": settings.OPENAI_MODEL,
|
||||
},
|
||||
},
|
||||
)
|
||||
@@ -310,7 +283,7 @@ def parse_llm_config() -> LLMConfig:
|
||||
|
||||
try:
|
||||
with open(config_path, encoding="utf-8") as f:
|
||||
config_dict = load_jsonc(f)
|
||||
config_dict = interpolate_config_value(load_jsonc(f))
|
||||
expanded_dict = _expand_default_into_nodes(config_dict)
|
||||
return LLMConfig.model_validate(expanded_dict)
|
||||
except Exception as e:
|
||||
@@ -361,7 +334,7 @@ def load_llm_config_override(path: Path | str) -> LLMConfig:
|
||||
if resolved_path.exists():
|
||||
logger.info(f"Loading custom LLM config from {resolved_path.resolve()}...")
|
||||
with open(resolved_path, encoding="utf-8") as f:
|
||||
override_config_dict = load_jsonc(f)
|
||||
override_config_dict = interpolate_config_value(load_jsonc(f))
|
||||
else:
|
||||
logger.warning(f"Custom LLM config not found at {path} - using default config")
|
||||
|
||||
|
||||
+21
-71
@@ -26,16 +26,10 @@ from artemis.config.constants import (
|
||||
DEFAULT_ADB_PORT,
|
||||
DEFAULT_MODEL,
|
||||
DEFAULT_PROFILE,
|
||||
ENV_ANTHROPIC_API_KEY,
|
||||
ENV_DATA_ENGINE_DB_PATH,
|
||||
ENV_GCP_API_KEY,
|
||||
ENV_GEMINI_API_KEY,
|
||||
ENV_GOOGLE_API_KEY,
|
||||
ENV_OCR_API_KEY,
|
||||
ENV_OPEN_ROUTER_API_KEY,
|
||||
ENV_OPENAI_API_KEY,
|
||||
ENV_VISION_API_KEY,
|
||||
ENV_XAI_API_KEY,
|
||||
)
|
||||
from artemis.config.paths import (
|
||||
GLOBAL_APP_DIR,
|
||||
@@ -94,12 +88,6 @@ class Settings(BaseSettings):
|
||||
|
||||
# LLM Provider Authentication
|
||||
OPENAI_API_KEY: SecretStr | None = None
|
||||
GOOGLE_API_KEY: SecretStr | None = None
|
||||
GEMINI_API_KEY: SecretStr | None = None
|
||||
GCP_API_KEY: SecretStr | None = None
|
||||
ANTHROPIC_API_KEY: SecretStr | None = None
|
||||
XAI_API_KEY: SecretStr | None = None
|
||||
OPEN_ROUTER_API_KEY: SecretStr | None = None
|
||||
|
||||
# Google Cloud Vision OCR Authentication
|
||||
OCR_API_KEY: SecretStr | None = None
|
||||
@@ -108,6 +96,19 @@ class Settings(BaseSettings):
|
||||
|
||||
# Custom Provider Endpoints
|
||||
OPENAI_BASE_URL: str | None = None
|
||||
OPENAI_MODEL: str = Field(default="gpt-5-mini", min_length=1)
|
||||
OPENAI_TIMEOUT_SECONDS: float = Field(default=120.0, gt=0)
|
||||
OPENAI_MAX_RETRIES: int = Field(default=0, ge=0)
|
||||
ARTEMIS_CREDENTIAL_TIMEOUT_SECONDS: float = Field(default=12.0, gt=0)
|
||||
ARTEMIS_MODEL_CACHE_TTL_SECONDS: float = Field(default=10.0, ge=0)
|
||||
ARTEMIS_VIDEO_TIMEOUT_SECONDS: float = Field(default=120.0, gt=0)
|
||||
ARTEMIS_VIDEO_MAX_RETRIES: int = Field(default=1, ge=0)
|
||||
ARTEMIS_VIDEO_CHUNK_SECONDS: float = Field(default=60.0, gt=0)
|
||||
ARTEMIS_CHECKPOINT_TIMEOUT_SECONDS: float = Field(default=180.0, gt=0)
|
||||
ARTEMIS_SETTLEMENT_TIMEOUT_SECONDS: float = Field(default=120.0, gt=0)
|
||||
ARTEMIS_VIDEO_ENABLED: bool = True
|
||||
ARTEMIS_OUTPUTTER_ENABLED: bool = True
|
||||
ARTEMIS_STEP_SUMMARIZER_ENABLED: bool = True
|
||||
|
||||
# Android ADB Connectivity
|
||||
ADB_HOST: str | None = Field(default=DEFAULT_ADB_HOST)
|
||||
@@ -125,16 +126,6 @@ class Settings(BaseSettings):
|
||||
ARTEMIS_DEFAULT_PROFILE: str = Field(default=DEFAULT_PROFILE)
|
||||
ARTEMIS_DEFAULT_MODEL: str = Field(default=DEFAULT_MODEL)
|
||||
|
||||
# Explorer Tool Settings (the tier itself is configured in artemis.jsonc or
|
||||
# via ARTEMIS_EXPLORER_VERSION; see artemis.config.agent.ExplorerConfig)
|
||||
EXPLORER_CACHING: bool | None = Field(
|
||||
default=None,
|
||||
description=(
|
||||
"Environment-level override for Explorer context caching; unset"
|
||||
" defers to the agent configuration and the tier default."
|
||||
),
|
||||
)
|
||||
|
||||
# LLM Reliability
|
||||
LLM_PAUSE_TIMEOUT_SECONDS: float = Field(
|
||||
default=900.0,
|
||||
@@ -154,16 +145,10 @@ class Settings(BaseSettings):
|
||||
|
||||
@model_validator(mode="after")
|
||||
def fallback_api_keys(self) -> "Settings":
|
||||
"""Normalize Google / Gemini / GCP and OCR API keys and filter placeholders."""
|
||||
"""Normalize OpenAI and OCR API keys and filter placeholders."""
|
||||
# Sanitize any placeholder values loaded from environment or .env
|
||||
for attr in (
|
||||
"OPENAI_API_KEY",
|
||||
"GOOGLE_API_KEY",
|
||||
"GEMINI_API_KEY",
|
||||
"GCP_API_KEY",
|
||||
"ANTHROPIC_API_KEY",
|
||||
"XAI_API_KEY",
|
||||
"OPEN_ROUTER_API_KEY",
|
||||
"OCR_API_KEY",
|
||||
"VISION_API_KEY",
|
||||
"API_KEY",
|
||||
@@ -172,12 +157,6 @@ class Settings(BaseSettings):
|
||||
if val and is_placeholder_key(val):
|
||||
setattr(self, attr, None)
|
||||
|
||||
if not self.GOOGLE_API_KEY:
|
||||
if self.GEMINI_API_KEY:
|
||||
self.GOOGLE_API_KEY = self.GEMINI_API_KEY
|
||||
elif self.GCP_API_KEY:
|
||||
self.GOOGLE_API_KEY = self.GCP_API_KEY
|
||||
|
||||
# Fallback for OCR & Vision API keys
|
||||
if not self.OCR_API_KEY and self.VISION_API_KEY:
|
||||
self.OCR_API_KEY = self.VISION_API_KEY
|
||||
@@ -187,27 +166,18 @@ class Settings(BaseSettings):
|
||||
"""Get API key for a specified provider.
|
||||
|
||||
Args:
|
||||
provider: Provider name (e.g., 'google', 'gemini', 'openai', 'ocr', 'vision').
|
||||
provider: Provider name ('openai', 'ocr', or 'vision').
|
||||
|
||||
Returns:
|
||||
SecretStr containing the API key or None if not configured.
|
||||
"""
|
||||
from artemis.llm.google.provider import is_google_family_provider
|
||||
|
||||
provider_lower = provider.lower()
|
||||
key: SecretStr | None = None
|
||||
if is_google_family_provider(provider_lower):
|
||||
key = self.GOOGLE_API_KEY or self.GEMINI_API_KEY or self.GCP_API_KEY
|
||||
elif provider_lower in ("ocr", "vision", "google_vision"):
|
||||
if provider_lower in ("ocr", "vision", "google_vision"):
|
||||
key = self.OCR_API_KEY or self.VISION_API_KEY
|
||||
elif provider_lower == "openai":
|
||||
key = self.OPENAI_API_KEY
|
||||
elif provider_lower in ("anthropic", "claude"):
|
||||
key = self.ANTHROPIC_API_KEY
|
||||
elif provider_lower == "openrouter":
|
||||
key = self.OPEN_ROUTER_API_KEY
|
||||
elif provider_lower in ("xai", "grok"):
|
||||
key = self.XAI_API_KEY
|
||||
|
||||
if key and not is_placeholder_key(key):
|
||||
return key
|
||||
@@ -222,35 +192,14 @@ class Settings(BaseSettings):
|
||||
persist_to_env: Whether to save the key to the app directory's .env file.
|
||||
"""
|
||||
secret = SecretStr(key)
|
||||
from artemis.llm.google.provider import is_google_family_provider
|
||||
|
||||
provider_lower = provider.lower()
|
||||
env_key_name = None
|
||||
|
||||
if is_google_family_provider(provider_lower):
|
||||
self.GOOGLE_API_KEY = secret
|
||||
self.GEMINI_API_KEY = secret
|
||||
self.GCP_API_KEY = secret
|
||||
env_key_name = ENV_GEMINI_API_KEY
|
||||
os.environ[ENV_GOOGLE_API_KEY] = key
|
||||
os.environ[ENV_GEMINI_API_KEY] = key
|
||||
os.environ[ENV_GCP_API_KEY] = key
|
||||
elif provider_lower == "openai":
|
||||
if provider_lower == "openai":
|
||||
self.OPENAI_API_KEY = secret
|
||||
env_key_name = ENV_OPENAI_API_KEY
|
||||
os.environ[ENV_OPENAI_API_KEY] = key
|
||||
elif provider_lower in ("anthropic", "claude"):
|
||||
self.ANTHROPIC_API_KEY = secret
|
||||
env_key_name = ENV_ANTHROPIC_API_KEY
|
||||
os.environ[ENV_ANTHROPIC_API_KEY] = key
|
||||
elif provider_lower == "openrouter":
|
||||
self.OPEN_ROUTER_API_KEY = secret
|
||||
env_key_name = ENV_OPEN_ROUTER_API_KEY
|
||||
os.environ[ENV_OPEN_ROUTER_API_KEY] = key
|
||||
elif provider_lower == "xai":
|
||||
self.XAI_API_KEY = secret
|
||||
env_key_name = ENV_XAI_API_KEY
|
||||
os.environ[ENV_XAI_API_KEY] = key
|
||||
elif provider_lower in ("ocr", "vision", "google_vision"):
|
||||
self.OCR_API_KEY = secret
|
||||
self.VISION_API_KEY = secret
|
||||
@@ -258,6 +207,9 @@ class Settings(BaseSettings):
|
||||
os.environ[ENV_OCR_API_KEY] = key
|
||||
os.environ[ENV_VISION_API_KEY] = key
|
||||
|
||||
else:
|
||||
raise ValueError(f"Unsupported credential provider: {provider}")
|
||||
|
||||
if persist_to_env and env_key_name:
|
||||
target_env_files = [get_env_file()]
|
||||
seen_paths = set()
|
||||
@@ -275,9 +227,7 @@ class Settings(BaseSettings):
|
||||
|
||||
# Determine keys to update
|
||||
keys_to_update = [env_key_name]
|
||||
if is_google_family_provider(provider_lower):
|
||||
keys_to_update = [ENV_GEMINI_API_KEY, ENV_GOOGLE_API_KEY, ENV_GCP_API_KEY]
|
||||
elif provider_lower in ("ocr", "vision", "google_vision"):
|
||||
if provider_lower in ("ocr", "vision", "google_vision"):
|
||||
keys_to_update = [ENV_OCR_API_KEY, ENV_VISION_API_KEY]
|
||||
|
||||
new_lines = []
|
||||
|
||||
@@ -20,28 +20,3 @@ VALIDATOR_POLL_TIMEOUT = 2.0
|
||||
VALIDATOR_POLL_INTERVAL = 0.2
|
||||
VALIDATOR_OCR_INTERVAL = 1.0
|
||||
VALIDATOR_UI_HIERARCHY_TIMEOUT = 1.0
|
||||
|
||||
from google.genai import types as _genai_types
|
||||
|
||||
SAFETY_SETTINGS_BLOCK_NONE = [
|
||||
_genai_types.SafetySetting(
|
||||
category=_genai_types.HarmCategory.HARM_CATEGORY_HATE_SPEECH,
|
||||
threshold=_genai_types.HarmBlockThreshold.BLOCK_NONE,
|
||||
),
|
||||
_genai_types.SafetySetting(
|
||||
category=_genai_types.HarmCategory.HARM_CATEGORY_HARASSMENT,
|
||||
threshold=_genai_types.HarmBlockThreshold.BLOCK_NONE,
|
||||
),
|
||||
_genai_types.SafetySetting(
|
||||
category=_genai_types.HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,
|
||||
threshold=_genai_types.HarmBlockThreshold.BLOCK_NONE,
|
||||
),
|
||||
_genai_types.SafetySetting(
|
||||
category=_genai_types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
|
||||
threshold=_genai_types.HarmBlockThreshold.BLOCK_NONE,
|
||||
),
|
||||
_genai_types.SafetySetting(
|
||||
category=_genai_types.HarmCategory.HARM_CATEGORY_CIVIC_INTEGRITY,
|
||||
threshold=_genai_types.HarmBlockThreshold.BLOCK_NONE,
|
||||
),
|
||||
]
|
||||
@@ -26,11 +26,11 @@ from artemis.core.diagnostics.schema import (
|
||||
|
||||
|
||||
class LLMCredentialsProbe(BaseProbe):
|
||||
"""Probe verifying Gemini and multi-provider multimodal LLM credentials."""
|
||||
"""Readiness depends on an OpenAI credential, never merely a base URL."""
|
||||
|
||||
@property
|
||||
def probe_id(self) -> str:
|
||||
return "gemini_api_key"
|
||||
return "openai_api_key"
|
||||
|
||||
@property
|
||||
def category(self) -> ProbeCategory:
|
||||
@@ -40,192 +40,38 @@ class LLMCredentialsProbe(BaseProbe):
|
||||
def is_blocker(self) -> bool:
|
||||
return True
|
||||
|
||||
def _mask_key(self, key_str: str) -> str:
|
||||
"""Helper to safely mask an API credential for display."""
|
||||
if len(key_str) > 10:
|
||||
return f"{key_str[:6]}...{key_str[-4:]}"
|
||||
return "***"
|
||||
|
||||
async def probe(self) -> ProbeResult:
|
||||
import os
|
||||
|
||||
from artemis.config.settings import is_placeholder_key
|
||||
|
||||
gemini_key = settings.get_api_key("google")
|
||||
openai_key = settings.get_api_key("openai")
|
||||
claude_key = settings.get_api_key("anthropic")
|
||||
openrouter_key = settings.get_api_key("openrouter")
|
||||
xai_key = settings.get_api_key("xai")
|
||||
ocr_key = settings.get_api_key("ocr")
|
||||
|
||||
configured_providers: list[dict[str, Any]] = []
|
||||
api_keys_map: dict[str, str] = {}
|
||||
if gemini_key and not is_placeholder_key(gemini_key):
|
||||
g_val = gemini_key.get_secret_value()
|
||||
configured_providers.append(
|
||||
{
|
||||
"provider": "google",
|
||||
"label": "Gemini",
|
||||
"masked": self._mask_key(g_val),
|
||||
"raw_key": g_val,
|
||||
"key": g_val,
|
||||
}
|
||||
)
|
||||
api_keys_map["google"] = g_val
|
||||
api_keys_map["gemini"] = g_val
|
||||
if openai_key and not is_placeholder_key(openai_key):
|
||||
o_val = openai_key.get_secret_value()
|
||||
configured_providers.append(
|
||||
key = settings.get_api_key("openai")
|
||||
configured = bool(key)
|
||||
providers = (
|
||||
[
|
||||
{
|
||||
"provider": "openai",
|
||||
"label": "ChatGPT",
|
||||
"masked": self._mask_key(o_val),
|
||||
"raw_key": o_val,
|
||||
"key": o_val,
|
||||
"label": "OpenAI Responses",
|
||||
"masked": "****" + key.get_secret_value()[-4:],
|
||||
}
|
||||
)
|
||||
api_keys_map["openai"] = o_val
|
||||
if claude_key and not is_placeholder_key(claude_key):
|
||||
c_val = claude_key.get_secret_value()
|
||||
configured_providers.append(
|
||||
{
|
||||
"provider": "anthropic",
|
||||
"label": "Claude",
|
||||
"masked": self._mask_key(c_val),
|
||||
"raw_key": c_val,
|
||||
"key": c_val,
|
||||
}
|
||||
)
|
||||
api_keys_map["anthropic"] = c_val
|
||||
if openrouter_key and not is_placeholder_key(openrouter_key):
|
||||
or_val = openrouter_key.get_secret_value()
|
||||
configured_providers.append(
|
||||
{
|
||||
"provider": "openrouter",
|
||||
"label": "OpenRouter",
|
||||
"masked": self._mask_key(or_val),
|
||||
"raw_key": or_val,
|
||||
"key": or_val,
|
||||
}
|
||||
)
|
||||
api_keys_map["openrouter"] = or_val
|
||||
if xai_key and not is_placeholder_key(xai_key):
|
||||
x_val = xai_key.get_secret_value()
|
||||
configured_providers.append(
|
||||
{
|
||||
"provider": "xai",
|
||||
"label": "xAI (Grok)",
|
||||
"masked": self._mask_key(x_val),
|
||||
"raw_key": x_val,
|
||||
"key": x_val,
|
||||
}
|
||||
)
|
||||
api_keys_map["xai"] = x_val
|
||||
|
||||
# Detect any custom model endpoints or environment variables defined in files
|
||||
for env_var, label, prov_id in [
|
||||
("DEEPSEEK_API_KEY", "DeepSeek", "deepseek"),
|
||||
("GROQ_API_KEY", "Groq", "groq"),
|
||||
("OPENAI_BASE_URL", "Custom OpenAI Endpoint", "custom"),
|
||||
("OLLAMA_BASE_URL", "Local Ollama", "ollama"),
|
||||
("VLLM_BASE_URL", "vLLM Endpoint", "vllm"),
|
||||
("VERTEX_AI_PROJECT", "Google Cloud Vertex AI", "vertexai"),
|
||||
]:
|
||||
val = os.environ.get(env_var)
|
||||
if val and val.strip() and not is_placeholder_key(val.strip()):
|
||||
configured_providers.append(
|
||||
{
|
||||
"provider": prov_id,
|
||||
"label": label,
|
||||
"masked": self._mask_key(val.strip()),
|
||||
"raw_key": val.strip(),
|
||||
"key": val.strip(),
|
||||
}
|
||||
)
|
||||
api_keys_map[prov_id] = val.strip()
|
||||
|
||||
if ocr_key and not is_placeholder_key(ocr_key):
|
||||
api_keys_map["ocr"] = ocr_key.get_secret_value()
|
||||
|
||||
current_active_key = (
|
||||
gemini_key.get_secret_value()
|
||||
if gemini_key
|
||||
else (configured_providers[0]["key"] if configured_providers else "")
|
||||
]
|
||||
if key
|
||||
else []
|
||||
)
|
||||
|
||||
metadata = {
|
||||
"configured_count": len(configured_providers),
|
||||
"providers": configured_providers,
|
||||
"has_ocr_key": ocr_key is not None,
|
||||
"current_key": current_active_key,
|
||||
"current_gemini_key": gemini_key.get_secret_value() if gemini_key else "",
|
||||
"api_keys": api_keys_map,
|
||||
}
|
||||
|
||||
# Case 1: Gemini API Key configured (Standard / Recommended)
|
||||
if gemini_key:
|
||||
masked = self._mask_key(gemini_key.get_secret_value())
|
||||
return ProbeResult(
|
||||
id=self.probe_id,
|
||||
category=self.category,
|
||||
title="Multimodal LLM API Key",
|
||||
status=ProbeStatus.PASS,
|
||||
is_blocker=self.is_blocker,
|
||||
summary="Active (Gemini)",
|
||||
description=f"Gemini multimodal API credential is active ({masked}) and ready for vision and reasoning.",
|
||||
metadata=metadata,
|
||||
actions=[
|
||||
ProbeAction(
|
||||
action_type="hint",
|
||||
label="Provider Active",
|
||||
payload="Gemini 2.5 Flash / Pro Multimodal Engine is enabled.",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
# Case 2: Other LLM provider configured
|
||||
if configured_providers:
|
||||
first_p = configured_providers[0]
|
||||
return ProbeResult(
|
||||
id=self.probe_id,
|
||||
category=self.category,
|
||||
title="Multimodal LLM API Key",
|
||||
status=ProbeStatus.PASS,
|
||||
is_blocker=self.is_blocker,
|
||||
summary=f"Active ({first_p['label']})",
|
||||
description=f"{first_p['label']} multimodal API credential is active ({first_p['masked']}).",
|
||||
metadata=metadata,
|
||||
actions=[
|
||||
ProbeAction(
|
||||
action_type="hint",
|
||||
label="Provider Active",
|
||||
payload=f"{first_p['label']} multimodal vision endpoint is active.",
|
||||
)
|
||||
],
|
||||
)
|
||||
|
||||
# Case 3: No LLM key configured
|
||||
return ProbeResult(
|
||||
id=self.probe_id,
|
||||
category=self.category,
|
||||
title="Multimodal LLM API Key",
|
||||
status=ProbeStatus.FAIL,
|
||||
is_blocker=self.is_blocker,
|
||||
summary="Key Missing",
|
||||
description="No Multimodal LLM credential (e.g. GEMINI_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY) found in environment or .env file.",
|
||||
metadata=metadata,
|
||||
actions=[
|
||||
ProbeAction(
|
||||
action_type="command",
|
||||
label="Run Artemis Init",
|
||||
payload="artemis init",
|
||||
),
|
||||
ProbeAction(
|
||||
action_type="link",
|
||||
label="Get Google AI Studio Key",
|
||||
payload="https://aistudio.google.com/app/apikey",
|
||||
),
|
||||
],
|
||||
title="OpenAI API 密钥",
|
||||
status=ProbeStatus.PASS if configured else ProbeStatus.FAIL,
|
||||
is_blocker=True,
|
||||
summary="已配置 OpenAI Responses" if configured else "缺少 API 密钥",
|
||||
description="模型统一通过 OpenAI Responses 接口调用。"
|
||||
if configured
|
||||
else "请在环境变量或 .env 中设置 OPENAI_API_KEY。",
|
||||
metadata={
|
||||
"configured_count": len(providers),
|
||||
"providers": providers,
|
||||
"has_ocr_key": bool(settings.get_api_key("ocr")),
|
||||
},
|
||||
actions=[]
|
||||
if configured
|
||||
else [ProbeAction(action_type="command", label="初始化配置", payload="artemis init")],
|
||||
)
|
||||
|
||||
|
||||
@@ -267,7 +113,7 @@ class VisionOCRProbe(BaseProbe):
|
||||
is_blocker=False,
|
||||
summary="Active & Configured",
|
||||
description=f"Google Cloud Vision OCR ({masked}) is active for image text recognition.",
|
||||
metadata={"configured": True, "masked_key": masked, "key": val, "raw_key": val},
|
||||
metadata={"configured": True, "masked_key": masked},
|
||||
actions=[
|
||||
ProbeAction(
|
||||
action_type="hint",
|
||||
|
||||
@@ -17,7 +17,7 @@
|
||||
import platform
|
||||
import sys
|
||||
|
||||
from artemis.config import parse_llm_config
|
||||
from artemis.config import parse_llm_config, settings
|
||||
from artemis.core.diagnostics.probes.base import BaseProbe
|
||||
from artemis.core.diagnostics.schema import (
|
||||
ProbeAction,
|
||||
@@ -67,7 +67,7 @@ class PythonRuntimeProbe(BaseProbe):
|
||||
title="Python Runtime Environment",
|
||||
status=ProbeStatus.PASS,
|
||||
is_blocker=self.is_blocker,
|
||||
summary=f"Python {py_ver_str} Ready",
|
||||
summary=f"Python {py_ver_str} 已就绪",
|
||||
description=f"Python {py_ver_str} ({arch}, 64-bit) detected at {executable}. Satisfies requirement (>= 3.12).",
|
||||
metadata=metadata,
|
||||
actions=[
|
||||
@@ -121,7 +121,7 @@ class SystemConfigProbe(BaseProbe):
|
||||
async def probe(self) -> ProbeResult:
|
||||
try:
|
||||
cfg = parse_llm_config()
|
||||
planner_model = getattr(getattr(cfg, "planner", None), "model", "Gemini 2.5 Flash")
|
||||
planner_model = getattr(getattr(cfg, "planner", None), "model", settings.OPENAI_MODEL)
|
||||
metadata = {
|
||||
"valid": True,
|
||||
"config_file": "config/artemis.jsonc",
|
||||
@@ -133,7 +133,7 @@ class SystemConfigProbe(BaseProbe):
|
||||
title="Artemis System Configuration",
|
||||
status=ProbeStatus.PASS,
|
||||
is_blocker=self.is_blocker,
|
||||
summary="Config Valid",
|
||||
summary="配置有效",
|
||||
description=f"System configuration loaded and validated (Planner model: {planner_model}).",
|
||||
metadata=metadata,
|
||||
actions=[
|
||||
|
||||
@@ -72,7 +72,7 @@ class ToolchainProbe(BaseProbe):
|
||||
title="Video Recording Toolchain",
|
||||
status=ProbeStatus.PASS,
|
||||
is_blocker=self.is_blocker,
|
||||
summary="Ready (FFmpeg + scrcpy)",
|
||||
summary="录屏工具已就绪(FFmpeg + scrcpy)",
|
||||
description=f"FFmpeg ({ffmpeg_path}) and scrcpy ({scrcpy_path}) are available. Live stream and high-speed video replays are active.",
|
||||
metadata=metadata,
|
||||
actions=[
|
||||
@@ -132,7 +132,7 @@ class ToolchainProbe(BaseProbe):
|
||||
title="Video Recording Toolchain",
|
||||
status=ProbeStatus.FAIL,
|
||||
is_blocker=self.is_blocker,
|
||||
summary=f"Missing {missing_str}",
|
||||
summary=f"缺少 {missing_str}",
|
||||
description=f"Video toolchain is required for screen streaming and test replay recording. Missing: {missing_str}.",
|
||||
metadata=metadata,
|
||||
actions=actions,
|
||||
|
||||
@@ -39,7 +39,7 @@ CHECK_ORDER: dict[str, int] = {
|
||||
"python_runtime": 0,
|
||||
"system_config": 1,
|
||||
"integration_host": 2,
|
||||
"gemini_api_key": 3,
|
||||
"openai_api_key": 3,
|
||||
"android_adb": 4,
|
||||
"toolchain": 5,
|
||||
"vision_ocr_key": 6,
|
||||
|
||||
@@ -78,7 +78,7 @@ class ProbeResult(BaseModel):
|
||||
"""Outcome of a single readiness probe check."""
|
||||
|
||||
id: str = Field(
|
||||
description="Unique identifier for the probe (e.g. 'android_adb', 'gemini_api_key')"
|
||||
description="Unique identifier for the probe (e.g. 'android_adb', 'openai_api_key')"
|
||||
)
|
||||
category: ProbeCategory = Field(description="Category of the probe")
|
||||
title: str = Field(description="Human-readable title (e.g. 'Device / Emulator Connected')")
|
||||
|
||||
@@ -36,21 +36,7 @@ def init_command() -> None:
|
||||
)
|
||||
)
|
||||
|
||||
# 1. Select Provider
|
||||
console.print("\n[bold]Step 1: Choose Primary LLM Provider[/bold]")
|
||||
console.print(" [1] Google Gemini (Recommended - Default)")
|
||||
console.print(" [2] OpenAI (GPT-4o)")
|
||||
console.print(" [3] Anthropic (Claude 3.5 Sonnet)")
|
||||
console.print(" [4] OpenRouter")
|
||||
|
||||
choice = Prompt.ask("Select provider", choices=["1", "2", "3", "4"], default="1")
|
||||
provider_map = {
|
||||
"1": ("GEMINI_API_KEY", "Google Gemini"),
|
||||
"2": ("OPENAI_API_KEY", "OpenAI"),
|
||||
"3": ("ANTHROPIC_API_KEY", "Anthropic"),
|
||||
"4": ("OPEN_ROUTER_API_KEY", "OpenRouter"),
|
||||
}
|
||||
env_key, provider_name = provider_map[choice]
|
||||
env_key, provider_name = "OPENAI_API_KEY", "OpenAI Responses"
|
||||
|
||||
# 2. Enter API Key
|
||||
console.print(f"\n[bold]Step 2: Enter your {provider_name} API Key[/bold]")
|
||||
@@ -126,7 +112,7 @@ def init_command() -> None:
|
||||
"",
|
||||
"# Execution Defaults",
|
||||
"ARTEMIS_DEFAULT_PROFILE=pro",
|
||||
"ARTEMIS_DEFAULT_MODEL=gemini-2.5-flash",
|
||||
"OPENAI_MODEL=gpt-5-mini",
|
||||
"ARTEMIS_TRACES_DIR=./traces",
|
||||
"",
|
||||
]
|
||||
|
||||
@@ -503,7 +503,7 @@ def run_command(
|
||||
Panel(
|
||||
f"[bold red]✖ Authentication Error:[/bold red] {err_msg}\n\n"
|
||||
"💡 [bold cyan]Quick Fix:[/bold cyan] Run [bold green]artemis init[/bold green] to configure your API key in 10 seconds,\n"
|
||||
"or add [bold]GEMINI_API_KEY=your_key[/bold] to [dim].env[/dim].",
|
||||
"or add [bold]OPENAI_API_KEY=your_key[/bold] to [dim].env[/dim].",
|
||||
title="Missing API Key",
|
||||
expand=False,
|
||||
)
|
||||
|
||||
@@ -1,57 +0,0 @@
|
||||
# Copyright 2026 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Google / Gemini specific knowledge shared by every call site.
|
||||
|
||||
Two concerns live here so they are defined exactly once:
|
||||
|
||||
* :mod:`artemis.llm.google.provider` - "is this endpoint Google?" and the
|
||||
Gemini model capability table (thinking levels, agentic video).
|
||||
* :mod:`artemis.llm.google.usage` - token-usage normalization across the
|
||||
LangChain, ``generate_content`` and Interactions API response shapes.
|
||||
|
||||
Both modules are dependency-free (no settings, no SDK client) so they can be
|
||||
imported from configuration code without creating import cycles.
|
||||
"""
|
||||
|
||||
from artemis.llm.google.provider import (
|
||||
VideoProcessing,
|
||||
gemini_version,
|
||||
is_agentic_video_auto_eligible,
|
||||
is_gemini_model,
|
||||
is_google_chat_model,
|
||||
is_google_family_provider,
|
||||
is_google_provider,
|
||||
resolve_video_processing,
|
||||
strip_provider_prefix,
|
||||
supports_agentic_video,
|
||||
supports_thinking_level,
|
||||
)
|
||||
from artemis.llm.google.usage import normalize_usage, usage_from_message
|
||||
|
||||
__all__ = [
|
||||
"VideoProcessing",
|
||||
"gemini_version",
|
||||
"is_agentic_video_auto_eligible",
|
||||
"is_gemini_model",
|
||||
"is_google_chat_model",
|
||||
"is_google_family_provider",
|
||||
"is_google_provider",
|
||||
"normalize_usage",
|
||||
"resolve_video_processing",
|
||||
"strip_provider_prefix",
|
||||
"supports_agentic_video",
|
||||
"supports_thinking_level",
|
||||
"usage_from_message",
|
||||
]
|
||||
@@ -1,185 +0,0 @@
|
||||
# Copyright 2026 Google LLC
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Google provider identity and the Gemini model capability table.
|
||||
|
||||
Provider identity has two tiers:
|
||||
|
||||
* :func:`is_google_provider` - the Gemini Developer API (``google`` /
|
||||
``gemini``). Native ``google-genai`` features (Files API, explicit caching,
|
||||
the Interactions API) are only reachable here.
|
||||
* :func:`is_google_family_provider` - additionally Vertex AI, which shares
|
||||
Google credentials but not the developer-API-only features.
|
||||
|
||||
Model IDs use ``gemini-<major>.<minor>-<tier>[<suffix>]``. Release suffixes
|
||||
(``preview``, ``latest``, ``exp``, and numeric tokens) retain base-family
|
||||
capabilities. Variants such as ``image`` and ``tts`` are excluded from agentic
|
||||
video support, but their version still determines thinking-level support.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Any, Literal, NamedTuple
|
||||
|
||||
VideoProcessing = Literal["auto", "agentic", "static"]
|
||||
ResolvedVideoProcessing = Literal["agentic", "static"]
|
||||
|
||||
_GOOGLE_API_PROVIDERS = frozenset({"google", "gemini"})
|
||||
_GOOGLE_FAMILY_PROVIDERS = _GOOGLE_API_PROVIDERS | frozenset({"vertexai", "vertex"})
|
||||
|
||||
_GEMINI_MODEL_RE = re.compile(r"gemini-(\d+)\.(\d+)-(flash-lite|flash|pro)", re.IGNORECASE)
|
||||
_RELEASE_SUFFIX_TOKENS = frozenset({"preview", "latest", "exp"})
|
||||
|
||||
|
||||
class _GeminiModel(NamedTuple):
|
||||
version: tuple[int, int]
|
||||
tier: str
|
||||
#: False for variant ids (``-image``, ``-tts``, ...) that carry their own
|
||||
#: capabilities and must not inherit the family's answers.
|
||||
is_base: bool
|
||||
|
||||
|
||||
# Agentic video understanding (Interactions API, ``processing: "agentic"``)
|
||||
# launched on Gemini 3.6/3.7/3.8 Flash and 3.5 Flash-Lite. Later releases in
|
||||
# the same families are assumed to keep it.
|
||||
_AGENTIC_VIDEO_MIN_FLASH = (3, 6)
|
||||
_AGENTIC_VIDEO_MIN_FLASH_LITE = (3, 5)
|
||||
|
||||
|
||||
def _normalize_provider(value: Any) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
value = getattr(value, "value", value)
|
||||
return str(value).lower().replace("_", "").replace("-", "").strip()
|
||||
|
||||
|
||||
def is_google_provider(value: Any) -> bool:
|
||||
"""True for the Gemini Developer API (``google`` / ``gemini``).
|
||||
|
||||
Accepts a provider name, a provider enum, or anything exposing ``.value``.
|
||||
"""
|
||||
return _normalize_provider(value) in _GOOGLE_API_PROVIDERS
|
||||
|
||||
|
||||
def is_google_family_provider(value: Any) -> bool:
|
||||
"""True for every provider that authenticates with Google credentials."""
|
||||
return _normalize_provider(value) in _GOOGLE_FAMILY_PROVIDERS
|
||||
|
||||
|
||||
def is_google_chat_model(model: Any) -> bool:
|
||||
"""True when a LangChain chat model is backed by Google (by class name).
|
||||
|
||||
Only used when no endpoint metadata is available; the class-name sniff is
|
||||
the historical fallback and is kept in one place here.
|
||||
"""
|
||||
cls = getattr(model, "__class__", None)
|
||||
name = cls.__name__ if cls else ""
|
||||
return "Google" in name or "Gemini" in name or "VertexAI" in name
|
||||
|
||||
|
||||
def strip_provider_prefix(model_name: Any) -> str:
|
||||
"""Drops a ``provider/`` prefix (``google/gemini-3.8-flash`` -> ``gemini-3.8-flash``)."""
|
||||
name = str(model_name or "")
|
||||
return name.rsplit("/", 1)[-1] if "/" in name else name
|
||||
|
||||
|
||||
def is_gemini_model(model_name: Any) -> bool:
|
||||
"""True when the model name refers to any Gemini model."""
|
||||
return "gemini" in strip_provider_prefix(model_name).lower()
|
||||
|
||||
|
||||
def _is_release_suffix(rest: str) -> bool:
|
||||
"""Whether what follows the tier is a release suffix (see the module docstring)."""
|
||||
if not rest:
|
||||
return True
|
||||
if not rest.startswith("-"):
|
||||
return False
|
||||
return all(token in _RELEASE_SUFFIX_TOKENS or token.isdigit() for token in rest[1:].split("-"))
|
||||
|
||||
|
||||
def _parse_gemini_model(model_name: Any) -> _GeminiModel | None:
|
||||
name = strip_provider_prefix(model_name).lower()
|
||||
match = _GEMINI_MODEL_RE.search(name)
|
||||
if not match:
|
||||
return None
|
||||
return _GeminiModel(
|
||||
version=(int(match.group(1)), int(match.group(2))),
|
||||
tier=match.group(3),
|
||||
is_base=_is_release_suffix(name[match.end() :]),
|
||||
)
|
||||
|
||||
|
||||
def gemini_version(model_name: Any) -> tuple[int, int] | None:
|
||||
"""``(major, minor)`` parsed from a ``gemini-X.Y-*`` name, else ``None``."""
|
||||
parsed = _parse_gemini_model(model_name)
|
||||
return parsed.version if parsed else None
|
||||
|
||||
|
||||
def _gemini_tier(model_name: Any) -> str | None:
|
||||
"""The tier of a *base* model id; ``None`` for variants and non-Gemini names."""
|
||||
parsed = _parse_gemini_model(model_name)
|
||||
return parsed.tier if parsed and parsed.is_base else None
|
||||
|
||||
|
||||
def supports_thinking_level(model_name: Any) -> bool:
|
||||
"""Whether the model accepts ``thinking_level`` (Gemini 3 and later).
|
||||
|
||||
Gemini 1.x/2.x only understand ``thinking_budget``. Names that do not
|
||||
carry a parsable version (e.g. the robotics ER models) are assumed to be
|
||||
current and therefore accept it.
|
||||
"""
|
||||
version = gemini_version(model_name)
|
||||
return version is None or version[0] >= 3
|
||||
|
||||
|
||||
def supports_agentic_video(model_name: Any) -> bool:
|
||||
"""Whether the Gemini API accepts ``processing: "agentic"`` for this model.
|
||||
|
||||
Only base Flash / Flash-Lite ids qualify; variants (``-image``, ``-tts``,
|
||||
``-live``, ...) do not inherit the family's support.
|
||||
"""
|
||||
parsed = _parse_gemini_model(model_name)
|
||||
if parsed is None or not parsed.is_base:
|
||||
return False
|
||||
if parsed.tier == "flash":
|
||||
return parsed.version >= _AGENTIC_VIDEO_MIN_FLASH
|
||||
if parsed.tier == "flash-lite":
|
||||
return parsed.version >= _AGENTIC_VIDEO_MIN_FLASH_LITE
|
||||
return False
|
||||
|
||||
|
||||
def is_agentic_video_auto_eligible(model_name: Any) -> bool:
|
||||
"""Models that get agentic video processing under ``processing: "auto"``.
|
||||
|
||||
Flash-Lite is deliberately excluded: it accepts the mode but mis-reads
|
||||
timestamps in practice, so it must be opted into explicitly.
|
||||
"""
|
||||
return supports_agentic_video(model_name) and _gemini_tier(model_name) == "flash"
|
||||
|
||||
|
||||
def resolve_video_processing(configured: Any, model_name: Any) -> ResolvedVideoProcessing:
|
||||
"""Turns the configured video-processing knob into a concrete mode.
|
||||
|
||||
``static`` is always honoured. ``agentic`` is honoured only for models
|
||||
that support it and silently degrades otherwise. ``auto`` (and any
|
||||
unrecognised value) enables agentic processing for the auto-eligible
|
||||
models and keeps static processing for everything else.
|
||||
"""
|
||||
mode = str(configured or "auto").strip().lower() if isinstance(configured, str) else "auto"
|
||||
if mode == "static":
|
||||
return "static"
|
||||
if mode == "agentic":
|
||||
return "agentic" if supports_agentic_video(model_name) else "static"
|
||||
return "agentic" if is_agentic_video_auto_eligible(model_name) else "static"
|
||||
+43
-323
@@ -12,12 +12,7 @@
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Dynamic Multi-Provider Model Router for ARTEMIS.
|
||||
|
||||
Provides role-based LLM/VLM dispatching, dynamic fallback chains,
|
||||
and unified provider configuration across Gemini, Vertex AI, OpenAI,
|
||||
Anthropic, OpenRouter, and local Ollama/vLLM endpoints.
|
||||
"""
|
||||
"""Single transport for all Artemis models: OpenAI Responses API."""
|
||||
|
||||
from enum import StrEnum
|
||||
import hashlib
|
||||
@@ -28,94 +23,34 @@ from langchain_core.language_models.chat_models import BaseChatModel
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from artemis.config.settings import settings
|
||||
from artemis.llm.google.provider import supports_thinking_level
|
||||
from artemis.utils.logger import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
|
||||
|
||||
class ModelProvider(StrEnum):
|
||||
GOOGLE = "google"
|
||||
GEMINI = "google"
|
||||
VERTEX_AI = "vertexai"
|
||||
OPENAI = "openai"
|
||||
ANTHROPIC = "anthropic"
|
||||
OPENROUTER = "openrouter"
|
||||
XAI = "xai"
|
||||
OLLAMA = "ollama"
|
||||
VLLM = "vllm"
|
||||
CUSTOM = "custom"
|
||||
|
||||
@classmethod
|
||||
def from_string(cls, val: Any) -> "ModelProvider":
|
||||
"""Converts a string or enum into a canonical ModelProvider.
|
||||
|
||||
``None``/empty means "use the default" (Google); an unrecognized
|
||||
non-empty value raises instead of silently routing to Gemini with
|
||||
the wrong credentials.
|
||||
"""
|
||||
if isinstance(val, cls):
|
||||
return val
|
||||
if val is None:
|
||||
return cls.GOOGLE
|
||||
s = str(val).lower().replace("_", "").replace("-", "").strip()
|
||||
if not s:
|
||||
return cls.GOOGLE
|
||||
mapping = {
|
||||
"google": cls.GOOGLE,
|
||||
"gemini": cls.GOOGLE,
|
||||
"vertexai": cls.VERTEX_AI,
|
||||
"vertex": cls.VERTEX_AI,
|
||||
"openai": cls.OPENAI,
|
||||
"anthropic": cls.ANTHROPIC,
|
||||
"claude": cls.ANTHROPIC,
|
||||
"openrouter": cls.OPENROUTER,
|
||||
"xai": cls.XAI,
|
||||
"grok": cls.XAI,
|
||||
"ollama": cls.OLLAMA,
|
||||
"vllm": cls.VLLM,
|
||||
"custom": cls.CUSTOM,
|
||||
}
|
||||
provider = mapping.get(s)
|
||||
if provider is None:
|
||||
raise ValueError(
|
||||
f"Unknown LLM provider {val!r}. Valid providers: {sorted(set(mapping))}"
|
||||
)
|
||||
return provider
|
||||
if val is None or str(val).strip().lower() in ("", "openai"):
|
||||
return cls.OPENAI
|
||||
raise ValueError(f"Unsupported LLM provider {val!r}; only OpenAI Responses is supported")
|
||||
|
||||
|
||||
class ModelEndpoint(BaseModel):
|
||||
"""Configuration definition for an LLM/VLM model endpoint."""
|
||||
|
||||
provider: ModelProvider = Field(default=ModelProvider.GOOGLE, description="Model provider")
|
||||
model_name: str = Field(default="gemini-2.5-flash", description="Model name identifier")
|
||||
api_key: str | None = Field(default=None, description="API Key or secret")
|
||||
api_base: str | None = Field(default=None, description="Custom API endpoint base URL")
|
||||
temperature: float = Field(default=0.0, description="Sampling temperature")
|
||||
max_tokens: int | None = Field(default=None, description="Maximum completion tokens")
|
||||
timeout_seconds: float = Field(default=60.0, description="Request timeout in seconds")
|
||||
is_multimodal: bool = Field(
|
||||
default=True, description="Whether endpoint accepts image/video inputs"
|
||||
)
|
||||
reasoning_effort: str | None = Field(
|
||||
default=None, description="Reasoning effort: 'low', 'medium', 'high'"
|
||||
)
|
||||
thinking_budget: int | None = Field(default=None, description="Thinking budget token count")
|
||||
thinking_level: str | None = Field(
|
||||
default=None, description="Thinking level: 'minimal', 'low', 'medium', 'high'"
|
||||
)
|
||||
include_thoughts: bool | None = Field(
|
||||
default=None, description="Whether to include thought traces in response"
|
||||
)
|
||||
enable_grounding: bool = Field(
|
||||
default=False,
|
||||
description="Whether to enable Google Search grounding for Gemini endpoints",
|
||||
)
|
||||
provider: ModelProvider = ModelProvider.OPENAI
|
||||
model_name: str = Field(default_factory=lambda: settings.OPENAI_MODEL)
|
||||
api_key: str | None = Field(default=None, repr=False)
|
||||
api_base: str | None = None
|
||||
temperature: float | None = None
|
||||
max_tokens: int | None = Field(default=None, gt=0)
|
||||
timeout_seconds: float = Field(default_factory=lambda: settings.OPENAI_TIMEOUT_SECONDS, gt=0)
|
||||
max_retries: int = Field(default_factory=lambda: settings.OPENAI_MAX_RETRIES, ge=0)
|
||||
is_multimodal: bool = True
|
||||
reasoning_effort: str | None = None
|
||||
enable_grounding: bool = False
|
||||
|
||||
def cache_key(self) -> tuple:
|
||||
"""Key the client cache by endpoint settings, hashing the API key."""
|
||||
api_key_digest = (
|
||||
hashlib.sha256(self.api_key.encode("utf-8")).hexdigest()[:16] if self.api_key else None
|
||||
key = self.api_key or (
|
||||
settings.OPENAI_API_KEY.get_secret_value() if settings.OPENAI_API_KEY else ""
|
||||
)
|
||||
return (
|
||||
self.provider.value,
|
||||
@@ -123,81 +58,15 @@ class ModelEndpoint(BaseModel):
|
||||
self.temperature,
|
||||
self.max_tokens,
|
||||
self.timeout_seconds,
|
||||
self.thinking_budget,
|
||||
self.thinking_level,
|
||||
self.include_thoughts,
|
||||
self.max_retries,
|
||||
self.reasoning_effort,
|
||||
self.enable_grounding,
|
||||
self.api_base,
|
||||
api_key_digest,
|
||||
self.api_base or settings.OPENAI_BASE_URL,
|
||||
hashlib.sha256(key.encode()).hexdigest(),
|
||||
)
|
||||
|
||||
|
||||
def _patch_langchain_google_genai():
|
||||
"""Patches ChatGoogleGenerativeAI._process_tool_config to preserve and ensure include_server_side_tool_invocations."""
|
||||
try:
|
||||
from langchain_google_genai.chat_models import ChatGoogleGenerativeAI
|
||||
from google.genai.types import ToolConfig
|
||||
|
||||
if getattr(ChatGoogleGenerativeAI, "_is_artemis_patched", False):
|
||||
return
|
||||
|
||||
original_process = ChatGoogleGenerativeAI._process_tool_config
|
||||
|
||||
def patched_process_tool_config(self, tool_choice, tool_config, formatted_tools):
|
||||
config = original_process(self, tool_choice, tool_config, formatted_tools)
|
||||
|
||||
has_builtin = False
|
||||
has_functions = False
|
||||
if formatted_tools:
|
||||
for tool in formatted_tools:
|
||||
tool_dict = tool if isinstance(tool, dict) else getattr(tool, "__dict__", {})
|
||||
if any(
|
||||
tool_dict.get(k) is not None
|
||||
for k in (
|
||||
"google_search",
|
||||
"code_execution",
|
||||
"google_maps",
|
||||
"google_search_retrieval",
|
||||
)
|
||||
):
|
||||
has_builtin = True
|
||||
if tool_dict.get("function_declarations"):
|
||||
has_functions = True
|
||||
|
||||
if tool_config:
|
||||
normalized = (
|
||||
ToolConfig.model_validate(tool_config)
|
||||
if isinstance(tool_config, dict)
|
||||
else tool_config
|
||||
)
|
||||
if getattr(normalized, "include_server_side_tool_invocations", None) is not None:
|
||||
if config is None:
|
||||
config = ToolConfig(
|
||||
include_server_side_tool_invocations=normalized.include_server_side_tool_invocations
|
||||
)
|
||||
else:
|
||||
config.include_server_side_tool_invocations = (
|
||||
normalized.include_server_side_tool_invocations
|
||||
)
|
||||
|
||||
if has_builtin and has_functions:
|
||||
if config is None:
|
||||
config = ToolConfig(include_server_side_tool_invocations=True)
|
||||
else:
|
||||
config.include_server_side_tool_invocations = True
|
||||
|
||||
return config
|
||||
|
||||
ChatGoogleGenerativeAI._process_tool_config = patched_process_tool_config
|
||||
ChatGoogleGenerativeAI._is_artemis_patched = True
|
||||
except Exception as e:
|
||||
logger.warning(f"Could not patch ChatGoogleGenerativeAI._process_tool_config: {e}")
|
||||
|
||||
|
||||
class ModelFactory:
|
||||
"""Unified factory for instantiating and caching LangChain chat models across providers."""
|
||||
|
||||
_cache: dict[tuple, BaseChatModel] = {}
|
||||
|
||||
@classmethod
|
||||
@@ -209,181 +78,32 @@ class ModelFactory:
|
||||
|
||||
@classmethod
|
||||
def create_model(cls, endpoint: ModelEndpoint) -> BaseChatModel:
|
||||
ModelProvider.from_string(endpoint.provider)
|
||||
if os.environ.get("ARTEMIS_FAKE_LLM") == "1":
|
||||
from artemis.llm.fake_model import FakeChatModel
|
||||
|
||||
delay = float(os.environ.get("ARTEMIS_FAKE_LLM_DELAY_S", "0") or 0)
|
||||
logger.warning(
|
||||
f"ARTEMIS_FAKE_LLM=1 — returning FakeChatModel (delay={delay}s) "
|
||||
f"instead of {endpoint.provider}/{endpoint.model_name}"
|
||||
)
|
||||
return FakeChatModel(delay_s=delay)
|
||||
|
||||
provider = ModelProvider.from_string(endpoint.provider)
|
||||
|
||||
if provider == ModelProvider.GOOGLE:
|
||||
_patch_langchain_google_genai()
|
||||
from langchain_google_genai import (
|
||||
ChatGoogleGenerativeAI,
|
||||
HarmBlockThreshold,
|
||||
HarmCategory,
|
||||
return FakeChatModel(
|
||||
delay_s=float(os.environ.get("ARTEMIS_FAKE_LLM_DELAY_S", "0") or 0)
|
||||
)
|
||||
|
||||
api_key = (
|
||||
endpoint.api_key
|
||||
or (settings.GOOGLE_API_KEY.get_secret_value() if settings.GOOGLE_API_KEY else None)
|
||||
or os.environ.get("GOOGLE_API_KEY")
|
||||
or os.environ.get("GEMINI_API_KEY")
|
||||
)
|
||||
# Gemini 1.x/2.x only understand thinking_budget.
|
||||
thinking_level = (
|
||||
endpoint.thinking_level if supports_thinking_level(endpoint.model_name) else None
|
||||
)
|
||||
from langchain_openai import ChatOpenAI
|
||||
|
||||
kwargs: dict[str, Any] = {
|
||||
"model": endpoint.model_name,
|
||||
"temperature": endpoint.temperature,
|
||||
"max_output_tokens": endpoint.max_tokens,
|
||||
"api_key": api_key,
|
||||
"timeout": endpoint.timeout_seconds,
|
||||
"thinking_budget": endpoint.thinking_budget,
|
||||
"thinking_level": thinking_level,
|
||||
"include_thoughts": endpoint.include_thoughts,
|
||||
"safety_settings": {
|
||||
HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_NONE,
|
||||
HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_NONE,
|
||||
HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
|
||||
HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,
|
||||
},
|
||||
}
|
||||
return ChatGoogleGenerativeAI(**{k: v for k, v in kwargs.items() if v is not None})
|
||||
|
||||
elif provider == ModelProvider.VERTEX_AI:
|
||||
from langchain_google_vertexai import (
|
||||
ChatVertexAI,
|
||||
HarmBlockThreshold,
|
||||
HarmCategory,
|
||||
)
|
||||
|
||||
kwargs = {
|
||||
"model_name": endpoint.model_name,
|
||||
"temperature": endpoint.temperature,
|
||||
"max_output_tokens": endpoint.max_tokens,
|
||||
"timeout": endpoint.timeout_seconds,
|
||||
"thinking_budget": endpoint.thinking_budget,
|
||||
"safety_settings": {
|
||||
HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_NONE,
|
||||
HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_NONE,
|
||||
HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
|
||||
HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,
|
||||
},
|
||||
}
|
||||
return ChatVertexAI(**{k: v for k, v in kwargs.items() if v is not None})
|
||||
|
||||
elif provider == ModelProvider.OPENAI:
|
||||
from langchain_openai import ChatOpenAI
|
||||
|
||||
api_key = (
|
||||
endpoint.api_key
|
||||
or (settings.OPENAI_API_KEY.get_secret_value() if settings.OPENAI_API_KEY else None)
|
||||
or os.environ.get("OPENAI_API_KEY", "EMPTY")
|
||||
)
|
||||
base_url = endpoint.api_base or (
|
||||
str(settings.OPENAI_BASE_URL) if settings.OPENAI_BASE_URL else None
|
||||
)
|
||||
kwargs = {
|
||||
"model": endpoint.model_name,
|
||||
"temperature": endpoint.temperature,
|
||||
"max_tokens": endpoint.max_tokens,
|
||||
"api_key": api_key,
|
||||
"base_url": base_url,
|
||||
"timeout": endpoint.timeout_seconds,
|
||||
}
|
||||
if endpoint.reasoning_effort:
|
||||
kwargs["reasoning_effort"] = endpoint.reasoning_effort
|
||||
return ChatOpenAI(**{k: v for k, v in kwargs.items() if v is not None})
|
||||
|
||||
elif provider == ModelProvider.ANTHROPIC:
|
||||
from langchain_anthropic import ChatAnthropic
|
||||
|
||||
api_key = (
|
||||
endpoint.api_key
|
||||
or (
|
||||
settings.ANTHROPIC_API_KEY.get_secret_value()
|
||||
if settings.ANTHROPIC_API_KEY
|
||||
else None
|
||||
)
|
||||
or os.environ.get("ANTHROPIC_API_KEY")
|
||||
)
|
||||
kwargs = {
|
||||
"model": endpoint.model_name,
|
||||
"temperature": endpoint.temperature,
|
||||
"api_key": api_key,
|
||||
"timeout": endpoint.timeout_seconds,
|
||||
}
|
||||
budget = endpoint.thinking_budget
|
||||
if not budget and endpoint.reasoning_effort:
|
||||
effort_map = {"low": 2048, "medium": 8192, "high": 32768}
|
||||
budget = effort_map.get(endpoint.reasoning_effort.lower())
|
||||
if budget:
|
||||
kwargs["thinking"] = {"type": "enabled", "budget_tokens": budget}
|
||||
kwargs["temperature"] = 1.0
|
||||
return ChatAnthropic(**{k: v for k, v in kwargs.items() if v is not None})
|
||||
|
||||
elif provider == ModelProvider.OPENROUTER:
|
||||
from langchain_openai import ChatOpenAI
|
||||
|
||||
api_key = (
|
||||
endpoint.api_key
|
||||
or (
|
||||
settings.OPEN_ROUTER_API_KEY.get_secret_value()
|
||||
if settings.OPEN_ROUTER_API_KEY
|
||||
else None
|
||||
)
|
||||
or os.environ.get("OPEN_ROUTER_API_KEY")
|
||||
)
|
||||
return ChatOpenAI(
|
||||
model=endpoint.model_name,
|
||||
temperature=endpoint.temperature,
|
||||
api_key=api_key,
|
||||
base_url=endpoint.api_base or "https://openrouter.ai/api/v1",
|
||||
timeout=endpoint.timeout_seconds,
|
||||
)
|
||||
|
||||
elif provider == ModelProvider.XAI:
|
||||
from langchain_openai import ChatOpenAI
|
||||
|
||||
api_key = (
|
||||
endpoint.api_key
|
||||
or (settings.XAI_API_KEY.get_secret_value() if settings.XAI_API_KEY else None)
|
||||
or os.environ.get("XAI_API_KEY")
|
||||
)
|
||||
return ChatOpenAI(
|
||||
model=endpoint.model_name,
|
||||
temperature=endpoint.temperature,
|
||||
api_key=api_key,
|
||||
base_url=endpoint.api_base or "https://api.x.ai/v1",
|
||||
timeout=endpoint.timeout_seconds,
|
||||
)
|
||||
|
||||
elif provider in (ModelProvider.OLLAMA, ModelProvider.VLLM, ModelProvider.CUSTOM):
|
||||
from langchain_openai import ChatOpenAI
|
||||
|
||||
api_key = endpoint.api_key or os.environ.get("OPENAI_API_KEY", "EMPTY")
|
||||
base_url = endpoint.api_base or os.environ.get(
|
||||
"OPENAI_BASE_URL", "http://localhost:8000/v1"
|
||||
)
|
||||
kwargs = {
|
||||
"model": endpoint.model_name,
|
||||
"temperature": endpoint.temperature,
|
||||
"max_tokens": endpoint.max_tokens,
|
||||
"api_key": api_key,
|
||||
"base_url": base_url,
|
||||
"timeout": endpoint.timeout_seconds,
|
||||
}
|
||||
if endpoint.reasoning_effort:
|
||||
kwargs["reasoning_effort"] = endpoint.reasoning_effort
|
||||
return ChatOpenAI(**{k: v for k, v in kwargs.items() if v is not None})
|
||||
|
||||
else:
|
||||
raise ValueError(f"Unsupported model provider: {provider}")
|
||||
api_key = endpoint.api_key or (
|
||||
settings.OPENAI_API_KEY.get_secret_value() if settings.OPENAI_API_KEY else None
|
||||
)
|
||||
if not api_key:
|
||||
raise ValueError("请在环境变量或 .env 中配置 OPENAI_API_KEY")
|
||||
kwargs = {
|
||||
"model": endpoint.model_name,
|
||||
"api_key": api_key,
|
||||
"base_url": endpoint.api_base or settings.OPENAI_BASE_URL or None,
|
||||
"timeout": endpoint.timeout_seconds,
|
||||
"max_retries": endpoint.max_retries,
|
||||
"temperature": endpoint.temperature,
|
||||
"max_tokens": endpoint.max_tokens,
|
||||
"reasoning_effort": endpoint.reasoning_effort,
|
||||
"use_responses_api": True,
|
||||
"output_version": "responses/v1",
|
||||
"store": False,
|
||||
}
|
||||
return ChatOpenAI(**{k: v for k, v in kwargs.items() if v is not None})
|
||||
File renamed without changes.
@@ -35,6 +35,7 @@ context is rebuilt only when a compression event replaces or folds turns.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
from artemis.config.settings import settings
|
||||
from dataclasses import dataclass, field
|
||||
import json
|
||||
from pathlib import Path
|
||||
@@ -45,7 +46,6 @@ from uuid import uuid4
|
||||
|
||||
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
|
||||
|
||||
from artemis.llm.google import is_google_provider
|
||||
from artemis.memory.step_memory import JobKey, StepLens, StepMemoryService
|
||||
from artemis.memory.transcript import format_session_offset
|
||||
from artemis.utils.logger import get_logger
|
||||
@@ -298,7 +298,7 @@ class StepCapsuleLens(StepLens):
|
||||
fallback_model_name: str | None = None,
|
||||
fallback_llm: Any | None = None,
|
||||
):
|
||||
self._model_name = model_name or "gemini-3.8-flash"
|
||||
self._model_name = model_name or settings.OPENAI_MODEL
|
||||
self._llm = llm
|
||||
self._ctx = ctx
|
||||
# Availability hardening: `chunking.model` is a dedicated model with no
|
||||
@@ -323,16 +323,16 @@ class StepCapsuleLens(StepLens):
|
||||
|
||||
def _get_llm(self):
|
||||
if self._llm is None:
|
||||
from artemis.services.llm import get_google_llm
|
||||
from artemis.services.llm import get_openai_llm
|
||||
|
||||
self._llm = get_google_llm(model_name=self._model_name, temperature=0.0)
|
||||
self._llm = get_openai_llm(model_name=self._model_name, temperature=0.0)
|
||||
return self._llm
|
||||
|
||||
def _get_fallback_llm(self):
|
||||
if self._fallback_llm is None and self._fallback_model_name:
|
||||
from artemis.services.llm import get_google_llm
|
||||
from artemis.services.llm import get_openai_llm
|
||||
|
||||
self._fallback_llm = get_google_llm(
|
||||
self._fallback_llm = get_openai_llm(
|
||||
model_name=self._fallback_model_name, temperature=0.0
|
||||
)
|
||||
return self._fallback_llm
|
||||
@@ -891,7 +891,7 @@ class HistoryChunkManager:
|
||||
self._max_steps = int(getattr(cc, "max_steps", 12) or 12)
|
||||
self._min_steps = max(1, min(self._max_steps, int(getattr(cc, "min_steps", 3) or 3)))
|
||||
self._target_source_tokens = int(getattr(cc, "target_source_tokens", 2000) or 2000)
|
||||
self._model_name = getattr(cc, "model", None) or "gemini-3.8-flash"
|
||||
self._model_name = getattr(cc, "model", None) or settings.OPENAI_MODEL
|
||||
self._max_chunks = int(getattr(cc, "max_chunks", 8) or 8)
|
||||
# None uses max_chunks as the era cap.
|
||||
self._max_eras = int(getattr(cc, "max_eras", None) or self._max_chunks)
|
||||
@@ -965,7 +965,7 @@ class HistoryChunkManager:
|
||||
fallback = getattr(getattr(llm_cfg, "summarizer", None), "fallback", None)
|
||||
provider = str(getattr(fallback, "provider", "") or "")
|
||||
model = getattr(fallback, "model", None)
|
||||
if model and is_google_provider(provider) and model != self._model_name:
|
||||
if model and str(provider).lower() == "openai" and model != self._model_name:
|
||||
return str(model)
|
||||
except Exception as exc:
|
||||
logger.debug(f"Capsule fallback model resolution skipped: {exc}", exc_info=True)
|
||||
|
||||
@@ -1,299 +1,122 @@
|
||||
// 模型仅支持 OpenAI Responses;端点和密钥从 .env 读取。
|
||||
{
|
||||
// ==============================================================================
|
||||
// ☕ ARTEMIS Autonomous Agent & Framework Configuration
|
||||
// ==============================================================================
|
||||
|
||||
// ------------------------------------------------------------------------------
|
||||
// 1. Global Default Model (Applied to all sub-agents unless overridden below)
|
||||
// ------------------------------------------------------------------------------
|
||||
"default": {
|
||||
"provider": "google",
|
||||
"model": "gemini-3.8-flash",
|
||||
"thinking_level": "medium",
|
||||
"provider": "openai",
|
||||
"model": "${env:OPENAI_MODEL:-gpt-5-mini}",
|
||||
"timeout": "${env:OPENAI_TIMEOUT_SECONDS:-120}",
|
||||
"max_retries": "${env:OPENAI_MAX_RETRIES:-0}",
|
||||
"fallback": {
|
||||
"provider": "google",
|
||||
"model": "gemini-3.7-flash",
|
||||
"thinking_level": "medium"
|
||||
"provider": "openai",
|
||||
"model": "${env:OPENAI_MODEL:-gpt-5-mini}"
|
||||
}
|
||||
},
|
||||
|
||||
// ------------------------------------------------------------------------------
|
||||
// 2. Ready-to-Use Presets (Switch easily in code or CLI)
|
||||
// ------------------------------------------------------------------------------
|
||||
"presets": {
|
||||
"gemini-flagship": {
|
||||
"provider": "google",
|
||||
"model": "gemini-3.8-flash",
|
||||
"fallback": { "provider": "google", "model": "gemini-3.7-flash" }
|
||||
},
|
||||
"gemini-flash": {
|
||||
"provider": "google",
|
||||
"model": "gemini-2.5-flash",
|
||||
"fallback": { "provider": "google", "model": "gemini-2.0-flash" }
|
||||
},
|
||||
"openai-gpt4o": {
|
||||
"provider": "openai",
|
||||
"model": "gpt-4o",
|
||||
"fallback": { "provider": "openai", "model": "gpt-4o-mini" }
|
||||
},
|
||||
"cost-saving": {
|
||||
"provider": "google",
|
||||
"model": "gemini-3.5-flash-lite",
|
||||
"fallback": { "provider": "google", "model": "gemini-3.1-flash-lite" }
|
||||
},
|
||||
"local-ollama": {
|
||||
"provider": "openai",
|
||||
"model": "llama3.2-vision",
|
||||
"fallback": { "provider": "openai", "model": "llama3.2-vision" }
|
||||
}
|
||||
},
|
||||
|
||||
// ------------------------------------------------------------------------------
|
||||
// 3. Agent & Sub-Agent Node Customizations (Optional - Inherits from 'default')
|
||||
// ------------------------------------------------------------------------------
|
||||
"presets": {},
|
||||
"nodes": {
|
||||
// 🧠 Planner: High-level goal decomposition and strategic step planning
|
||||
"planner": {
|
||||
"thinking_level": "high"
|
||||
},
|
||||
|
||||
// 🎯 Operator: Screen perception, element interaction, and direct execution
|
||||
"operator": {
|
||||
"thinking_level": "medium",
|
||||
"include_thoughts": true
|
||||
},
|
||||
|
||||
// 🎬 Video Analyzer: Multimodal screen recording verification and temporal analysis
|
||||
// NOTE: Dual-path engine automatically routes:
|
||||
// - Native Gemini Files API for Gemini models with Google API keys (long video file stream);
|
||||
// Gemini 3.6+ Flash sub-agents use agentic video understanding by default (see video_analyzer.processing below)
|
||||
// - Universal Multi-Model Engine for Claude 3.5/3.7, GPT-4o, OpenRouter, Vertex AI (dynamic keyframe extraction + multimodal tool calling)
|
||||
"planner": {},
|
||||
"operator": {},
|
||||
"video_analyzer": {},
|
||||
|
||||
// 👁️ Object Detector & Explorer (Flash Mode): Visual coordinates grounding and bounding boxes
|
||||
// CRITICAL REQUIREMENT: Standard LLMs (including standard Gemini Flash, GPT-4o, Claude 3.7) lack sub-pixel spatial coordinate fine-tuning.
|
||||
// To achieve accurate [x, y] point and bounding box localization, this node MUST use specialized Gemini ER (Embodied Reasoning / Robotics) models:
|
||||
// e.g., "gemini-robotics-er-2-preview". Non-ER models will fail spatial coordinate detection.
|
||||
"object_detector": {
|
||||
"model": "gemini-robotics-er-2-preview",
|
||||
"fallback": {
|
||||
"provider": "google",
|
||||
"model": "gemini-robotics-er-2-preview"
|
||||
}
|
||||
},
|
||||
|
||||
// 🔍 Explorer: Deep element grounding and multi-candidate visual search
|
||||
// NOTE:
|
||||
// - "flash" version delegates directly to object_detector (requires Gemini ER model above).
|
||||
// - "pro" / "ultra" versions run multi-turn ReAct loops (supports both Native Gemini and Universal Multi-Model LangChain engines).
|
||||
"explorer": {
|
||||
"thinking_level": "medium"
|
||||
},
|
||||
|
||||
// 📋 Summarizer: Aggregates step history and generates structured outcome
|
||||
"object_detector": {},
|
||||
"explorer": {},
|
||||
"summarizer": {},
|
||||
|
||||
// 🔽 Hopper: Fast lightweight package name locator
|
||||
"hopper": {
|
||||
"model": "gemini-3.5-flash-lite",
|
||||
"fallback": {
|
||||
"provider": "google",
|
||||
"model": "gemini-3.1-flash-lite"
|
||||
}
|
||||
},
|
||||
|
||||
// 📝 Outputter: Synthesizes execution trace into final output report
|
||||
"hopper": {},
|
||||
"outputter": {},
|
||||
|
||||
// 🪵 Log Analyzer: Error log diagnosis with search grounding
|
||||
"log_analyzer": {
|
||||
"enable_grounding": true
|
||||
},
|
||||
|
||||
// 🩺 Diagnoser Expert: Failure root cause diagnosis with search grounding
|
||||
"diagnoser_expert": {
|
||||
"enable_grounding": true
|
||||
},
|
||||
// 🏛️ Committee Council Members
|
||||
"log_analyzer": {},
|
||||
"diagnoser_expert": {},
|
||||
"planner_avatar": {},
|
||||
"history_analyzer_expert": {},
|
||||
// 🔎 Checker: Plan-declared checkpoint verification and exit final review.
|
||||
// Thoughts are streamed so the verifier's reasoning shows live in the timeline.
|
||||
"checker": {
|
||||
"thinking_level": "medium",
|
||||
"include_thoughts": true
|
||||
}
|
||||
// 🎯 Planner Validation: Advisory review of milestone changes.
|
||||
// Defaults to the same lightweight model as the pixel safety net
|
||||
// (gemini-3.5-flash-lite, temperature 0); add "planner_validation": {...} here to override.
|
||||
"checker": {}
|
||||
},
|
||||
|
||||
// ------------------------------------------------------------------------------
|
||||
// 4. Runtime Mode & Agent Capabilities Configuration (Dual-Engine: Flash vs. Pro)
|
||||
// ------------------------------------------------------------------------------
|
||||
"agent": {
|
||||
// ==========================================================================
|
||||
// ⚡ 4.1 FLASH Profile Options (Reactive Single-Agent Lightweight Loop)
|
||||
// ==========================================================================
|
||||
// Best for: Simple, fast, deterministic tasks; runs unbounded by default.
|
||||
// Execution: Observe-Think-Act single LLM reactive loop without graph overhead.
|
||||
"flash": {
|
||||
"max_turns": 0, // Maximum reactive turns before stopping (0 = unlimited; history is bounded by transcript compression, not by a turn cap)
|
||||
"explorer_mode": "flash", // Explorer tier for the Flash runner: 'flash' | 'pro' | 'ultra' (see top-level "explorer")
|
||||
// 🧠 Step Summarizer: Asynchronous visual context compressor
|
||||
// Summarizes intermediate steps in background without blocking the runner,
|
||||
// replacing images only after strictly objective, high-density summaries are ready.
|
||||
"max_turns": 0,
|
||||
"explorer_mode": "flash",
|
||||
"step_summarizer": {
|
||||
"enabled": true, // Whether background step summarization is active
|
||||
"model": "gemini-3.5-flash-lite", // Lightweight model for fast background inference
|
||||
"prune_history_xml": true // Prune outdated heavy XML trees from historical steps
|
||||
"enabled": "${env:ARTEMIS_STEP_SUMMARIZER_ENABLED:-true}",
|
||||
"model": "${env:OPENAI_MODEL:-gpt-5-mini}",
|
||||
"prune_history_xml": true
|
||||
}
|
||||
},
|
||||
// ==========================================================================
|
||||
// 🚀 4.2 PRO Profile Options (Deep Multi-Agent Graph Closed-Loop)
|
||||
// ==========================================================================
|
||||
// Best for: Complex, exploratory, dynamic multi-branch tasks.
|
||||
// Execution: LangGraph multi-node graph with a living task plan, a pre-execution safety net (single actions) / fast-action bursts (multi-action turns),
|
||||
// execution-incident recovery handled by the Operator itself (no separate repair agent), and a read-only Checker for checkpoint verification + exit final review.
|
||||
"pro": {
|
||||
// 🧭 Explorer: Perception sub-agent under Pro profile ('flash', 'pro', or 'ultra')
|
||||
"explorer": {
|
||||
"mode": "flash" // Explorer tier for Pro nodes (Operator/Validator): 'flash' | 'pro' | 'ultra'
|
||||
"mode": "flash"
|
||||
},
|
||||
// 🎯 Planner Validation: Async verification when task plan milestones change
|
||||
"planner_validation": {
|
||||
"enabled": true // On by default: every milestone text change is reviewed (advisory: hint + reason only, never rolled back)
|
||||
"enabled": true
|
||||
},
|
||||
// 🏛️ Committee: Multi-Agent Council debate tool for Operator
|
||||
"committee": {
|
||||
"enabled": false,
|
||||
"debate_rounds": 2
|
||||
},
|
||||
// 🔎 Checker: Plan-declared checkpoint verification (optimistic async) + exit final review
|
||||
"checker": {
|
||||
"enabled": true, // Master switch (compat alias): false disables both gates below
|
||||
"midway_checks": false, // Plan-declared midway checkpoints (off in the factory layering)
|
||||
"final_check": true, // Exit final review against the user's original goal (on by default)
|
||||
"enabled": true,
|
||||
"midway_checks": false,
|
||||
"final_check": true,
|
||||
"max_iterations": 20,
|
||||
"final_check_max_attempts": 3,
|
||||
"checkpoint_max_repairs": 2,
|
||||
"max_concurrent_checkpoints": 3,
|
||||
"checkpoint_timeout": 180.0,
|
||||
"settlement_timeout": 120.0,
|
||||
"assert_failure_policy": "continue", // 'continue' or 'halt' on assert failures
|
||||
"device_probes": true // Checker's enumerated read-only device probes
|
||||
"checkpoint_timeout": "${env:ARTEMIS_CHECKPOINT_TIMEOUT_SECONDS:-180}",
|
||||
"settlement_timeout": "${env:ARTEMIS_SETTLEMENT_TIMEOUT_SECONDS:-120}",
|
||||
"assert_failure_policy": "continue",
|
||||
"device_probes": true
|
||||
},
|
||||
// 🎬 Video Analyzer & Screen Recording: Continuous screen recording via scrcpy and temporal analysis
|
||||
"video_analyzer": {
|
||||
"enabled": true, // Whether continuous video recording and video tools are active (default: auto-detected)
|
||||
"enabled": "${env:ARTEMIS_VIDEO_ENABLED:-true}",
|
||||
"enable_ledger": true
|
||||
}
|
||||
},
|
||||
// ==========================================================================
|
||||
// 🌐 4.3 Shared Capabilities & Global Overrides
|
||||
// ==========================================================================
|
||||
// Shared step memory and history compression
|
||||
// Transcript ledger + layered compression for the Pro operator.
|
||||
"memory": {
|
||||
// Background summarization runtime (visual/capsule lenses)
|
||||
"runtime": {
|
||||
"max_concurrency": 2, // 1→2 per 2026-09-01 baseline: serial queue pushed summary-ready P90 past the scrub window
|
||||
"max_concurrency": 2,
|
||||
"retry_limit": 3,
|
||||
"flush_timeout_s": 30
|
||||
},
|
||||
"transcript": {
|
||||
// ROLLBACK SWITCH: set false to restore the legacy per-turn 2-message
|
||||
// prompt path byte-for-byte (L2/L3 chunk compression rides this same
|
||||
// flag). See services/token_meter.py for measured session token usage
|
||||
// and cache-hit ratios.
|
||||
"enabled": true,
|
||||
"context_budget_tokens": 80000,
|
||||
// Below start_ratio of the operator's own measured prompt size, closed
|
||||
// segments only prepare their summary; the original turns are kept in
|
||||
// context and replaced once occupancy reaches this ratio. Measurements
|
||||
// come from the operator; other agents' calls do not affect this gate.
|
||||
"start_ratio": 0.35,
|
||||
"soft_ratio": 0.7,
|
||||
"hard_ratio": 0.9,
|
||||
"min_active_steps": 5,
|
||||
// Screenshot edge: 6 while the operator's measured context is below
|
||||
// start_ratio (nothing is being replaced yet), 3 from there up. A
|
||||
// screenshot reaching the edge is replaced in place by its visual
|
||||
// summary (the 'Current Screenshot' label leaves with it) and never
|
||||
// restored; the depth only decides how long new ones stay.
|
||||
"image_scrub_depth": 3,
|
||||
"image_scrub_depth_relaxed": 6,
|
||||
"pending_grace_steps": 3,
|
||||
// Text edge: once a newer observation exists (depth 1) the UI element
|
||||
// list, the plan recitation and the per-turn ephemeral blocks
|
||||
// (reminders, hints, guidance wrappers) leave the older observation.
|
||||
// User instruction bodies remain in history. The text edge stays
|
||||
// shallow on purpose: only the live observation carries an indexed
|
||||
// element list, so a stale [n] index can never be picked up as a
|
||||
// target.
|
||||
"xml_scrub_depth": 1,
|
||||
// Local historical-state hint (no model call); distance calibrated
|
||||
// 2026-09-01 from 460 dHash-stamped on-device steps (same screen <=4,
|
||||
// different screens >=7).
|
||||
"similarity_hint": true,
|
||||
"similarity_max_distance": 5
|
||||
},
|
||||
// Segment (L2) chunk compression: three-band chunks, era folding
|
||||
"chunking": {
|
||||
"max_steps": 12,
|
||||
// Size/pressure closes wait until at least this many turns are
|
||||
// eligible, so heavy turns do not become one-step chunks (one capsule
|
||||
// call each); milestone closes are exempt, the hard line waives it.
|
||||
"min_steps": 3,
|
||||
"target_source_tokens": 2000,
|
||||
"model": "gemini-3.8-flash",
|
||||
"model": "${env:OPENAI_MODEL:-gpt-5-mini}",
|
||||
"max_chunks": 8
|
||||
// "max_eras": 8 // independent era cap; omitted = follows max_chunks
|
||||
},
|
||||
// search_history: deterministic keyword / step-range lookup over stored history
|
||||
"recall": {
|
||||
"enabled": true,
|
||||
"max_results": 5,
|
||||
"max_text_tokens": 2000,
|
||||
"screen_scan_steps": 150
|
||||
},
|
||||
// replay_steps: full step replays (whole trailing steps are dropped over budget)
|
||||
"replay": {
|
||||
"max_steps": 5,
|
||||
"max_tokens": 12000
|
||||
}
|
||||
},
|
||||
// 🎬 Screen Recording & Video Analyzer Configuration
|
||||
"video_analyzer": {
|
||||
"enabled": true, // Whether continuous screen recording via scrcpy is active (default: auto-detected)
|
||||
"enabled": "${env:ARTEMIS_VIDEO_ENABLED:-true}",
|
||||
"enable_ledger": true
|
||||
// "processing": "auto", // How Gemini sub-agents watch a clip: auto (agentic on Gemini 3.6+ Flash, static elsewhere) | agentic | static
|
||||
// "agentic_chunk_size_seconds": 600, // Longest clip one agentic sub-agent takes; static mode keeps chunk_size_seconds (60)
|
||||
// "agentic_call_timeout_seconds": 300 // Hard timeout for one agentic sub-agent response
|
||||
},
|
||||
// 📝 Outputter: Post-execution report synthesis and structured schema extraction
|
||||
// Reviews action history, notes, and verifies goal achievement to synthesize final output.
|
||||
"outputter": {
|
||||
"enabled": true, // Whether Outputter is mounted to synthesize final outputs
|
||||
"force_synthesis": false // Force Outputter synthesis even without explicit output schema
|
||||
"enabled": "${env:ARTEMIS_OUTPUTTER_ENABLED:-true}",
|
||||
"force_synthesis": false
|
||||
},
|
||||
// 🧭 Explorer tier: the perception sub-agent behind `ask_explorer`.
|
||||
// Tiers are a user setting; calling agents never see which tier is active.
|
||||
// flash - one-shot visual detection on the screenshot (fastest)
|
||||
// pro - 3-turn reasoning loop: UI-tree search + coordinate audit + detection
|
||||
// ultra - 8-turn loop adding zoom, OCR and pixel-level image processing (slowest)
|
||||
// `flash.explorer_mode` and `pro.explorer.mode` above are the per-profile knobs;
|
||||
// `caching` (Gemini explicit context caching) defaults per tier: off for pro, on for ultra.
|
||||
"explorer": {
|
||||
"default_version": "flash" // Fallback tier when no profile knob applies
|
||||
// "caching": true, // Uncomment to force context caching for pro/ultra
|
||||
"default_version": "flash"
|
||||
},
|
||||
// Advanced: per-agent tier override, e.g. {"validator": "ultra"}. Takes
|
||||
// precedence over the profile knobs; leave empty to let them decide.
|
||||
"explorer_versions": {},
|
||||
// 🚫 Per-agent Tool Denylists
|
||||
"denylisted_tools": {
|
||||
"explorer": [] // Perception tools to withhold from the Explorer, e.g. "ask_image_processor"
|
||||
"explorer": []
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -15,7 +15,7 @@
|
||||
limitations under the License.
|
||||
-->
|
||||
|
||||
<html lang="en" data-beasties-container>
|
||||
<html lang="zh-CN" data-beasties-container>
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<title>artemis</title>
|
||||
@@ -23,15 +23,15 @@
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||
<style>@font-face{font-family:'Material Symbols Outlined';font-style:normal;font-weight:100 700;src:url(https://fonts.gstatic.com/s/materialsymbolsoutlined/v368/kJEhBvYX7BgnkSrUwT8OhrdQw4oELdPIeeII9v6oFsI.woff2) format('woff2');}.material-symbols-outlined{font-family:'Material Symbols Outlined';font-weight:normal;font-style:normal;font-size:24px;line-height:1;letter-spacing:normal;text-transform:none;display:inline-block;white-space:nowrap;word-wrap:normal;direction:ltr;-webkit-font-feature-settings:'liga';-webkit-font-smoothing:antialiased;}</style>
|
||||
<style>@font-face{font-family:'Material Symbols Outlined';font-style:normal;font-weight:100 700;src:url(https://fonts.gstatic.com/s/materialsymbolsoutlined/v374/kJEhBvYX7BgnkSrUwT8OhrdQw4oELdPIeeII9v6oFsI.woff2) format('woff2');}.material-symbols-outlined{font-family:'Material Symbols Outlined';font-weight:normal;font-style:normal;font-size:24px;line-height:1;letter-spacing:normal;text-transform:none;display:inline-block;white-space:nowrap;word-wrap:normal;direction:ltr;-webkit-font-feature-settings:'liga';-webkit-font-smoothing:antialiased;}</style>
|
||||
<style>@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format('woff2');unicode-range:U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format('woff2');unicode-range:U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format('woff2');unicode-range:U+1F00-1FFF;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format('woff2');unicode-range:U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format('woff2');unicode-range:U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format('woff2');unicode-range:U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format('woff2');unicode-range:U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format('woff2');unicode-range:U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format('woff2');unicode-range:U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format('woff2');unicode-range:U+1F00-1FFF;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format('woff2');unicode-range:U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format('woff2');unicode-range:U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format('woff2');unicode-range:U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format('woff2');unicode-range:U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;}@font-face{font-family:'Inter';font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format('woff2');unicode-range:U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;}@font-face{font-family:'Inter';font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format('woff2');unicode-range:U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;}@font-face{font-family:'Inter';font-style:normal;font-weight:500;font-display:swap;src:url(https://fLine truncated
|
||||
<link rel="icon" type="image/png" sizes="192x192" href="/logo.png?v=2">
|
||||
<link rel="icon" type="image/png" sizes="32x32" href="/logo.png?v=2">
|
||||
<link rel="icon" type="image/svg+xml" href="/logo.svg?v=2">
|
||||
<link rel="shortcut icon" href="/favicon.ico?v=2">
|
||||
<link rel="apple-touch-icon" href="/logo.png?v=2">
|
||||
<style>@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:500;Line truncated
|
||||
<style>@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:500;Line truncated
|
||||
<body>
|
||||
<app-root></app-root>
|
||||
<script src="polyfills-5CFQRCPP.js" type="module"></script><script src="main-BGAFACUZ.js" type="module"></script></body>
|
||||
<script src="polyfills-5CFQRCPP.js" type="module"></script><script src="main-WGGFA77O.js" type="module"></script></body>
|
||||
</html>
|
||||
@@ -1,18 +0,0 @@
|
||||
var aO=Object.defineProperty,sO=Object.defineProperties;var cO=Object.getOwnPropertyDescriptors;var vm=Object.getOwnPropertySymbols;var lO=Object.prototype.hasOwnProperty,dO=Object.prototype.propertyIsEnumerable;var ym=(n,t,e)=>t in n?aO(n,t,{enumerable:!0,configurable:!0,writable:!0,value:e}):n[t]=e,S=(n,t)=>{for(var e in t||={})lO.call(t,e)&&ym(n,e,t[e]);if(vm)for(var e of vm(t))dO.call(t,e)&&ym(n,e,t[e]);return n},F=(n,t)=>sO(n,cO(t));var be=(n,t,e)=>new Promise((o,i)=>{var r=c=>{try{s(e.next(c))}catch(p){i(p)}},a=c=>{try{s(e.throw(c))}catch(p){i(p)}},s=c=>c.done?o(c.value):Promise.resolve(c.value).then(r,a);s((e=e.apply(n,t)).next())});var mt=null,Ds=!1,$o=1,uO=null,nt=Symbol("SIGNAL");function Y(n){let t=mt;return mt=n,t}function Is(){return mt}var po={version:0,lastCleanEpoch:0,dirty:!1,producers:void 0,producersTail:void 0,consumers:void 0,consumersTail:void 0,recomputing:!1,consumerAllowSignalWrites:!1,consumerIsAlwaysLive:!1,kind:"unknown",producerMustRecompute:()=>!1,producerRecomputeValue:()=>{},consumerMarkedDirty:()=>{},consumerOnSignalRead:()=>{}};function Ho(n){if(Ds)throw new Error("");if(mt===null)return;mt.consumerOnSignalRead(n);let t=mt.producersTail;if(t!==void 0&&t.producer===n)return;let e,o=mt.recomputing;if(o&&(e=t!==void 0?t.nextProducer:mt.producers,e!==void 0&&e.producer===n)){mt.producersTail=e,e.lastReadVersion=n.version,e.knownValidAtEpoch=$o;return}let i=n.consumersTail;if(i!==void 0&&i.consumer===mt&&(!o||i.knownValidAtEpoch===$o))return;let r=$i(mt),a={producer:n,consumer:mt,nextProducer:e,prevConsumer:void 0,knownValidAtEpoch:$o,lastReadVersion:n.version,nextConsumer:void 0};mt.producersTail=a,t!==void 0?t.nextProducer=a:mt.producers=a,r&&km(n,a)}function xm(){$o++}function Uo(n){if(!($i(n)&&!n.dirty)&&!(!n.dirty&&n.lastCleanEpoch===$o)){if(!n.producerMustRecompute(n)&&!Wr(n)){ji(n);return}n.producerRecomputeValue(n),ji(n)}}function eu(n){if(n.consumers===void 0)return;let t=Ds;Ds=!0;try{for(let e=n.consumers;e!==void 0;e=e.nextConsumer){let o=e.consumer;o.dirty||gO(o)}}finally{Ds=t}}function tu(){return mt?.consumerAllowSignalWrites!==!1}function gO(n){n.dirty=!0,eu(n),n.consumerMarkedDirty?.(n)}function ji(n){n.dirty=!1,n.lastCleanEpoch=$o}function fo(n){return n&&wm(n),Y(n)}function wm(n){if(n.producersTail?.knownValidAtEpoch===$o){let t=n.producers;for(;t!==void 0;)t.knownValidAtEpoch=null,t=t.nextProducer}n.producersTail=void 0,n.recomputing=!0}function Go(n,t){Y(t),n&&Sm(n)}function Sm(n){n.recomputing=!1;let t=n.producersTail,e=t!==void 0?t.nextProducer:n.producers;if(e!==void 0){if($i(n))do e=nu(e);while(e!==void 0);t!==void 0?t.nextProducer=void 0:n.producers=void 0}}function Wr(n){for(let t=n.producers;t!==void 0;t=t.nextProducer){let e=t.producer,o=t.lastReadVersion;if(o!==e.version||(Uo(e),o!==e.version))return!0}return!1}function qo(n){if($i(n)){let t=n.producers;for(;t!==void 0;)t=nu(t)}n.producers=void 0,n.producersTail=void 0,n.consumers=void 0,n.consumersTail=void 0}function km(n,t){let e=n.consumersTail,o=$i(n);if(e!==void 0?(t.nextConsumer=e.nextConsumer,e.nextConsumer=t):(t.nextConsumer=void 0,n.consumers=t),t.prevConsumer=e,n.consumersTail=t,!o)for(let i=n.producers;i!==void 0;i=i.nextProducer)km(i.producer,i)}function nu(n){let t=n.producer,e=n.nextProducer,o=n.nextConsumer,i=n.prevConsumer;if(n.nextConsumer=void 0,n.prevConsumer=void 0,o!==void 0?o.prevConsumer=i:t.consumersTail=i,i!==void 0)i.nextConsumer=o;else if(t.consumers=o,!$i(t)){let r=t.producers;for(;r!==void 0;)r=nu(r)}return e}function $i(n){return n.consumerIsAlwaysLive||n.consumers!==void 0}function Kr(n){uO?.(n)}function Yr(n,t){return Object.is(n,t)}function Ts(n,t){let e=Object.create(pO);e.computation=n,t!==void 0&&(e.equal=t);let o=()=>{if(Uo(e),Ho(e),e.value===Cn)throw e.error;return e.value};return o[nt]=e,Kr(e),o}var zo=Symbol("UNSET"),Bo=Symbol("COMPUTING"),Cn=Symbol("ERRORED"),pO=F(S({},po),{value:zo,dirty:!0,error:null,equal:Yr,kind:"computed",producerMustRecompute(n){return n.value===zo||n.value===Bo},producerRecomputeValue(n){if(n.value===Bo)throw new Error("");let t=n.value;n.value=Bo;let e=fo(n),o,i=!1;try{o=n.computation(),Y(null),i=t!==zo&&t!==Cn&&o!==Cn&&n.equal(t,o)}catch(r){o=Cn,n.error=r}finally{Go(n,e)}if(i){n.value=t;return}n.value=o,n.version++}});function fO(){throw new Error}var Em=fO;function Dm(n){Em(n)}function ou(n){Em=n}var mO=null;function iu(n,t){let e=Object.create(Ns);e.value=n,t!==void 0&&(e.equal=t);let o=()=>Im(e);return o[nt]=e,Kr(e),[o,a=>Wo(e,a),a=>As(e,a)]}function Im(n){return Ho(n),n.value}function Wo(n,t){tu()||Dm(n),n.equal(n.value,t)||(n.value=t,_O(n))}function As(n,t){tu()||Dm(n),Wo(n,t(n.value))}var Ns=F(S({},po),{equal:Yr,value:void 0,kind:"signal"});function _O(n){n.version++,xm(),eu(n),mO?.(n)}var ru=F(S({},po),{consumerIsAlwaysLive:!0,consumerAllowSignalWrites:!0,dirty:!0,kind:"effect"});function au(n){if(n.dirty=!1,n.version>0&&!Wr(n))return;n.version++;let t=fo(n);try{n.cleanup(),n.fn()}finally{Go(n,t)}}var su;function Rs(){return
Line truncated
|
||||
${e.map((o,i)=>`${i+1}) ${o.toString()}`).join(`
|
||||
`)}`:"",this.name="UnsubscriptionError",this.errors=e});function Zr(n,t){if(n){let e=n.indexOf(t);0<=e&&n.splice(e,1)}}var Be=class n{constructor(t){this.initialTeardown=t,this.closed=!1,this._parentage=null,this._finalizers=null}unsubscribe(){let t;if(!this.closed){this.closed=!0;let{_parentage:e}=this;if(e)if(this._parentage=null,Array.isArray(e))for(let r of e)r.remove(this);else e.remove(this);let{initialTeardown:o}=this;if(ae(o))try{o()}catch(r){t=r instanceof Fs?r.errors:[r]}let{_finalizers:i}=this;if(i){this._finalizers=null;for(let r of i)try{Rm(r)}catch(a){t=t??[],a instanceof Fs?t=[...t,...a.errors]:t.push(a)}}if(t)throw new Fs(t)}}add(t){var e;if(t&&t!==this)if(this.closed)Rm(t);else{if(t instanceof n){if(t.closed||t._hasParent(this))return;t._addParent(this)}(this._finalizers=(e=this._finalizers)!==null&&e!==void 0?e:[]).push(t)}}_hasParent(t){let{_parentage:e}=this;return e===t||Array.isArray(e)&&e.includes(t)}_addParent(t){let{_parentage:e}=this;this._parentage=Array.isArray(e)?(e.push(t),e):e?[e,t]:t}_removeParent(t){let{_parentage:e}=this;e===t?this._parentage=null:Array.isArray(e)&&Zr(e,t)}remove(t){let{_finalizers:e}=this;e&&Zr(e,t),t instanceof n&&t._removeParent(this)}};Be.EMPTY=(()=>{let n=new Be;return n.closed=!0,n})();var du=Be.EMPTY;function Ls(n){return n instanceof Be||n&&"closed"in n&&ae(n.remove)&&ae(n.add)&&ae(n.unsubscribe)}function Rm(n){ae(n)?n():n.unsubscribe()}var en={onUnhandledError:null,onStoppedNotification:null,Promise:void 0,useDeprecatedSynchronousErrorHandling:!1,useDeprecatedNextContext:!1};var Hi={setTimeout(n,t,...e){let{delegate:o}=Hi;return o?.setTimeout?o.setTimeout(n,t,...e):setTimeout(n,t,...e)},clearTimeout(n){let{delegate:t}=Hi;return(t?.clearTimeout||clearTimeout)(n)},delegate:void 0};function Vs(n){Hi.setTimeout(()=>{let{onUnhandledError:t}=en;if(t)t(n);else throw n})}function Qr(){}var Fm=uu("C",void 0,void 0);function Lm(n){return uu("E",void 0,n)}function Vm(n){return uu("N",n,void 0)}function uu(n,t,e){return{kind:n,value:t,error:e}}var Ko=null;function Ui(n){if(en.useDeprecatedSynchronousErrorHandling){let t=!Ko;if(t&&(Ko={errorThrown:!1,error:null}),n(),t){let{errorThrown:e,error:o}=Ko;if(Ko=null,e)throw o}}else n()}function jm(n){en.useDeprecatedSynchronousErrorHandling&&Ko&&(Ko.errorThrown=!0,Ko.error=n)}var Yo=class extends Be{constructor(t){super(),this.isStopped=!1,t?(this.destination=t,Ls(t)&&t.add(this)):this.destination=bO}static create(t,e,o){return new zn(t,e,o)}next(t){this.isStopped?pu(Vm(t),this):this._next(t)}error(t){this.isStopped?pu(Lm(t),this):(this.isStopped=!0,this._error(t))}complete(){this.isStopped?pu(Fm,this):(this.isStopped=!0,this._complete())}unsubscribe(){this.closed||(this.isStopped=!0,super.unsubscribe(),this.destination=null)}_next(t){this.destination.next(t)}_error(t){try{this.destination.error(t)}finally{this.unsubscribe()}}_complete(){try{this.destination.complete()}finally{this.unsubscribe()}}},CO=Function.prototype.bind;function gu(n,t){return CO.call(n,t)}var fu=class{constructor(t){this.partialObserver=t}next(t){let{partialObserver:e}=this;if(e.next)try{e.next(t)}catch(o){js(o)}}error(t){let{partialObserver:e}=this;if(e.error)try{e.error(t)}catch(o){js(o)}else js(t)}complete(){let{partialObserver:t}=this;if(t.complete)try{t.complete()}catch(e){js(e)}}},zn=class extends Yo{constructor(t,e,o){super();let i;if(ae(t)||!t)i={next:t??void 0,error:e??void 0,complete:o??void 0};else{let r;this&&en.useDeprecatedNextContext?(r=Object.create(t),r.unsubscribe=()=>this.unsubscribe(),i={next:t.next&&gu(t.next,r),error:t.error&&gu(t.error,r),complete:t.complete&&gu(t.complete,r)}):i=t}this.destination=new fu(i)}};function js(n){en.useDeprecatedSynchronousErrorHandling?jm(n):Vs(n)}function MO(n){throw n}function pu(n,t){let{onStoppedNotification:e}=en;e&&Hi.setTimeout(()=>e(n,t))}var bO={closed:!0,next:Qr,error:MO,complete:Qr};var Gi=typeof Symbol=="function"&&Symbol.observable||"@@observable";function tn(n){return n}function $s(...n){return mu(n)}function mu(n){return n.length===0?tn:n.length===1?n[0]:function(e){return n.reduce((o,i)=>i(o),e)}}var ie=class n{constructor(t){t&&(this._subscribe=t)}lift(t){let e=new n;return e.source=this,e.operator=t,e}subscribe(t,e,o){let i=OO(t)?t:new zn(t,e,o);return Ui(()=>{let{operator:r,source:a}=this;i.add(r?r.call(i,a):a?this._subscribe(i):this._trySubscribe(i))}),i}_trySubscribe(t){try{return this._subscribe(t)}catch(e){t.error(e)}}forEach(t,e){return e=$m(e),new e((o,i)=>{let r=new zn({next:a=>{try{t(a)}catch(s){i(s),r.unsubscribe()}},error:i,complete:o});this.subscribe(r)})}_subscribe(t){var e;return(e=this.source)===null||e===void 0?void 0:e.subscribe(t)}[Gi](){return this}pipe(...t){return mu(t)(this)}toPromise(t){return t=$m(t),new t((e,o)=>{let i;this.subscribe(r=>i=r,r=>o(r),()=>e(i))})}};ie.create=n=>new ie(n);function $m(n){var t;return(t=n??en.Promise)!==null&&t!==void 0?t:Promise}function PO(n){return n&&ae(n.next)&&ae(n.error)&&ae(n.complete)}function OO(n){return n&&nLine truncated
|
||||
`);return o>=0?e.slice(0,o):e}function fc(n,t){return n?t?`${n} ${t}`:n:t||""}var zO=he({__forward_ref__:he});function on(n){return n.__forward_ref__=on,n}function lt(n){return Lu(n)?n():n}function Lu(n){return typeof n=="function"&&Object.hasOwn(n,zO)&&n.__forward_ref__===on}function te(n){return{token:n.token,providedIn:n.providedIn||null,factory:n.factory,value:void 0}}function Pn(n){return{providers:n.providers||[],imports:n.imports||[]}}function sa(n){return BO(n,mc)}function Vu(n){return sa(n)!==null}function BO(n,t){return Object.hasOwn(n,t)&&n[t]||null}function HO(n){let t=n?.[mc]??null;return t||null}function xu(n){return n&&Object.hasOwn(n,cc)?n[cc]:null}var mc=he({\u0275prov:he}),cc=he({\u0275inj:he}),B=class{_desc;ngMetadataName="InjectionToken";\u0275prov;constructor(t,e){this._desc=t,this.\u0275prov=void 0,typeof e=="number"?this.__NG_ELEMENT_ID__=e:e!==void 0&&(this.\u0275prov=te({token:this,providedIn:e.providedIn||"root",factory:e.factory}))}get multi(){return this}toString(){return`InjectionToken ${this._desc}`}};function ju(n){return n&&!!n.\u0275providers}var ca=he({\u0275cmp:he}),la=he({\u0275dir:he}),$u=he({\u0275pipe:he}),zu=he({\u0275mod:he}),na=he({\u0275fac:he}),oi=he({__NG_ELEMENT_ID__:he}),s_=he({__NG_ENV_ID__:he});function p_(n){return _c(n,"@NgModule"),n[zu]||null}function ii(n){return _c(n,"@Component"),n[ca]||null}function Bu(n){return _c(n,"@Directive"),n[la]||null}function f_(n){return _c(n,"@Pipe"),n[$u]||null}function _c(n,t){if(n==null)throw new z(-919,!1)}function rn(n){return typeof n=="string"?n:n==null?"":String(n)}var m_=he({ngErrorCode:he}),UO=he({ngErrorMessage:he}),GO=he({ngTokenPath:he});function Hu(n,t){return __("",-200,t)}function hc(n,t){throw new z(-201,!1)}function __(n,t,e){let o=new z(t,n);return o[m_]=t,o[UO]=n,e&&(o[GO]=e),o}function qO(n){return n[m_]}var wu;function h_(){return wu}function _t(n){let t=wu;return wu=n,t}function Uu(n,t,e){let o=sa(n);if(o&&o.providedIn=="root")return o.value===void 0?o.value=o.factory():o.value;if(e&8)return null;if(t!==void 0)return t;hc(n,"")}var an=globalThis;var WO={},Xo=WO,KO="__NG_DI_FLAG__",Su=class{injector;constructor(t){this.injector=t}retrieve(t,e){let o=ei(e)||0;try{return this.injector.get(t,o&8?null:Xo,o)}catch(i){if(zi(i))return i;throw i}}};function YO(n,t=0){let e=Rs();if(e===void 0)throw new z(-203,!1);if(e===null)return Uu(n,void 0,t);{let o=ZO(t),i=e.retrieve(n,o);if(zi(i)){if(o.optional)return null;throw i}return i}}function Q(n,t=0){return(h_()||YO)(lt(n),t)}function D(n,t){return Q(n,ei(t))}function ei(n){return typeof n>"u"||typeof n=="number"?n:0|(n.optional&&8)|(n.host&&1)|(n.self&&2)|(n.skipSelf&&4)}function ZO(n){return{optional:!!(n&8),host:!!(n&1),self:!!(n&2),skipSelf:!!(n&4)}}function ku(n){let t=[];for(let e=0;e<n.length;e++){let o=lt(n[e]);if(Array.isArray(o)){if(o.length===0)throw new z(900,!1);let i,r=0;for(let a=0;a<o.length;a++){let s=o[a],c=QO(s);typeof c=="number"?c===-1?i=s.token:r|=c:i=s}t.push(Q(i,r))}else t.push(Q(o))}return t}function QO(n){return n[KO]}function Co(n,t){let e=Object.hasOwn(n,na);return e?n[na]:null}function C_(n,t,e){if(n.length!==t.length)return!1;for(let o=0;o<n.length;o++){let i=n[o],r=t[o];if(e&&(i=e(i),r=e(r)),r!==i)return!1}return!0}function M_(n){return n.flat(Number.POSITIVE_INFINITY)}function Cc(n,t){n.forEach(e=>Array.isArray(e)?Cc(e,t):t(e))}function Gu(n,t,e){t>=n.length?n.push(e):n.splice(t,0,e)}function da(n,t){return t>=n.length-1?n.pop():n.splice(t,1)[0]}function b_(n,t,e,o){let i=n.length;if(i==t)n.push(e,o);else if(i===1)n.push(o,n[0]),n[0]=e;else{for(i--,n.push(n[i-1],n[i]);i>t;){let r=i-2;n[i]=n[r],i--}n[t]=e,n[t+1]=o}}function Mc(n,t,e){let o=Ji(n,t);return o>=0?n[o|1]=e:(o=~o,b_(n,o,t,e)),o}function bc(n,t){let e=Ji(n,t);if(e>=0)return n[e|1]}function Ji(n,t){return JO(n,t,1)}function JO(n,t,e){let o=0,i=n.length>>e;for(;i!==o;){let r=o+(i-o>>1),a=n[r<<e];if(t===a)return r<<e;a>t?i=r:o=r+1}return~(i<<e)}var Mo={},ht=[],ri=new B(""),ua=new B("",-1),qu=new B(""),Zi=class{get(t,e=Xo){if(e===Xo){let i=__("",-201);throw i.name="\u0275NotFound",i}return e}};function bo(n){return{\u0275providers:n}}function P_(...n){return{\u0275providers:Wu(!0,n),\u0275fromNgModule:!0}}function Wu(n,...t){let e=[],o=new Set,i,r=a=>{e.push(a)};return Cc(t,a=>{let s=a;lc(s,r,[],o)&&(i||=[],i.push(s))}),i!==void 0&&O_(i,r),e}function O_(n,t){for(let e=0;e<n.length;e++){let{ngModule:o,providers:i}=n[e];Ku(i,r=>{t(r,o)})}}function lc(n,t,e,o){if(n=lt(n),!n)return!1;let i=null,r=xu(n),a=!r&&ii(n);if(!r&&!a){let c=n.ngModule;if(r=xu(c),r)i=c;else return!1}else{if(a&&!a.standalone)return!1;i=n}let s=o.has(i);if(a){if(s)return!1;if(o.add(i),a.dependencies){let c=typeof a.dependencies=="function"?a.dependencies():a.dependencies;for(let p of c)lc(p,t,e,o)}}else if(r){if(r.imports!=null&&!s){o.add(i);let p;Cc(r.imports,f=>{lc(f,t,e,o)&&(p||=[],p.push(f))}),p!==void 0&&O_(p,t)}if(!s){let p=Co(i)||(()=>new i);t({provide:i,useFactory:p,deps:ht},i),t({provide:qu,useValue:i,
Line truncated
|
||||
`).forEach(e=>{let o=e.indexOf(":");if(o>0){let i=e.slice(0,o),r=e.slice(o+1).trim();this.addHeaderEntry(i,r)}})}:typeof Headers<"u"&&t instanceof Headers?(this.headers=new Map,t.forEach((e,o)=>{this.addHeaderEntry(o,e)})):this.lazyInit=()=>{this.headers=new Map,Object.entries(t).forEach(([e,o])=>{this.setHeaderEntries(e,o)})}:this.headers=new Map}has(t){return this.init(),this.headers.has(t.toLowerCase())}get(t){this.init();let e=this.headers.get(t.toLowerCase());return e&&e.length>0?e[0]:null}keys(){return this.init(),Array.from(this.normalizedNames.values())}getAll(t){return this.init(),this.headers.get(t.toLowerCase())||null}append(t,e){return this.clone({name:t,value:e,op:"a"})}set(t,e){return this.clone({name:t,value:e,op:"s"})}delete(t,e){return this.clone({name:t,value:e,op:"d"})}maybeSetNormalizedName(t,e){this.normalizedNames.has(e)||this.normalizedNames.set(e,t)}init(){this.lazyInit&&(this.lazyInit instanceof n?this.copyFrom(this.lazyInit):this.lazyInit(),this.lazyInit=null,this.lazyUpdate&&(this.lazyUpdate.forEach(t=>this.applyUpdate(t)),this.lazyUpdate=null))}copyFrom(t){t.init();for(let[e,o]of t.headers.entries())this.headers.set(e,o),this.normalizedNames.set(e,t.normalizedNames.get(e))}clone(t){let e=new n;return e.lazyInit=this.lazyInit&&this.lazyInit instanceof n?this.lazyInit:this,e.lazyUpdate=(this.lazyUpdate||[]).concat([t]),e}applyUpdate(t){let e=t.name.toLowerCase();switch(t.op){case"a":case"s":let o=t.value;if(typeof o=="string"&&(o=[o]),o.length===0)return;this.maybeSetNormalizedName(t.name,e);let i=t.op==="a"?(this.headers.get(e)||[]).slice():[];i.push(...o),this.headers.set(e,i);break;case"d":let r=t.value;if(r===void 0)this.headers.delete(e),this.normalizedNames.delete(e);else{let a=Array.isArray(r)?r:[r],s=this.headers.get(e);if(!s)return;s=s.filter(c=>a.indexOf(c)===-1),s.length===0?(this.headers.delete(e),this.normalizedNames.delete(e)):this.headers.set(e,s)}break}}addHeaderEntry(t,e){let o=t.toLowerCase();this.maybeSetNormalizedName(t,o),this.headers.has(o)?this.headers.get(o).push(e):this.headers.set(o,[e])}setHeaderEntries(t,e){let o=(Array.isArray(e)?e:[e]).map(r=>r.toString()),i=t.toLowerCase();this.headers.set(i,o),this.maybeSetNormalizedName(t,i)}forEach(t){this.init(),Array.from(this.normalizedNames.keys()).forEach(e=>t(this.normalizedNames.get(e),this.headers.get(e)))}};var Wl=class{map=new Map;set(t,e){return this.map.set(t,e),this}get(t){return this.map.has(t)||this.map.set(t,t.defaultValue()),this.map.get(t)}delete(t){return this.map.delete(t),this}has(t){return this.map.has(t)}keys(){return this.map.keys()}},Kl=class{encodeKey(t){return y0(t)}encodeValue(t){return y0(t)}decodeKey(t){return decodeURIComponent(t)}decodeValue(t){return decodeURIComponent(t)}};function Ck(n,t){let e=new Map;return n.length>0&&n.replace(/^\?/,"").split("&").forEach(i=>{let r=i.indexOf("="),[a,s]=r==-1?[t.decodeKey(i),""]:[t.decodeKey(i.slice(0,r)),t.decodeValue(i.slice(r+1))],c=e.get(a)||[];c.push(s),e.set(a,c)}),e}var Mk=/%(\d[a-f0-9])/gi,bk={40:"@","3A":":",24:"$","2C":",","3B":";","3D":"=","3F":"?","2F":"/"};function y0(n){return encodeURIComponent(n).replace(Mk,(t,e)=>bk[e]??t)}function ql(n){return`${n}`}var ao=class n{map;encoder;updates=null;cloneFrom=null;constructor(t={}){if(this.encoder=t.encoder||new Kl,t.fromString){if(t.fromObject)throw new z(2805,!1);this.map=Ck(t.fromString,this.encoder)}else t.fromObject?(this.map=new Map,Object.keys(t.fromObject).forEach(e=>{let o=t.fromObject[e],i=Array.isArray(o)?o.map(ql):[ql(o)];this.map.set(e,i)})):this.map=null}has(t){return this.init(),this.map.has(t)}get(t){this.init();let e=this.map.get(t);return e?e[0]:null}getAll(t){return this.init(),this.map.get(t)||null}keys(){return this.init(),Array.from(this.map.keys())}append(t,e){return this.clone({param:t,value:e,op:"a"})}appendAll(t){let e=[];return Object.keys(t).forEach(o=>{let i=t[o];Array.isArray(i)?i.forEach(r=>{e.push({param:o,value:r,op:"a"})}):e.push({param:o,value:i,op:"a"})}),this.clone(e)}set(t,e){return this.clone({param:t,value:e,op:"s"})}delete(t,e){return this.clone({param:t,value:e,op:"d"})}toString(){return this.init(),this.keys().map(t=>{let e=this.encoder.encodeKey(t);return this.map.get(t).map(o=>e+"="+this.encoder.encodeValue(o)).join("&")}).filter(t=>t!=="").join("&")}clone(t){let e=new n({encoder:this.encoder});return e.cloneFrom=this.cloneFrom||this,e.updates=(this.updates||[]).concat(t),e}init(){if(this.map===null&&(this.map=new Map),this.cloneFrom!==null){this.cloneFrom.init();for(let[t,e]of this.cloneFrom.map.entries())this.map.set(t,e);this.updates.forEach(t=>{switch(t.op){case"a":case"s":let e=t.op==="a"?(this.map.get(t.param)||[]).slice():[];e.push(ql(t.value)),this.map.set(t.param,e);break;case"d":if(t.value!==void 0){let o=(this.map.get(t.param)||[]).slice(),i=o.indexOf(ql(t.value));i!==-1&&o.splice(i,1),o.length>0?this.map.set(t.param,o):this.map.delete(t.param)}else{this.map.delete(t.param);break}}}),this.cloneFrom=this.updates=null}}};function Pk(
Line truncated
|
||||
`)||"","boot_logs"))}),l(8,"span",29),u(9),d(),l(10,"span"),u(11),d()()(),l(12,"pre",202),M(13,yI,2,0)(14,xI,2,0,"div",203),d()()}if(n&2){let e=m(3);g(9),O(e.copiedId()==="boot_logs"?"check":"content_copy"),g(2),O(e.copiedId()==="boot_logs"?"Copied":"Copy Logs"),g(2),b(e.emulatorLaunchState()?.logs?.length?13:14)}}function SI(n,t){if(n&1){let e=L();l(0,"div",65)(1,"div",168)(2,"div",169)(3,"div",170)(4,"div",171)(5,"span",172),u(6,"sync"),d()(),l(7,"div",173)(8,"div",174)(9,"h4",175),u(10," Launching Android Emulator: "),l(11,"span",176),u(12),d()(),l(13,"span",177),u(14),d()(),l(15,"p",178),u(16),d()()(),l(17,"div",179)(18,"button",180),k("click",function(){v(e);let i=m(2);return y(i.toggleLaunchLogs())}),l(19,"span",29),u(20,"terminal"),d(),l(21,"span"),u(22),d()(),l(23,"button",181),k("click",function(){v(e);let i=m(2);return y(i.stopEmulator())}),l(24,"span",29),u(25,"cancel"),d(),l(26,"span"),u(27,"Stop"),d()()()(),l(28,"div",182)(29,"div",183)(30,"div",184),M(31,mI,2,0,"span",185)(32,_I,1,0,"span",186),d(),l(33,"span",187),u(34,"1. Process Start"),d()(),$(35,"div",188),l(36,"div",183)(37,"div",184),M(38,hI,2,0,"span",185)(39,CI,1,0,"span",186)(40,MI,2,0,"span"),d(),l(41,"span",187),u(42,"2. ADB Handshake"),d()(),$(43,"div",188),l(44,"div",183)(45,"div",184),M(46,bI,2,0,"span",185)(47,PI,1,0,"span",186)(48,OI,2,0,"span"),d(),l(49,"span",187),u(50,"3. Android OS Boot"),d()(),$(51,"div",188),l(52,"div",183)(53,"div",184)(54,"span",185),u(55,"check_circle"),d()(),l(56,"span",187),u(57,"4. Ready"),d()()(),l(58,"div",189)(59,"div",190),$(60,"div",191),d(),l(61,"div",192)(62,"span",193),u(63),d(),l(64,"span",194)(65,"span",195),u(66,"timer"),d(),u(67),d()()(),l(68,"div",196)(69,"span",197),u(70,"info"),d(),l(71,"span"),u(72,"Android virtualization cold boot usually takes 20-45 seconds. Live progress is actively tracked above\u2014please do not click launch repeatedly."),d()(),M(73,wI,15,3,"div",198),d()()}if(n&2){let e=m(2);g(12),O(e.emulatorLaunchState()?.avd_name||e.activeDevice()?.serial||"Virtual Device"),g(2),O(e.emulatorLaunchState()?.status==="booting"?"OS Booting":"Starting"),g(2),O(e.emulatorLaunchState()?.stage_message||"Initializing Android Virtual Device..."),g(2),w("active",e.showLaunchLogs()),g(4),O(e.showLaunchLogs()?"Hide Logs":"View Boot Logs"),g(7),w("done",e.emulatorLaunchState()?.progress_percent>=35)("active",e.emulatorLaunchState()?.status==="starting"),g(2),b(e.emulatorLaunchState()?.progress_percent>=35?31:32),g(4),w("done",e.emulatorLaunchState()?.progress_percent>=35),g(),w("done",e.emulatorLaunchState()?.progress_percent>=65)("active",e.emulatorLaunchState()?.status==="waiting_for_adb"),g(2),b(e.emulatorLaunchState()?.progress_percent>=65?38:e.emulatorLaunchState()?.status==="waiting_for_adb"?39:40),g(5),w("done",e.emulatorLaunchState()?.progress_percent>=65),g(),w("done",e.emulatorLaunchState()?.status==="ready")("active",e.emulatorLaunchState()?.status==="booting"||e.adbProbe()?.summary==="Device Booting"),g(2),b(e.emulatorLaunchState()?.status==="ready"?46:e.emulatorLaunchState()?.status==="booting"||e.adbProbe()?.summary==="Device Booting"?47:48),g(5),w("done",e.emulatorLaunchState()?.status==="ready"),g(),w("done",e.emulatorLaunchState()?.status==="ready"),g(8),Se("width",e.emulatorLaunchState()?.progress_percent||25,"%"),g(3),V("",e.emulatorLaunchState()?.progress_percent||25,"% Complete"),g(4),V(" Elapsed: ",e.emulatorLaunchState()?.elapsed_seconds||0,"s "),g(6),b(e.showLaunchLogs()?73:-1)}}function kI(n,t){if(n&1&&(l(0,"div",204),u(1),d()),n&2){let e=t.$implicit;g(),O(e)}}function EI(n,t){if(n&1){let e=L();l(0,"div",214)(1,"div",199)(2,"span"),u(3,"Process Output / Failure Logs:"),d(),l(4,"button",201),k("click",function(){v(e);let i=m(3);return y(i.copyToClipboard(i.emulatorLaunchState()?.logs?.join(`
|
||||
`)||"","fail_logs"))}),l(5,"span",29),u(6),d(),l(7,"span"),u(8),d()()(),l(9,"pre",219),G(10,kI,2,1,"div",204,Ie),d()()}if(n&2){let e=m(3);g(6),O(e.copiedId()==="fail_logs"?"check":"content_copy"),g(2),O(e.copiedId()==="fail_logs"?"Copied":"Copy Logs"),g(2),q(e.emulatorLaunchState()?.logs)}}function DI(n,t){if(n&1){let e=L();l(0,"button",220),k("click",function(){v(e);let i=m(3);return y(i.launchAvdEmulator(i.emulatorLaunchState().avd_name))}),l(1,"span",29),u(2,"refresh"),d(),l(3,"span"),u(4),d()()}if(n&2){let e=m(3);R("disabled",e.isRemoteAdbServer()),g(4),V("Retry Launch (",e.emulatorLaunchState()?.avd_name,")")}}function II(n,t){if(n&1){let e=L();l(0,"div",66)(1,"div",205)(2,"div",206)(3,"div",207)(4,"div",208)(5,"span",209),u(6,"error"),d()(),l(7,"div",210)(8,"h4",211),u(9,"Emulator Launch Failed"),d(),l(10,"p",212),u(11),d()()(),l(12,"button",213),k("click",function(){v(e);let i=m(2);return y(i.dismissEmulatorStatus())}),l(13,"span",29),u(14,"close"),d()()(),M(15,EI,12,2,"div",214),l(16,"div",215),M(17,DI,5,2,"button",216),l(18,"button",217),k("click",function(){v(e);let i=m(2);return y(i.restartAdbServer())}),l(19,"span",29),u(20,"sync"),d(),l(21,"span"),u(22,"Restart ADB Server"),d()(),l(23,"button",218),k("click",function(){v(e);let i=m(2);return y(i.dismissEmulatorStatus())}),l(24,"span"),u(25,"Dismiss"),d()()()()()}if(n&2){let e=m(2);g(11),O(e.emulatorLaunchState()?.error||"The Android emulator process failed to start or crashed during initialization."),g(4),b(e.emulatorLaunchState()?.logs?.length?15:-1),g(2),b(e.emulatorLaunchState()?.avd_name?17:-1)}}function TI(n,t){if(n&1&&u(0),n&2){let e=m(3);We(" ",e.installedAvds().length," local virtual device",e.installedAvds().length>1?"s":""," available ")}}function AI(n,t){n&1&&u(0," Launch a local Android Virtual Device ")}function NI(n,t){if(n&1){let e=L();l(0,"div",231)(1,"div",234)(2,"span",29),u(3,"hub"),d(),l(4,"span"),u(5," Artemis is currently using "),l(6,"strong"),u(7),d(),u(8,". Switch to local ADB before using a local emulator, USB cable, or Wireless ADB. "),d()(),l(9,"button",235),k("click",function(){v(e);let i=m(3);return y(i.switchToLocalAdbServer())}),l(10,"span",29),u(11,"sync"),d(),l(12,"span"),u(13),d()()()}if(n&2){let e=m(3);g(7),We("",e.adbServerStatus()?.endpoint?.host,":",e.adbServerStatus()?.endpoint?.port),g(2),R("disabled",e.isSwitchingToLocalAdb()),g(),w("spinning",e.isSwitchingToLocalAdb()),g(3),O(e.isSwitchingToLocalAdb()?"Switching...":"Use Local ADB")}}function RI(n,t){if(n&1&&($(0,"span",252),l(1,"span"),u(2),d()),n&2){let e=m(6);g(2),We("",e.emulatorLaunchState()?.status==="booting"?"Booting OS":"Starting"," (",e.emulatorLaunchState()?.elapsed_seconds||0,"s)")}}function FI(n,t){n&1&&(l(0,"span",253),u(1,"play_arrow"),d(),l(2,"span"),u(3,"Launch"),d())}function LI(n,t){if(n&1){let e=L();l(0,"div",243)(1,"div",244)(2,"div",245)(3,"span",29),u(4,"smartphone"),d()(),l(5,"div",246)(6,"div",247)(7,"span",248),u(8),d(),l(9,"span",249),u(10,"Virtual Device"),d()()()(),l(11,"div",250)(12,"button",251),k("click",function(){let i=v(e).$implicit,r=m(5);return y(r.launchAvdEmulator(i))}),M(13,RI,3,2)(14,FI,4,0),d()()()}if(n&2){let e=t.$implicit,o=m(5);g(8),O(e),g(4),w("launching",o.launchingAvd()===e||o.isEmulatorLaunching()&&o.emulatorLaunchState()?.avd_name===e),R("disabled",o.isEmulatorLaunching()||o.isRemoteAdbServer())("title",o.isEmulatorLaunching()?"An emulator is currently launching...":"Launch "+e),g(),b(o.launchingAvd()===e||o.isEmulatorLaunching()&&o.emulatorLaunchState()?.avd_name===e?13:14)}}function VI(n,t){if(n&1&&(l(0,"div",236)(1,"div",238)(2,"div",239)(3,"span",120),u(4,"devices"),d(),l(5,"span",240),u(6,"Installed Virtual Devices (AVD)"),d(),l(7,"span",241),u(8),d()()(),l(9,"div",242),G(10,LI,15,6,"div",243,Ie),d()()),n&2){let e=m(4);g(8),V("",e.installedAvds().length," Available"),g(2),q(e.installedAvds())}}function jI(n,t){if(n&1&&(l(0,"div",264)(1,"div",266)(2,"div",267)(3,"span",268),u(4,"1"),d(),l(5,"span",269),u(6,"download"),d()(),l(7,"div",270),u(8,"Install Android Studio"),d(),l(9,"div",271),u(10),l(11,"a",272),u(12,"developer.android.com/studio"),d(),u(13," and install. "),d()(),l(14,"div",266)(15,"div",267)(16,"span",268),u(17,"2"),d(),l(18,"span",269),u(19,"developer_mode"),d()(),l(20,"div",270),u(21,"Open Device Manager"),d(),l(22,"div",271),u(23," On Studio welcome screen, open "),l(24,"strong"),u(25,"More Actions > Virtual Device Manager"),d(),u(26," (or "),l(27,"strong"),u(28,"Tools > Device Manager"),d(),u(29,"). "),d()(),l(30,"div",266)(31,"div",267)(32,"span",268),u(33,"3"),d(),l(34,"span",269),u(35,"add_circle"),d()(),l(36,"div",270),u(37,"Create Virtual Device"),d(),l(38,"div",271),u(39," Click "),l(40,"strong"),u(41,"Create Device"),d(),u(42,", choose "),l(43,"strong"),u(44,"Pixel 8"),d(),u(45,", select "),l(46,"strong"),u(47,"Android 14 (API 34)"),d(),u(48,", and click "),l(49,"strong"),u(50,"Finish"),d(),u(51,". "),d()(),l(52,"div",266)(53,"div",267)(54,"span",268),u(55,"4"),d(),l(56,"spa
Line truncated
|
||||
`).map(t=>{let e=t.trim(),o=t.length-t.trimStart().length;if(e.startsWith("- [x] ")||e.startsWith("- [X] ")){let a=e.substring(6);return{type:"checked",segments:$n(a),indent:o}}if(e.startsWith("- [/] ")){let a=e.substring(6);return{type:"progress",segments:$n(a),indent:o}}if(e.startsWith("- [ ] ")){let a=e.substring(6);return{type:"unchecked",segments:$n(a),indent:o}}let i=e.match(/^[-*]\s*(verify|assert)(@end)?\s*:\s*(.*)$/i);if(i){let a=i[1].toLowerCase(),s=!!i[2],c=i[3].trim();return{type:"verify",checkKind:a,atEnd:s,segments:$n(c),indent:o}}let r=e.match(/^[-*]\s*finding\s*:\s*(.*)$/i);if(r){let a=r[1].trim();return{type:"finding",checkKind:"finding",segments:$n(a),indent:o}}if(e.startsWith("- ")||e.startsWith("* ")){let a=e.substring(2);return{type:"list-item",segments:$n(a),indent:o}}if(e.startsWith("# ")){let a=e.substring(2);return{type:"h1",segments:$n(a),indent:o}}if(e.startsWith("## ")){let a=e.substring(3);return{type:"h2",segments:$n(a),indent:o}}if(e.startsWith("### ")){let a=e.substring(4);return{type:"h3",segments:$n(a),indent:o}}return e?{type:"text",segments:$n(t),indent:o}:{type:"empty",segments:[],indent:0}}):[]}function ks(n){let t=Ss(n),e=[],o=[],i=null,r=null,a=0;for(let s of t){if(s.type==="empty")continue;if((s.type==="h1"||s.type==="h2")&&!i){i=s.segments.map(p=>p.text).join("");continue}["checked","progress","unchecked"].includes(s.type)&&(!r||s.indent<=(r._indent??0))?(a++,r={index:a,type:s.type,segments:s.segments,subSteps:[],checks:[],_indent:s.indent},e.push(r)):r&&s.indent>(r._indent??0)?s.type==="verify"||s.type==="assert"||s.type==="finding"?r.checks.push(s):r.subSteps.push(s):o.push(s)}return{title:i,milestones:e,otherLines:o}}function WP(n){if(!n)return null;let t=n.trim();if(t.includes("```json")){let e=t.split("```json");e.length>1&&(t=e[1].split("```")[0].trim())}else if(t.includes("```")){let e=t.split("```");e.length>1&&(t=e[1].split("```")[0].trim())}try{let e=JSON.parse(t);if(e&&typeof e=="object"&&"success"in e)return{success:!!e.success,reason:e.reason||"No reason provided."}}catch(e){}return null}function Vi(n){return n?n.replace(/`([^`]+)`/g,'<code class="inline-code">$1</code>').replace(/\*\*\*([^*]+)\*\*\*/g,"<strong><em>$1</em></strong>").replace(/___([^_]+)___/g,"<strong><em>$1</em></strong>").replace(/\*\*([^*]+)\*\*/g,"<strong>$1</strong>").replace(/__([^_]+)__/g,"<strong>$1</strong>").replace(/\*([^*]+)\*/g,"<em>$1</em>").replace(/\b_([^_]+)_\b/g,"<em>$1</em>").replace(/~~([^~]+)~~/g,"<del>$1</del>"):""}function Om(n){if(!n)return"";let o=n.replace(/<\/?(thought|short_term_memory|strong_term_memory|reasoning|plan|task_plan|call_tool)[^\n>]*>?/gi,"").replace(/\r\n/g,`
|
||||
`).replace(/\r/g,`
|
||||
`).replace(/\n{3,}/g,`
|
||||
|
||||
`).trim().replace(/&/g,"&").replace(/</g,"<").replace(/>/g,">").split(`
|
||||
`),i=[],r=null,a=!1,s="",c=[],p=()=>{r&&(i.push(`</${r}>`),r=null)};for(let f=0;f<o.length;f++){let _=o[f],C=_.trim();if(C.startsWith("```")){if(a){let A=c.join(`
|
||||
`);i.push(`<pre class="code-block"><code class="lang-${s}">${A}</code></pre>`),a=!1,c=[],s=""}else p(),a=!0,s=C.substring(3).trim(),c=[];continue}if(a){c.push(_);continue}let h=C.match(/^(#{1,6})\s+(.*)$/);if(h){p();let A=h[1].length,N=Vi(h[2]);i.push(`<h${A}>${N}</h${A}>`);continue}if(/^(\-{3,}|\*{3,}|_{3,})$/.test(C)){p(),i.push('<hr class="md-hr" />');continue}if(C.startsWith("> ")||C.startsWith(">")){p();let A=Vi(C.replace(/^>\s?/,""));i.push(`<blockquote>${A}</blockquote>`);continue}let P=_.match(/^(\s*)([\*\-])\s+(.*)$/);if(P){r!=="ul"&&(p(),r="ul",i.push("<ul>"));let A=P[1].length,E=Math.max(0,Math.floor((A-2)/2))*12,U=E?` style="margin-left: ${E}px"`:"",ne=P[3],J=ne.match(/^(verify|assert)(@end)?\s*:\s*(.*)$/i),X=ne.match(/^finding\s*:\s*(.*)$/i);if(J){let W=J[1].toLowerCase(),Me=J[2]?"@end":"",Ee=W==="assert"?" assert":"",ft=Vi(J[3].trim());i.push(`<li${U} class="md-verify-item"><span class="verify-badge${Ee}">${W}${Me}</span>${ft}</li>`)}else if(X){let W=Vi(X[1].trim());i.push(`<li${U} class="md-finding-item"><span class="finding-badge">finding</span>${W}</li>`)}else{let W=Vi(ne);i.push(`<li${U}>${W}</li>`)}continue}let T=_.match(/^(\s*)(\d+)\.\s+(.*)$/);if(T){r!=="ol"&&(p(),r="ol",i.push("<ol>"));let A=T[1].length,E=Math.max(0,Math.floor((A-2)/2))*12,U=E?` style="margin-left: ${E}px"`:"",ne=Vi(T[3]);i.push(`<li${U}>${ne}</li>`);continue}if(p(),C==="")i.push('<div class="md-para-gap"></div>');else{let A=Vi(_);i.push(`<div>${A}</div>`)}}if(a){let f=c.join(`
|
||||
`);i.push(`<pre class="code-block"><code class="lang-${s}">${f}</code></pre>`)}return p(),i.join(`
|
||||
`)}function Gr(n,t=0){if(n!=null){if(typeof n=="number")return n<1e11?n*1e3:n;let e=new Date(n).getTime();if(!isNaN(e)&&e>0)return e}return t}function KP(n,t){if(!t)return-1;let e=String(t);return n.findIndex(o=>o.type==="checker"&&o.data?.trace_id&&String(o.data.trace_id)===e)}function zT(n){for(let t=n.length-1;t>=0;t--)if(n[t].type==="checker"&&n[t].data?.isCompleted===!1)return t;return-1}function BT(n,t){let e=t.data||{};if(!e.event)return;if(e.event==="run_outcome"){let s="checker-outcome",c=n.findIndex(_=>_.id===s),p=F(S(S({},c>-1?n[c].data:{}),e),{phase:"outcome",status:"done",isCompleted:!0,generic_tools:[]}),f={id:s,type:"checker",timestamp:t.timestamp,data:p};c>-1?n[c]=f:n.push(f);return}if(!e.attempt_id)return;let o=`checker-${e.attempt_id}`,i=n.findIndex(s=>s.id===o),r=i>-1?n[i]:null,a=typeof e.ts=="number"?e.ts:t.timestamp?new Date(t.timestamp).getTime()/1e3:void 0;if(e.event==="attempt_started"){let s=F(S(S({},r?.data||{}),e),{status:r?.data?.isCompleted?r.data.status:"running",isCompleted:r?.data?.isCompleted??!1,started_at:r?.data?.started_at??a,verdicts:r?.data?.verdicts||[],findings:r?.data?.findings||[],generic_tools:r?.data?.generic_tools||[]}),c={id:o,type:"checker",timestamp:r?.timestamp||t.timestamp,data:s};r?n[i]=c:n.push(c);return}if(e.event==="attempt_finished"){let s=r?.data?.started_at??a,c=Array.isArray(e.verdicts)?e.verdicts:r?.data?.verdicts||[],p=r?.data?.items?.length?r.data.items:c.map(C=>({kind:C.kind,text:C.item_text,when:C.when})),f=F(S(S({},r?.data||{}),e),{status:e.status||"done",verdicts:c,items:p,findings:Array.isArray(e.findings)?e.findings:r?.data?.findings||[],generic_tools:r?.data?.generic_tools||[],started_at:s,finished_at:a,duration:s!==void 0&&a!==void 0?Math.max(0,a-s):void 0,isCompleted:!0}),_={id:o,type:"checker",timestamp:r?.timestamp||t.timestamp,data:f};r?n[i]=_:n.push(_)}}function HT(n){let t=new Map;n.forEach((e,o)=>{e.type==="checker"&&e.data?.trace_id&&t.set(String(e.data.trace_id),o)}),t.size!==0&&n.forEach((e,o)=>{if(e.type==="checker")return;let i=e.data?.generic_tools||[];if(i.length===0)return;let r=[],a=!1;for(let s of i){let c=s?.trace_id?String(s.trace_id):"",p=s?.parent_trace_id?String(s.parent_trace_id):"";if(c&&t.has(c)&&s?.type==="agent"){a=!0;continue}let f=p?t.get(p):void 0;if(f===void 0){r.push(s);continue}let _=n[f],C=_.data.generic_tools||[];C.some(h=>h.trace_id&&h.trace_id===s.trace_id)||(n[f]=F(S({},_),{data:F(S({},_.data),{generic_tools:[...C,s]})})),a=!0}a&&(n[o]=F(S({},e),{data:F(S({},e.data),{generic_tools:r})}))})}function ZP(n){if(!n||n.length===0)return[];let t=[];return n.forEach(e=>{if(e){if(e.type==="checker_event"){BT(t,e);return}if(e.type==="llm_stream"){let o=e.data?.execution_id||e.id,i=e.data?.step_id,r=e.data?.stream_type||"text",a=e.data?.text||"",s=e.data?.isCompleted??!1,c=KP(t,e.data?.parent_trace_id);if(c>-1){let f=t[c],_=S({},f.data),C=Array.isArray(_.stream_segments)?[..._.stream_segments]:[],h=C.findIndex(A=>A.execution_id===o&&A.stream_type===r),P={execution_id:o,stream_type:r==="thinking"?"thinking":"text",text:a,timestamp:h>-1?C[h].timestamp:e.timestamp,isCompleted:s};h>-1?C[h]=P:C.push(P),_.stream_segments=C;let T=A=>C.filter(N=>N.stream_type===A&&N.text.trim()).map(N=>N.text).join(`
|
||||
|
||||
`);_.operator_native_thinking=T("thinking"),_.operator_raw_thinking=T("text"),r==="thinking"?_.operator_native_thinking_timestamp=_.operator_native_thinking_timestamp||e.timestamp:_.operator_raw_thinking_timestamp=_.operator_raw_thinking_timestamp||e.timestamp,o&&!_.execution_id&&(_.execution_id=o),t[c]=F(S({},f),{data:_});return}let p=-1;if(i&&(p=t.findIndex(f=>f.id===`step-${i}`||f.data?.step_id===i)),p===-1&&o&&(p=t.findIndex(f=>f.data?.execution_id===o||f.id===`stream-${o}`)),p>-1){let f=t[p],_=S({},f.data);r==="thinking"?(_.operator_native_thinking=a,_.operator_native_thinking_timestamp=_.operator_native_thinking_timestamp||e.timestamp):(_.operator_raw_thinking=a,_.operator_raw_thinking_timestamp=_.operator_raw_thinking_timestamp||e.timestamp),i&&!_.step_id&&(_.step_id=i),o&&!_.execution_id&&(_.execution_id=o),_.isCompleted=s,t[p]=F(S({},f),{data:_})}else{let f=i?`step-${i}`:`stream-${o}`,_={execution_id:o,step_id:i,isCompleted:s,generic_tools:[]};r==="thinking"?(_.operator_native_thinking=a,_.operator_native_thinking_timestamp=e.timestamp):(_.operator_raw_thinking=a,_.operator_raw_thinking_timestamp=e.timestamp),t.push({id:f,type:i?"step":"llm_stream",timestamp:e.timestamp,data:_})}}else if(e.type==="step_recorded"||e.type==="step_updated"){let o=e.data.step_id||e.data.step_number||"unknown",i=t.findIndex(r=>r.id===`step-${o}`||r.data?.step_id===o);if(i>-1){let r=t[i],a=r.data.generic_tools||[],s=e.data.generic_tools||[],c=[...a];s.forEach(f=>{let _=c.findIndex(C=>C.trace_id&&f.trace_id&&C.trace_id===f.trace_id);_>-1?c[_]=S(S({},c[_]),f):c.push(f)});let p=F(S(S({},r.data),e.data),{operator_native_thinking:e.data.operator_native_thinking||r.data.operator_native_thinking||void 0,operator_raw_thinking:e.data.operator_raw_thinking||r.data.operator_raw_thinking||void 0,operator_native_thinking_timestamp:r.data.operator_native_thinking_timestamp||e.data.operator_native_thinking_timestamp||(e.data.operator_native_thinking?e.timestamp:void 0),operator_raw_thinking_timestamp:r.data.operator_raw_thinking_timestamp||e.data.operator_raw_thinking_timestamp||(e.data.operator_raw_thinking?e.timestamp:void 0),action_taken:e.data.action_taken||r.data.action_taken||void 0,last_execution_result:e.data.last_execution_result||r.data.last_execution_result||void 0,generic_tools:c,isCompleted:!0});t[i]=F(S({},r),{id:`step-${o}`,type:"step",timestamp:r.timestamp||e.timestamp,data:p})}else t.push({id:`step-${o}`,type:"step",timestamp:e.timestamp,data:F(S({},e.data),{isCompleted:!0,operator_native_thinking_timestamp:e.data.operator_native_thinking_timestamp||(e.data.operator_native_thinking?e.timestamp:void 0),operator_raw_thinking_timestamp:e.data.operator_raw_thinking_timestamp||(e.data.operator_raw_thinking?e.timestamp:void 0),generic_tools:e.data.generic_tools||[]})})}else if(e.type==="trace_recorded"){let o=e.data.type==="action",i=e.data.type==="tool"||o,r=e.data.type==="llm_call"&&(e.data.status==="failed"||e.data.status==="retrying");if(!i&&!r)return;let a=e.data.step_id,s=-1,c=KP(t,e.data.parent_trace_id);if(c===-1&&String(e.data.agent_name||"").toLowerCase()==="checker"&&(c=zT(t)),c>-1&&(s=c),s===-1&&e.data.trace_id&&(s=t.findIndex(p=>p.data?.generic_tools?.some(f=>f.trace_id===e.data.trace_id))),s===-1&&a&&(s=t.findIndex(p=>p.id===`step-${a}`||p.data?.step_id===a)),s===-1&&!a&&(s=t.findIndex(p=>p.id==="step-pre-planning"||p.data?.step_id==="pre-planning")),s===-1&&!a){let p=e.timestamp?new Date(e.timestamp).getTime():0,f=-1;for(let _=0;_<t.length;_++){let C=t[_],h=C.timestamp?new Date(C.timestamp).getTime():0;h<=p&&h>=f&&(f=h,s=_)}}if(s>-1){let p=t[s].data,f=p.generic_tools?[...p.generic_tools]:[],_=e.data.trace_id?f.findIndex(C=>C.trace_id===e.data.trace_id):-1;_>-1?f[_]=S(S({},f[_]),e.data):f.push(e.data),t[s]=F(S({},t[s]),{data:F(S({},p),{generic_tools:f})})}else{let p=a||"pre-planning";t.push({id:`step-${p}`,type:"step",timestamp:e.timestamp,data:{step_id:p,isCompleted:!0,generic_tools:[e.data]}})}}}}),HT(t),t.sort((e,o)=>{let i=e.timestamp?new Date(e.timestamp).getTime():0,r=o.timestamp?new Date(o.timestamp).getTime():0;if(Math.abs(i-r)>50)return i-r;let a=p=>p.data?.step_number!==void 0&&p.data.step_number!==null?Number(p.data.step_number):p.data?.step_id==="pre-planning"||p.data?.step_type==="planning"?0:99999,s=a(e),c=a(o);return s!==c&&s!==99999&&c!==99999?s-c:i-r}),t}var YP=new WeakMap;function UT(n){if(typeof n=="string")return{isImage:n.includes("data:image/")||n.includes("base64,"),length:n.length};if(n===null||typeof n!="object")return{isImage:!1,length:String(n).length};let t=YP.get(n);if(t)return t;let e="";try{e=JSON.stringify(n)||""}catch(i){e=String(n)}let o={isImage:e.includes("data:image/")||e.includes("base64,"),length:e.length};return YP.set(n,o),o}function GT(n){if(!n||!n.data)return{total:0,prompt:0,completion:0};let t=n.data;if(t.total_tokens!==void 0&&t.total_tokens!==null&&Number(t.total_tokens)>0){let s=t.token_usage?.prompt_tokens||t.token_usage?.input_tokens||0,c=t.token_usage?.complet
Line truncated
|
||||
@@ -0,0 +1,157 @@
|
||||
var Ay=Object.defineProperty,Ry=Object.defineProperties;var Fy=Object.getOwnPropertyDescriptors;var eh=Object.getOwnPropertySymbols;var Ny=Object.prototype.hasOwnProperty,Ly=Object.prototype.propertyIsEnumerable;var th=(n,t,e)=>t in n?Ay(n,t,{enumerable:!0,configurable:!0,writable:!0,value:e}):n[t]=e,x=(n,t)=>{for(var e in t||={})Ny.call(t,e)&&th(n,e,t[e]);if(eh)for(var e of eh(t))Ly.call(t,e)&&th(n,e,t[e]);return n},F=(n,t)=>Ry(n,Fy(t));var ke=(n,t,e)=>new Promise((o,i)=>{var r=c=>{try{s(e.next(c))}catch(u){i(u)}},a=c=>{try{s(e.throw(c))}catch(u){i(u)}},s=c=>c.done?o(c.value):Promise.resolve(c.value).then(r,a);s((e=e.apply(n,t)).next())});var It=null,nc=!1,ci=1,Vy=null,lt=Symbol("SIGNAL");function Y(n){let t=It;return It=n,t}function oc(){return It}var To={version:0,lastCleanEpoch:0,dirty:!1,producers:void 0,producersTail:void 0,consumers:void 0,consumersTail:void 0,recomputing:!1,consumerAllowSignalWrites:!1,consumerIsAlwaysLive:!1,kind:"unknown",producerMustRecompute:()=>!1,producerRecomputeValue:()=>{},consumerMarkedDirty:()=>{},consumerOnSignalRead:()=>{}};function ui(n){if(nc)throw new Error("");if(It===null)return;It.consumerOnSignalRead(n);let t=It.producersTail;if(t!==void 0&&t.producer===n)return;let e,o=It.recomputing;if(o&&(e=t!==void 0?t.nextProducer:It.producers,e!==void 0&&e.producer===n)){It.producersTail=e,e.lastReadVersion=n.version,e.knownValidAtEpoch=ci;return}let i=n.consumersTail;if(i!==void 0&&i.consumer===It&&(!o||i.knownValidAtEpoch===ci))return;let r=lr(It),a={producer:n,consumer:It,nextProducer:e,prevConsumer:void 0,knownValidAtEpoch:ci,lastReadVersion:n.version,nextConsumer:void 0};It.producersTail=a,t!==void 0?t.nextProducer=a:It.producers=a,r&&rh(n,a)}function nh(){ci++}function gi(n){if(!(lr(n)&&!n.dirty)&&!(!n.dirty&&n.lastCleanEpoch===ci)){if(!n.producerMustRecompute(n)&&!fa(n)){cr(n);return}n.producerRecomputeValue(n),cr(n)}}function Yu(n){if(n.consumers===void 0)return;let t=nc;nc=!0;try{for(let e=n.consumers;e!==void 0;e=e.nextConsumer){let o=e.consumer;o.dirty||By(o)}}finally{nc=t}}function Ku(){return It?.consumerAllowSignalWrites!==!1}function By(n){n.dirty=!0,Yu(n),n.consumerMarkedDirty?.(n)}function cr(n){n.dirty=!1,n.lastCleanEpoch=ci}function Ao(n){return n&&oh(n),Y(n)}function oh(n){if(n.producersTail?.knownValidAtEpoch===ci){let t=n.producers;for(;t!==void 0;)t.knownValidAtEpoch=null,t=t.nextProducer}n.producersTail=void 0,n.recomputing=!0}function pi(n,t){Y(t),n&&ih(n)}function ih(n){n.recomputing=!1;let t=n.producersTail,e=t!==void 0?t.nextProducer:n.producers;if(e!==void 0){if(lr(n))do e=Zu(e);while(e!==void 0);t!==void 0?t.nextProducer=void 0:n.producers=void 0}}function fa(n){for(let t=n.producers;t!==void 0;t=t.nextProducer){let e=t.producer,o=t.lastReadVersion;if(o!==e.version||(gi(e),o!==e.version))return!0}return!1}function fi(n){if(lr(n)){let t=n.producers;for(;t!==void 0;)t=Zu(t)}n.producers=void 0,n.producersTail=void 0,n.consumers=void 0,n.consumersTail=void 0}function rh(n,t){let e=n.consumersTail,o=lr(n);if(e!==void 0?(t.nextConsumer=e.nextConsumer,e.nextConsumer=t):(t.nextConsumer=void 0,n.consumers=t),t.prevConsumer=e,n.consumersTail=t,!o)for(let i=n.producers;i!==void 0;i=i.nextProducer)rh(i.producer,i)}function Zu(n){let t=n.producer,e=n.nextProducer,o=n.nextConsumer,i=n.prevConsumer;if(n.nextConsumer=void 0,n.prevConsumer=void 0,o!==void 0?o.prevConsumer=i:t.consumersTail=i,i!==void 0)i.nextConsumer=o;else if(t.consumers=o,!lr(t)){let r=t.producers;for(;r!==void 0;)r=Zu(r)}return e}function lr(n){return n.consumerIsAlwaysLive||n.consumers!==void 0}function ma(n){Vy?.(n)}function _a(n,t){return Object.is(n,t)}function ic(n,t){let e=Object.create(jy);e.computation=n,t!==void 0&&(e.equal=t);let o=()=>{if(gi(e),ui(e),e.value===zn)throw e.error;return e.value};return o[lt]=e,ma(e),o}var li=Symbol("UNSET"),di=Symbol("COMPUTING"),zn=Symbol("ERRORED"),jy=F(x({},To),{value:li,dirty:!0,error:null,equal:_a,kind:"computed",producerMustRecompute(n){return n.value===li||n.value===di},producerRecomputeValue(n){if(n.value===di)throw new Error("");let t=n.value;n.value=di;let e=Ao(n),o,i=!1;try{o=n.computation(),Y(null),i=t!==li&&t!==zn&&o!==zn&&n.equal(t,o)}catch(r){o=zn,n.error=r}finally{pi(n,e)}if(i){n.value=t;return}n.value=o,n.version++}});function zy(){throw new Error}var ah=zy;function sh(n){ah(n)}function Xu(n){ah=n}var $y=null;function Qu(n,t){let e=Object.create(ac);e.value=n,t!==void 0&&(e.equal=t);let o=()=>ch(e);return o[lt]=e,ma(e),[o,a=>mi(e,a),a=>rc(e,a)]}function ch(n){return ui(n),n.value}function mi(n,t){Ku()||sh(n),n.equal(n.value,t)||(n.value=t,Hy(n))}function rc(n,t){Ku()||sh(n),mi(n,t(n.value))}var ac=F(x({},To),{equal:_a,value:void 0,kind:"signal"});function Hy(n){n.version++,nh(),Yu(n),$y?.(n)}var Ju=F(x({},To),{consumerIsAlwaysLive:!0,consumerAllowSignalWrites:!0,dirty:!0,kind:"effect"});function eg(n){if(n.dirty=!1,n.version>0&&!fa(n))return;n.version++;let t=Ao(n);try{n.cleanup(),n.fn()}finally{pi(n,t)}}var tg;function sc(){return
Line truncated
|
||||
${e.map((o,i)=>`${i+1}) ${o.toString()}`).join(`
|
||||
`)}`:"",this.name="UnsubscriptionError",this.errors=e});function _i(n,t){if(n){let e=n.indexOf(t);0<=e&&n.splice(e,1)}}var Oe=class n{constructor(t){this.initialTeardown=t,this.closed=!1,this._parentage=null,this._finalizers=null}unsubscribe(){let t;if(!this.closed){this.closed=!0;let{_parentage:e}=this;if(e)if(this._parentage=null,Array.isArray(e))for(let r of e)r.remove(this);else e.remove(this);let{initialTeardown:o}=this;if(de(o))try{o()}catch(r){t=r instanceof cc?r.errors:[r]}let{_finalizers:i}=this;if(i){this._finalizers=null;for(let r of i)try{gh(r)}catch(a){t=t??[],a instanceof cc?t=[...t,...a.errors]:t.push(a)}}if(t)throw new cc(t)}}add(t){var e;if(t&&t!==this)if(this.closed)gh(t);else{if(t instanceof n){if(t.closed||t._hasParent(this))return;t._addParent(this)}(this._finalizers=(e=this._finalizers)!==null&&e!==void 0?e:[]).push(t)}}_hasParent(t){let{_parentage:e}=this;return e===t||Array.isArray(e)&&e.includes(t)}_addParent(t){let{_parentage:e}=this;this._parentage=Array.isArray(e)?(e.push(t),e):e?[e,t]:t}_removeParent(t){let{_parentage:e}=this;e===t?this._parentage=null:Array.isArray(e)&&_i(e,t)}remove(t){let{_finalizers:e}=this;e&&_i(e,t),t instanceof n&&t._removeParent(this)}};Oe.EMPTY=(()=>{let n=new Oe;return n.closed=!0,n})();var ig=Oe.EMPTY;function lc(n){return n instanceof Oe||n&&"closed"in n&&de(n.remove)&&de(n.add)&&de(n.unsubscribe)}function gh(n){de(n)?n():n.unsubscribe()}var vn={onUnhandledError:null,onStoppedNotification:null,Promise:void 0,useDeprecatedSynchronousErrorHandling:!1,useDeprecatedNextContext:!1};var gr={setTimeout(n,t,...e){let{delegate:o}=gr;return o?.setTimeout?o.setTimeout(n,t,...e):setTimeout(n,t,...e)},clearTimeout(n){let{delegate:t}=gr;return(t?.clearTimeout||clearTimeout)(n)},delegate:void 0};function dc(n){gr.setTimeout(()=>{let{onUnhandledError:t}=vn;if(t)t(n);else throw n})}function ha(){}var ph=rg("C",void 0,void 0);function fh(n){return rg("E",void 0,n)}function mh(n){return rg("N",n,void 0)}function rg(n,t,e){return{kind:n,value:t,error:e}}var hi=null;function pr(n){if(vn.useDeprecatedSynchronousErrorHandling){let t=!hi;if(t&&(hi={errorThrown:!1,error:null}),n(),t){let{errorThrown:e,error:o}=hi;if(hi=null,e)throw o}}else n()}function _h(n){vn.useDeprecatedSynchronousErrorHandling&&hi&&(hi.errorThrown=!0,hi.error=n)}var Ci=class extends Oe{constructor(t){super(),this.isStopped=!1,t?(this.destination=t,lc(t)&&t.add(this)):this.destination=qy}static create(t,e,o){return new lo(t,e,o)}next(t){this.isStopped?sg(mh(t),this):this._next(t)}error(t){this.isStopped?sg(fh(t),this):(this.isStopped=!0,this._error(t))}complete(){this.isStopped?sg(ph,this):(this.isStopped=!0,this._complete())}unsubscribe(){this.closed||(this.isStopped=!0,super.unsubscribe(),this.destination=null)}_next(t){this.destination.next(t)}_error(t){try{this.destination.error(t)}finally{this.unsubscribe()}}_complete(){try{this.destination.complete()}finally{this.unsubscribe()}}},Wy=Function.prototype.bind;function ag(n,t){return Wy.call(n,t)}var cg=class{constructor(t){this.partialObserver=t}next(t){let{partialObserver:e}=this;if(e.next)try{e.next(t)}catch(o){uc(o)}}error(t){let{partialObserver:e}=this;if(e.error)try{e.error(t)}catch(o){uc(o)}else uc(t)}complete(){let{partialObserver:t}=this;if(t.complete)try{t.complete()}catch(e){uc(e)}}},lo=class extends Ci{constructor(t,e,o){super();let i;if(de(t)||!t)i={next:t??void 0,error:e??void 0,complete:o??void 0};else{let r;this&&vn.useDeprecatedNextContext?(r=Object.create(t),r.unsubscribe=()=>this.unsubscribe(),i={next:t.next&&ag(t.next,r),error:t.error&&ag(t.error,r),complete:t.complete&&ag(t.complete,r)}):i=t}this.destination=new cg(i)}};function uc(n){vn.useDeprecatedSynchronousErrorHandling?_h(n):dc(n)}function Gy(n){throw n}function sg(n,t){let{onStoppedNotification:e}=vn;e&&gr.setTimeout(()=>e(n,t))}var qy={closed:!0,next:ha,error:Gy,complete:ha};var fr=typeof Symbol=="function"&&Symbol.observable||"@@observable";function yn(n){return n}function gc(...n){return lg(n)}function lg(n){return n.length===0?yn:n.length===1?n[0]:function(e){return n.reduce((o,i)=>i(o),e)}}var te=class n{constructor(t){t&&(this._subscribe=t)}lift(t){let e=new n;return e.source=this,e.operator=t,e}subscribe(t,e,o){let i=Ky(t)?t:new lo(t,e,o);return pr(()=>{let{operator:r,source:a}=this;i.add(r?r.call(i,a):a?this._subscribe(i):this._trySubscribe(i))}),i}_trySubscribe(t){try{return this._subscribe(t)}catch(e){t.error(e)}}forEach(t,e){return e=hh(e),new e((o,i)=>{let r=new lo({next:a=>{try{t(a)}catch(s){i(s),r.unsubscribe()}},error:i,complete:o});this.subscribe(r)})}_subscribe(t){var e;return(e=this.source)===null||e===void 0?void 0:e.subscribe(t)}[fr](){return this}pipe(...t){return lg(t)(this)}toPromise(t){return t=hh(t),new t((e,o)=>{let i;this.subscribe(r=>i=r,r=>o(r),()=>e(i))})}};te.create=n=>new te(n);function hh(n){var t;return(t=n??vn.Promise)!==null&&t!==void 0?t:Promise}function Yy(n){return n&&de(n.next)&&de(n.error)&&de(n.complete)}function Ky(n){return n&&nLine truncated
|
||||
`);return o>=0?e.slice(0,o):e}function Gc(n,t){return n?t?`${n} ${t}`:n:t||""}var fx=we({__forward_ref__:we});function yt(n){return n.__forward_ref__=yt,n}function Mt(n){return Ig(n)?n():n}function Ig(n){return typeof n=="function"&&Object.hasOwn(n,fx)&&n.__forward_ref__===yt}function ee(n){return{token:n.token,providedIn:n.providedIn||null,factory:n.factory,value:void 0}}function at(n){return{providers:n.providers||[],imports:n.imports||[]}}function Ea(n){return mx(n,qc)}function Tg(n){return Ea(n)!==null}function mx(n,t){return Object.hasOwn(n,t)&&n[t]||null}function _x(n){let t=n?.[qc]??null;return t||null}function Mg(n){return n&&Object.hasOwn(n,jc)?n[jc]:null}var qc=we({\u0275prov:we}),jc=we({\u0275inj:we}),$=class{_desc;ngMetadataName="InjectionToken";\u0275prov;constructor(t,e){this._desc=t,this.\u0275prov=void 0,typeof e=="number"?this.__NG_ELEMENT_ID__=e:e!==void 0&&(this.\u0275prov=ee({token:this,providedIn:e.providedIn||"root",factory:e.factory}))}get multi(){return this}toString(){return`InjectionToken ${this._desc}`}};function Ag(n){return n&&!!n.\u0275providers}var Da=we({\u0275cmp:we}),Ia=we({\u0275dir:we}),Rg=we({\u0275pipe:we}),Fg=we({\u0275mod:we}),ya=we({\u0275fac:we}),wi=we({__NG_ELEMENT_ID__:we}),zh=we({__NG_ENV_ID__:we});function qh(n){return Yc(n,"@NgModule"),n[Fg]||null}function Lo(n){return Yc(n,"@Component"),n[Da]||null}function Ng(n){return Yc(n,"@Directive"),n[Ia]||null}function Yh(n){return Yc(n,"@Pipe"),n[Rg]||null}function Yc(n,t){if(n==null)throw new j(-919,!1)}function wn(n){return typeof n=="string"?n:n==null?"":String(n)}var Kh=we({ngErrorCode:we}),hx=we({ngErrorMessage:we}),Cx=we({ngTokenPath:we});function Lg(n,t){return Zh("",-200,t)}function Kc(n,t){throw new j(-201,!1)}function Zh(n,t,e){let o=new j(t,n);return o[Kh]=t,o[hx]=n,e&&(o[Cx]=e),o}function Mx(n){return n[Kh]}var bg;function Xh(){return bg}function Rt(n){let t=bg;return bg=n,t}function Vg(n,t,e){let o=Ea(n);if(o&&o.providedIn=="root")return o.value===void 0?o.value=o.factory():o.value;if(e&8)return null;if(t!==void 0)return t;Kc(n,"")}var Sn=globalThis;var bx={},Oi=bx,Px="__NG_DI_FLAG__",Pg=class{injector;constructor(t){this.injector=t}retrieve(t,e){let o=vi(e)||0;try{return this.injector.get(t,o&8?null:Oi,o)}catch(i){if(dr(i))return i;throw i}}};function Ox(n,t=0){let e=sc();if(e===void 0)throw new j(-203,!1);if(e===null)return Vg(n,void 0,t);{let o=vx(t),i=e.retrieve(n,o);if(dr(i)){if(o.optional)return null;throw i}return i}}function Q(n,t=0){return(Xh()||Ox)(Mt(n),t)}function v(n,t){return Q(n,vi(t))}function vi(n){return typeof n>"u"||typeof n=="number"?n:0|(n.optional&&8)|(n.host&&1)|(n.self&&2)|(n.skipSelf&&4)}function vx(n){return{optional:!!(n&8),host:!!(n&1),self:!!(n&2),skipSelf:!!(n&4)}}function Og(n){let t=[];for(let e=0;e<n.length;e++){let o=Mt(n[e]);if(Array.isArray(o)){if(o.length===0)throw new j(900,!1);let i,r=0;for(let a=0;a<o.length;a++){let s=o[a],c=yx(s);typeof c=="number"?c===-1?i=s.token:r|=c:i=s}t.push(Q(i,r))}else t.push(Q(o))}return t}function yx(n){return n[Px]}function No(n,t){let e=Object.hasOwn(n,ya);return e?n[ya]:null}function Qh(n,t,e){if(n.length!==t.length)return!1;for(let o=0;o<n.length;o++){let i=n[o],r=t[o];if(e&&(i=e(i),r=e(r)),r!==i)return!1}return!0}function Jh(n){return n.flat(Number.POSITIVE_INFINITY)}function Zc(n,t){n.forEach(e=>Array.isArray(e)?Zc(e,t):t(e))}function Bg(n,t,e){t>=n.length?n.push(e):n.splice(t,0,e)}function Ta(n,t){return t>=n.length-1?n.pop():n.splice(t,1)[0]}function eC(n,t,e,o){let i=n.length;if(i==t)n.push(e,o);else if(i===1)n.push(o,n[0]),n[0]=e;else{for(i--,n.push(n[i-1],n[i]);i>t;){let r=i-2;n[i]=n[r],i--}n[t]=e,n[t+1]=o}}function Xc(n,t,e){let o=Or(n,t);return o>=0?n[o|1]=e:(o=~o,eC(n,o,t,e)),o}function Qc(n,t){let e=Or(n,t);if(e>=0)return n[e|1]}function Or(n,t){return xx(n,t,1)}function xx(n,t,e){let o=0,i=n.length>>e;for(;i!==o;){let r=o+(i-o>>1),a=n[r<<e];if(t===a)return r<<e;a>t?i=r:o=r+1}return~(i<<e)}var Vo={},Ft=[],Si=new $(""),Aa=new $("",-1),jg=new $(""),br=class{get(t,e=Oi){if(e===Oi){let i=Zh("",-201);throw i.name="\u0275NotFound",i}return e}};function Bo(n){return{\u0275providers:n}}function tC(...n){return{\u0275providers:zg(!0,n),\u0275fromNgModule:!0}}function zg(n,...t){let e=[],o=new Set,i,r=a=>{e.push(a)};return Zc(t,a=>{let s=a;zc(s,r,[],o)&&(i||=[],i.push(s))}),i!==void 0&&nC(i,r),e}function nC(n,t){for(let e=0;e<n.length;e++){let{ngModule:o,providers:i}=n[e];$g(i,r=>{t(r,o)})}}function zc(n,t,e,o){if(n=Mt(n),!n)return!1;let i=null,r=Mg(n),a=!r&&Lo(n);if(!r&&!a){let c=n.ngModule;if(r=Mg(c),r)i=c;else return!1}else{if(a&&!a.standalone)return!1;i=n}let s=o.has(i);if(a){if(s)return!1;if(o.add(i),a.dependencies){let c=typeof a.dependencies=="function"?a.dependencies():a.dependencies;for(let u of c)zc(u,t,e,o)}}else if(r){if(r.imports!=null&&!s){o.add(i);let u;Zc(r.imports,f=>{zc(f,t,e,o)&&(u||=[],u.push(f))}),u!==void 0&&nC(u,t)}if(!s){let u=No(i)||(()=>new i);t({provide:i,useFactory:u,deps:Ft},i),t({provide:jg,useValue:i,
Line truncated
|
||||
`).forEach(e=>{let o=e.indexOf(":");if(o>0){let i=e.slice(0,o),r=e.slice(o+1).trim();this.addHeaderEntry(i,r)}})}:typeof Headers<"u"&&t instanceof Headers?(this.headers=new Map,t.forEach((e,o)=>{this.addHeaderEntry(o,e)})):this.lazyInit=()=>{this.headers=new Map,Object.entries(t).forEach(([e,o])=>{this.setHeaderEntries(e,o)})}:this.headers=new Map}has(t){return this.init(),this.headers.has(t.toLowerCase())}get(t){this.init();let e=this.headers.get(t.toLowerCase());return e&&e.length>0?e[0]:null}keys(){return this.init(),Array.from(this.normalizedNames.values())}getAll(t){return this.init(),this.headers.get(t.toLowerCase())||null}append(t,e){return this.clone({name:t,value:e,op:"a"})}set(t,e){return this.clone({name:t,value:e,op:"s"})}delete(t,e){return this.clone({name:t,value:e,op:"d"})}maybeSetNormalizedName(t,e){this.normalizedNames.has(e)||this.normalizedNames.set(e,t)}init(){this.lazyInit&&(this.lazyInit instanceof n?this.copyFrom(this.lazyInit):this.lazyInit(),this.lazyInit=null,this.lazyUpdate&&(this.lazyUpdate.forEach(t=>this.applyUpdate(t)),this.lazyUpdate=null))}copyFrom(t){t.init();for(let[e,o]of t.headers.entries())this.headers.set(e,o),this.normalizedNames.set(e,t.normalizedNames.get(e))}clone(t){let e=new n;return e.lazyInit=this.lazyInit&&this.lazyInit instanceof n?this.lazyInit:this,e.lazyUpdate=(this.lazyUpdate||[]).concat([t]),e}applyUpdate(t){let e=t.name.toLowerCase();switch(t.op){case"a":case"s":let o=t.value;if(typeof o=="string"&&(o=[o]),o.length===0)return;this.maybeSetNormalizedName(t.name,e);let i=t.op==="a"?(this.headers.get(e)||[]).slice():[];i.push(...o),this.headers.set(e,i);break;case"d":let r=t.value;if(r===void 0)this.headers.delete(e),this.normalizedNames.delete(e);else{let a=Array.isArray(r)?r:[r],s=this.headers.get(e);if(!s)return;s=s.filter(c=>a.indexOf(c)===-1),s.length===0?(this.headers.delete(e),this.normalizedNames.delete(e)):this.headers.set(e,s)}break}}addHeaderEntry(t,e){let o=t.toLowerCase();this.maybeSetNormalizedName(t,o),this.headers.has(o)?this.headers.get(o).push(e):this.headers.set(o,[e])}setHeaderEntries(t,e){let o=(Array.isArray(e)?e:[e]).map(r=>r.toString()),i=t.toLowerCase();this.headers.set(i,o),this.maybeSetNormalizedName(t,i)}forEach(t){this.init(),Array.from(this.normalizedNames.keys()).forEach(e=>t(this.normalizedNames.get(e),this.headers.get(e)))}};var Sd=class{map=new Map;set(t,e){return this.map.set(t,e),this}get(t){return this.map.has(t)||this.map.set(t,t.defaultValue()),this.map.get(t)}delete(t){return this.map.delete(t),this}has(t){return this.map.has(t)}keys(){return this.map.keys()}},kd=class{encodeKey(t){return cP(t)}encodeValue(t){return cP(t)}decodeKey(t){return decodeURIComponent(t)}decodeValue(t){return decodeURIComponent(t)}};function WE(n,t){let e=new Map;return n.length>0&&n.replace(/^\?/,"").split("&").forEach(i=>{let r=i.indexOf("="),[a,s]=r==-1?[t.decodeKey(i),""]:[t.decodeKey(i.slice(0,r)),t.decodeValue(i.slice(r+1))],c=e.get(a)||[];c.push(s),e.set(a,c)}),e}var GE=/%(\d[a-f0-9])/gi,qE={40:"@","3A":":",24:"$","2C":",","3B":";","3D":"=","3F":"?","2F":"/"};function cP(n){return encodeURIComponent(n).replace(GE,(t,e)=>qE[e]??t)}function wd(n){return`${n}`}var yo=class n{map;encoder;updates=null;cloneFrom=null;constructor(t={}){if(this.encoder=t.encoder||new kd,t.fromString){if(t.fromObject)throw new j(2805,!1);this.map=WE(t.fromString,this.encoder)}else t.fromObject?(this.map=new Map,Object.keys(t.fromObject).forEach(e=>{let o=t.fromObject[e],i=Array.isArray(o)?o.map(wd):[wd(o)];this.map.set(e,i)})):this.map=null}has(t){return this.init(),this.map.has(t)}get(t){this.init();let e=this.map.get(t);return e?e[0]:null}getAll(t){return this.init(),this.map.get(t)||null}keys(){return this.init(),Array.from(this.map.keys())}append(t,e){return this.clone({param:t,value:e,op:"a"})}appendAll(t){let e=[];return Object.keys(t).forEach(o=>{let i=t[o];Array.isArray(i)?i.forEach(r=>{e.push({param:o,value:r,op:"a"})}):e.push({param:o,value:i,op:"a"})}),this.clone(e)}set(t,e){return this.clone({param:t,value:e,op:"s"})}delete(t,e){return this.clone({param:t,value:e,op:"d"})}toString(){return this.init(),this.keys().map(t=>{let e=this.encoder.encodeKey(t);return this.map.get(t).map(o=>e+"="+this.encoder.encodeValue(o)).join("&")}).filter(t=>t!=="").join("&")}clone(t){let e=new n({encoder:this.encoder});return e.cloneFrom=this.cloneFrom||this,e.updates=(this.updates||[]).concat(t),e}init(){if(this.map===null&&(this.map=new Map),this.cloneFrom!==null){this.cloneFrom.init();for(let[t,e]of this.cloneFrom.map.entries())this.map.set(t,e);this.updates.forEach(t=>{switch(t.op){case"a":case"s":let e=t.op==="a"?(this.map.get(t.param)||[]).slice():[];e.push(wd(t.value)),this.map.set(t.param,e);break;case"d":if(t.value!==void 0){let o=(this.map.get(t.param)||[]).slice(),i=o.indexOf(wd(t.value));i!==-1&&o.splice(i,1),o.length>0?this.map.set(t.param,o):this.map.delete(t.param)}else{this.map.delete(t.param);break}}}),this.cloneFrom=this.updates=null}}};function YE(
Line truncated
|
||||
|
||||
`);n.operator_native_thinking=o("thinking"),n.operator_raw_thinking=o("text");let i=e("thinking")[0],r=e("text")[0];i&&!n.operator_native_thinking_timestamp&&(n.operator_native_thinking_timestamp=i.timestamp),r&&!n.operator_raw_thinking_timestamp&&(n.operator_raw_thinking_timestamp=r.timestamp)}function oA(n,t){let e=t.data||{};if(!e.event)return;if(e.event==="run_outcome"){let s="checker-outcome",c=n.findIndex(_=>_.id===s),u=F(x(x({},c>-1?n[c].data:{}),e),{phase:"outcome",status:"done",isCompleted:!0,generic_tools:[]}),f={id:s,type:"checker",timestamp:t.timestamp,data:u};c>-1?n[c]=f:n.push(f);return}if(!e.attempt_id)return;let o=`checker-${e.attempt_id}`,i=n.findIndex(s=>s.id===o),r=i>-1?n[i]:null,a=typeof e.ts=="number"?e.ts:t.timestamp?new Date(t.timestamp).getTime()/1e3:void 0;if(e.event==="attempt_started"){let s=F(x(x({},r?.data||{}),e),{status:r?.data?.isCompleted?r.data.status:"running",isCompleted:r?.data?.isCompleted??!1,started_at:r?.data?.started_at??a,verdicts:r?.data?.verdicts||[],findings:r?.data?.findings||[],generic_tools:r?.data?.generic_tools||[]}),c={id:o,type:"checker",timestamp:r?.timestamp||t.timestamp,data:s};r?n[i]=c:n.push(c);return}if(e.event==="attempt_finished"){let s=r?.data?.started_at??a,c=Array.isArray(e.verdicts)?e.verdicts:r?.data?.verdicts||[],u=r?.data?.items?.length?r.data.items:c.map(C=>({kind:C.kind,text:C.item_text,when:C.when})),f=nA(r?.data?.stream_segments,e.stream_segments),_=F(x(x({},r?.data||{}),e),{status:e.status||"done",verdicts:c,items:u,findings:Array.isArray(e.findings)?e.findings:r?.data?.findings||[],generic_tools:r?.data?.generic_tools||[],stream_segments:f,started_at:s,finished_at:a,duration:s!==void 0&&a!==void 0?Math.max(0,a-s):void 0,isCompleted:!0});f&&jv(_,f);let h={id:o,type:"checker",timestamp:r?.timestamp||t.timestamp,data:_};r?n[i]=h:n.push(h)}}function iA(n){let t=new Map;n.forEach((e,o)=>{e.type==="checker"&&e.data?.trace_id&&t.set(String(e.data.trace_id),o)}),t.size!==0&&n.forEach((e,o)=>{if(e.type==="checker")return;let i=e.data?.generic_tools||[];if(i.length===0)return;let r=[],a=!1;for(let s of i){let c=s?.trace_id?String(s.trace_id):"",u=s?.parent_trace_id?String(s.parent_trace_id):"";if(c&&t.has(c)&&s?.type==="agent"){a=!0;continue}let f=u?t.get(u):void 0;if(f===void 0){r.push(s);continue}let _=n[f],h=_.data.generic_tools||[];h.some(C=>C.trace_id&&C.trace_id===s.trace_id)||(n[f]=F(x({},_),{data:F(x({},_.data),{generic_tools:[...h,s]})})),a=!0}a&&(n[o]=F(x({},e),{data:F(x({},e.data),{generic_tools:r})}))})}function k_(n){return n?.type==="log"&&n?.name==="hierarchy_backend"&&!!n?.payload?.previous_backend}function zv(n){if(!n||n.length===0)return[];let t=[];return n.forEach(e=>{if(e){if(e.type==="checker_event"){oA(t,e);return}if(e.type==="llm_stream"){let o=e.data?.execution_id||e.id,i=e.data?.step_id,r=e.data?.stream_type||"text",a=e.data?.text||"",s=e.data?.isCompleted??!1,c=e.data?.isReset??!1,u=e.data?.resetMessage,f=Nv(t,e.data?.parent_trace_id);if(f>-1){let h=t[f],C=x({},h.data),O=Array.isArray(C.stream_segments)?[...C.stream_segments]:[],P=O.findIndex(R=>R.execution_id===o&&R.stream_type===r),T={execution_id:o,stream_type:r==="thinking"?"thinking":"text",text:a,timestamp:P>-1?O[P].timestamp:e.timestamp,isCompleted:s,isReset:c,resetMessage:u};P>-1?O[P]=T:O.push(T),C.stream_segments=O,r==="thinking"?C.operator_native_thinking_timestamp=C.operator_native_thinking_timestamp||e.timestamp:C.operator_raw_thinking_timestamp=C.operator_raw_thinking_timestamp||e.timestamp,jv(C,O),o&&!C.execution_id&&(C.execution_id=o),t[f]=F(x({},h),{data:C});return}let _=-1;if(i&&(_=t.findIndex(h=>h.id===`step-${i}`||h.data?.step_id===i)),_===-1&&o&&(_=t.findIndex(h=>h.data?.execution_id===o||h.id===`stream-${o}`)),_===-1&&!i&&(_=t.findIndex(h=>h.type==="llm_stream"&&h.data?.isReset)),_>-1){let h=t[_],C=x({},h.data),O=Array.isArray(C.stream_resets)?[...C.stream_resets]:[];if(c){C.isReset=!0,C.resetMessage=u;let P=u||Do,T=o||`reset-${e.timestamp||C.step_id||"stream"}`,R=O.findIndex(D=>D.id===T);R>-1?O[R]={id:T,message:P,isWaiting:!0,streamType:r}:O.push({id:T,message:P,isWaiting:!0,streamType:r}),C.stream_resets=O}else o&&C.execution_id!==o?(C.stream_resets=O.map(P=>F(x({},P),{isWaiting:!1})),C.isReset=!1,C.resetMessage=void 0,C.execution_id=o):!c&&a&&a.length>0&&O.some(P=>P.isWaiting)&&(C.stream_resets=O.map(P=>F(x({},P),{isWaiting:!1})),C.isReset=!1,C.resetMessage=void 0);r==="thinking"?(C.operator_native_thinking=a,C.operator_native_thinking_timestamp=C.operator_native_thinking_timestamp||e.timestamp):(C.operator_raw_thinking=a,C.operator_raw_thinking_timestamp=C.operator_raw_thinking_timestamp||e.timestamp),i&&!C.step_id&&(C.step_id=i),o&&!C.execution_id&&(C.execution_id=o),C.isCompleted=s,s&&C.stream_resets&&(C.stream_resets=C.stream_resets.map(P=>F(x({},P),{isWaiting:!1}))),t[_]=F(x({},h),{data:C})}else{let h=i?`step-${i}`:`stream-${o}`,C={execution_id:o,step_id:i,isCompleted:s,isReset:c,resetMessage:u,stream_resets:c?[{id:o||`reset-${e.timestamp|Line truncated
|
||||
`)||"","boot_logs"))}),l(8,"span",29),g(9),d(),l(10,"span"),g(11),d()()(),l(12,"pre",123),M(13,UA,2,0)(14,WA,2,0,"div",124),d()()}if(n&2){let e=m(3);p(9),y(e.copiedId()==="boot_logs"?"check":"content_copy"),p(2),y(e.copiedId()==="boot_logs"?"Copied":"Copy Logs"),p(2),b(e.emulatorLaunchState()?.logs?.length?13:14)}}function qA(n,t){if(n&1){let e=B();l(0,"div",47)(1,"div",89)(2,"div",90)(3,"div",91)(4,"div",92)(5,"span",93),g(6,"sync"),d()(),l(7,"div",94)(8,"div",95)(9,"h4",96),g(10," \u6B63\u5728\u542F\u52A8 Android \u6A21\u62DF\u5668\uFF1A "),l(11,"span",97),g(12),d()(),l(13,"span",98),g(14),d()(),l(15,"p",99),g(16),d()()(),l(17,"div",100)(18,"button",101),I("click",function(){S(e);let i=m(2);return k(i.toggleLaunchLogs())}),l(19,"span",29),g(20,"terminal"),d(),l(21,"span"),g(22),d()(),l(23,"button",102),I("click",function(){S(e);let i=m(2);return k(i.stopEmulator())}),l(24,"span",29),g(25,"cancel"),d(),l(26,"span"),g(27,"\u505C\u6B62"),d()()()(),l(28,"div",103)(29,"div",104)(30,"div",105),M(31,FA,2,0,"span",106)(32,NA,1,0,"span",107),d(),l(33,"span",108),g(34,"1. \u542F\u52A8\u8FDB\u7A0B"),d()(),z(35,"div",109),l(36,"div",104)(37,"div",105),M(38,LA,2,0,"span",106)(39,VA,1,0,"span",107)(40,BA,2,0,"span"),d(),l(41,"span",108),g(42,"2. ADB \u8FDE\u63A5"),d()(),z(43,"div",109),l(44,"div",104)(45,"div",105),M(46,jA,2,0,"span",106)(47,zA,1,0,"span",107)(48,$A,2,0,"span"),d(),l(49,"span",108),g(50,"3. \u7CFB\u7EDF\u542F\u52A8"),d()(),z(51,"div",109),l(52,"div",104)(53,"div",105)(54,"span",106),g(55,"check_circle"),d()(),l(56,"span",108),g(57,"4. \u5C31\u7EEA"),d()()(),l(58,"div",110)(59,"div",111),z(60,"div",112),d(),l(61,"div",113)(62,"span",114),g(63),d(),l(64,"span",115)(65,"span",116),g(66,"timer"),d(),g(67),d()()(),l(68,"div",117)(69,"span",118),g(70,"info"),d(),l(71,"span"),g(72,"Android \u6A21\u62DF\u5668\u51B7\u542F\u52A8\u901A\u5E38\u9700\u8981 20\u201345 \u79D2\u3002\u4E0A\u65B9\u4F1A\u663E\u793A\u5B9E\u65F6\u8FDB\u5EA6\uFF0C\u8BF7\u52FF\u91CD\u590D\u70B9\u51FB\u542F\u52A8\u3002"),d()(),M(73,GA,15,3,"div",119),d()()}if(n&2){let e=m(2);p(12),y(e.emulatorLaunchState()?.avd_name||e.activeDevice()?.serial||"\u865A\u62DF\u8BBE\u5907"),p(2),y(e.emulatorLaunchState()?.status==="booting"?"OS Booting":"Starting"),p(2),y(e.emulatorLaunchState()?.stage_message||"Initializing Android Virtual Device..."),p(2),E("active",e.showLaunchLogs()),p(4),y(e.showLaunchLogs()?"Hide Logs":"View Boot Logs"),p(7),E("done",e.emulatorLaunchState()?.progress_percent>=35)("active",e.emulatorLaunchState()?.status==="starting"),p(2),b(e.emulatorLaunchState()?.progress_percent>=35?31:32),p(4),E("done",e.emulatorLaunchState()?.progress_percent>=35),p(),E("done",e.emulatorLaunchState()?.progress_percent>=65)("active",e.emulatorLaunchState()?.status==="waiting_for_adb"),p(2),b(e.emulatorLaunchState()?.progress_percent>=65?38:e.emulatorLaunchState()?.status==="waiting_for_adb"?39:40),p(5),E("done",e.emulatorLaunchState()?.progress_percent>=65),p(),E("done",e.emulatorLaunchState()?.status==="ready")("active",e.emulatorLaunchState()?.status==="booting"||e.adbProbe()?.summary==="Device Booting"),p(2),b(e.emulatorLaunchState()?.status==="ready"?46:e.emulatorLaunchState()?.status==="booting"||e.adbProbe()?.summary==="Device Booting"?47:48),p(5),E("done",e.emulatorLaunchState()?.status==="ready"),p(),E("done",e.emulatorLaunchState()?.status==="ready"),p(8),Ie("width",e.emulatorLaunchState()?.progress_percent||25,"%"),p(3),L("",e.emulatorLaunchState()?.progress_percent||25,"% Complete"),p(4),L(" Elapsed: ",e.emulatorLaunchState()?.elapsed_seconds||0,"s "),p(6),b(e.showLaunchLogs()?73:-1)}}function YA(n,t){if(n&1&&(l(0,"div",125),g(1),d()),n&2){let e=t.$implicit;p(),y(e)}}function KA(n,t){if(n&1){let e=B();l(0,"div",135)(1,"div",120)(2,"span"),g(3,"\u8FDB\u7A0B\u8F93\u51FA\u4E0E\u9519\u8BEF\u65E5\u5FD7\uFF1A"),d(),l(4,"button",122),I("click",function(){S(e);let i=m(3);return k(i.copyToClipboard(i.emulatorLaunchState()?.logs?.join(`
|
||||
`)||"","fail_logs"))}),l(5,"span",29),g(6),d(),l(7,"span"),g(8),d()()(),l(9,"pre",140),W(10,YA,2,1,"div",125,ze),d()()}if(n&2){let e=m(3);p(6),y(e.copiedId()==="fail_logs"?"check":"content_copy"),p(2),y(e.copiedId()==="fail_logs"?"Copied":"Copy Logs"),p(2),G(e.emulatorLaunchState()?.logs)}}function ZA(n,t){if(n&1){let e=B();l(0,"button",141),I("click",function(){S(e);let i=m(3);return k(i.launchAvdEmulator(i.emulatorLaunchState().avd_name))}),l(1,"span",29),g(2,"refresh"),d(),l(3,"span"),g(4),d()()}if(n&2){let e=m(3);N("disabled",e.isRemoteAdbServer()),p(4),L("Retry Launch (",e.emulatorLaunchState()?.avd_name,")")}}function XA(n,t){if(n&1){let e=B();l(0,"div",48)(1,"div",126)(2,"div",127)(3,"div",128)(4,"div",129)(5,"span",130),g(6,"error"),d()(),l(7,"div",131)(8,"h4",132),g(9,"\u6A21\u62DF\u5668\u542F\u52A8\u5931\u8D25"),d(),l(10,"p",133),g(11),d()()(),l(12,"button",134),I("click",function(){S(e);let i=m(2);return k(i.dismissEmulatorStatus())}),l(13,"span",29),g(14,"close"),d()()(),M(15,KA,12,2,"div",135),l(16,"div",136),M(17,ZA,5,2,"button",137),l(18,"button",138),I("click",function(){S(e);let i=m(2);return k(i.restartAdbServer())}),l(19,"span",29),g(20,"sync"),d(),l(21,"span"),g(22,"\u91CD\u542F ADB \u670D\u52A1\u5668"),d()(),l(23,"button",139),I("click",function(){S(e);let i=m(2);return k(i.dismissEmulatorStatus())}),l(24,"span"),g(25,"\u5173\u95ED\u63D0\u793A"),d()()()()()}if(n&2){let e=m(2);p(11),y(e.emulatorLaunchState()?.error||"The Android emulator process failed to start or crashed during initialization."),p(4),b(e.emulatorLaunchState()?.logs?.length?15:-1),p(2),b(e.emulatorLaunchState()?.avd_name?17:-1)}}function QA(n,t){if(n&1&&g(0),n&2){let e=m(3);L(" ",e.installedAvds().length," \u4E2A\u672C\u5730\u865A\u62DF\u8BBE\u5907\u53EF\u7528 ")}}function JA(n,t){n&1&&g(0," \u542F\u52A8\u672C\u5730 Android \u865A\u62DF\u8BBE\u5907 ")}function e5(n,t){if(n&1){let e=B();l(0,"div",152)(1,"div",155)(2,"span",29),g(3,"hub"),d(),l(4,"span"),g(5," Artemis \u5F53\u524D\u4F7F\u7528 "),l(6,"strong"),g(7),d(),g(8,"\u3002\u4F7F\u7528\u672C\u5730\u6A21\u62DF\u5668\u3001USB \u6216\u65E0\u7EBF ADB \u524D\uFF0C\u8BF7\u5148\u5207\u6362\u81F3\u672C\u5730 ADB\u3002 "),d()(),l(9,"button",156),I("click",function(){S(e);let i=m(3);return k(i.switchToLocalAdbServer())}),l(10,"span",29),g(11,"sync"),d(),l(12,"span"),g(13),d()()()}if(n&2){let e=m(3);p(7),tt("",e.adbServerStatus()?.endpoint?.host,":",e.adbServerStatus()?.endpoint?.port),p(2),N("disabled",e.isSwitchingToLocalAdb()),p(),E("spinning",e.isSwitchingToLocalAdb()),p(3),y(e.isSwitchingToLocalAdb()?"Switching...":"Use Local ADB")}}function t5(n,t){if(n&1&&(z(0,"span",174),l(1,"span"),g(2),d()),n&2){let e=m(6);p(2),tt("",e.emulatorLaunchState()?.status==="booting"?"Booting OS":"Starting"," (",e.emulatorLaunchState()?.elapsed_seconds||0,"s)")}}function n5(n,t){n&1&&(l(0,"span",175),g(1,"play_arrow"),d(),l(2,"span"),g(3,"\u542F\u52A8"),d())}function o5(n,t){if(n&1){let e=B();l(0,"div",165)(1,"div",166)(2,"div",167)(3,"span",29),g(4,"smartphone"),d()(),l(5,"div",168)(6,"div",169)(7,"span",170),g(8),d(),l(9,"span",171),g(10,"\u865A\u62DF\u8BBE\u5907"),d()()()(),l(11,"div",172)(12,"button",173),I("click",function(){let i=S(e).$implicit,r=m(5);return k(r.launchAvdEmulator(i))}),M(13,t5,3,2)(14,n5,4,0),d()()()}if(n&2){let e=t.$implicit,o=m(5);p(8),y(e),p(4),E("launching",o.launchingAvd()===e||o.isEmulatorLaunching()&&o.emulatorLaunchState()?.avd_name===e),N("disabled",o.isEmulatorLaunching()||o.isRemoteAdbServer())("title",o.isEmulatorLaunching()?"\u6B63\u5728\u542F\u52A8\u6A21\u62DF\u5668\u2026":"\u542F\u52A8"+e),p(),b(o.launchingAvd()===e||o.isEmulatorLaunching()&&o.emulatorLaunchState()?.avd_name===e?13:14)}}function i5(n,t){if(n&1&&(l(0,"div",157)(1,"div",159)(2,"div",160)(3,"span",161),g(4,"devices"),d(),l(5,"span",162),g(6,"\u5DF2\u5B89\u88C5\u7684\u865A\u62DF\u8BBE\u5907\uFF08AVD\uFF09"),d(),l(7,"span",163),g(8),d()()(),l(9,"div",164),W(10,o5,15,6,"div",165,ze),d()()),n&2){let e=m(4);p(8),L("",e.installedAvds().length," \u4E2A\u53EF\u7528"),p(2),G(e.installedAvds())}}function r5(n,t){if(n&1&&(l(0,"div",186)(1,"div",188)(2,"div",189)(3,"span",190),g(4,"1"),d(),l(5,"span",191),g(6,"download"),d()(),l(7,"div",192),g(8,"\u5B89\u88C5 Android Studio"),d(),l(9,"div",193),g(10),l(11,"a",194),g(12,"developer.android.com/studio"),d(),g(13," \u5E76\u5B89\u88C5\u3002 "),d()(),l(14,"div",188)(15,"div",189)(16,"span",190),g(17,"2"),d(),l(18,"span",191),g(19,"developer_mode"),d()(),l(20,"div",192),g(21,"\u6253\u5F00\u8BBE\u5907\u7BA1\u7406\u5668"),d(),l(22,"div",193),g(23," \u5728 Android Studio \u6B22\u8FCE\u9875\u9762\u6253\u5F00 "),l(24,"strong"),g(25,"\u66F4\u591A\u64CD\u4F5C > \u865A\u62DF\u8BBE\u5907\u7BA1\u7406\u5668"),d(),g(26," \uFF08\u6216 "),l(27,"strong"),g(28,"\u5DE5\u5177 > \u8BBE\u5907\u7BA1\u7406\u5668"),d(),g(29,"). "),d()(),l(30,"div",188)(31,"div",189)(32,"span",190),g(33,"3"),d(),l(34,"span",191),g(35,"add_circle"),d()(),l(36,"div",192),g(37,"\u521B\u5EFA\u865A\u62D
Line truncated
|
||||
pointer-events: none;
|
||||
top: 0;
|
||||
left: 0;
|
||||
height: 100%;
|
||||
width: 100%;
|
||||
}
|
||||
|
||||
.cdk-overlay-container {
|
||||
position: fixed;
|
||||
}
|
||||
@layer cdk-overlay {
|
||||
.cdk-overlay-container {
|
||||
z-index: 1000;
|
||||
}
|
||||
}
|
||||
.cdk-overlay-container:empty {
|
||||
display: none;
|
||||
}
|
||||
|
||||
.cdk-global-overlay-wrapper {
|
||||
display: flex;
|
||||
position: absolute;
|
||||
}
|
||||
@layer cdk-overlay {
|
||||
.cdk-global-overlay-wrapper {
|
||||
z-index: 1000;
|
||||
}
|
||||
}
|
||||
|
||||
.cdk-overlay-pane {
|
||||
position: absolute;
|
||||
pointer-events: auto;
|
||||
box-sizing: border-box;
|
||||
display: flex;
|
||||
max-width: 100%;
|
||||
max-height: 100%;
|
||||
}
|
||||
@layer cdk-overlay {
|
||||
.cdk-overlay-pane {
|
||||
z-index: 1000;
|
||||
}
|
||||
}
|
||||
|
||||
.cdk-overlay-backdrop {
|
||||
position: absolute;
|
||||
top: 0;
|
||||
bottom: 0;
|
||||
left: 0;
|
||||
right: 0;
|
||||
pointer-events: auto;
|
||||
-webkit-tap-highlight-color: transparent;
|
||||
opacity: 0;
|
||||
touch-action: manipulation;
|
||||
}
|
||||
@layer cdk-overlay {
|
||||
.cdk-overlay-backdrop {
|
||||
z-index: 1000;
|
||||
transition: opacity 400ms cubic-bezier(0.25, 0.8, 0.25, 1);
|
||||
}
|
||||
}
|
||||
@media (prefers-reduced-motion) {
|
||||
.cdk-overlay-backdrop {
|
||||
transition-duration: 1ms;
|
||||
}
|
||||
}
|
||||
|
||||
.cdk-overlay-backdrop-showing {
|
||||
opacity: 1;
|
||||
}
|
||||
@media (forced-colors: active) {
|
||||
.cdk-overlay-backdrop-showing {
|
||||
opacity: 0.6;
|
||||
}
|
||||
}
|
||||
|
||||
@layer cdk-overlay {
|
||||
.cdk-overlay-dark-backdrop {
|
||||
background: rgba(0, 0, 0, 0.32);
|
||||
}
|
||||
}
|
||||
|
||||
.cdk-overlay-transparent-backdrop {
|
||||
transition: visibility 1ms linear, opacity 1ms linear;
|
||||
visibility: hidden;
|
||||
opacity: 1;
|
||||
}
|
||||
.cdk-overlay-transparent-backdrop.cdk-overlay-backdrop-showing, .cdk-high-contrast-active .cdk-overlay-transparent-backdrop {
|
||||
opacity: 0;
|
||||
visibility: visible;
|
||||
}
|
||||
|
||||
.cdk-overlay-backdrop-noop-animation {
|
||||
transition: none;
|
||||
}
|
||||
|
||||
.cdk-overlay-connected-position-bounding-box {
|
||||
position: absolute;
|
||||
display: flex;
|
||||
flex-direction: column;
|
||||
min-width: 1px;
|
||||
min-height: 1px;
|
||||
}
|
||||
@layer cdk-overlay {
|
||||
.cdk-overlay-connected-position-bounding-box {
|
||||
z-index: 1000;
|
||||
}
|
||||
}
|
||||
|
||||
.cdk-global-scrollblock {
|
||||
position: fixed;
|
||||
width: 100%;
|
||||
overflow-y: scroll;
|
||||
}
|
||||
|
||||
.cdk-overlay-popover {
|
||||
background: none;
|
||||
border: none;
|
||||
padding: 0;
|
||||
outline: 0;
|
||||
overflow: visible;
|
||||
position: fixed;
|
||||
pointer-events: none;
|
||||
white-space: normal;
|
||||
color: inherit;
|
||||
text-decoration: none;
|
||||
width: 100%;
|
||||
height: 100%;
|
||||
inset: auto;
|
||||
top: 0;
|
||||
left: 0;
|
||||
}
|
||||
.cdk-overlay-popover::backdrop {
|
||||
display: none;
|
||||
}
|
||||
.cdk-overlay-popover .cdk-overlay-backdrop {
|
||||
position: fixed;
|
||||
z-index: auto;
|
||||
}
|
||||
`],encapsulation:2})}return n})(),hy=(()=>{class n{_platform=v(ir);_containerElement;_document=v(Me);_styleLoader=v(A_);ngOnDestroy(){this._containerElement?.remove()}getContainerElement(){return this._loadStyles(),this._containerElement||this._createContainer(),this._containerElement}_createContainer(){let e="cdk-overlay-container";if(this._platform.isBrowser||T_()){let i=this._document.querySelectorAll(`.${e}[platform="server"], .${e}[platform="test"]`);for(let r=0;r<i.length;r++)i[r].remove()}let o=this._document.createElement("div");o.classList.add(e),T_()?o.setAttribute("platform","test"):this._platform.isBrowser||o.setAttribute("platform","server"),this._document.body.appendChild(o),this._containerElement=o}_loadStyles(){this._styleLoader.load(_y)}static \u0275fac=function(o){return new(o||n)};static \u0275prov=oe({token:n,factory:n.\u0275fac})}return n})(),$_=class{_renderer;_ngZone;element;_cleanupClick;_cleanupTransitionEnd;_fallbackTimeout;constructor(t,e,o,i){this._renderer=e,this._ngZone=o,this.element=t.createElement("div"),this.element.classList.add("cdk-overlay-backdrop"),this._cleanupClick=e.listen(this.element,"click",i)}detach(){this._ngZone.runOutsideAngular(()=>{let t=this.element;clearTimeout(this._fallbackTimeout),this._cleanupTransitionEnd?.(),this._cleanupTransitionEnd=this._renderer.listen(t,"transitionend",this.dispose),this._fallbackTimeout=setTimeout(this.dispose,500),t.style.pointerEvents="none",t.classList.remove("cdk-overlay-backdrop-showing")})}dispose=()=>{clearTimeout(this._fallbackTimeout),this._cleanupClick?.(),this._cleanupTransitionEnd?.(),this._cleanupClick=this._cleanupTransitionEnd=this._fallbackTimeout=void 0,this.element.remove()}};function U_(n){return n&&n.nodeType===1}var Vu=class{_portalOutlet;_host;_pane;_config;_ngZone;_keyboardDispatcher;_document;_location;_outsideClickDispatcher;_animationsDisabled;_injector;_renderer;_backdropClick=new fe;_attachments=new fe;_detachments=new fe;_positionStrategy;_scrollStrategy;_locationChanges=Oe.EMPTY;_backdropRef=null;_detachContentMutationObserver;_detachContentAfterRenderRef;_disposed=!1;_previousHostParent;_keydownEvents=new fe;_outsidePointerEvents=new fe;_afterNextRenderRef;constructor(t,e,o,i,r,a,s,c,u,f=!1,_,h){this._portalOutlet=t,this._host=e,this._pane=o,this._config=i,this._ngZone=r,this._keyboardDispatcher=a,this._document=s,this._location=c,this._outsideClickDispatcher=u,this._animationsDisabled=f,this._injector=_,this._renderer=h,i.scrollStrategy&&(this._scrollStrategy=i.scrollStrategy,this._scrollStrategy.attach(this)),this._positionStrategy=i.positionStrategy}get overlayElement(){return this._pane}get backdropElement(){return this._backdropRef?.element||null}get hostElement(){return this._host}get eventPredicate(){return this._config?.eventPredicate||null}attach(t){if(this._disposed)return null;this._attachHost();let e=this._portalOutlet.attach(t);return this._positionStrategy?.attach(this),this._updateStackingOrder(),this._updateElementSize(),this._updateElementDirection(),this._scrollStrategy&&this._scrollStrategy.enable(),this._afterNextRenderRef?.destroy(),this._afterNextRenderRef=Po(()=>{this.hasAttached()&&this.updatePosition()},{injector:this._injector}),this._togglePointerEvents(!0),this._config.hasBackdrop&&this._attachBackdrop(),this._config.panelClass&&this._toggleClasses(this._pane,this._config.panelClass,!0),this._attachments.next(),this._completeDetachContent(),this._keyboardDispatcher.add(this),this._config.disposeOnNavigation&&(this._locationChanges=this._location.subscribe(()=>this.dispose())),this._outsideClickDispatcher.add(this),typeof e?.onDestroy=="function"&&e.onDestroy(()=>{this.hasAttached()&&this._ngZone.runOutsideAngular(()=>Promise.resolve().then(()=>this.detach()))}),e}detach(){if(!this.hasAttached())return;this.detachBackdrop(),this._togglePointerEvents(!1),this._positionStrategy&&this._positionStrategy.detach&&this._positionStrategy.detach(),this._scrollStrategy&&this._scrollStrategy.disable();let t=this._portalOutlet.detach();return this._detachments.next(),this._completeDetachContent(),this._keyboardDispatcher.remove(this),this._detachContentWhenEmpty(),this._locationChanges.unsubscribe(),this._outsideClickDispatcher.remove(this),t}dispose(){if(this._disposed)return;let t=this.hasAttached();this._positionStrategy&&this._positionStrategy.dispose(),this._disposeScrollStrategy(),this._backdropRef?.dispose(),this._locationChanges.unsubscribe(),this._keyboardDispatcher.remove(this),this._portalOutlet.dispose(),this._attachments.complete(),this._backdropClick.complete(),this._keydownEvents.complete(),this._outsidePointerEvents.complete(),this._outsideClickDispatcher.remove(this),this._host?.remove(),this._afterNextRenderRef?.destroy(),this._previousHostParent=this._pane=this._host=this._backdropRef=null,t&&this._detachments.next(),this._detachments.complete(),this._completeDetachContent(),this._disposed=!0}hasAttached(){return this._portalOutlet.hasAttached()}backdropClick(){
Line truncated
|
||||
`).map(t=>{let e=t.trim(),o=t.length-t.trimStart().length;if(e.startsWith("- [x] ")||e.startsWith("- [X] ")){let a=e.substring(6);return{type:"checked",segments:co(a),indent:o}}if(e.startsWith("- [/] ")){let a=e.substring(6);return{type:"progress",segments:co(a),indent:o}}if(e.startsWith("- [ ] ")){let a=e.substring(6);return{type:"unchecked",segments:co(a),indent:o}}let i=e.match(/^[-*]\s*(verify|assert)(@end)?\s*:\s*(.*)$/i);if(i){let a=i[1].toLowerCase(),s=!!i[2],c=i[3].trim();return{type:"verify",checkKind:a,atEnd:s,segments:co(c),indent:o}}let r=e.match(/^[-*]\s*finding\s*:\s*(.*)$/i);if(r){let a=r[1].trim();return{type:"finding",checkKind:"finding",segments:co(a),indent:o}}if(e.startsWith("- ")||e.startsWith("* ")){let a=e.substring(2);return{type:"list-item",segments:co(a),indent:o}}if(e.startsWith("# ")){let a=e.substring(2);return{type:"h1",segments:co(a),indent:o}}if(e.startsWith("## ")){let a=e.substring(3);return{type:"h2",segments:co(a),indent:o}}if(e.startsWith("### ")){let a=e.substring(4);return{type:"h3",segments:co(a),indent:o}}return e?{type:"text",segments:co(t),indent:o}:{type:"empty",segments:[],indent:0}}):[]}function ec(n){let t=Js(n),e=[],o=[],i=null,r=null,a=0;for(let s of t){if(s.type==="empty")continue;if((s.type==="h1"||s.type==="h2")&&!i){i=s.segments.map(u=>u.text).join("");continue}["checked","progress","unchecked"].includes(s.type)&&(!r||s.indent<=(r._indent??0))?(a++,r={index:a,type:s.type,segments:s.segments,subSteps:[],checks:[],_indent:s.indent},e.push(r)):r&&s.indent>(r._indent??0)?s.type==="verify"||s.type==="assert"||s.type==="finding"?r.checks.push(s):r.subSteps.push(s):o.push(s)}return{title:i,milestones:e,otherLines:o}}function Sy(n){if(!n)return null;let t=n.trim();if(t.includes("```json")){let e=t.split("```json");e.length>1&&(t=e[1].split("```")[0].trim())}else if(t.includes("```")){let e=t.split("```");e.length>1&&(t=e[1].split("```")[0].trim())}try{let e=JSON.parse(t);if(e&&typeof e=="object"&&"success"in e)return{success:!!e.success,reason:e.reason||"\u672A\u63D0\u4F9B\u539F\u56E0\u3002"}}catch(e){}return null}function sr(n){return n?n.replace(/`([^`]+)`/g,'<code class="inline-code">$1</code>').replace(/\*\*\*([^*]+)\*\*\*/g,"<strong><em>$1</em></strong>").replace(/___([^_]+)___/g,"<strong><em>$1</em></strong>").replace(/\*\*([^*]+)\*\*/g,"<strong>$1</strong>").replace(/__([^_]+)__/g,"<strong>$1</strong>").replace(/\*([^*]+)\*/g,"<em>$1</em>").replace(/\b_([^_]+)_\b/g,"<em>$1</em>").replace(/~~([^~]+)~~/g,"<del>$1</del>"):""}function X_(n){if(!n)return"";let o=n.replace(/<\/?(thought|short_term_memory|strong_term_memory|reasoning|plan|task_plan|call_tool)[^\n>]*>?/gi,"").replace(/\r\n/g,`
|
||||
`).replace(/\r/g,`
|
||||
`).replace(/\n{3,}/g,`
|
||||
|
||||
`).trim().replace(/&/g,"&").replace(/</g,"<").replace(/>/g,">").split(`
|
||||
`),i=[],r=null,a=!1,s="",c=[],u=()=>{r&&(i.push(`</${r}>`),r=null)};for(let f=0;f<o.length;f++){let _=o[f],h=_.trim();if(h.startsWith("```")){if(a){let T=c.join(`
|
||||
`);i.push(`<pre class="code-block"><code class="lang-${s}">${T}</code></pre>`),a=!1,c=[],s=""}else u(),a=!0,s=h.substring(3).trim(),c=[];continue}if(a){c.push(_);continue}let C=h.match(/^(#{1,6})\s+(.*)$/);if(C){u();let T=C[1].length,R=sr(C[2]);i.push(`<h${T}>${R}</h${T}>`);continue}if(/^(\-{3,}|\*{3,}|_{3,})$/.test(h)){u(),i.push('<hr class="md-hr" />');continue}if(h.startsWith("> ")||h.startsWith(">")){u();let T=sr(h.replace(/^>\s?/,""));i.push(`<blockquote>${T}</blockquote>`);continue}let O=_.match(/^(\s*)([\*\-])\s+(.*)$/);if(O){r!=="ul"&&(u(),r="ul",i.push("<ul>"));let T=O[1].length,D=Math.max(0,Math.floor((T-2)/2))*12,H=D?` style="margin-left: ${D}px"`:"",ue=O[3],J=ue.match(/^(verify|assert)(@end)?\s*:\s*(.*)$/i),X=ue.match(/^finding\s*:\s*(.*)$/i);if(J){let q=J[1].toLowerCase(),he=J[2]?"@end":"",Ve=q==="assert"?" assert":"",Dt=sr(J[3].trim());i.push(`<li${H} class="md-verify-item"><span class="verify-badge${Ve}">${q}${he}</span>${Dt}</li>`)}else if(X){let q=sr(X[1].trim());i.push(`<li${H} class="md-finding-item"><span class="finding-badge">finding</span>${q}</li>`)}else{let q=sr(ue);i.push(`<li${H}>${q}</li>`)}continue}let P=_.match(/^(\s*)(\d+)\.\s+(.*)$/);if(P){r!=="ol"&&(u(),r="ol",i.push("<ol>"));let T=P[1].length,D=Math.max(0,Math.floor((T-2)/2))*12,H=D?` style="margin-left: ${D}px"`:"",ue=sr(P[3]);i.push(`<li${H}>${ue}</li>`);continue}if(u(),h==="")i.push('<div class="md-para-gap"></div>');else{let T=sr(_);i.push(`<div>${T}</div>`)}}if(a){let f=c.join(`
|
||||
`);i.push(`<pre class="code-block"><code class="lang-${s}">${f}</code></pre>`)}return u(),i.join(`
|
||||
`)}var MR=n=>[n],Ey=(n,t)=>t.id||n;function bR(n,t){n&1&&(l(0,"div",28),g(1," \u6CA1\u6709\u6B63\u5728\u6267\u884C\u6216\u6392\u961F\u7684\u4EFB\u52A1 "),d())}function PR(n,t){n&1&&z(0,"span",36)}function OR(n,t){if(n&1&&(l(0,"span",38)(1,"span",47),g(2,"phone_android"),d(),l(3,"span",48),g(4),d()()),n&2){let e=m().$implicit,o=m(2);N("title","Device: "+o.getDeviceSerial(e)),p(4),y(o.getDeviceSerial(e))}}function vR(n,t){n&1&&(l(0,"span",39),g(1,"\xB7"),d())}function yR(n,t){if(n&1){let e=B();l(0,"button",49),I("click",function(i){S(e);let r=m().$implicit,a=m(2);return k(a.stopTask(r.session_id,i))}),l(1,"span",25),g(2,"stop_circle"),d()()}}function xR(n,t){if(n&1){let e=B();l(0,"button",50),I("click",function(i){S(e);let r=m().$implicit,a=m(2);return k(a.deleteTask(r.session_id,i))}),l(1,"span",25),g(2,"delete"),d()()}}function wR(n,t){if(n&1){let e=B();l(0,"div",33),I("click",function(i){let r=S(e).$implicit,a=m(2);return k(a.selectTask(r.session_id,i))}),l(1,"div",34)(2,"span",35),M(3,PR,1,0,"span",36),g(4),mt(5,"uiLabel"),d(),l(6,"div",37),M(7,OR,5,2,"span",38),M(8,vR,2,0,"span",39),l(9,"span",40),g(10),mt(11,"date"),d(),M(12,yR,3,0,"button",41),M(13,xR,3,0,"button",42),d()(),l(14,"div",43),g(15),d(),l(16,"div",44)(17,"span",45),g(18),d(),l(19,"span",46),g(20),mt(21,"date"),d()()()}if(n&2){let e=t.$implicit,o=m(2);E("selected",o.agentService.currentSessionId()===e.session_id)("running",o.getTaskStatus(e)==="running"),p(2),N("ngClass",o.getTaskStatus(e)),p(),b(o.getTaskStatus(e)==="running"?3:-1),p(),L(" ",qi(5,18,o.getTaskStatus(e))," "),p(3),b(o.getDeviceSerial(e)?7:-1),p(),b(o.getDeviceSerial(e)?8:-1),p(2),y(qt(11,20,e.start_time*1e3,"shortTime")),p(2),b(o.getTaskStatus(e)==="running"||o.getTaskStatus(e)==="paused"?12:-1),p(),b(o.getTaskStatus(e)!=="running"&&o.getTaskStatus(e)!=="paused"?13:-1),p(),N("title",Gi(e.initial_goal)),p(),L(" ",e.initial_goal||"\u672A\u547D\u540D\u4EFB\u52A1"," "),p(2),N("title",e.session_id),p(),L("ID: ",e.session_id),p(2),y(qt(21,23,e.start_time*1e3,"MM/dd/yy"))}}function SR(n,t){n&1&&(l(0,"div",28),g(1," \u6682\u65E0\u5386\u53F2\u4EFB\u52A1 "),d())}function kR(n,t){n&1&&z(0,"span",36)}function ER(n,t){if(n&1&&(l(0,"span",38)(1,"span",47),g(2,"phone_android"),d(),l(3,"span",48),g(4),d()()),n&2){let e=m().$implicit,o=m(2);N("title","Device: "+o.getDeviceSerial(e)),p(4),y(o.getDeviceSerial(e))}}function DR(n,t){n&1&&(l(0,"span",39),g(1,"\xB7"),d())}function IR(n,t){if(n&1){let e=B();l(0,"button",50),I("click",function(i){S(e);let r=m().$implicit,a=m(2);return k(a.deleteTask(r.session_id,i))}),l(1,"span",25),g(2,"delete"),d()()}}function TR(n,t){if(n&1){let e=B();l(0,"div",33),I("click",function(i){let r=S(e).$implicit,a=m(2);return k(a.selectTask(r.session_id,i))}),l(1,"div",34)(2,"span",35),M(3,kR,1,0,"span",36),g(4),mt(5,"uiLabel"),d(),l(6,"div",37),M(7,ER,5,2,"span",38),M(8,DR,2,0,"span",39),l(9,"span",40),g(10),mt(11,"date"),d(),M(12,IR,3,0,"button",42),d()(),l(13,"div",43),g(14),d(),l(15,"div",44)(16,"span",45),g(17),d(),l(18,"span",46),g(19),mt(20,"date"),d()()()}if(n&2){let e=t.$implicit,o=m(2);E("selected",o.agentService.currentSessionId()===e.session_id)("running",o.getTaskStatus(e)==="running"),p(2),N("ngClass",o.getTaskStatus(e)),p(),b(o.getTaskStatus(e)==="running"?3:-1),p(),L(" ",qi(5,17,o.getTaskStatus(e))," "),p(3),b(o.getDeviceSerial(e)?7:-1),p(),b(o.getDeviceSerial(e)?8:-1),p(2),y(qt(11,19,e.start_time*1e3,"shortTime")),p(2),b(o.getTaskStatus(e)!=="running"&&o.getTaskStatus(e)!=="paused"?12:-1),p(),N("title",Gi(e.initial_goal)),p(),L(" ",e.initial_goal||"\u672A\u547D\u540D\u4EFB\u52A1"," "),p(2),N("title",e.session_id),p(),L("ID: ",e.session_id),p(2),y(qt(20,22,e.start_time*1e3,"MM/dd/yy"))}}function AR(n,t){if(n&1&&(l(0,"div",23)(1,"div",24)(2,"span",25),g(3,"list_alt"),d(),l(4,"span"),g(5,"\u4EFB\u52A1\u961F\u5217"),d()(),l(6,"div",26)(7,"span",27),g(8,"bolt"),d(),l(9,"span"),g(10),d()(),M(11,bR,2,0,"div",28),W(12,wR,22,26,"div",29,je().trackSession,!0),l(14,"div",30)(15,"span",31),g(16,"history"),d(),l(17,"span"),g(18),d()(),M(19,SR,2,0,"div",28),l(20,"div",32),W(21,TR,21,25,"div",29,je().trackSession,!0),d()()),n&2){let e=m();p(10),L("Active Queue (",e.activeQueue().length,")"),p(),b(e.activeQueue().length===0?11:-1),p(),G(e.activeQueue()),p(6),L("Past Runs & History (",e.historyTasks().length,")"),p(),b(e.historyTasks().length===0?19:-1),p(2),G(e.historyTasks())}}function RR(n,t){if(n&1){let e=B();l(0,"button",56),I("click",function(i){let r=S(e).$implicit,a=m(3);return k(a.selectNote(r,i))}),l(1,"span",57),g(2,"draft"),d(),g(3),d()}if(n&2){let e=t.$implicit,o=m(3);E("active",o.agentService.selectedNoteKey()===e),p(3),L(" ",e," ")}}function FR(n,t){if(n&1&&(l(0,"div",52)(1,"div",54),W(2,RR,4,3,"button",55,je().trackNoteKey,!0),d()()),n&2){let e=m(2);p(2),G(e.noteKeys())}}function NR(n,t){n&1&&(l(0,"div",28)(1,"span",58),g(2,"description"),d(),l(3,"span"),g(4,"\u6B64\u4F1A\u8BDD\u6682\u65E0\u7B14\u8BB0\u3002"),d()())}function LR(n,t){if(n&1&&(l(0,"h3",5
Line truncated
|
||||
+1
-1
@@ -1 +1 @@
|
||||
@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-diLine truncated
|
||||
@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-diLine truncated
|
||||
+10
-52
@@ -38,8 +38,6 @@ import uuid
|
||||
|
||||
from adbutils import AdbClient
|
||||
from dotenv import load_dotenv
|
||||
from google import genai
|
||||
from langchain_google_genai import ChatGoogleGenerativeAI
|
||||
from PIL import Image
|
||||
from pydantic import BaseModel
|
||||
|
||||
@@ -186,6 +184,16 @@ class Agent:
|
||||
self._initialized = False
|
||||
self._task_lock = asyncio.Lock()
|
||||
|
||||
async def _prewarm_llm_connections(self, api_key: str | None = None):
|
||||
from artemis.llm.router import ModelEndpoint, ModelFactory
|
||||
|
||||
try:
|
||||
ModelFactory.get_model(ModelEndpoint(api_key=api_key))
|
||||
message = "模型连接已就绪"
|
||||
except Exception:
|
||||
message = "模型连接将在首次调用时初始化"
|
||||
publish_startup_progress("model_ready", message, session_id=self._session_id)
|
||||
|
||||
async def init(
|
||||
self,
|
||||
api_key: str | None = None,
|
||||
@@ -273,56 +281,6 @@ class Agent:
|
||||
|
||||
return True
|
||||
|
||||
async def _prewarm_llm_connections(self, api_key: str | None = None):
|
||||
"""Pre-warms the HTTP2/gRPC connection pools for both Native GenAI and LangChain clients in the background."""
|
||||
if os.environ.get("ARTEMIS_FAKE_LLM") == "1":
|
||||
logger.info("ARTEMIS_FAKE_LLM=1 — skipping real LLM connection pre-warming.")
|
||||
publish_startup_progress(
|
||||
"model_ready", "Model connection is ready (fake LLM)", session_id=self._session_id
|
||||
)
|
||||
return
|
||||
publish_startup_progress(
|
||||
"model_warmup", "Warming the model connection", session_id=self._session_id
|
||||
)
|
||||
logger.info("Starting background pre-warming of Gemini API connection pools...")
|
||||
try:
|
||||
key = api_key
|
||||
if not key and settings.GOOGLE_API_KEY:
|
||||
key = settings.GOOGLE_API_KEY.get_secret_value()
|
||||
|
||||
if not key:
|
||||
logger.warning("Skipping LLM pre-warming: No API key available.")
|
||||
publish_startup_progress(
|
||||
"model_ready",
|
||||
"Model connection will initialize on first use",
|
||||
session_id=self._session_id,
|
||||
)
|
||||
return
|
||||
|
||||
# 1. Pre-warm Native SDK client
|
||||
client = genai.Client(api_key=key)
|
||||
|
||||
# 2. Pre-warm LangChain client
|
||||
chat = ChatGoogleGenerativeAI(model="gemini-3.8-flash", google_api_key=key)
|
||||
|
||||
# Fire both calls concurrently in the background
|
||||
await asyncio.gather(
|
||||
client.aio.models.count_tokens(model="gemini-3.8-flash", contents="ping"),
|
||||
chat.ainvoke("ping"),
|
||||
return_exceptions=True,
|
||||
)
|
||||
logger.success("Gemini API connection pools successfully pre-warmed.")
|
||||
publish_startup_progress(
|
||||
"model_ready", "Model connection is ready", session_id=self._session_id
|
||||
)
|
||||
except Exception as e:
|
||||
logger.warning(f"Failed to pre-warm LLM connections: {e}")
|
||||
publish_startup_progress(
|
||||
"model_ready",
|
||||
"Model connection will initialize on first use",
|
||||
session_id=self._session_id,
|
||||
)
|
||||
|
||||
async def install_apk(self, apk_path: str | Path) -> None:
|
||||
"""Install an APK on the connected device.
|
||||
|
||||
|
||||
@@ -20,7 +20,7 @@ and analyze video content from a mobile device screen.
|
||||
The agent can:
|
||||
1. Start a screen recording
|
||||
2. Perform actions while recording
|
||||
3. Stop the recording and analyze its content using Gemini models
|
||||
3. Stop the recording and analyze its content using sampled frames with Responses
|
||||
|
||||
Use case: Recording a video playing on the screen and transcribing its content.
|
||||
"""
|
||||
@@ -28,6 +28,7 @@ Use case: Recording a video playing on the screen and transcribing its content.
|
||||
import asyncio
|
||||
|
||||
from artemis.config import LLM, LLMConfig, LLMConfigUtils, LLMWithFallback
|
||||
from artemis.config import settings
|
||||
from artemis.sdk.agent import Agent
|
||||
from artemis.sdk.builders.agent_config_builder import AgentConfigBuilder
|
||||
from artemis.sdk.types.agent import AgentConfig
|
||||
@@ -37,35 +38,29 @@ from artemis.sdk.types.task import AgentProfile, TaskRequest
|
||||
def get_video_capable_llm_config() -> LLMConfig:
|
||||
"""Returns an LLM config with video_analyzer configured.
|
||||
|
||||
The video_analyzer must use a video-capable Gemini model:
|
||||
- gemini-3.8-flash (recommended - fast and capable)
|
||||
- gemini-3.7-flash
|
||||
- gemini-3-pro-preview
|
||||
- gemini-2.5-flash
|
||||
- gemini-2.5-pro
|
||||
- gemini-2.0-flash
|
||||
Set OPENAI_MODEL to a Responses model with image input support.
|
||||
"""
|
||||
return LLMConfig(
|
||||
planner=LLMWithFallback(
|
||||
provider="openai",
|
||||
model="gpt-5-nano",
|
||||
fallback=LLM(provider="openai", model="gpt-5-mini"),
|
||||
model=settings.OPENAI_MODEL,
|
||||
fallback=LLM(provider="openai", model=settings.OPENAI_MODEL),
|
||||
),
|
||||
utils=LLMConfigUtils(
|
||||
outputter=LLMWithFallback(
|
||||
provider="openai",
|
||||
model="gpt-5-nano",
|
||||
fallback=LLM(provider="openai", model="gpt-5-mini"),
|
||||
model=settings.OPENAI_MODEL,
|
||||
fallback=LLM(provider="openai", model=settings.OPENAI_MODEL),
|
||||
),
|
||||
hopper=LLMWithFallback(
|
||||
provider="openai",
|
||||
model="gpt-5-nano",
|
||||
fallback=LLM(provider="openai", model="gpt-5-mini"),
|
||||
model=settings.OPENAI_MODEL,
|
||||
fallback=LLM(provider="openai", model=settings.OPENAI_MODEL),
|
||||
),
|
||||
video_analyzer=LLMWithFallback(
|
||||
provider="google",
|
||||
model="gemini-3.8-flash",
|
||||
fallback=LLM(provider="google", model="gemini-3.7-flash"),
|
||||
provider="openai",
|
||||
model=settings.OPENAI_MODEL,
|
||||
fallback=LLM(provider="openai", model=settings.OPENAI_MODEL),
|
||||
),
|
||||
),
|
||||
)
|
||||
|
||||
+25
-193
@@ -45,7 +45,6 @@ from artemis.config import (
|
||||
)
|
||||
from artemis.context import ArtemisContext
|
||||
from artemis.data_engine.trace import CURRENT_TRACE_ID, DataEngineCallbackHandler
|
||||
from artemis.llm.google import is_google_chat_model, is_google_provider
|
||||
from artemis.llm.reliability import (
|
||||
CircuitBreaker,
|
||||
FailureCategory,
|
||||
@@ -241,62 +240,6 @@ def _record_llm_event(name: str, payload: dict[str, Any], *, status: str = "retr
|
||||
llm_logger.warning("Failed to publish LLM event %s: %s", name, publish_error)
|
||||
|
||||
|
||||
class _ProviderRetryTelemetryHandler(logging.Handler):
|
||||
"""Turn provider SDK retry log records into structured Artemis telemetry."""
|
||||
|
||||
_DELAY_PATTERN = re.compile(r"\bin\s+(?P<delay>\d+(?:\.\d+)?)\s+seconds?\b", re.IGNORECASE)
|
||||
_ERROR_PATTERN = re.compile(r"\bas it raised\s+(?P<error>.+)$", re.IGNORECASE)
|
||||
_RETRYABLE_MARKERS = (
|
||||
"503",
|
||||
"429",
|
||||
"unavailable",
|
||||
"resource_exhausted",
|
||||
"high demand",
|
||||
"quota",
|
||||
"throttled",
|
||||
"overloaded",
|
||||
)
|
||||
|
||||
def emit(self, record: logging.LogRecord) -> None:
|
||||
try:
|
||||
request = _ACTIVE_LLM_REQUEST.get()
|
||||
if not request or request.get("provider") != ModelProvider.GOOGLE.value:
|
||||
return
|
||||
|
||||
message = record.getMessage()
|
||||
lowered = message.lower()
|
||||
if "retrying" not in lowered or not any(
|
||||
marker in lowered for marker in self._RETRYABLE_MARKERS
|
||||
):
|
||||
return
|
||||
|
||||
delay_match = self._DELAY_PATTERN.search(message)
|
||||
error_match = self._ERROR_PATTERN.search(message)
|
||||
delay = float(delay_match.group("delay")) if delay_match else 0.0
|
||||
error = error_match.group("error").strip() if error_match else message
|
||||
_record_llm_retry(
|
||||
error,
|
||||
delay,
|
||||
provider=request["provider"],
|
||||
source="provider_sdk",
|
||||
)
|
||||
except Exception:
|
||||
# Telemetry must never interfere with the provider's own retry.
|
||||
self.handleError(record)
|
||||
|
||||
|
||||
def _install_provider_retry_telemetry() -> None:
|
||||
provider_logger = logging.getLogger("google_genai._api_client")
|
||||
if any(
|
||||
isinstance(handler, _ProviderRetryTelemetryHandler) for handler in provider_logger.handlers
|
||||
):
|
||||
return
|
||||
provider_logger.addHandler(_ProviderRetryTelemetryHandler(level=logging.INFO))
|
||||
|
||||
|
||||
_install_provider_retry_telemetry()
|
||||
|
||||
|
||||
def _handle_llm_pause_and_resume(last_error: Exception) -> Path:
|
||||
"""Handles pausing task execution upon persistent failure until resumed."""
|
||||
err_msg = str(last_error)
|
||||
@@ -550,40 +493,12 @@ class RobustChatModelWrapper:
|
||||
|
||||
processed_tools = list(tools_list) if tools_list is not None else []
|
||||
|
||||
# Determine if the underlying model provider is Google / Gemini
|
||||
if self.endpoint is not None:
|
||||
google_backed = is_google_provider(self.endpoint.provider)
|
||||
else:
|
||||
google_backed = is_google_chat_model(self.base_model)
|
||||
|
||||
has_explicit_google_search = any(
|
||||
isinstance(t, dict) and "google_search" in t for t in processed_tools
|
||||
)
|
||||
should_ground = (
|
||||
self.endpoint is not None and self.endpoint.enable_grounding
|
||||
) or has_explicit_google_search
|
||||
|
||||
if google_backed:
|
||||
if should_ground:
|
||||
# Add native Google Search Grounding tool if not already present
|
||||
if not has_explicit_google_search:
|
||||
processed_tools.append({"google_search": {}})
|
||||
|
||||
# Ensure include_server_side_tool_invocations is True in tool_config
|
||||
# to satisfy Gemini API requirements when mixing built-in tools and function calling
|
||||
tool_config = kwargs.get("tool_config")
|
||||
if tool_config is None:
|
||||
kwargs["tool_config"] = {"include_server_side_tool_invocations": True}
|
||||
elif isinstance(tool_config, dict):
|
||||
tool_config = dict(tool_config)
|
||||
tool_config.setdefault("include_server_side_tool_invocations", True)
|
||||
kwargs["tool_config"] = tool_config
|
||||
else:
|
||||
# If not Gemini/Google (e.g. OpenAI, Anthropic, Ollama, OpenRouter, XAI):
|
||||
# Ignore grounding and filter out any Gemini-specific built-in dict tools to avoid provider errors
|
||||
processed_tools = [
|
||||
t for t in processed_tools if not (isinstance(t, dict) and "google_search" in t)
|
||||
]
|
||||
if self.endpoint is not None and self.endpoint.enable_grounding:
|
||||
if not any(
|
||||
isinstance(t, dict) and t.get("type") in ("web_search", "web_search_preview")
|
||||
for t in processed_tools
|
||||
):
|
||||
processed_tools.append({"type": "web_search"})
|
||||
|
||||
bound = self.base_model.bind_tools(processed_tools, *other_args, **kwargs)
|
||||
return RobustChatModelWrapper(bound, self.ctx, endpoint=self.endpoint)
|
||||
@@ -597,6 +512,9 @@ class RobustChatModelWrapper:
|
||||
|
||||
def with_structured_output(self, *args, **kwargs):
|
||||
if hasattr(self.base_model, "with_structured_output"):
|
||||
# Compatible Responses endpoints may support functions but not text.format=json_schema.
|
||||
# Keep schema validation in LangChain/Pydantic without changing the API transport.
|
||||
kwargs.setdefault("method", "function_calling")
|
||||
return RobustChatModelWrapper(
|
||||
self.base_model.with_structured_output(*args, **kwargs),
|
||||
self.ctx,
|
||||
@@ -907,100 +825,18 @@ async def invoke_llm_with_timeout_message[T](
|
||||
|
||||
|
||||
# Backward compatible factory functions delegating to ModelFactory
|
||||
def get_google_llm(
|
||||
model_name: str = "gemini-3.8-flash",
|
||||
temperature: float | None = None,
|
||||
timeout: float | None = None,
|
||||
thinking_budget: int | None = None,
|
||||
thinking_level: str | None = "medium",
|
||||
include_thoughts: bool | None = None,
|
||||
enable_grounding: bool = False,
|
||||
) -> BaseChatModel:
|
||||
ep = ModelEndpoint(
|
||||
provider=ModelProvider.GOOGLE,
|
||||
model_name=model_name,
|
||||
temperature=temperature or 0.0,
|
||||
timeout_seconds=timeout or 60.0,
|
||||
thinking_budget=thinking_budget,
|
||||
thinking_level=thinking_level,
|
||||
include_thoughts=include_thoughts,
|
||||
enable_grounding=enable_grounding,
|
||||
)
|
||||
return ModelFactory.create_model(ep)
|
||||
|
||||
|
||||
def get_vertex_llm(
|
||||
model_name: str = "gemini-3.8-flash",
|
||||
temperature: float | None = None,
|
||||
timeout: float | None = None,
|
||||
thinking_budget: int | None = None,
|
||||
) -> BaseChatModel:
|
||||
ep = ModelEndpoint(
|
||||
provider=ModelProvider.VERTEX_AI,
|
||||
model_name=model_name,
|
||||
temperature=temperature or 0.0,
|
||||
timeout_seconds=timeout or 60.0,
|
||||
thinking_budget=thinking_budget,
|
||||
)
|
||||
return ModelFactory.create_model(ep)
|
||||
|
||||
|
||||
def get_openai_llm(
|
||||
model_name: str = "o3",
|
||||
model_name: str | None = None,
|
||||
temperature: float | None = None,
|
||||
timeout: float | None = None,
|
||||
) -> BaseChatModel:
|
||||
ep = ModelEndpoint(
|
||||
provider=ModelProvider.OPENAI,
|
||||
model_name=model_name,
|
||||
temperature=temperature or 0.0,
|
||||
timeout_seconds=timeout or 60.0,
|
||||
)
|
||||
return ModelFactory.create_model(ep)
|
||||
|
||||
|
||||
def get_openrouter_llm(
|
||||
model_name: str,
|
||||
temperature: float | None = None,
|
||||
timeout: float | None = None,
|
||||
) -> BaseChatModel:
|
||||
ep = ModelEndpoint(
|
||||
provider=ModelProvider.OPENROUTER,
|
||||
model_name=model_name,
|
||||
temperature=temperature or 0.0,
|
||||
timeout_seconds=timeout or 60.0,
|
||||
)
|
||||
return ModelFactory.create_model(ep)
|
||||
|
||||
|
||||
def get_grok_llm(
|
||||
model_name: str,
|
||||
temperature: float | None = None,
|
||||
timeout: float | None = None,
|
||||
) -> BaseChatModel:
|
||||
ep = ModelEndpoint(
|
||||
provider=ModelProvider.XAI,
|
||||
model_name=model_name,
|
||||
temperature=temperature or 0.0,
|
||||
timeout_seconds=timeout or 60.0,
|
||||
)
|
||||
return ModelFactory.create_model(ep)
|
||||
|
||||
|
||||
def get_anthropic_llm(
|
||||
model_name: str,
|
||||
temperature: float | None = None,
|
||||
timeout: float | None = None,
|
||||
thinking_budget: int | None = None,
|
||||
reasoning_effort: str | None = None,
|
||||
) -> BaseChatModel:
|
||||
ep = ModelEndpoint(
|
||||
provider=ModelProvider.ANTHROPIC,
|
||||
model_name=model_name,
|
||||
temperature=temperature or 0.0,
|
||||
timeout_seconds=timeout or 60.0,
|
||||
thinking_budget=thinking_budget,
|
||||
reasoning_effort=reasoning_effort,
|
||||
model_name=model_name or settings.OPENAI_MODEL,
|
||||
temperature=temperature,
|
||||
timeout_seconds=timeout if timeout is not None else settings.OPENAI_TIMEOUT_SECONDS,
|
||||
)
|
||||
return ModelFactory.create_model(ep)
|
||||
|
||||
@@ -1010,21 +846,15 @@ def get_cached_raw_model(
|
||||
model_name: str,
|
||||
temperature: float | None = None,
|
||||
timeout: float | None = None,
|
||||
thinking_budget: int | None = None,
|
||||
thinking_level: str | None = None,
|
||||
include_thoughts: bool | None = None,
|
||||
reasoning_effort: str | None = None,
|
||||
enable_grounding: bool = False,
|
||||
) -> BaseChatModel:
|
||||
"""Retrieves or instantiates a cached raw LangChain chat model."""
|
||||
ep = ModelEndpoint(
|
||||
provider=ModelProvider.from_string(provider),
|
||||
model_name=model_name,
|
||||
temperature=temperature or 0.0,
|
||||
timeout_seconds=timeout or 60.0,
|
||||
thinking_budget=thinking_budget,
|
||||
thinking_level=thinking_level,
|
||||
include_thoughts=include_thoughts,
|
||||
model_name=model_name or settings.OPENAI_MODEL,
|
||||
temperature=temperature,
|
||||
timeout_seconds=timeout if timeout is not None else settings.OPENAI_TIMEOUT_SECONDS,
|
||||
reasoning_effort=reasoning_effort,
|
||||
enable_grounding=enable_grounding,
|
||||
)
|
||||
@@ -1061,18 +891,20 @@ def _resolve_endpoint(
|
||||
val = getattr(obj, attr, None)
|
||||
return val if isinstance(val, expected_type) else None
|
||||
|
||||
provider_val = getattr(cfg, "provider", "google")
|
||||
model_val = getattr(cfg, "model", "gemini-2.5-flash")
|
||||
provider_val = getattr(cfg, "provider", "openai")
|
||||
model_val = getattr(cfg, "model", settings.OPENAI_MODEL)
|
||||
|
||||
return ModelEndpoint(
|
||||
provider=ModelProvider.from_string(provider_val),
|
||||
model_name=str(model_val),
|
||||
temperature=_get_val(cfg, "temperature", (int, float)) or 0.0,
|
||||
timeout_seconds=_get_val(cfg, "timeout", (int, float)) or 60.0,
|
||||
thinking_budget=_get_val(cfg, "thinking_budget", int),
|
||||
thinking_level=_get_val(cfg, "thinking_level", str),
|
||||
temperature=_get_val(cfg, "temperature", (int, float)),
|
||||
timeout_seconds=_get_val(cfg, "timeout", (int, float)) or settings.OPENAI_TIMEOUT_SECONDS,
|
||||
max_retries=_get_val(cfg, "max_retries", int)
|
||||
if _get_val(cfg, "max_retries", int) is not None
|
||||
else settings.OPENAI_MAX_RETRIES,
|
||||
max_tokens=_get_val(cfg, "max_tokens", int),
|
||||
api_base=_get_val(cfg, "api_base", str),
|
||||
reasoning_effort=_get_val(cfg, "reasoning_effort", str),
|
||||
include_thoughts=_get_val(cfg, "include_thoughts", bool),
|
||||
enable_grounding=_get_val(cfg, "enable_grounding", bool) or False,
|
||||
)
|
||||
|
||||
|
||||
@@ -40,7 +40,7 @@ import threading
|
||||
from typing import Any
|
||||
|
||||
from artemis.data_engine.context_vars import CURRENT_NODE_NAME
|
||||
from artemis.llm.google import usage_from_message
|
||||
from artemis.llm.usage import usage_from_message
|
||||
from artemis.utils.logger import get_logger
|
||||
|
||||
logger = get_logger(__name__)
|
||||
@@ -62,7 +62,7 @@ OPERATOR_NODE_NAMES = frozenset({"operator", "flashrunner"})
|
||||
def extract_usage(response: Any) -> dict[str, int] | None:
|
||||
"""Extracts unified token usage from an LLM response message, best-effort.
|
||||
|
||||
Thin wrapper over :func:`artemis.llm.google.usage.usage_from_message`
|
||||
Thin wrapper over :func:`artemis.llm.usage.usage_from_message`
|
||||
that keeps the meter's historical four-key shape. Returns None when the
|
||||
response carries no usable usage numbers.
|
||||
"""
|
||||
|
||||
@@ -25,7 +25,6 @@ import concurrent.futures
|
||||
import inspect
|
||||
from typing import Annotated, Any, Literal
|
||||
|
||||
from google.genai import types as genai_types
|
||||
from langchain_core.tools import BaseTool, StructuredTool
|
||||
from langgraph.prebuilt import InjectedState
|
||||
from pydantic import BaseModel
|
||||
@@ -148,45 +147,6 @@ class ArtemisTool:
|
||||
parameters={"type": "object", "properties": properties, "required": required},
|
||||
)
|
||||
|
||||
def to_genai_declaration(self) -> genai_types.FunctionDeclaration:
|
||||
"""Exports this tool to a Google GenAI FunctionDeclaration schema."""
|
||||
flat_properties, required = self._flat_schema()
|
||||
properties = {}
|
||||
for prop_name, prop in flat_properties.items():
|
||||
genai_type = _GENAI_TYPES.get(prop["type"], genai_types.Type.STRING)
|
||||
items_schema = None
|
||||
if genai_type == genai_types.Type.ARRAY:
|
||||
item_type = (prop.get("items") or {}).get("type", "string")
|
||||
items_schema = genai_types.Schema(
|
||||
type=_GENAI_TYPES.get(item_type, genai_types.Type.STRING)
|
||||
)
|
||||
properties[prop_name] = genai_types.Schema(
|
||||
type=genai_type,
|
||||
description=prop.get("description", ""),
|
||||
items=items_schema,
|
||||
enum=[str(v) for v in prop["enum"]] if prop.get("enum") else None,
|
||||
)
|
||||
|
||||
return genai_types.FunctionDeclaration(
|
||||
name=self.name,
|
||||
description=self.description,
|
||||
parameters=genai_types.Schema(
|
||||
type=genai_types.Type.OBJECT,
|
||||
properties=properties,
|
||||
required=required,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
_GENAI_TYPES = {
|
||||
"string": genai_types.Type.STRING,
|
||||
"integer": genai_types.Type.INTEGER,
|
||||
"number": genai_types.Type.NUMBER,
|
||||
"boolean": genai_types.Type.BOOLEAN,
|
||||
"array": genai_types.Type.ARRAY,
|
||||
"object": genai_types.Type.OBJECT,
|
||||
}
|
||||
|
||||
|
||||
def _flatten_property(prop_data: dict[str, Any]) -> dict[str, Any]:
|
||||
"""One Pydantic JSON-schema property as a flat single-type entry.
|
||||
|
||||
@@ -12,9 +12,9 @@
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
"""Video Analyzer utility for analyzing video content using Gemini models.
|
||||
"""Video Analyzer utility for analyzing video content using OpenAI Responses models.
|
||||
|
||||
This utility sends video files to video-capable Gemini models for analysis
|
||||
This utility sends sampled video frames to image-capable Responses models for analysis
|
||||
and returns text descriptions based on the provided prompt.
|
||||
It also exposes a tool for agents to perform structured video analysis.
|
||||
"""
|
||||
@@ -111,7 +111,7 @@ DIAGNOSER_VIDEO_ANALYZER_DOCSTRING = (
|
||||
|
||||
|
||||
class VideoAnalyzerTool(ArtemisTool):
|
||||
"""Universal tool for analyzing video content using Gemini models."""
|
||||
"""Universal tool for analyzing video content using OpenAI Responses models."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
|
||||
@@ -16,7 +16,7 @@
|
||||
|
||||
import httpx
|
||||
|
||||
from artemis.llm.google.provider import is_google_family_provider
|
||||
from artemis.config.settings import settings
|
||||
|
||||
from artemis.utils.logger import get_logger
|
||||
|
||||
@@ -60,7 +60,7 @@ async def validate_api_key(
|
||||
provider: str,
|
||||
api_key: str,
|
||||
base_url: str | None = None,
|
||||
timeout: float = 12.0,
|
||||
timeout: float | None = None,
|
||||
) -> tuple[bool, str]:
|
||||
"""Tests if the provided API key is valid and usable with the corresponding provider.
|
||||
|
||||
@@ -90,19 +90,10 @@ async def validate_api_key(
|
||||
return False, "Please enter a real API key instead of the placeholder."
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=timeout) as client:
|
||||
if is_google_family_provider(clean_provider):
|
||||
# Test Google Gemini via generativelanguage API
|
||||
# The key travels in a header, never in the URL: httpx logs request
|
||||
# URLs at INFO level and those logs end up in files.
|
||||
url = "https://generativelanguage.googleapis.com/v1beta/models?pageSize=1"
|
||||
resp = await client.get(url, headers={"x-goog-api-key": clean_key})
|
||||
if resp.status_code == 200:
|
||||
return True, "Google Gemini API key verified successfully."
|
||||
err_msg = _extract_error_message(resp)
|
||||
return False, f"Gemini API verification failed ({resp.status_code}): {err_msg}"
|
||||
|
||||
elif clean_provider in ("ocr", "vision", "google_vision"):
|
||||
async with httpx.AsyncClient(
|
||||
timeout=timeout if timeout is not None else settings.ARTEMIS_CREDENTIAL_TIMEOUT_SECONDS
|
||||
) as client:
|
||||
if clean_provider in ("ocr", "vision", "google_vision"):
|
||||
# Test Google Cloud Vision OCR API
|
||||
url = "https://vision.googleapis.com/v1/images:annotate"
|
||||
payload = {
|
||||
@@ -124,68 +115,25 @@ async def validate_api_key(
|
||||
return False, f"Vision OCR API verification failed ({resp.status_code}): {err_msg}"
|
||||
|
||||
elif clean_provider == "openai":
|
||||
endpoint = (
|
||||
f"{base_url.rstrip('/')}/models"
|
||||
if base_url
|
||||
else "https://api.openai.com/v1/models"
|
||||
base = (base_url or settings.OPENAI_BASE_URL or "https://api.openai.com/v1").rstrip(
|
||||
"/"
|
||||
)
|
||||
resp = await client.get(endpoint, headers={"Authorization": f"Bearer {clean_key}"})
|
||||
if resp.status_code == 200:
|
||||
return True, "OpenAI API key verified successfully."
|
||||
err_msg = _extract_error_message(resp)
|
||||
return False, f"OpenAI API verification failed ({resp.status_code}): {err_msg}"
|
||||
|
||||
elif clean_provider in ("anthropic", "claude"):
|
||||
url = "https://api.anthropic.com/v1/models"
|
||||
resp = await client.get(
|
||||
url,
|
||||
headers={
|
||||
"x-api-key": clean_key,
|
||||
"anthropic-version": "2023-06-01",
|
||||
resp = await client.post(
|
||||
f"{base}/responses",
|
||||
headers={"Authorization": f"Bearer {clean_key}"},
|
||||
json={
|
||||
"model": settings.OPENAI_MODEL,
|
||||
"input": "Reply with OK.",
|
||||
"max_output_tokens": 16,
|
||||
"store": False,
|
||||
},
|
||||
)
|
||||
if resp.status_code == 200:
|
||||
return True, "Anthropic Claude API key verified successfully."
|
||||
err_msg = _extract_error_message(resp)
|
||||
return False, f"Anthropic API verification failed ({resp.status_code}): {err_msg}"
|
||||
|
||||
elif clean_provider == "openrouter":
|
||||
url = "https://openrouter.ai/api/v1/auth/key"
|
||||
resp = await client.get(url, headers={"Authorization": f"Bearer {clean_key}"})
|
||||
if resp.status_code == 200:
|
||||
return True, "OpenRouter API key verified successfully."
|
||||
err_msg = _extract_error_message(resp)
|
||||
return False, f"OpenRouter API verification failed ({resp.status_code}): {err_msg}"
|
||||
|
||||
elif clean_provider in ("xai", "grok"):
|
||||
url = "https://api.x.ai/v1/models"
|
||||
resp = await client.get(url, headers={"Authorization": f"Bearer {clean_key}"})
|
||||
if resp.status_code == 200:
|
||||
return True, "xAI API key verified successfully."
|
||||
err_msg = _extract_error_message(resp)
|
||||
return False, f"xAI API verification failed ({resp.status_code}): {err_msg}"
|
||||
|
||||
elif clean_provider in ("ollama", "vllm", "custom"):
|
||||
if base_url:
|
||||
endpoint = f"{base_url.rstrip('/')}/models"
|
||||
headers = (
|
||||
{"Authorization": f"Bearer {clean_key}"} if clean_key != "EMPTY" else {}
|
||||
)
|
||||
resp = await client.get(endpoint, headers=headers)
|
||||
if resp.status_code == 200:
|
||||
return True, f"Custom endpoint ({base_url}) verified successfully."
|
||||
err_msg = _extract_error_message(resp)
|
||||
return (
|
||||
False,
|
||||
f"Custom endpoint verification failed ({resp.status_code}): {err_msg}",
|
||||
)
|
||||
return True, "Custom endpoint configured."
|
||||
if resp.status_code == 200 and resp.json().get("output"):
|
||||
return True, "OpenAI Responses 连接验证成功。"
|
||||
return False, f"Responses 验证失败(HTTP {resp.status_code})"
|
||||
|
||||
else:
|
||||
# Unknown provider - fallback to basic length check
|
||||
if len(clean_key) >= 8:
|
||||
return True, f"Credentials for {provider} configured."
|
||||
return False, f"Invalid API key format for {provider}."
|
||||
return False, "仅支持 OpenAI Responses 和可选 OCR 凭据。"
|
||||
|
||||
except httpx.TimeoutException:
|
||||
logger.warning(f"Timeout verifying API key for {provider}")
|
||||
|
||||
Reference in new issue
Block a user