feat: add Chinese UI, Responses models, and mirrored Windows setup
Localize the web console and persist custom model and capability settings. Use OpenAI Responses exclusively and move runtime limits into environment configuration. Bootstrap uv and the virtual environment on Windows, support optional China mirrors, and package source distributions without environments, build output, or credentials. Update regression tests and repository quality checks for the new behavior.
This commit is contained in:
1 parent
371aa6df56
commit
2ddec53954
183 files changed
+3427
-11857
No files matched your search
+28
-21
@@ -1,32 +1,39 @@
|
|||||||
# ==============================================================================
|
# 复制为 .env,或在首页「模型配置」「功能开关」中填写并保存。
|
||||||
# Artemis Environment Configuration Template
|
# 所有模型调用仅使用 OpenAI Responses API;环境变量优先于 .env。
|
||||||
# ==============================================================================
|
|
||||||
# Copy this file to .env and fill in your API credentials:
|
|
||||||
# cp .env.example .env
|
|
||||||
|
|
||||||
# LLM Providers (Configure at least one)
|
|
||||||
GEMINI_API_KEY=
|
|
||||||
GOOGLE_API_KEY=
|
|
||||||
OPENAI_API_KEY=
|
OPENAI_API_KEY=
|
||||||
ANTHROPIC_API_KEY=
|
# 接口根地址:通常以 /v1 结尾,不要添加 /responses 或 /chat/completions。
|
||||||
OPEN_ROUTER_API_KEY=
|
OPENAI_BASE_URL=https://api.openai.com/v1
|
||||||
XAI_API_KEY=
|
OPENAI_MODEL=gpt-5-mini
|
||||||
|
# 单次模型 HTTP 请求超时(秒)
|
||||||
|
OPENAI_TIMEOUT_SECONDS=120
|
||||||
|
# SDK 内部重试次数;默认 0,由 Artemis 的分类恢复机制处理重试。
|
||||||
|
OPENAI_MAX_RETRIES=0
|
||||||
|
|
||||||
# Google Cloud Vision OCR (Optional - for advanced OCR processing)
|
# 功能开关:保存后对新任务生效
|
||||||
|
ARTEMIS_VIDEO_ENABLED=true
|
||||||
|
ARTEMIS_OUTPUTTER_ENABLED=true
|
||||||
|
ARTEMIS_STEP_SUMMARIZER_ENABLED=true
|
||||||
|
|
||||||
|
# 视频分段、重试与单次分析超时(秒)
|
||||||
|
ARTEMIS_VIDEO_CHUNK_SECONDS=60
|
||||||
|
ARTEMIS_VIDEO_TIMEOUT_SECONDS=120
|
||||||
|
ARTEMIS_VIDEO_MAX_RETRIES=1
|
||||||
|
# 检查点、退出汇总与凭据验证超时(秒)
|
||||||
|
ARTEMIS_CHECKPOINT_TIMEOUT_SECONDS=180
|
||||||
|
ARTEMIS_SETTLEMENT_TIMEOUT_SECONDS=120
|
||||||
|
ARTEMIS_CREDENTIAL_TIMEOUT_SECONDS=12
|
||||||
|
ARTEMIS_MODEL_CACHE_TTL_SECONDS=10
|
||||||
|
# 模型重试耗尽后的暂停等待上限(秒);0 表示无限等待
|
||||||
|
LLM_PAUSE_TIMEOUT_SECONDS=900
|
||||||
|
|
||||||
|
# 可选 Google Cloud Vision OCR,独立于模型调用,不配置也可运行。
|
||||||
OCR_API_KEY=
|
OCR_API_KEY=
|
||||||
|
|
||||||
# Android ADB Connectivity (Optional - defaults to 127.0.0.1:5037)
|
# Android ADB 连接
|
||||||
ADB_HOST=127.0.0.1
|
ADB_HOST=127.0.0.1
|
||||||
ADB_PORT=5037
|
ADB_PORT=5037
|
||||||
# ADB_DEVICE_SERIAL=
|
# ADB_DEVICE_SERIAL=
|
||||||
# Keep the display awake, falling back to a wake lock or safe heartbeat when needed.
|
|
||||||
# Set to false if Artemis is used with a personal device whose screen should time out.
|
|
||||||
ARTEMIS_KEEP_DEVICE_AWAKE=true
|
ARTEMIS_KEEP_DEVICE_AWAKE=true
|
||||||
# UI hierarchy backend: auto (Accessibility Helper, UIAutomator2 fallback) | helper | uiautomator
|
|
||||||
ARTEMIS_HIERARCHY_BACKEND=auto
|
ARTEMIS_HIERARCHY_BACKEND=auto
|
||||||
# Let tasks install / upgrade the Accessibility Helper APK on a device they hold.
|
|
||||||
# Set to false on shared or personal phones: then only `artemis helper install` installs it.
|
|
||||||
ARTEMIS_HELPER_AUTO_INSTALL=true
|
ARTEMIS_HELPER_AUTO_INSTALL=true
|
||||||
|
|
||||||
# Artemis Execution Defaults (flash or pro)
|
|
||||||
ARTEMIS_DEFAULT_PROFILE=flash
|
ARTEMIS_DEFAULT_PROFILE=flash
|
||||||
@@ -109,6 +109,7 @@ live_screenshot*.png
|
|||||||
*.key
|
*.key
|
||||||
|
|
||||||
# OS Artifacts
|
# OS Artifacts
|
||||||
|
artifacts/
|
||||||
.DS_Store
|
.DS_Store
|
||||||
.DS_Store?
|
.DS_Store?
|
||||||
._*
|
._*
|
||||||
|
|||||||
@@ -20,7 +20,7 @@
|
|||||||
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/Python-3.12+-3776AB.svg?logo=python&logoColor=white" alt="Python 3.12+"></a>
|
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/Python-3.12+-3776AB.svg?logo=python&logoColor=white" alt="Python 3.12+"></a>
|
||||||
<a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License: Apache-2.0"></a>
|
<a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License: Apache-2.0"></a>
|
||||||
<a href="https://modelcontextprotocol.io/"><img src="https://img.shields.io/badge/MCP-Native%20Server-8A2BE2.svg" alt="MCP Native"></a>
|
<a href="https://modelcontextprotocol.io/"><img src="https://img.shields.io/badge/MCP-Native%20Server-8A2BE2.svg" alt="MCP Native"></a>
|
||||||
<a href="https://ai.google.dev/"><img src="https://img.shields.io/badge/Multimodal-Gemini%20%7C%20Claude%20%7C%20GPT--4o%20%7C%20Qwen--VL-4285F4.svg" alt="Multi-Model"></a>
|
<a href="https://developers.openai.com/api/docs/guides/responses"><img src="https://img.shields.io/badge/API-OpenAI%20Responses-4285F4.svg" alt="OpenAI Responses"></a>
|
||||||
<a href="https://github.com/google-research/android_world"><img src="https://img.shields.io/badge/AndroidWorld-99%25%2B%20SOTA-success.svg" alt="AndroidWorld SOTA"></a>
|
<a href="https://github.com/google-research/android_world"><img src="https://img.shields.io/badge/AndroidWorld-99%25%2B%20SOTA-success.svg" alt="AndroidWorld SOTA"></a>
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
@@ -104,6 +104,14 @@ cd artemis
|
|||||||
|
|
||||||
> **Tip**: Opens `http://localhost:8000` in your default browser with a device connection wizard, live screen mirroring, prompt sandbox, and execution replays. You can also run directly from CLI: `uv run artemis run "Open Settings, find Battery and tell me current level" --profile flash`.
|
> **Tip**: Opens `http://localhost:8000` in your default browser with a device connection wizard, live screen mirroring, prompt sandbox, and execution replays. You can also run directly from CLI: `uv run artemis run "Open Settings, find Battery and tell me current level" --profile flash`.
|
||||||
|
|
||||||
|
### Model and feature settings
|
||||||
|
|
||||||
|
The web console now uses Simplified Chinese. Open **系统设置与运行准备 → 模型配置** to edit and save the Responses base URL, model, API key, request timeout, and retry count. **测试连接** tests the current form without saving; **保存配置** persists it to `.env`. Leave the key blank to retain the existing credential. Existing keys are never returned to the browser.
|
||||||
|
|
||||||
|
All models use **OpenAI Responses**, including custom gateways. Set `OPENAI_API_KEY`, `OPENAI_BASE_URL` (the API root, usually ending in `/v1`), and `OPENAI_MODEL` (for example `mimo-v2.6-flash`). Gemini, Anthropic and Chat Completions transports are no longer supported. Video analysis sends sampled image frames and requires image input support. Structured output uses Responses function calling with local schema validation.
|
||||||
|
|
||||||
|
The **功能开关** section persists video analysis/recording, final reports, and Flash step summaries. Changes apply to new tasks and survive restart. Environment variables take precedence over `.env`; explicit per-node or task overrides take precedence over global defaults. See [`.env.example`](.env.example) for timeout settings and the [Chinese setup guide](README_CN.md#网页模型配置与功能开关) for details.
|
||||||
|
|
||||||
<a id="mcp-setup"></a>
|
<a id="mcp-setup"></a>
|
||||||
<a id="mcp"></a>
|
<a id="mcp"></a>
|
||||||
<details>
|
<details>
|
||||||
|
|||||||
+66
-1
@@ -20,7 +20,7 @@
|
|||||||
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/Python-3.12+-3776AB.svg?logo=python&logoColor=white" alt="Python 3.12+"></a>
|
<a href="https://www.python.org/downloads/"><img src="https://img.shields.io/badge/Python-3.12+-3776AB.svg?logo=python&logoColor=white" alt="Python 3.12+"></a>
|
||||||
<a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License: Apache-2.0"></a>
|
<a href="LICENSE"><img src="https://img.shields.io/badge/License-Apache%202.0-blue.svg" alt="License: Apache-2.0"></a>
|
||||||
<a href="https://modelcontextprotocol.io/"><img src="https://img.shields.io/badge/MCP-Native%20Server-8A2BE2.svg" alt="MCP Native"></a>
|
<a href="https://modelcontextprotocol.io/"><img src="https://img.shields.io/badge/MCP-Native%20Server-8A2BE2.svg" alt="MCP Native"></a>
|
||||||
<a href="https://ai.google.dev/"><img src="https://img.shields.io/badge/Multimodal-Gemini%20%7C%20Claude%20%7C%20GPT--4o%20%7C%20Qwen--VL-4285F4.svg" alt="Multi-Model"></a>
|
<a href="https://developers.openai.com/api/docs/guides/responses"><img src="https://img.shields.io/badge/API-OpenAI%20Responses-4285F4.svg" alt="OpenAI Responses"></a>
|
||||||
<a href="https://github.com/google-research/android_world"><img src="https://img.shields.io/badge/AndroidWorld-99%25%2B%20SOTA-success.svg" alt="AndroidWorld SOTA"></a>
|
<a href="https://github.com/google-research/android_world"><img src="https://img.shields.io/badge/AndroidWorld-99%25%2B%20SOTA-success.svg" alt="AndroidWorld SOTA"></a>
|
||||||
</p>
|
</p>
|
||||||
|
|
||||||
@@ -104,8 +104,73 @@ cd artemis
|
|||||||
|
|
||||||
> PowerShell 默认不会从当前目录查找可执行脚本,因此必须使用 `.\start.bat`,且命令末尾不要添加 `\`。如果使用传统命令提示符(CMD),则运行 `start.bat`。
|
> PowerShell 默认不会从当前目录查找可执行脚本,因此必须使用 `.\start.bat`,且命令末尾不要添加 `\`。如果使用传统命令提示符(CMD),则运行 `start.bat`。
|
||||||
|
|
||||||
|
首次启动会检测 uv:已有则复用,缺少则从 Astral 官方安装。随后检查项目 `.venv`,缺少时自动安装所需的 Python 3.12 并创建环境;已有且满足 Python 3.12+ 要求的环境直接复用。依赖与 `uv.lock` 一致时跳过安装,否则只同步缺失或变化的依赖,保留已有开发工具。安装失败会停止启动并显示原因,不会删除已有环境。
|
||||||
|
|
||||||
|
国内网络可直接双击 **`start-cn.bat`**,或在 PowerShell 中运行:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
.\start-cn.bat
|
||||||
|
# 仅安装 Python 环境,也支持镜像模式
|
||||||
|
.\start-cn.bat -SetupOnly
|
||||||
|
# 与上面的入口等价
|
||||||
|
.\start.bat -Mirror China
|
||||||
|
# 本次使用官方源
|
||||||
|
.\start.bat -Mirror Official
|
||||||
|
```
|
||||||
|
|
||||||
|
下载配置位于 `config/downloads.json`。将 `mode` 改成 `China` 后,普通 `start.bat` 也默认使用国内源;命令行 `-Mirror` 优先。`timeout_seconds` 控制下载超时(默认 180 秒,范围 10~3600 秒)。国内模式使用:
|
||||||
|
|
||||||
|
| 组件 | 下载来源 |
|
||||||
|
| --- | --- |
|
||||||
|
| uv 与 Python 依赖 | 清华 PyPI 镜像;uv 直接从 wheel 提取并校验 SHA256,无需预装 Python |
|
||||||
|
| Python 运行时 | 南京大学 python-build-standalone 镜像 |
|
||||||
|
| Node.js 与 npm 包 | npmmirror |
|
||||||
|
| ADB | 腾讯 Android SDK 镜像 |
|
||||||
|
| FFmpeg | `imageio-ffmpeg` Python 包附带的程序 |
|
||||||
|
| scrcpy | 默认仍使用 GitHub 官方发布,可用下述配置替换 |
|
||||||
|
|
||||||
|
镜像模式从 `uv.lock` 导出固定版本和哈希,再通过所选索引安装,避免锁文件中的官方直链绕过镜像;不会改写锁文件或升级已锁定版本。npm 使用本次启动进程的 registry 配置。所有设置仅影响当前启动进程,不修改用户全局 pip、uv、npm 或 nvm 配置。已有 `UV_DEFAULT_INDEX`(或 `UV_INDEX_URL`)、`UV_PYTHON_INSTALL_MIRROR`、`npm_config_registry` 环境变量优先。
|
||||||
|
|
||||||
|
需要公司内网源或替换不可用镜像时,在 `config/downloads.json` 的 `overrides` 中填写对应 HTTPS 地址;空字符串表示使用所选模式的默认值。`python_index` 是以 `/simple` 结尾的索引,`python_download_mirror` 是 Python 发布目录,`node_mirror` 是包含版本子目录的 Node.js 下载根路径,`adb_url` 和 `scrcpy_url` 是可直接下载的 Windows ZIP 地址。scrcpy 暂无内置国内镜像,GitHub 不可达时请配置可信的 `scrcpy_url` 或预先安装 scrcpy。`uv_installer_url` 只用于没有设置 Python 镜像时的 PowerShell 安装器路径。镜像缺包或下载失败会显示错误;修正地址后重试即可,不会静默切换到官方源。
|
||||||
|
|
||||||
|
镜像说明参考:[清华 PyPI](https://mirrors.tuna.tsinghua.edu.cn/help/pypi/)、[uv 下载镜像配置](https://docs.astral.sh/uv/reference/environment/#uv_python_install_mirror)。
|
||||||
|
|
||||||
|
也可先单独准备 Python 环境,不启动页面、不安装设备工具、不配置 MCP:
|
||||||
|
|
||||||
|
```powershell
|
||||||
|
.\start.bat -SetupOnly
|
||||||
|
# 环境准备好后正常启动
|
||||||
|
.\start.bat
|
||||||
|
# 可选:指定端口、不自动打开浏览器
|
||||||
|
.\start.bat -Port 8001 -NoOpen
|
||||||
|
```
|
||||||
|
|
||||||
|
源码压缩包不携带 `.venv`、`node_modules`、前端编译结果或 APK。解压到可写目录后运行 `start.bat`,首次完整启动会安装所需 Node.js/npm 并构建网页,需要联网;已有构建时跳过。`.env` 不存在时才从 `.env.example` 创建,已有配置不会覆盖。Android 辅助 APK 的源码位于 `packages/artemis-accessibility-helper`,需要该功能时按该目录的构建脚本生成。
|
||||||
|
|
||||||
|
重新打源码包可运行 `uv run --no-sync python scripts/package_source.py`,输出到 `artifacts/`;脚本会排除本地密钥、运行数据、缓存、依赖目录与编译产物,并检查 ZIP 完整性。
|
||||||
|
|
||||||
> **提示**:启动后将自动在默认浏览器中打开 Web 控制台(`http://localhost:8000`),提供设备连接向导、实时投屏、任务演练与状态回放面板。你也可以通过命令行直接运行:`uv run artemis run "打开系统设置,找到电池选项并告诉我当前电量" --profile flash`。
|
> **提示**:启动后将自动在默认浏览器中打开 Web 控制台(`http://localhost:8000`),提供设备连接向导、实时投屏、任务演练与状态回放面板。你也可以通过命令行直接运行:`uv run artemis run "打开系统设置,找到电池选项并告诉我当前电量" --profile flash`。
|
||||||
|
|
||||||
|
### 网页模型配置与功能开关
|
||||||
|
|
||||||
|
Web 控制台使用简体中文。在首页打开「系统设置与运行准备」,可以修改接口地址、模型名称、API Key、请求超时和重试次数。点击「测试连接」会使用当前表单发送一次真实的 Responses 请求;点击「保存配置」才会写入工作目录的 `.env`。密钥留空会保留现有值,网页不会读取或显示已有密钥。
|
||||||
|
|
||||||
|
所有模型调用统一使用 **OpenAI Responses API**,不再提供 Gemini、Anthropic 或 Chat Completions 调用路径。自定义网关必须实现 `/responses`,接口地址填写根地址(通常以 `/v1` 结尾)。例如:
|
||||||
|
|
||||||
|
```dotenv
|
||||||
|
OPENAI_API_KEY=你的密钥
|
||||||
|
OPENAI_BASE_URL=https://你的网关/v1
|
||||||
|
OPENAI_MODEL=mimo-v2.6-flash
|
||||||
|
OPENAI_TIMEOUT_SECONDS=120
|
||||||
|
OPENAI_MAX_RETRIES=0
|
||||||
|
```
|
||||||
|
|
||||||
|
「功能开关」提供视频分析与录屏、最终报告生成、Flash 步骤摘要压缩,可分别保存。设置对**新任务**生效,重启后保留。环境变量优先于 `.env`;在网页保存时会同时更新当前服务进程的配置,服务的启动环境若显式定义了同名变量,后续重启仍以启动环境为准。手动配置的节点模型或任务级选项优先于全局默认值。
|
||||||
|
|
||||||
|
视频分析通过抽取画面帧调用模型,因此需要支持图片输入的模型。`mimo-v2.6-flash` 已验证文本、图片、工具调用、流式输出及结构化函数调用;对于不支持 `json_schema` 的网关,结构化结果使用 Responses 函数调用并在本地校验。音频输入是否可用取决于端点能力,拒绝音频时视频分析会尝试仅分析画面。
|
||||||
|
|
||||||
|
视频分段时长、分析超时、检查点超时等配置及单位见 [`.env.example`](.env.example)。可选 Cloud Vision OCR 独立于模型调用,未配置 OCR 密钥也可运行。
|
||||||
|
|
||||||
<a id="mcp-setup"></a>
|
<a id="mcp-setup"></a>
|
||||||
<a id="mcp"></a>
|
<a id="mcp"></a>
|
||||||
<details>
|
<details>
|
||||||
|
|||||||
@@ -86,14 +86,14 @@ class ReplayManager:
|
|||||||
It reconstructs historical execution states (SQLite database records, trace
|
It reconstructs historical execution states (SQLite database records, trace
|
||||||
histories, and screenshots) and replays a single agent step in an isolated,
|
histories, and screenshots) and replays a single agent step in an isolated,
|
||||||
deterministic sandbox environment while interacting live with a physical
|
deterministic sandbox environment while interacting live with a physical
|
||||||
Android device and Gemini.
|
Android device and OpenAI Responses.
|
||||||
|
|
||||||
Key Responsibilities:
|
Key Responsibilities:
|
||||||
- Reconstructs and preloads historical SQLite states up to a specific
|
- Reconstructs and preloads historical SQLite states up to a specific
|
||||||
step.
|
step.
|
||||||
- Establishes sandboxed `ArtemisContext` and `DataEngine` instances.
|
- Establishes sandboxed `ArtemisContext` and `DataEngine` instances.
|
||||||
- Integrates with physical devices via ADB and UIAutomator.
|
- Integrates with physical devices via ADB and UIAutomator.
|
||||||
- Hooks and spies on the Gemini API to capture thoughts and raw outputs.
|
- Records Responses model calls to capture thoughts and raw outputs.
|
||||||
- Reconciles live and preloaded traces into unified chronological trees.
|
- Reconciles live and preloaded traces into unified chronological trees.
|
||||||
|
|
||||||
Attributes:
|
Attributes:
|
||||||
@@ -1552,8 +1552,8 @@ class ReplayManager:
|
|||||||
# Update values
|
# Update values
|
||||||
conn_temp.execute(
|
conn_temp.execute(
|
||||||
"""
|
"""
|
||||||
UPDATE video_recordings
|
UPDATE video_recordings
|
||||||
SET local_video_path = ?, start_time = ?, end_time = ?
|
SET local_video_path = ?, start_time = ?, end_time = ?
|
||||||
WHERE session_id = ?
|
WHERE session_id = ?
|
||||||
""",
|
""",
|
||||||
(
|
(
|
||||||
@@ -1583,8 +1583,8 @@ class ReplayManager:
|
|||||||
|
|
||||||
conn_temp.execute(
|
conn_temp.execute(
|
||||||
"""
|
"""
|
||||||
UPDATE video_recordings
|
UPDATE video_recordings
|
||||||
SET local_video_path = ?, start_time = ?, end_time = ?
|
SET local_video_path = ?, start_time = ?, end_time = ?
|
||||||
WHERE session_id = ?
|
WHERE session_id = ?
|
||||||
""",
|
""",
|
||||||
(
|
(
|
||||||
@@ -1885,121 +1885,34 @@ class ReplayManager:
|
|||||||
|
|
||||||
data_engine = ctx.data_engine
|
data_engine = ctx.data_engine
|
||||||
|
|
||||||
print("Executing using Live Gemini Client...")
|
print("Executing using OpenAI Responses...")
|
||||||
|
|
||||||
from google.genai.models import AsyncModels
|
from langchain_openai import ChatOpenAI
|
||||||
from unittest.mock import patch
|
from unittest.mock import patch
|
||||||
|
|
||||||
original_generate_content = AsyncModels.generate_content
|
original_generate = ChatOpenAI._agenerate
|
||||||
|
|
||||||
async def wrapped_generate_content(self, *, model, contents, config=None, **kwargs):
|
async def wrapped_generate(model, messages, *args, **kwargs):
|
||||||
system_instruction = None
|
record = {
|
||||||
if config and config.system_instruction:
|
"prompt": {
|
||||||
if isinstance(config.system_instruction, str):
|
"model": model.model_name,
|
||||||
system_instruction = config.system_instruction
|
"contents": [
|
||||||
elif hasattr(config.system_instruction, "parts"):
|
{"role": message.type, "parts": message.content} for message in messages
|
||||||
parts = []
|
],
|
||||||
for p in config.system_instruction.parts:
|
},
|
||||||
if hasattr(p, "text") and p.text:
|
"response": None,
|
||||||
parts.append(p.text)
|
"error": None,
|
||||||
system_instruction = " ".join(parts)
|
|
||||||
else:
|
|
||||||
system_instruction = str(config.system_instruction)
|
|
||||||
|
|
||||||
prompt_data = {
|
|
||||||
"model": model,
|
|
||||||
"system_instruction": system_instruction,
|
|
||||||
"contents": [],
|
|
||||||
}
|
}
|
||||||
|
|
||||||
contents_list = contents if isinstance(contents, list) else [contents]
|
|
||||||
for content in contents_list:
|
|
||||||
parts_desc = []
|
|
||||||
if hasattr(content, "parts") and content.parts:
|
|
||||||
for part in content.parts:
|
|
||||||
if hasattr(part, "text") and part.text:
|
|
||||||
parts_desc.append({"type": "text", "text": part.text})
|
|
||||||
elif hasattr(part, "inline_data") and part.inline_data:
|
|
||||||
parts_desc.append(
|
|
||||||
{
|
|
||||||
"type": "image",
|
|
||||||
"mime_type": part.inline_data.mime_type,
|
|
||||||
"length": len(part.inline_data.data),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
elif hasattr(part, "function_call") and part.function_call:
|
|
||||||
parts_desc.append(
|
|
||||||
{
|
|
||||||
"type": "function_call",
|
|
||||||
"name": part.function_call.name,
|
|
||||||
"args": (
|
|
||||||
dict(part.function_call.args)
|
|
||||||
if part.function_call.args
|
|
||||||
else {}
|
|
||||||
),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
elif hasattr(part, "function_call") and part.function_response:
|
|
||||||
parts_desc.append(
|
|
||||||
{
|
|
||||||
"type": "function_response",
|
|
||||||
"name": part.function_response.name,
|
|
||||||
"response": (
|
|
||||||
dict(part.function_response.response)
|
|
||||||
if part.function_response.response
|
|
||||||
else {}
|
|
||||||
),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
parts_desc.append({"type": "raw", "value": str(content)})
|
|
||||||
|
|
||||||
prompt_data["contents"].append(
|
|
||||||
{
|
|
||||||
"role": getattr(content, "role", "unknown"),
|
|
||||||
"parts": parts_desc,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
record = {"prompt": prompt_data, "response": None, "error": None}
|
|
||||||
llm_calls_log.append(record)
|
llm_calls_log.append(record)
|
||||||
|
|
||||||
try:
|
try:
|
||||||
res = await original_generate_content(
|
result = await original_generate(model, messages, *args, **kwargs)
|
||||||
self,
|
record["response"] = [
|
||||||
model=model,
|
generation.message.model_dump(mode="json") for generation in result.generations
|
||||||
contents=contents,
|
]
|
||||||
config=config,
|
return result
|
||||||
**kwargs,
|
except Exception as error:
|
||||||
)
|
record["error"] = error.__class__.__name__
|
||||||
|
raise
|
||||||
thinking_parts = []
|
|
||||||
if hasattr(res, "candidates") and res.candidates and res.candidates[0].content:
|
|
||||||
for part in res.candidates[0].content.parts:
|
|
||||||
if getattr(part, "thought", False):
|
|
||||||
thinking_parts.append(part.text)
|
|
||||||
|
|
||||||
resp_data = {
|
|
||||||
"text": res.text if hasattr(res, "text") else None,
|
|
||||||
"function_calls": [],
|
|
||||||
"thought": ("\n".join(thinking_parts) if thinking_parts else None),
|
|
||||||
}
|
|
||||||
if hasattr(res, "function_calls") and res.function_calls:
|
|
||||||
for fc in res.function_calls:
|
|
||||||
resp_data["function_calls"].append(
|
|
||||||
{
|
|
||||||
"name": fc.name,
|
|
||||||
"args": dict(fc.args) if fc.args else {},
|
|
||||||
}
|
|
||||||
)
|
|
||||||
record["response"] = resp_data
|
|
||||||
return res
|
|
||||||
except Exception as err:
|
|
||||||
err_str = str(err)
|
|
||||||
record["error"] = (
|
|
||||||
f"{err.__class__.__name__}: {err_str}" if err_str else err.__class__.__name__
|
|
||||||
)
|
|
||||||
raise err
|
|
||||||
|
|
||||||
try:
|
try:
|
||||||
try:
|
try:
|
||||||
@@ -2042,7 +1955,7 @@ class ReplayManager:
|
|||||||
val = resolved_val
|
val = resolved_val
|
||||||
tool_args[name] = val
|
tool_args[name] = val
|
||||||
|
|
||||||
with patch.object(AsyncModels, "generate_content", wrapped_generate_content):
|
with patch.object(ChatOpenAI, "_agenerate", wrapped_generate):
|
||||||
outcome = await tool_fn(**tool_args)
|
outcome = await tool_fn(**tool_args)
|
||||||
|
|
||||||
replay_result["success"] = True
|
replay_result["success"] = True
|
||||||
|
|||||||
@@ -31,6 +31,63 @@ from artemis.core.diagnostics.schema import SystemReadinessReport
|
|||||||
|
|
||||||
router = APIRouter(prefix="/api/system", tags=["system"])
|
router = APIRouter(prefix="/api/system", tags=["system"])
|
||||||
|
|
||||||
|
from apps.admin_console.services.model_settings_service import (
|
||||||
|
ModelSettingsUpdate,
|
||||||
|
get_model_settings,
|
||||||
|
save_model_settings,
|
||||||
|
test_model_settings,
|
||||||
|
CapabilitySettings,
|
||||||
|
get_capability_settings,
|
||||||
|
save_capability_settings,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/capability-settings")
|
||||||
|
async def read_capability_settings():
|
||||||
|
return get_capability_settings()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/capability-settings")
|
||||||
|
async def write_capability_settings(payload: CapabilitySettings):
|
||||||
|
try:
|
||||||
|
return save_capability_settings(payload)
|
||||||
|
except OSError as exc:
|
||||||
|
raise HTTPException(
|
||||||
|
status_code=500, detail="无法保存功能开关,请检查 .env 文件权限"
|
||||||
|
) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/model-settings")
|
||||||
|
async def read_model_settings():
|
||||||
|
return get_model_settings()
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/model-settings")
|
||||||
|
async def write_model_settings(payload: ModelSettingsUpdate):
|
||||||
|
try:
|
||||||
|
result = save_model_settings(payload)
|
||||||
|
except OSError as exc:
|
||||||
|
raise HTTPException(status_code=500, detail="无法写入 .env,请检查文件权限") from exc
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=400, detail="API Key 格式不正确,请检查后重试") from exc
|
||||||
|
await readiness_engine.run_all(force_refresh=True)
|
||||||
|
return {**result, "message": "配置已保存,新任务立即生效,重启后自动加载。"}
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/model-settings/test")
|
||||||
|
async def check_model_settings(payload: ModelSettingsUpdate):
|
||||||
|
try:
|
||||||
|
return await test_model_settings(payload)
|
||||||
|
except Exception as exc:
|
||||||
|
# SDK exceptions may contain URLs or request data; never echo credentials.
|
||||||
|
status = getattr(exc, "status_code", None)
|
||||||
|
detail = (
|
||||||
|
f"Responses 连接失败(HTTP {status})"
|
||||||
|
if status
|
||||||
|
else "Responses 连接失败,请检查接口地址、模型、密钥和超时设置。"
|
||||||
|
)
|
||||||
|
raise HTTPException(status_code=400, detail=detail) from exc
|
||||||
|
|
||||||
|
|
||||||
def _require_local_admin_request(request: Request) -> None:
|
def _require_local_admin_request(request: Request) -> None:
|
||||||
"""Keep endpoint probing and mutation on the local administration boundary."""
|
"""Keep endpoint probing and mutation on the local administration boundary."""
|
||||||
@@ -264,8 +321,8 @@ class UpdateCredentialsRequest(BaseModel):
|
|||||||
"""Payload to update and configure LLM or Vision OCR API credentials."""
|
"""Payload to update and configure LLM or Vision OCR API credentials."""
|
||||||
|
|
||||||
provider: str = Field(
|
provider: str = Field(
|
||||||
default="google",
|
default="openai",
|
||||||
description="Provider identifier (e.g. google, gemini, openai, anthropic, openrouter, ocr)",
|
description="Provider identifier (openai or ocr)",
|
||||||
)
|
)
|
||||||
api_key: str = Field(description="The secret API key string to configure")
|
api_key: str = Field(description="The secret API key string to configure")
|
||||||
persist_to_env: bool = Field(
|
persist_to_env: bool = Field(
|
||||||
@@ -277,8 +334,8 @@ class ValidateCredentialsRequest(BaseModel):
|
|||||||
"""Payload to test and verify LLM or Vision OCR API credentials without saving."""
|
"""Payload to test and verify LLM or Vision OCR API credentials without saving."""
|
||||||
|
|
||||||
provider: str = Field(
|
provider: str = Field(
|
||||||
default="google",
|
default="openai",
|
||||||
description="Provider identifier (e.g. google, gemini, openai, anthropic, openrouter, ocr)",
|
description="Provider identifier (openai or ocr)",
|
||||||
)
|
)
|
||||||
api_key: str = Field(description="The secret API key string to test")
|
api_key: str = Field(description="The secret API key string to test")
|
||||||
base_url: str | None = Field(default=None, description="Optional custom base URL")
|
base_url: str | None = Field(default=None, description="Optional custom base URL")
|
||||||
@@ -294,9 +351,8 @@ async def get_credentials():
|
|||||||
"""
|
"""
|
||||||
from artemis.config import settings
|
from artemis.config import settings
|
||||||
|
|
||||||
providers = ("google", "openai", "anthropic", "openrouter", "ocr")
|
providers = ("openai", "ocr")
|
||||||
status = {name: bool(settings.get_api_key(name)) for name in providers}
|
status = {name: bool(settings.get_api_key(name)) for name in providers}
|
||||||
status["gemini"] = status["google"]
|
|
||||||
return {
|
return {
|
||||||
"providers": [
|
"providers": [
|
||||||
{"name": name, "configured": configured} for name, configured in status.items()
|
{"name": name, "configured": configured} for name, configured in status.items()
|
||||||
@@ -415,13 +471,7 @@ async def get_model_config_and_env():
|
|||||||
return "****"
|
return "****"
|
||||||
return f"****{k[-4:]}"
|
return f"****{k[-4:]}"
|
||||||
|
|
||||||
gemini_real = get_real_env_value("GEMINI_API_KEY", "google") or get_real_env_value(
|
|
||||||
"GOOGLE_API_KEY", "google"
|
|
||||||
)
|
|
||||||
openai_real = get_real_env_value("OPENAI_API_KEY", "openai")
|
openai_real = get_real_env_value("OPENAI_API_KEY", "openai")
|
||||||
anthropic_real = get_real_env_value("ANTHROPIC_API_KEY", "anthropic")
|
|
||||||
openrouter_real = get_real_env_value("OPEN_ROUTER_API_KEY", "openrouter")
|
|
||||||
xai_real = get_real_env_value("XAI_API_KEY", "xai")
|
|
||||||
base_url_val = settings.OPENAI_BASE_URL or os.environ.get("OPENAI_BASE_URL")
|
base_url_val = settings.OPENAI_BASE_URL or os.environ.get("OPENAI_BASE_URL")
|
||||||
if base_url_val and is_placeholder_key(base_url_val):
|
if base_url_val and is_placeholder_key(base_url_val):
|
||||||
base_url_val = None
|
base_url_val = None
|
||||||
@@ -430,47 +480,19 @@ async def get_model_config_and_env():
|
|||||||
)
|
)
|
||||||
|
|
||||||
env_vars = [
|
env_vars = [
|
||||||
{
|
|
||||||
"name": "GEMINI_API_KEY",
|
|
||||||
"provider": "google",
|
|
||||||
"is_set": bool(gemini_real),
|
|
||||||
"preview": mask_key(gemini_real),
|
|
||||||
"description": "Google Gemini multimodal vision API key (free tier available)",
|
|
||||||
},
|
|
||||||
{
|
{
|
||||||
"name": "OPENAI_API_KEY",
|
"name": "OPENAI_API_KEY",
|
||||||
"provider": "openai",
|
"provider": "openai",
|
||||||
"is_set": bool(openai_real),
|
"is_set": bool(openai_real),
|
||||||
"preview": mask_key(openai_real),
|
"preview": mask_key(openai_real),
|
||||||
"description": "OpenAI API key (GPT-4o, GPT-4o-mini)",
|
"description": "OpenAI Responses API 密钥",
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "ANTHROPIC_API_KEY",
|
|
||||||
"provider": "anthropic",
|
|
||||||
"is_set": bool(anthropic_real),
|
|
||||||
"preview": mask_key(anthropic_real),
|
|
||||||
"description": "Anthropic Claude API key (Claude 3.5 Sonnet, Claude 3.7)",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "OPEN_ROUTER_API_KEY",
|
|
||||||
"provider": "openrouter",
|
|
||||||
"is_set": bool(openrouter_real),
|
|
||||||
"preview": mask_key(openrouter_real),
|
|
||||||
"description": "OpenRouter unified API gateway key",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "XAI_API_KEY",
|
|
||||||
"provider": "xai",
|
|
||||||
"is_set": bool(xai_real),
|
|
||||||
"preview": mask_key(xai_real),
|
|
||||||
"description": "xAI Grok vision API key",
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"name": "OPENAI_BASE_URL",
|
"name": "OPENAI_BASE_URL",
|
||||||
"provider": "custom",
|
"provider": "openai",
|
||||||
"is_set": bool(base_url_val),
|
"is_set": bool(base_url_val),
|
||||||
"preview": base_url_val,
|
"preview": base_url_val,
|
||||||
"description": "Custom API endpoint (for local Ollama, vLLM, DeepSeek, or proxies)",
|
"description": "OpenAI Responses 接口根地址(包含 /v1,不包含 /responses)",
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"name": "VISION_API_KEY",
|
"name": "VISION_API_KEY",
|
||||||
|
|||||||
@@ -12,6 +12,8 @@
|
|||||||
# See the License for the specific language governing permissions and
|
# See the License for the specific language governing permissions and
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
|
from artemis.config.settings import settings
|
||||||
|
|
||||||
import json
|
import json
|
||||||
import logging
|
import logging
|
||||||
import time
|
import time
|
||||||
@@ -26,7 +28,7 @@ class ModelService:
|
|||||||
# parse_llm_config re-reads and re-validates the config file on every call;
|
# parse_llm_config re-reads and re-validates the config file on every call;
|
||||||
# /api/sessions calls this once per session row, so cache (provider, model_id)
|
# /api/sessions calls this once per session row, so cache (provider, model_id)
|
||||||
# for a short TTL. Config edits take effect within _LLM_INFO_TTL seconds.
|
# for a short TTL. Config edits take effect within _LLM_INFO_TTL seconds.
|
||||||
_LLM_INFO_TTL = 10.0
|
_LLM_INFO_TTL = settings.ARTEMIS_MODEL_CACHE_TTL_SECONDS
|
||||||
_llm_info_cache: tuple[float, str, str] | None = None
|
_llm_info_cache: tuple[float, str, str] | None = None
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -35,8 +37,8 @@ class ModelService:
|
|||||||
now = time.monotonic()
|
now = time.monotonic()
|
||||||
if cached and now - cached[0] < cls._LLM_INFO_TTL:
|
if cached and now - cached[0] < cls._LLM_INFO_TTL:
|
||||||
return cached[1], cached[2]
|
return cached[1], cached[2]
|
||||||
provider = "google"
|
provider = "openai"
|
||||||
model_id = "gemini-3.7-flash"
|
model_id = settings.OPENAI_MODEL
|
||||||
try:
|
try:
|
||||||
from artemis.config import parse_llm_config
|
from artemis.config import parse_llm_config
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,161 @@
|
|||||||
|
"""Persist the single Responses endpoint without returning credentials to the browser."""
|
||||||
|
|
||||||
|
import os
|
||||||
|
import tempfile
|
||||||
|
import threading
|
||||||
|
from pathlib import Path
|
||||||
|
from urllib.parse import urlsplit
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field, SecretStr, field_validator
|
||||||
|
|
||||||
|
from artemis.config.paths import get_env_file
|
||||||
|
from artemis.config.settings import settings
|
||||||
|
|
||||||
|
_write_lock = threading.Lock()
|
||||||
|
|
||||||
|
|
||||||
|
class ModelSettingsUpdate(BaseModel):
|
||||||
|
base_url: str = "https://api.openai.com/v1"
|
||||||
|
model: str = Field(min_length=1, max_length=200)
|
||||||
|
api_key: SecretStr | None = None # Omitted/blank keeps the existing key.
|
||||||
|
timeout_seconds: float = Field(default=120, gt=0, le=3600)
|
||||||
|
max_retries: int = Field(default=0, ge=0, le=10)
|
||||||
|
|
||||||
|
@field_validator("base_url")
|
||||||
|
@classmethod
|
||||||
|
def validate_base_url(cls, value: str) -> str:
|
||||||
|
value = value.strip().rstrip("/")
|
||||||
|
parsed = urlsplit(value)
|
||||||
|
if (
|
||||||
|
parsed.scheme not in ("https", "http")
|
||||||
|
or not parsed.hostname
|
||||||
|
or parsed.username
|
||||||
|
or parsed.password
|
||||||
|
or parsed.query
|
||||||
|
or parsed.fragment
|
||||||
|
):
|
||||||
|
raise ValueError("请输入有效的 HTTP(S) 接口根地址,不要包含密钥、查询参数或片段")
|
||||||
|
if value.endswith(("/responses", "/chat/completions")):
|
||||||
|
raise ValueError("请输入接口根地址,例如 https://api.openai.com/v1")
|
||||||
|
return value
|
||||||
|
|
||||||
|
@field_validator("model")
|
||||||
|
@classmethod
|
||||||
|
def validate_model(cls, value: str) -> str:
|
||||||
|
value = value.strip()
|
||||||
|
if not value or any(c in value for c in "\r\n\x00"):
|
||||||
|
raise ValueError("模型名称不能为空或包含换行")
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def get_model_settings() -> dict:
|
||||||
|
return {
|
||||||
|
"provider": "openai",
|
||||||
|
"api_mode": "responses",
|
||||||
|
"base_url": settings.OPENAI_BASE_URL or "https://api.openai.com/v1",
|
||||||
|
"model": settings.OPENAI_MODEL,
|
||||||
|
"api_key_configured": bool(settings.OPENAI_API_KEY),
|
||||||
|
"timeout_seconds": settings.OPENAI_TIMEOUT_SECONDS,
|
||||||
|
"max_retries": settings.OPENAI_MAX_RETRIES,
|
||||||
|
"env_path": str(get_env_file()),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def save_model_settings(update: ModelSettingsUpdate) -> dict:
|
||||||
|
if update.api_key and any(c in update.api_key.get_secret_value() for c in "\r\n\x00"):
|
||||||
|
raise ValueError("API Key 不能包含换行")
|
||||||
|
values = {
|
||||||
|
"OPENAI_BASE_URL": update.base_url,
|
||||||
|
"OPENAI_MODEL": update.model,
|
||||||
|
"OPENAI_TIMEOUT_SECONDS": str(update.timeout_seconds),
|
||||||
|
"OPENAI_MAX_RETRIES": str(update.max_retries),
|
||||||
|
}
|
||||||
|
if update.api_key and update.api_key.get_secret_value().strip():
|
||||||
|
values["OPENAI_API_KEY"] = update.api_key.get_secret_value().strip()
|
||||||
|
persist_environment(values)
|
||||||
|
settings.OPENAI_BASE_URL = update.base_url
|
||||||
|
settings.OPENAI_MODEL = update.model
|
||||||
|
settings.OPENAI_TIMEOUT_SECONDS = update.timeout_seconds
|
||||||
|
settings.OPENAI_MAX_RETRIES = update.max_retries
|
||||||
|
if "OPENAI_API_KEY" in values:
|
||||||
|
settings.OPENAI_API_KEY = SecretStr(values["OPENAI_API_KEY"])
|
||||||
|
from artemis.llm.router import ModelFactory
|
||||||
|
from apps.admin_console.services.model_service import ModelService
|
||||||
|
|
||||||
|
ModelFactory._cache.clear()
|
||||||
|
ModelService._llm_info_cache = None
|
||||||
|
return get_model_settings()
|
||||||
|
|
||||||
|
|
||||||
|
def persist_environment(values: dict[str, str]) -> None:
|
||||||
|
with _write_lock:
|
||||||
|
env_path = get_env_file()
|
||||||
|
env_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
original = env_path.read_text(encoding="utf-8") if env_path.exists() else ""
|
||||||
|
# Preserve comments and unrelated variables; quote values to avoid dotenv interpolation.
|
||||||
|
kept = [
|
||||||
|
line
|
||||||
|
for line in original.splitlines()
|
||||||
|
if line.removeprefix("export ").split("=", 1)[0].strip() not in values
|
||||||
|
]
|
||||||
|
for key, value in values.items():
|
||||||
|
escaped = value.replace("\\", "\\\\").replace("'", "\\'")
|
||||||
|
kept.append(f"{key}='{escaped}'")
|
||||||
|
content = "\n".join(kept) + "\n"
|
||||||
|
fd, temp_name = tempfile.mkstemp(prefix=".env-", dir=env_path.parent)
|
||||||
|
try:
|
||||||
|
with os.fdopen(fd, "w", encoding="utf-8", newline="\n") as stream:
|
||||||
|
stream.write(content)
|
||||||
|
stream.flush()
|
||||||
|
os.fsync(stream.fileno())
|
||||||
|
os.replace(temp_name, env_path)
|
||||||
|
finally:
|
||||||
|
Path(temp_name).unlink(missing_ok=True)
|
||||||
|
for key, value in values.items():
|
||||||
|
os.environ[key] = value
|
||||||
|
|
||||||
|
|
||||||
|
class CapabilitySettings(BaseModel):
|
||||||
|
video_enabled: bool = True
|
||||||
|
outputter_enabled: bool = True
|
||||||
|
step_summarizer_enabled: bool = True
|
||||||
|
|
||||||
|
|
||||||
|
def get_capability_settings() -> CapabilitySettings:
|
||||||
|
return CapabilitySettings(
|
||||||
|
video_enabled=settings.ARTEMIS_VIDEO_ENABLED,
|
||||||
|
outputter_enabled=settings.ARTEMIS_OUTPUTTER_ENABLED,
|
||||||
|
step_summarizer_enabled=settings.ARTEMIS_STEP_SUMMARIZER_ENABLED,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def save_capability_settings(update: CapabilitySettings) -> CapabilitySettings:
|
||||||
|
values = {
|
||||||
|
"ARTEMIS_VIDEO_ENABLED": update.video_enabled,
|
||||||
|
"ARTEMIS_OUTPUTTER_ENABLED": update.outputter_enabled,
|
||||||
|
"ARTEMIS_STEP_SUMMARIZER_ENABLED": update.step_summarizer_enabled,
|
||||||
|
}
|
||||||
|
persist_environment({key: str(value).lower() for key, value in values.items()})
|
||||||
|
for key, value in values.items():
|
||||||
|
setattr(settings, key, value)
|
||||||
|
return get_capability_settings()
|
||||||
|
|
||||||
|
|
||||||
|
async def test_model_settings(update: ModelSettingsUpdate) -> dict:
|
||||||
|
from langchain_core.messages import HumanMessage
|
||||||
|
from artemis.llm.router import ModelEndpoint, ModelFactory
|
||||||
|
|
||||||
|
key = update.api_key.get_secret_value().strip() if update.api_key else None
|
||||||
|
endpoint = ModelEndpoint(
|
||||||
|
model_name=update.model,
|
||||||
|
api_key=key or None,
|
||||||
|
api_base=update.base_url,
|
||||||
|
timeout_seconds=update.timeout_seconds,
|
||||||
|
max_retries=0,
|
||||||
|
max_tokens=64,
|
||||||
|
)
|
||||||
|
model = ModelFactory.create_model(endpoint)
|
||||||
|
result = await model.ainvoke([HumanMessage(content="Reply with OK only.")])
|
||||||
|
if not result.content:
|
||||||
|
raise ValueError("Responses 接口返回空内容,请检查模型配置")
|
||||||
|
return {"success": True, "message": "Responses 连接测试成功", "model": update.model}
|
||||||
@@ -1,3 +1,6 @@
|
|||||||
|
import { registerLocaleData } from '@angular/common';
|
||||||
|
import zh from '@angular/common/locales/zh';
|
||||||
|
registerLocaleData(zh);
|
||||||
/**
|
/**
|
||||||
* Copyright 2026 Google LLC
|
* Copyright 2026 Google LLC
|
||||||
*
|
*
|
||||||
@@ -14,7 +17,7 @@
|
|||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { ApplicationConfig, provideZoneChangeDetection } from '@angular/core';
|
import { ApplicationConfig, LOCALE_ID, provideZoneChangeDetection } from '@angular/core';
|
||||||
import { provideRouter } from '@angular/router';
|
import { provideRouter } from '@angular/router';
|
||||||
import { provideHttpClient, withXhr } from '@angular/common/http';
|
import { provideHttpClient, withXhr } from '@angular/common/http';
|
||||||
|
|
||||||
@@ -22,6 +25,7 @@ import { routes } from './app.routes';
|
|||||||
|
|
||||||
export const appConfig: ApplicationConfig = {
|
export const appConfig: ApplicationConfig = {
|
||||||
providers: [
|
providers: [
|
||||||
|
{ provide: LOCALE_ID, useValue: 'zh-CN' },
|
||||||
provideZoneChangeDetection({ eventCoalescing: true }),
|
provideZoneChangeDetection({ eventCoalescing: true }),
|
||||||
provideRouter(routes),
|
provideRouter(routes),
|
||||||
provideHttpClient(withXhr())
|
provideHttpClient(withXhr())
|
||||||
|
|||||||
@@ -4,17 +4,17 @@ import { buildStartupWorkItems } from './agent-stream.component';
|
|||||||
describe('startup Work block', () => {
|
describe('startup Work block', () => {
|
||||||
it('shows only the three device preparation operations', () => {
|
it('shows only the three device preparation operations', () => {
|
||||||
const events: StartupProgressEvent[] = [
|
const events: StartupProgressEvent[] = [
|
||||||
{ stage: 'submitting', message: 'Submitting the task', timestamp: 100 },
|
{ stage: 'submitting', message: '正在提交任务', timestamp: 100 },
|
||||||
{ stage: 'queued', message: 'Task received and queued', timestamp: 100.9 },
|
{ stage: 'queued', message: 'Task received and queued', timestamp: 100.9 },
|
||||||
{ stage: 'launching', message: 'Starting the execution process', timestamp: 101 },
|
{ stage: 'launching', message: 'Starting the execution process', timestamp: 101 },
|
||||||
{ stage: 'configuration', message: 'Loading the run configuration', timestamp: 103.1 },
|
{ stage: 'configuration', message: 'Loading the run configuration', timestamp: 103.1 },
|
||||||
{ stage: 'device_check', message: 'Checking the Android device', timestamp: 103.2 },
|
{ stage: 'device_check', message: 'Checking the Android device', timestamp: 103.2 },
|
||||||
{ stage: 'device_ready', message: 'Android device connected', timestamp: 123 },
|
{ stage: 'device_ready', message: 'Android 设备已连接', timestamp: 123 },
|
||||||
{ stage: 'model_warmup', message: 'Warming the model connection', timestamp: 123 },
|
{ stage: 'model_warmup', message: 'Warming the model connection', timestamp: 123 },
|
||||||
{ stage: 'uiautomator', message: 'Connecting to UI Automator', timestamp: 124 },
|
{ stage: 'uiautomator', message: 'Connecting to UI Automator', timestamp: 124 },
|
||||||
{ stage: 'uiautomator_ready', message: 'UI Automator is ready', timestamp: 128 },
|
{ stage: 'uiautomator_ready', message: 'UI Automator is ready', timestamp: 128 },
|
||||||
{ stage: 'environment', message: 'Preparing the device environment', timestamp: 128 },
|
{ stage: 'environment', message: 'Preparing the device environment', timestamp: 128 },
|
||||||
{ stage: 'environment_ready', message: 'Device environment is ready', timestamp: 131 }
|
{ stage: 'environment_ready', message: '设备环境已就绪', timestamp: 131 }
|
||||||
];
|
];
|
||||||
|
|
||||||
const items = buildStartupWorkItems(events, 131, false, true);
|
const items = buildStartupWorkItems(events, 131, false, true);
|
||||||
@@ -90,7 +90,7 @@ describe('startup Work block', () => {
|
|||||||
|
|
||||||
const [item] = buildStartupWorkItems(events, 20, false, true);
|
const [item] = buildStartupWorkItems(events, 20, false, true);
|
||||||
|
|
||||||
expect(item.message).toBe('UI hierarchy service is ready');
|
expect(item.message).toBe('界面层级服务已就绪');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('uses old first_response events only as a hidden completion boundary', () => {
|
it('uses old first_response events only as a hidden completion boundary', () => {
|
||||||
@@ -101,7 +101,7 @@ describe('startup Work block', () => {
|
|||||||
|
|
||||||
const [item] = buildStartupWorkItems(events, 24, false, true);
|
const [item] = buildStartupWorkItems(events, 24, false, true);
|
||||||
|
|
||||||
expect(item.message).toBe('Device environment is ready');
|
expect(item.message).toBe('设备环境已就绪');
|
||||||
expect(item.elapsed).toBe('3.0s');
|
expect(item.elapsed).toBe('3.0s');
|
||||||
expect(item.isActive).toBeFalse();
|
expect(item.isActive).toBeFalse();
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -24,11 +24,11 @@ limitations under the License.
|
|||||||
class="vscode-tab-btn"
|
class="vscode-tab-btn"
|
||||||
[class.active]="isTaskDropdownOpen()"
|
[class.active]="isTaskDropdownOpen()"
|
||||||
(click)="toggleTaskDropdown($event)"
|
(click)="toggleTaskDropdown($event)"
|
||||||
title="Task Queue"
|
title="任务队列"
|
||||||
aria-label="Task Queue"
|
aria-label="任务队列"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined tab-icon">list_alt</span>
|
<span class="material-symbols-outlined tab-icon">list_alt</span>
|
||||||
<span class="tab-label">Task Queue</span>
|
<span class="tab-label">任务队列</span>
|
||||||
</button>
|
</button>
|
||||||
<button
|
<button
|
||||||
#notesOrigin="cdkOverlayOrigin"
|
#notesOrigin="cdkOverlayOrigin"
|
||||||
@@ -36,11 +36,11 @@ limitations under the License.
|
|||||||
class="vscode-tab-btn"
|
class="vscode-tab-btn"
|
||||||
[class.active]="isNotesDropdownOpen()"
|
[class.active]="isNotesDropdownOpen()"
|
||||||
(click)="toggleNotesDropdown($event)"
|
(click)="toggleNotesDropdown($event)"
|
||||||
title="Notes & Plans"
|
title="笔记与计划"
|
||||||
aria-label="Notes & Plans"
|
aria-label="笔记与计划"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined tab-icon">description</span>
|
<span class="material-symbols-outlined tab-icon">description</span>
|
||||||
<span class="tab-label">Notes & Plans</span>
|
<span class="tab-label">笔记与计划</span>
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -57,7 +57,7 @@ limitations under the License.
|
|||||||
<div class="vscode-dropdown-menu task-dropdown">
|
<div class="vscode-dropdown-menu task-dropdown">
|
||||||
<div class="popover-title-header">
|
<div class="popover-title-header">
|
||||||
<span class="material-symbols-outlined">list_alt</span>
|
<span class="material-symbols-outlined">list_alt</span>
|
||||||
<span>Task Queue</span>
|
<span>任务队列</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="dropdown-section-label">
|
<div class="dropdown-section-label">
|
||||||
<span class="material-symbols-outlined icon-bolt">bolt</span>
|
<span class="material-symbols-outlined icon-bolt">bolt</span>
|
||||||
@@ -65,7 +65,8 @@ limitations under the License.
|
|||||||
</div>
|
</div>
|
||||||
@if (activeQueue().length === 0) {
|
@if (activeQueue().length === 0) {
|
||||||
<div class="dropdown-empty-state">
|
<div class="dropdown-empty-state">
|
||||||
No active or queued tasks
|
|
||||||
|
没有正在执行或排队的任务
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@for (session of activeQueue(); track trackSession($index, session)) {
|
@for (session of activeQueue(); track trackSession($index, session)) {
|
||||||
@@ -80,7 +81,7 @@ limitations under the License.
|
|||||||
@if (getTaskStatus(session) === 'running') {
|
@if (getTaskStatus(session) === 'running') {
|
||||||
<span class="status-dot"></span>
|
<span class="status-dot"></span>
|
||||||
}
|
}
|
||||||
{{ getTaskStatus(session) | uppercase }}
|
{{ getTaskStatus(session) | uiLabel }}
|
||||||
</span>
|
</span>
|
||||||
<div class="item-header-actions">
|
<div class="item-header-actions">
|
||||||
@if (getDeviceSerial(session)) {
|
@if (getDeviceSerial(session)) {
|
||||||
@@ -98,8 +99,8 @@ limitations under the License.
|
|||||||
type="button"
|
type="button"
|
||||||
class="btn-stop-card"
|
class="btn-stop-card"
|
||||||
(click)="stopTask(session.session_id, $event)"
|
(click)="stopTask(session.session_id, $event)"
|
||||||
title="Stop this task"
|
title="停止此任务"
|
||||||
aria-label="Stop this task"
|
aria-label="停止此任务"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined">stop_circle</span>
|
<span class="material-symbols-outlined">stop_circle</span>
|
||||||
</button>
|
</button>
|
||||||
@@ -109,8 +110,8 @@ limitations under the License.
|
|||||||
type="button"
|
type="button"
|
||||||
class="btn-delete-card"
|
class="btn-delete-card"
|
||||||
(click)="deleteTask(session.session_id, $event)"
|
(click)="deleteTask(session.session_id, $event)"
|
||||||
title="Delete this task"
|
title="删除此任务"
|
||||||
aria-label="Delete this task"
|
aria-label="删除此任务"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined">delete</span>
|
<span class="material-symbols-outlined">delete</span>
|
||||||
</button>
|
</button>
|
||||||
@@ -118,7 +119,7 @@ limitations under the License.
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<div class="item-goal" title="{{ session.initial_goal }}">
|
<div class="item-goal" title="{{ session.initial_goal }}">
|
||||||
{{ session.initial_goal || 'Unnamed Task' }}
|
{{ session.initial_goal || '未命名任务' }}
|
||||||
</div>
|
</div>
|
||||||
<div class="item-footer">
|
<div class="item-footer">
|
||||||
<span class="item-id" [title]="session.session_id">ID: {{ session.session_id }}</span>
|
<span class="item-id" [title]="session.session_id">ID: {{ session.session_id }}</span>
|
||||||
@@ -132,7 +133,8 @@ limitations under the License.
|
|||||||
</div>
|
</div>
|
||||||
@if (historyTasks().length === 0) {
|
@if (historyTasks().length === 0) {
|
||||||
<div class="dropdown-empty-state">
|
<div class="dropdown-empty-state">
|
||||||
No past task history
|
|
||||||
|
暂无历史任务
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
<div class="dropdown-history-list">
|
<div class="dropdown-history-list">
|
||||||
@@ -148,7 +150,7 @@ limitations under the License.
|
|||||||
@if (getTaskStatus(session) === 'running') {
|
@if (getTaskStatus(session) === 'running') {
|
||||||
<span class="status-dot"></span>
|
<span class="status-dot"></span>
|
||||||
}
|
}
|
||||||
{{ getTaskStatus(session) | uppercase }}
|
{{ getTaskStatus(session) | uiLabel }}
|
||||||
</span>
|
</span>
|
||||||
<div class="item-header-actions">
|
<div class="item-header-actions">
|
||||||
@if (getDeviceSerial(session)) {
|
@if (getDeviceSerial(session)) {
|
||||||
@@ -166,8 +168,8 @@ limitations under the License.
|
|||||||
type="button"
|
type="button"
|
||||||
class="btn-delete-card"
|
class="btn-delete-card"
|
||||||
(click)="deleteTask(session.session_id, $event)"
|
(click)="deleteTask(session.session_id, $event)"
|
||||||
title="Delete this task"
|
title="删除此任务"
|
||||||
aria-label="Delete this task"
|
aria-label="删除此任务"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined">delete</span>
|
<span class="material-symbols-outlined">delete</span>
|
||||||
</button>
|
</button>
|
||||||
@@ -175,7 +177,7 @@ limitations under the License.
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<div class="item-goal" title="{{ session.initial_goal }}">
|
<div class="item-goal" title="{{ session.initial_goal }}">
|
||||||
{{ session.initial_goal || 'Unnamed Task' }}
|
{{ session.initial_goal || '未命名任务' }}
|
||||||
</div>
|
</div>
|
||||||
<div class="item-footer">
|
<div class="item-footer">
|
||||||
<span class="item-id" [title]="session.session_id">ID: {{ session.session_id }}</span>
|
<span class="item-id" [title]="session.session_id">ID: {{ session.session_id }}</span>
|
||||||
@@ -200,7 +202,7 @@ limitations under the License.
|
|||||||
<div class="vscode-dropdown-menu notes-dropdown">
|
<div class="vscode-dropdown-menu notes-dropdown">
|
||||||
<div class="popover-title-header">
|
<div class="popover-title-header">
|
||||||
<span class="material-symbols-outlined">description</span>
|
<span class="material-symbols-outlined">description</span>
|
||||||
<span>Notes & Plans</span>
|
<span>笔记与计划</span>
|
||||||
</div>
|
</div>
|
||||||
<!-- Notes Selector / Pills (Shown if there are note files) -->
|
<!-- Notes Selector / Pills (Shown if there are note files) -->
|
||||||
@if (noteKeys().length > 1) {
|
@if (noteKeys().length > 1) {
|
||||||
@@ -222,7 +224,7 @@ limitations under the License.
|
|||||||
@if (noteKeys().length === 0) {
|
@if (noteKeys().length === 0) {
|
||||||
<div class="dropdown-empty-state">
|
<div class="dropdown-empty-state">
|
||||||
<span class="material-symbols-outlined empty-icon">description</span>
|
<span class="material-symbols-outlined empty-icon">description</span>
|
||||||
<span>No notes available for this session.</span>
|
<span>此会话暂无笔记。</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
<!-- Rich Notes Content Viewer -->
|
<!-- Rich Notes Content Viewer -->
|
||||||
@@ -402,7 +404,7 @@ limitations under the License.
|
|||||||
(click)="toggleRunInfo($event)"
|
(click)="toggleRunInfo($event)"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined model-icon">{{ getModelIcon(model.name) }}</span>
|
<span class="material-symbols-outlined model-icon">{{ getModelIcon(model.name) }}</span>
|
||||||
<span class="model-label">Architecture:</span>
|
<span class="model-label">执行模式:</span>
|
||||||
<span class="model-name">{{ getModelDisplayName(model.name) }}</span>
|
<span class="model-name">{{ getModelDisplayName(model.name) }}</span>
|
||||||
</button>
|
</button>
|
||||||
|
|
||||||
@@ -415,7 +417,7 @@ limitations under the License.
|
|||||||
(backdropClick)="isRunInfoOpen.set(false)"
|
(backdropClick)="isRunInfoOpen.set(false)"
|
||||||
[cdkConnectedOverlayPositions]="dropdownPositions"
|
[cdkConnectedOverlayPositions]="dropdownPositions"
|
||||||
>
|
>
|
||||||
<div class="vscode-dropdown-menu run-info-dropdown" role="dialog" aria-label="Run info">
|
<div class="vscode-dropdown-menu run-info-dropdown" role="dialog" aria-label="运行信息">
|
||||||
<div class="run-info-header">
|
<div class="run-info-header">
|
||||||
<span class="material-symbols-outlined" [class.is-pro]="runInfoIsPro()">{{ getModelIcon(model.name) }}</span>
|
<span class="material-symbols-outlined" [class.is-pro]="runInfoIsPro()">{{ getModelIcon(model.name) }}</span>
|
||||||
<span class="run-info-title">{{ getModelDisplayName(model.name) }}</span>
|
<span class="run-info-title">{{ getModelDisplayName(model.name) }}</span>
|
||||||
@@ -425,16 +427,16 @@ limitations under the License.
|
|||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="run-info-row">
|
<div class="run-info-row">
|
||||||
<span class="run-info-label">Elapsed</span>
|
<span class="run-info-label">已用时间</span>
|
||||||
<span class="run-info-value">{{ runInfoElapsed() }}</span>
|
<span class="run-info-value">{{ runInfoElapsed() }}</span>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="run-info-row">
|
<div class="run-info-row">
|
||||||
<span class="run-info-label">Tokens</span>
|
<span class="run-info-label">Token 用量</span>
|
||||||
@if (runUsage(); as usage) {
|
@if (runUsage(); as usage) {
|
||||||
<span class="run-info-value">
|
<span class="run-info-value">
|
||||||
{{ formatRunTokens(usage.total_tokens) }}
|
{{ formatRunTokens(usage.total_tokens) }}
|
||||||
<span class="run-info-sub">{{ formatRunTokens(usage.prompt_tokens) }} in · {{ formatRunTokens(usage.completion_tokens) }} out</span>
|
<span class="run-info-sub">{{ formatRunTokens(usage.prompt_tokens) }} 输入 · {{ formatRunTokens(usage.completion_tokens) }} 输出</span>
|
||||||
</span>
|
</span>
|
||||||
} @else {
|
} @else {
|
||||||
<span class="run-info-value is-muted">—</span>
|
<span class="run-info-value is-muted">—</span>
|
||||||
@@ -443,11 +445,11 @@ limitations under the License.
|
|||||||
|
|
||||||
<div class="run-info-row is-stacked">
|
<div class="run-info-row is-stacked">
|
||||||
<div class="run-info-line">
|
<div class="run-info-line">
|
||||||
<span class="run-info-label">Context</span>
|
<span class="run-info-label">上下文</span>
|
||||||
@if (runInfoContextPercent() !== null) {
|
@if (runInfoContextPercent() !== null) {
|
||||||
<span class="run-info-value">
|
<span class="run-info-value">
|
||||||
{{ runInfoContextPercent() }}%
|
{{ runInfoContextPercent() }}%
|
||||||
<span class="run-info-sub-inline">of {{ formatRunTokens(runUsage()?.operator_context_window_tokens) }}</span>
|
<span class="run-info-sub-inline">/ {{ formatRunTokens(runUsage()?.operator_context_window_tokens) }}</span>
|
||||||
</span>
|
</span>
|
||||||
} @else {
|
} @else {
|
||||||
<span class="run-info-value is-muted">—</span>
|
<span class="run-info-value is-muted">—</span>
|
||||||
@@ -464,15 +466,15 @@ limitations under the License.
|
|||||||
|
|
||||||
@if (runInfoIsPro()) {
|
@if (runInfoIsPro()) {
|
||||||
<div class="run-info-row">
|
<div class="run-info-row">
|
||||||
<span class="run-info-label">Result check</span>
|
<span class="run-info-label">结果检查</span>
|
||||||
<span class="run-info-value">{{ runTuningLabel('verify', runInfoTuning()?.verification_level) }}</span>
|
<span class="run-info-value">{{ runTuningLabel('verify', runInfoTuning()?.verification_level) }}</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="run-info-row">
|
<div class="run-info-row">
|
||||||
<span class="run-info-label">Screen reading</span>
|
<span class="run-info-label">屏幕识别</span>
|
||||||
<span class="run-info-value">{{ runTuningLabel('explore', runInfoTuning()?.explorer_mode) }}</span>
|
<span class="run-info-value">{{ runTuningLabel('explore', runInfoTuning()?.explorer_mode) }}</span>
|
||||||
</div>
|
</div>
|
||||||
} @else {
|
} @else {
|
||||||
<div class="run-info-note">No extra settings for Flash.</div>
|
<div class="run-info-note">Flash 无需额外设置。</div>
|
||||||
}
|
}
|
||||||
</div>
|
</div>
|
||||||
</ng-template>
|
</ng-template>
|
||||||
@@ -520,22 +522,22 @@ limitations under the License.
|
|||||||
</span>
|
</span>
|
||||||
}
|
}
|
||||||
@if (agentService.isCurrentSessionRunning()) {
|
@if (agentService.isCurrentSessionRunning()) {
|
||||||
<span class="btn-text">Recording...</span>
|
<span class="btn-text">正在录制…</span>
|
||||||
}
|
}
|
||||||
@if (!agentService.isCurrentSessionRunning() && agentService.currentSessionRecordingStatus() === 'processing') {
|
@if (!agentService.isCurrentSessionRunning() && agentService.currentSessionRecordingStatus() === 'processing') {
|
||||||
<span class="btn-text">Preparing...</span>
|
<span class="btn-text">正在准备…</span>
|
||||||
}
|
}
|
||||||
@if (!agentService.isCurrentSessionRunning() && agentService.currentSessionRecordingStatus() !== 'processing' && agentService.currentSessionVideoUrl()) {
|
@if (!agentService.isCurrentSessionRunning() && agentService.currentSessionRecordingStatus() !== 'processing' && agentService.currentSessionVideoUrl()) {
|
||||||
<span class="btn-text">Play Recording</span>
|
<span class="btn-text">播放录像</span>
|
||||||
}
|
}
|
||||||
@if (!agentService.isCurrentSessionRunning() && agentService.currentSessionRecordingStatus() !== 'processing' && !agentService.currentSessionVideoUrl() && agentService.hasCurrentSessionStepFrames()) {
|
@if (!agentService.isCurrentSessionRunning() && agentService.currentSessionRecordingStatus() !== 'processing' && !agentService.currentSessionVideoUrl() && agentService.hasCurrentSessionStepFrames()) {
|
||||||
<span class="btn-text">Step Replay</span>
|
<span class="btn-text">步骤回放</span>
|
||||||
}
|
}
|
||||||
@if (!agentService.isCurrentSessionRunning() && agentService.currentSessionRecordingStatus() !== 'processing' && !agentService.currentSessionVideoUrl() && !agentService.hasCurrentSessionStepFrames() && agentService.currentSessionRecordingStatus() === 'failed') {
|
@if (!agentService.isCurrentSessionRunning() && agentService.currentSessionRecordingStatus() !== 'processing' && !agentService.currentSessionVideoUrl() && !agentService.hasCurrentSessionStepFrames() && agentService.currentSessionRecordingStatus() === 'failed') {
|
||||||
<span class="btn-text">Recording Failed</span>
|
<span class="btn-text">录屏失败</span>
|
||||||
}
|
}
|
||||||
@if (!agentService.isCurrentSessionRunning() && agentService.currentSessionRecordingStatus() !== 'processing' && !agentService.currentSessionVideoUrl() && !agentService.hasCurrentSessionStepFrames() && agentService.currentSessionRecordingStatus() !== 'failed') {
|
@if (!agentService.isCurrentSessionRunning() && agentService.currentSessionRecordingStatus() !== 'processing' && !agentService.currentSessionVideoUrl() && !agentService.hasCurrentSessionStepFrames() && agentService.currentSessionRecordingStatus() !== 'failed') {
|
||||||
<span class="btn-text">Screen Recording</span>
|
<span class="btn-text">屏幕录像</span>
|
||||||
}
|
}
|
||||||
@if (agentService.isCurrentSessionRunning()) {
|
@if (agentService.isCurrentSessionRunning()) {
|
||||||
<span class="video-pulse-dot red"></span>
|
<span class="video-pulse-dot red"></span>
|
||||||
@@ -554,15 +556,15 @@ limitations under the License.
|
|||||||
@if (agentService.isSessionContentLoading()) {
|
@if (agentService.isSessionContentLoading()) {
|
||||||
<div class="session-loading-state" aria-live="polite">
|
<div class="session-loading-state" aria-live="polite">
|
||||||
<span class="material-symbols-outlined spin-animation">progress_activity</span>
|
<span class="material-symbols-outlined spin-animation">progress_activity</span>
|
||||||
<span>Loading run history...</span>
|
<span>正在加载执行历史…</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
|
|
||||||
@if (!agentService.isSessionContentLoading() && consolidatedBlocks().length === 0 && !showPlanningLoader() && startupWorkItems().length === 0) {
|
@if (!agentService.isSessionContentLoading() && consolidatedBlocks().length === 0 && !showPlanningLoader() && startupWorkItems().length === 0) {
|
||||||
<div class="empty-state">
|
<div class="empty-state">
|
||||||
<div class="empty-icon"><span class="material-symbols-outlined">sensors</span></div>
|
<div class="empty-icon"><span class="material-symbols-outlined">sensors</span></div>
|
||||||
<h3>No Stream Activity Selected</h3>
|
<h3>尚未选择执行记录</h3>
|
||||||
<p>Please select a session from the Task List on the right to view its real-time log payloads.</p>
|
<p>请从右侧任务列表选择会话,查看实时执行记录。</p>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -572,9 +574,9 @@ limitations under the License.
|
|||||||
@if (startupWorkItems().length > 0) {
|
@if (startupWorkItems().length > 0) {
|
||||||
<div class="phase-container startup-phase" aria-live="polite">
|
<div class="phase-container startup-phase" aria-live="polite">
|
||||||
<div class="phase-header first-phase">
|
<div class="phase-header first-phase">
|
||||||
<span class="phase-worked-time">Worked for {{ startupPreparationDuration() }}s</span>
|
<span class="phase-worked-time">准备耗时 {{ startupPreparationDuration() }} 秒</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="phase-body" aria-label="Task startup progress">
|
<div class="phase-body" aria-label="任务启动进度">
|
||||||
<div class="block-item type-stream">
|
<div class="block-item type-stream">
|
||||||
<div class="step-body">
|
<div class="step-body">
|
||||||
<div class="step-actions-container">
|
<div class="step-actions-container">
|
||||||
@@ -661,7 +663,7 @@ limitations under the License.
|
|||||||
[class.collapsed]="isCheckerSegmentCollapsed(block, itemIdx)">
|
[class.collapsed]="isCheckerSegmentCollapsed(block, itemIdx)">
|
||||||
<div class="details-summary" (click)="toggleCheckerSegment(block, itemIdx)">
|
<div class="details-summary" (click)="toggleCheckerSegment(block, itemIdx)">
|
||||||
<span class="stream-status-dot" [class.streaming]="isCheckerSegmentRunning(block, itemIdx) && !isCheckerSegmentResetNotice(item)" [class.resetting]="isCheckerSegmentResetNotice(item)"></span>
|
<span class="stream-status-dot" [class.streaming]="isCheckerSegmentRunning(block, itemIdx) && !isCheckerSegmentResetNotice(item)" [class.resetting]="isCheckerSegmentResetNotice(item)"></span>
|
||||||
<span class="header-label">{{ isCheckerSegmentResetNotice(item) ? getCheckerSegmentResetMessage(item) : 'Thought' }}</span>
|
<span class="header-label">{{ isCheckerSegmentResetNotice(item) ? getCheckerSegmentResetMessage(item) : '思考过程' }}</span>
|
||||||
</div>
|
</div>
|
||||||
@if (item.data.text && item.data.text.trim()) {
|
@if (item.data.text && item.data.text.trim()) {
|
||||||
<div class="details-content-wrapper">
|
<div class="details-content-wrapper">
|
||||||
@@ -689,7 +691,7 @@ limitations under the License.
|
|||||||
[class.is-running]="isCheckerSegmentRunning(block, itemIdx)">
|
[class.is-running]="isCheckerSegmentRunning(block, itemIdx)">
|
||||||
<div class="report-header">
|
<div class="report-header">
|
||||||
<span class="stream-status-dot" [class.streaming]="isCheckerSegmentRunning(block, itemIdx)"></span>
|
<span class="stream-status-dot" [class.streaming]="isCheckerSegmentRunning(block, itemIdx)"></span>
|
||||||
<span class="header-label">Work</span>
|
<span class="header-label">执行过程</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="report-content stream-box">
|
<div class="report-content stream-box">
|
||||||
<div class="stream-text" [innerHTML]="renderMarkdown(item.data.text, block.id + '-seg-' + itemIdx)"></div>
|
<div class="stream-text" [innerHTML]="renderMarkdown(item.data.text, block.id + '-seg-' + itemIdx)"></div>
|
||||||
@@ -718,7 +720,7 @@ limitations under the License.
|
|||||||
[class.collapsed]="collapsedStreams().has(block.data.execution_id || block.data.step_id || block.id)">
|
[class.collapsed]="collapsedStreams().has(block.data.execution_id || block.data.step_id || block.id)">
|
||||||
<div class="details-summary" (click)="toggleCollapse(block.data.execution_id || block.data.step_id || block.id)">
|
<div class="details-summary" (click)="toggleCollapse(block.data.execution_id || block.data.step_id || block.id)">
|
||||||
<span class="stream-status-dot" [class.streaming]="isBlockRunning(block)"></span>
|
<span class="stream-status-dot" [class.streaming]="isBlockRunning(block)"></span>
|
||||||
<span class="header-label">Thought</span>
|
<span class="header-label">思考过程</span>
|
||||||
</div>
|
</div>
|
||||||
@if (getStreamDisplayText(block.id + '-native', item.data.text)) {
|
@if (getStreamDisplayText(block.id + '-native', item.data.text)) {
|
||||||
<div class="details-content-wrapper">
|
<div class="details-content-wrapper">
|
||||||
@@ -750,7 +752,7 @@ limitations under the License.
|
|||||||
[class.is-running]="isBlockRunning(block)">
|
[class.is-running]="isBlockRunning(block)">
|
||||||
<div class="report-header">
|
<div class="report-header">
|
||||||
<span class="stream-status-dot" [class.streaming]="isBlockRunning(block)"></span>
|
<span class="stream-status-dot" [class.streaming]="isBlockRunning(block)"></span>
|
||||||
<span class="header-label">Work</span>
|
<span class="header-label">执行过程</span>
|
||||||
</div>
|
</div>
|
||||||
@if (getStreamDisplayText(block.id, item.data.text)) {
|
@if (getStreamDisplayText(block.id, item.data.text)) {
|
||||||
<div class="report-content stream-box">
|
<div class="report-content stream-box">
|
||||||
@@ -772,13 +774,13 @@ limitations under the License.
|
|||||||
{{ getReportStatusValue(item.data) === 'failed' ? 'cancel' : 'task_alt' }}
|
{{ getReportStatusValue(item.data) === 'failed' ? 'cancel' : 'task_alt' }}
|
||||||
</span>
|
</span>
|
||||||
<span class="action-title">
|
<span class="action-title">
|
||||||
{{ getReportStatusValue(item.data) === 'failed' ? 'Task Execution Failed' : 'Task Completed' }}
|
{{ getReportStatusValue(item.data) === 'failed' ? '任务执行失败' : '任务已完成' }}
|
||||||
</span>
|
</span>
|
||||||
<span class="action-status-badge" [class.is-failed]="getReportStatusValue(item.data) === 'failed'" [class.is-completed]="getReportStatusValue(item.data) !== 'failed'">
|
<span class="action-status-badge" [class.is-failed]="getReportStatusValue(item.data) === 'failed'" [class.is-completed]="getReportStatusValue(item.data) !== 'failed'">
|
||||||
<span class="material-symbols-outlined status-icon">
|
<span class="material-symbols-outlined status-icon">
|
||||||
{{ getReportStatusValue(item.data) === 'failed' ? 'error' : 'check_circle' }}
|
{{ getReportStatusValue(item.data) === 'failed' ? 'error' : 'check_circle' }}
|
||||||
</span>
|
</span>
|
||||||
{{ getReportStatusValue(item.data) === 'failed' ? 'Failed' : 'Completed' }}
|
{{ getReportStatusValue(item.data) === 'failed' ? '失败' : '已完成' }}
|
||||||
</span>
|
</span>
|
||||||
<span class="action-expand-icon material-symbols-outlined">
|
<span class="action-expand-icon material-symbols-outlined">
|
||||||
{{ isActionExpanded('action-' + block.id + '-' + itemIdx) ? 'expand_less' : 'expand_more' }}
|
{{ isActionExpanded('action-' + block.id + '-' + itemIdx) ? 'expand_less' : 'expand_more' }}
|
||||||
@@ -788,7 +790,7 @@ limitations under the License.
|
|||||||
@if (!isActionExpanded('action-' + block.id + '-' + itemIdx) && getReportStatusExplanation(item.data)) {
|
@if (!isActionExpanded('action-' + block.id + '-' + itemIdx) && getReportStatusExplanation(item.data)) {
|
||||||
<div class="action-card-body">
|
<div class="action-card-body">
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">Summary:</span>
|
<span class="detail-label">摘要:</span>
|
||||||
<span class="detail-value">"{{ getReportStatusExplanation(item.data) }}"</span>
|
<span class="detail-value">"{{ getReportStatusExplanation(item.data) }}"</span>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -799,7 +801,7 @@ limitations under the License.
|
|||||||
<div class="expanded-grid">
|
<div class="expanded-grid">
|
||||||
@if (getReportStatusExplanation(item.data); as expText) {
|
@if (getReportStatusExplanation(item.data); as expText) {
|
||||||
<div class="grid-item full-width">
|
<div class="grid-item full-width">
|
||||||
<span class="grid-label">Report & Findings</span>
|
<span class="grid-label">报告与发现</span>
|
||||||
<span class="grid-value report-markdown-body" [innerHTML]="renderMarkdown(expText, 'report-' + block.id + '-' + itemIdx)"></span>
|
<span class="grid-value report-markdown-body" [innerHTML]="renderMarkdown(expText, 'report-' + block.id + '-' + itemIdx)"></span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -809,20 +811,20 @@ limitations under the License.
|
|||||||
<div class="expanded-screenshots-container">
|
<div class="expanded-screenshots-container">
|
||||||
@if (getStepPreImageUrl(block.data, item.data)) {
|
@if (getStepPreImageUrl(block.data, item.data)) {
|
||||||
<div class="screenshot-column">
|
<div class="screenshot-column">
|
||||||
<span class="screenshot-label">Final Verified State</span>
|
<span class="screenshot-label">最终验证状态</span>
|
||||||
<div class="image-wrapper" (click)="openImageModal(getStepPreImageUrl(block.data, item.data)!, item.data, $event)">
|
<div class="image-wrapper" (click)="openImageModal(getStepPreImageUrl(block.data, item.data)!, item.data, $event)">
|
||||||
<div class="img-relative-box">
|
<div class="img-relative-box">
|
||||||
<img [src]="getStepPreImageUrl(block.data, item.data)" alt="Final Screen Screenshot" />
|
<img [src]="getStepPreImageUrl(block.data, item.data)" alt="最终屏幕截图" />
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if (getStepPostImageUrl(block.data, item.data)) {
|
@if (getStepPostImageUrl(block.data, item.data)) {
|
||||||
<div class="screenshot-column">
|
<div class="screenshot-column">
|
||||||
<span class="screenshot-label">Post Verification</span>
|
<span class="screenshot-label">操作后验证</span>
|
||||||
<div class="image-wrapper" (click)="openImageModal(getStepPostImageUrl(block.data, item.data)!, null, $event)">
|
<div class="image-wrapper" (click)="openImageModal(getStepPostImageUrl(block.data, item.data)!, null, $event)">
|
||||||
<div class="img-relative-box">
|
<div class="img-relative-box">
|
||||||
<img [src]="getStepPostImageUrl(block.data, item.data)" alt="Post Screenshot" />
|
<img [src]="getStepPostImageUrl(block.data, item.data)" alt="操作后截图" />
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -846,7 +848,8 @@ limitations under the License.
|
|||||||
@if (isActionFailed(item.data, block.data)) {
|
@if (isActionFailed(item.data, block.data)) {
|
||||||
<span class="action-status-badge is-failed">
|
<span class="action-status-badge is-failed">
|
||||||
<span class="material-symbols-outlined status-icon">error</span>
|
<span class="material-symbols-outlined status-icon">error</span>
|
||||||
Failed
|
|
||||||
|
失败
|
||||||
</span>
|
</span>
|
||||||
}
|
}
|
||||||
<span class="action-expand-icon material-symbols-outlined">
|
<span class="action-expand-icon material-symbols-outlined">
|
||||||
@@ -858,7 +861,7 @@ limitations under the License.
|
|||||||
<div class="action-card-body">
|
<div class="action-card-body">
|
||||||
@if (getActionTargetText(item.data)) {
|
@if (getActionTargetText(item.data)) {
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">Target:</span>
|
<span class="detail-label">目标:</span>
|
||||||
<span class="detail-value">"{{ getActionTargetText(item.data) }}"</span>
|
<span class="detail-value">"{{ getActionTargetText(item.data) }}"</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -870,7 +873,7 @@ limitations under the License.
|
|||||||
}
|
}
|
||||||
@if (getActionCoords(item.data)) {
|
@if (getActionCoords(item.data)) {
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">Coordinates:</span>
|
<span class="detail-label">坐标:</span>
|
||||||
<span class="detail-value coordinate-badge">{{ getActionCoords(item.data) }}</span>
|
<span class="detail-value coordinate-badge">{{ getActionCoords(item.data) }}</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -882,7 +885,7 @@ limitations under the License.
|
|||||||
<div class="expanded-grid">
|
<div class="expanded-grid">
|
||||||
@if (getActionTargetText(item.data)) {
|
@if (getActionTargetText(item.data)) {
|
||||||
<div class="grid-item">
|
<div class="grid-item">
|
||||||
<span class="grid-label">Target</span>
|
<span class="grid-label">目标</span>
|
||||||
<span class="grid-value">"{{ getActionTargetText(item.data) }}"</span>
|
<span class="grid-value">"{{ getActionTargetText(item.data) }}"</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -894,19 +897,19 @@ limitations under the License.
|
|||||||
}
|
}
|
||||||
@if (getActionCoords(item.data)) {
|
@if (getActionCoords(item.data)) {
|
||||||
<div class="grid-item">
|
<div class="grid-item">
|
||||||
<span class="grid-label">Coordinates</span>
|
<span class="grid-label">坐标</span>
|
||||||
<span class="grid-value code-pill">{{ getActionCoords(item.data) }}</span>
|
<span class="grid-value code-pill">{{ getActionCoords(item.data) }}</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if (getActionBounds(item.data)) {
|
@if (getActionBounds(item.data)) {
|
||||||
<div class="grid-item">
|
<div class="grid-item">
|
||||||
<span class="grid-label">Target Bounds</span>
|
<span class="grid-label">目标边界</span>
|
||||||
<span class="grid-value code-pill">[{{ getActionBounds(item.data) }}]</span>
|
<span class="grid-value code-pill">[{{ getActionBounds(item.data) }}]</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if (getActionClass(item.data)) {
|
@if (getActionClass(item.data)) {
|
||||||
<div class="grid-item">
|
<div class="grid-item">
|
||||||
<span class="grid-label">Target Class</span>
|
<span class="grid-label">目标类型</span>
|
||||||
<span class="grid-value code-pill">{{ getActionClass(item.data) }}</span>
|
<span class="grid-value code-pill">{{ getActionClass(item.data) }}</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -919,7 +922,7 @@ limitations under the License.
|
|||||||
<!-- Fallback item if all detail fields are empty -->
|
<!-- Fallback item if all detail fields are empty -->
|
||||||
@if (!getActionTargetText(item.data) && !getActionInputText(item.data) && !getActionCoords(item.data) && !getActionBounds(item.data) && !getActionClass(item.data) && getActionExtraParams(item.data).length === 0) {
|
@if (!getActionTargetText(item.data) && !getActionInputText(item.data) && !getActionCoords(item.data) && !getActionBounds(item.data) && !getActionClass(item.data) && getActionExtraParams(item.data).length === 0) {
|
||||||
<div class="grid-item">
|
<div class="grid-item">
|
||||||
<span class="grid-label">Action</span>
|
<span class="grid-label">操作</span>
|
||||||
<span class="grid-value">{{ getActionTitle(item.data) }}</span>
|
<span class="grid-value">{{ getActionTitle(item.data) }}</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -929,10 +932,10 @@ limitations under the License.
|
|||||||
<div class="expanded-screenshots-container">
|
<div class="expanded-screenshots-container">
|
||||||
@if (getStepPreImageUrl(block.data, item.data)) {
|
@if (getStepPreImageUrl(block.data, item.data)) {
|
||||||
<div class="screenshot-column">
|
<div class="screenshot-column">
|
||||||
<span class="screenshot-label">Pre-Action (Before)</span>
|
<span class="screenshot-label">操作前</span>
|
||||||
<div class="image-wrapper" (click)="openImageModal(getStepPreImageUrl(block.data, item.data)!, item.data, $event)">
|
<div class="image-wrapper" (click)="openImageModal(getStepPreImageUrl(block.data, item.data)!, item.data, $event)">
|
||||||
<div class="img-relative-box">
|
<div class="img-relative-box">
|
||||||
<img #preImg [src]="getStepPreImageUrl(block.data, item.data)" (load)="onImageLoad(preImg, preOverlay, item.data)" alt="Pre-Action Screenshot" />
|
<img #preImg [src]="getStepPreImageUrl(block.data, item.data)" (load)="onImageLoad(preImg, preOverlay, item.data)" alt="操作前截图" />
|
||||||
<div #preOverlay class="overlay-layer"></div>
|
<div #preOverlay class="overlay-layer"></div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -940,10 +943,10 @@ limitations under the License.
|
|||||||
}
|
}
|
||||||
@if (getStepPostImageUrl(block.data, item.data)) {
|
@if (getStepPostImageUrl(block.data, item.data)) {
|
||||||
<div class="screenshot-column">
|
<div class="screenshot-column">
|
||||||
<span class="screenshot-label">Post-Action (After)</span>
|
<span class="screenshot-label">操作后</span>
|
||||||
<div class="image-wrapper" (click)="openImageModal(getStepPostImageUrl(block.data, item.data)!, null, $event)">
|
<div class="image-wrapper" (click)="openImageModal(getStepPostImageUrl(block.data, item.data)!, null, $event)">
|
||||||
<div class="img-relative-box">
|
<div class="img-relative-box">
|
||||||
<img [src]="getStepPostImageUrl(block.data, item.data)" alt="Post-Action Screenshot" />
|
<img [src]="getStepPostImageUrl(block.data, item.data)" alt="操作后截图" />
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -983,13 +986,15 @@ limitations under the License.
|
|||||||
<div class="llm-retry-note-text">
|
<div class="llm-retry-note-text">
|
||||||
@if (isLLMRetryEntryWaiting(retry)) {
|
@if (isLLMRetryEntryWaiting(retry)) {
|
||||||
<span class="llm-retry-status-line">
|
<span class="llm-retry-status-line">
|
||||||
LLM temporarily unavailable
|
|
||||||
|
模型暂时不可用
|
||||||
@if (getLLMRetryEntryDelay(retry); as retryDelay) {
|
@if (getLLMRetryEntryDelay(retry); as retryDelay) {
|
||||||
· Retrying in {{ retryDelay }}…
|
· Retrying in {{ retryDelay }}…
|
||||||
}
|
}
|
||||||
</span>
|
</span>
|
||||||
<span class="llm-retry-guidance">
|
<span class="llm-retry-guidance">
|
||||||
Frequent retries may indicate a low API rate limit. Consider increasing the limit or reducing request concurrency.
|
|
||||||
|
频繁重试可能是接口限流导致,可提高调用额度或降低并发数。
|
||||||
</span>
|
</span>
|
||||||
}
|
}
|
||||||
@if (!isLLMRetryEntryWaiting(retry)) {
|
@if (!isLLMRetryEntryWaiting(retry)) {
|
||||||
@@ -1011,16 +1016,18 @@ limitations under the License.
|
|||||||
(click)="hasLLMFailureRetryEntries(item.data) && toggleActionExpanded('llm-failure-' + block.id + '-' + itemIdx, $event)">
|
(click)="hasLLMFailureRetryEntries(item.data) && toggleActionExpanded('llm-failure-' + block.id + '-' + itemIdx, $event)">
|
||||||
<div class="action-card-header llm-failure-header">
|
<div class="action-card-header llm-failure-header">
|
||||||
<span class="material-symbols-outlined action-icon is-failed">error</span>
|
<span class="material-symbols-outlined action-icon is-failed">error</span>
|
||||||
<span class="action-title">LLM Request Failed</span>
|
<span class="action-title">模型请求失败</span>
|
||||||
@if (isTerminalLLMFailure(item.data)) {
|
@if (isTerminalLLMFailure(item.data)) {
|
||||||
<span class="action-status-badge is-paused">
|
<span class="action-status-badge is-paused">
|
||||||
<span class="material-symbols-outlined status-icon">pause_circle</span>
|
<span class="material-symbols-outlined status-icon">pause_circle</span>
|
||||||
Paused
|
|
||||||
|
已暂停
|
||||||
</span>
|
</span>
|
||||||
} @else {
|
} @else {
|
||||||
<span class="action-status-badge is-failed">
|
<span class="action-status-badge is-failed">
|
||||||
<span class="material-symbols-outlined status-icon">warning</span>
|
<span class="material-symbols-outlined status-icon">warning</span>
|
||||||
Failed
|
|
||||||
|
失败
|
||||||
</span>
|
</span>
|
||||||
}
|
}
|
||||||
@if (hasLLMFailureRetryEntries(item.data)) {
|
@if (hasLLMFailureRetryEntries(item.data)) {
|
||||||
@@ -1031,22 +1038,22 @@ limitations under the License.
|
|||||||
</div>
|
</div>
|
||||||
<div class="action-card-body">
|
<div class="action-card-body">
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">What happened:</span>
|
<span class="detail-label">发生了什么:</span>
|
||||||
<span class="detail-value">The AI request could not be completed after automatic retries.</span>
|
<span class="detail-value">自动重试后仍无法完成模型请求。</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">Failure reason:</span>
|
<span class="detail-label">失败原因:</span>
|
||||||
<span class="detail-value">{{ getLLMErrorText(item.data) }}</span>
|
<span class="detail-value">{{ getLLMErrorText(item.data) }}</span>
|
||||||
</div>
|
</div>
|
||||||
@if (getLLMFailureWaited(item.data); as waited) {
|
@if (getLLMFailureWaited(item.data); as waited) {
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">Waited:</span>
|
<span class="detail-label">已等待:</span>
|
||||||
<span class="detail-value">{{ waited }}</span>
|
<span class="detail-value">{{ waited }}</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if (isActionExpanded('llm-failure-' + block.id + '-' + itemIdx)) {
|
@if (isActionExpanded('llm-failure-' + block.id + '-' + itemIdx)) {
|
||||||
<div class="failure-retry-details">
|
<div class="failure-retry-details">
|
||||||
<div class="failure-retry-title">Retry details</div>
|
<div class="failure-retry-title">重试详情</div>
|
||||||
@for (retry of getLLMFailureRetryEntries(item.data); track retry; let retryIndex = $index) {
|
@for (retry of getLLMFailureRetryEntries(item.data); track retry; let retryIndex = $index) {
|
||||||
<div class="failure-retry-row">
|
<div class="failure-retry-row">
|
||||||
<span>Retry {{ retryIndex + 1 }}</span>
|
<span>Retry {{ retryIndex + 1 }}</span>
|
||||||
@@ -1066,14 +1073,14 @@ limitations under the License.
|
|||||||
type="button"
|
type="button"
|
||||||
class="btn-continue-task"
|
class="btn-continue-task"
|
||||||
(click)="resumePausedTask($event)"
|
(click)="resumePausedTask($event)"
|
||||||
title="Continue the paused task"
|
title="继续已暂停的任务"
|
||||||
aria-label="Continue the paused task"
|
aria-label="继续已暂停的任务"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined continue-icon">play_arrow</span>
|
<span class="material-symbols-outlined continue-icon">play_arrow</span>
|
||||||
<span>Continue task</span>
|
<span>继续任务</span>
|
||||||
<span class="material-symbols-outlined continue-arrow">arrow_forward</span>
|
<span class="material-symbols-outlined continue-arrow">arrow_forward</span>
|
||||||
</button>
|
</button>
|
||||||
<span class="llm-resume-hint">Task paused · no steps are running</span>
|
<span class="llm-resume-hint">任务已暂停 · 当前没有执行中的步骤</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
</div>
|
</div>
|
||||||
@@ -1096,7 +1103,7 @@ limitations under the License.
|
|||||||
<span
|
<span
|
||||||
class="tool-key-pill"
|
class="tool-key-pill"
|
||||||
(click)="onToolKeyClick(key)"
|
(click)="onToolKeyClick(key)"
|
||||||
title="Click to view note content"
|
title="点击查看笔记内容"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined pill-icon">draft</span>
|
<span class="material-symbols-outlined pill-icon">draft</span>
|
||||||
{{ key }}
|
{{ key }}
|
||||||
@@ -1134,7 +1141,8 @@ limitations under the License.
|
|||||||
@if (isToolFailed(item.data)) {
|
@if (isToolFailed(item.data)) {
|
||||||
<span class="action-status-badge is-failed">
|
<span class="action-status-badge is-failed">
|
||||||
<span class="material-symbols-outlined status-icon">error</span>
|
<span class="material-symbols-outlined status-icon">error</span>
|
||||||
Failed
|
|
||||||
|
失败
|
||||||
</span>
|
</span>
|
||||||
}
|
}
|
||||||
<span class="action-expand-icon material-symbols-outlined">
|
<span class="action-expand-icon material-symbols-outlined">
|
||||||
@@ -1148,19 +1156,19 @@ limitations under the License.
|
|||||||
@if (isAdbCommandTool(item.data)) {
|
@if (isAdbCommandTool(item.data)) {
|
||||||
@if (getAdbCommandLine(item.data)) {
|
@if (getAdbCommandLine(item.data)) {
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">Command:</span>
|
<span class="detail-label">命令:</span>
|
||||||
<span class="detail-value command-value"><code>{{ getAdbCommandLine(item.data) }}</code></span>
|
<span class="detail-value command-value"><code>{{ getAdbCommandLine(item.data) }}</code></span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if (getAdbCwd(item.data)) {
|
@if (getAdbCwd(item.data)) {
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">Cwd:</span>
|
<span class="detail-label">工作目录:</span>
|
||||||
<span class="detail-value">"{{ getAdbCwd(item.data) }}"</span>
|
<span class="detail-value">"{{ getAdbCwd(item.data) }}"</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if (getAdbTerminalId(item.data)) {
|
@if (getAdbTerminalId(item.data)) {
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">Terminal ID:</span>
|
<span class="detail-label">终端 ID:</span>
|
||||||
<span class="detail-value">"{{ getAdbTerminalId(item.data) }}"</span>
|
<span class="detail-value">"{{ getAdbTerminalId(item.data) }}"</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -1169,7 +1177,7 @@ limitations under the License.
|
|||||||
@if (!isAdbCommandTool(item.data)) {
|
@if (!isAdbCommandTool(item.data)) {
|
||||||
@if (getToolTargetText(item.data)) {
|
@if (getToolTargetText(item.data)) {
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">Target:</span>
|
<span class="detail-label">目标:</span>
|
||||||
<span class="detail-value">"{{ getToolTargetText(item.data) }}"</span>
|
<span class="detail-value">"{{ getToolTargetText(item.data) }}"</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -1181,19 +1189,19 @@ limitations under the License.
|
|||||||
}
|
}
|
||||||
@if (getToolCoords(item.data)) {
|
@if (getToolCoords(item.data)) {
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">Coordinates:</span>
|
<span class="detail-label">坐标:</span>
|
||||||
<span class="detail-value coordinate-badge">{{ getToolCoords(item.data) }}</span>
|
<span class="detail-value coordinate-badge">{{ getToolCoords(item.data) }}</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if (getToolAnalysisText(item.data)) {
|
@if (getToolAnalysisText(item.data)) {
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">Analysis:</span>
|
<span class="detail-label">分析:</span>
|
||||||
<span class="detail-value">"{{ getToolAnalysisText(item.data) }}"</span>
|
<span class="detail-value">"{{ getToolAnalysisText(item.data) }}"</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if (getToolGenericDetails(item.data)) {
|
@if (getToolGenericDetails(item.data)) {
|
||||||
<div class="action-detail-row">
|
<div class="action-detail-row">
|
||||||
<span class="detail-label">Details:</span>
|
<span class="detail-label">详情:</span>
|
||||||
<span class="detail-value">"{{ getToolGenericDetails(item.data) }}"</span>
|
<span class="detail-value">"{{ getToolGenericDetails(item.data) }}"</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -1207,19 +1215,19 @@ limitations under the License.
|
|||||||
@if (isAdbCommandTool(item.data)) {
|
@if (isAdbCommandTool(item.data)) {
|
||||||
@if (getAdbCommandLine(item.data)) {
|
@if (getAdbCommandLine(item.data)) {
|
||||||
<div class="grid-item full-width">
|
<div class="grid-item full-width">
|
||||||
<span class="grid-label">Command</span>
|
<span class="grid-label">命令</span>
|
||||||
<span class="grid-value command-value"><code>{{ getAdbCommandLine(item.data) }}</code></span>
|
<span class="grid-value command-value"><code>{{ getAdbCommandLine(item.data) }}</code></span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if (getAdbCwd(item.data)) {
|
@if (getAdbCwd(item.data)) {
|
||||||
<div class="grid-item">
|
<div class="grid-item">
|
||||||
<span class="grid-label">Working Directory</span>
|
<span class="grid-label">工作目录</span>
|
||||||
<span class="grid-value">"{{ getAdbCwd(item.data) }}"</span>
|
<span class="grid-value">"{{ getAdbCwd(item.data) }}"</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if (getAdbTerminalId(item.data)) {
|
@if (getAdbTerminalId(item.data)) {
|
||||||
<div class="grid-item">
|
<div class="grid-item">
|
||||||
<span class="grid-label">Terminal ID</span>
|
<span class="grid-label">终端 ID</span>
|
||||||
<span class="grid-value code-pill">{{ getAdbTerminalId(item.data) }}</span>
|
<span class="grid-value code-pill">{{ getAdbTerminalId(item.data) }}</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -1227,7 +1235,7 @@ limitations under the License.
|
|||||||
@if (!isAdbCommandTool(item.data)) {
|
@if (!isAdbCommandTool(item.data)) {
|
||||||
@if (getToolTargetText(item.data)) {
|
@if (getToolTargetText(item.data)) {
|
||||||
<div class="grid-item">
|
<div class="grid-item">
|
||||||
<span class="grid-label">Target</span>
|
<span class="grid-label">目标</span>
|
||||||
<span class="grid-value">"{{ getToolTargetText(item.data) }}"</span>
|
<span class="grid-value">"{{ getToolTargetText(item.data) }}"</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -1239,19 +1247,19 @@ limitations under the License.
|
|||||||
}
|
}
|
||||||
@if (getToolCoords(item.data)) {
|
@if (getToolCoords(item.data)) {
|
||||||
<div class="grid-item">
|
<div class="grid-item">
|
||||||
<span class="grid-label">Coordinates</span>
|
<span class="grid-label">坐标</span>
|
||||||
<span class="grid-value code-pill">{{ getToolCoords(item.data) }}</span>
|
<span class="grid-value code-pill">{{ getToolCoords(item.data) }}</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if (getToolAnalysisText(item.data)) {
|
@if (getToolAnalysisText(item.data)) {
|
||||||
<div class="grid-item">
|
<div class="grid-item">
|
||||||
<span class="grid-label">Analysis</span>
|
<span class="grid-label">分析</span>
|
||||||
<span class="grid-value">"{{ getToolAnalysisText(item.data) }}"</span>
|
<span class="grid-value">"{{ getToolAnalysisText(item.data) }}"</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@if (getToolGenericDetails(item.data)) {
|
@if (getToolGenericDetails(item.data)) {
|
||||||
<div class="grid-item">
|
<div class="grid-item">
|
||||||
<span class="grid-label">Details</span>
|
<span class="grid-label">详情</span>
|
||||||
<span class="grid-value">"{{ getToolGenericDetails(item.data) }}"</span>
|
<span class="grid-value">"{{ getToolGenericDetails(item.data) }}"</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -1268,10 +1276,10 @@ limitations under the License.
|
|||||||
<div class="expanded-screenshots-container">
|
<div class="expanded-screenshots-container">
|
||||||
@if (getStepPreImageUrl(block.data, item.data)) {
|
@if (getStepPreImageUrl(block.data, item.data)) {
|
||||||
<div class="screenshot-column">
|
<div class="screenshot-column">
|
||||||
<span class="screenshot-label">Pre-Action (Before)</span>
|
<span class="screenshot-label">操作前</span>
|
||||||
<div class="image-wrapper" (click)="openImageModal(getStepPreImageUrl(block.data, item.data)!, item.data, $event)">
|
<div class="image-wrapper" (click)="openImageModal(getStepPreImageUrl(block.data, item.data)!, item.data, $event)">
|
||||||
<div class="img-relative-box">
|
<div class="img-relative-box">
|
||||||
<img #preImgTool [src]="getStepPreImageUrl(block.data, item.data)" (load)="onImageLoad(preImgTool, preOverlayTool, item.data)" alt="Pre-Action Screenshot" />
|
<img #preImgTool [src]="getStepPreImageUrl(block.data, item.data)" (load)="onImageLoad(preImgTool, preOverlayTool, item.data)" alt="操作前截图" />
|
||||||
<div #preOverlayTool class="overlay-layer"></div>
|
<div #preOverlayTool class="overlay-layer"></div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -1279,10 +1287,10 @@ limitations under the License.
|
|||||||
}
|
}
|
||||||
@if (getStepPostImageUrl(block.data, item.data)) {
|
@if (getStepPostImageUrl(block.data, item.data)) {
|
||||||
<div class="screenshot-column">
|
<div class="screenshot-column">
|
||||||
<span class="screenshot-label">Post-Action (After)</span>
|
<span class="screenshot-label">操作后</span>
|
||||||
<div class="image-wrapper" (click)="openImageModal(getStepPostImageUrl(block.data, item.data)!, null, $event)">
|
<div class="image-wrapper" (click)="openImageModal(getStepPostImageUrl(block.data, item.data)!, null, $event)">
|
||||||
<div class="img-relative-box">
|
<div class="img-relative-box">
|
||||||
<img [src]="getStepPostImageUrl(block.data, item.data)" alt="Post-Action Screenshot" />
|
<img [src]="getStepPostImageUrl(block.data, item.data)" alt="操作后截图" />
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -1337,12 +1345,12 @@ limitations under the License.
|
|||||||
<div class="synthesis-card-header">
|
<div class="synthesis-card-header">
|
||||||
<div class="header-left">
|
<div class="header-left">
|
||||||
<span class="material-symbols-outlined icon-sparkle">assignment_turned_in</span>
|
<span class="material-symbols-outlined icon-sparkle">assignment_turned_in</span>
|
||||||
<span class="main-title">Task Report</span>
|
<span class="main-title">任务报告</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="header-right">
|
<div class="header-right">
|
||||||
<button type="button" class="btn-open-note" (click)="openOutputterNote($event)">
|
<button type="button" class="btn-open-note" (click)="openOutputterNote($event)">
|
||||||
<span class="material-symbols-outlined">visibility</span>
|
<span class="material-symbols-outlined">visibility</span>
|
||||||
<span>View in Notes</span>
|
<span>在笔记中查看</span>
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
@@ -1423,11 +1431,11 @@ limitations under the License.
|
|||||||
@if (selectedZoomImage()) {
|
@if (selectedZoomImage()) {
|
||||||
<div class="image-modal-backdrop" (click)="closeImageModal($event)">
|
<div class="image-modal-backdrop" (click)="closeImageModal($event)">
|
||||||
<div class="image-modal-content" (click)="$event.stopPropagation()">
|
<div class="image-modal-content" (click)="$event.stopPropagation()">
|
||||||
<button class="image-modal-close" (click)="closeImageModal($event)" aria-label="Close image preview">
|
<button class="image-modal-close" (click)="closeImageModal($event)" aria-label="关闭图片预览">
|
||||||
<span class="material-symbols-outlined">close</span>
|
<span class="material-symbols-outlined">close</span>
|
||||||
</button>
|
</button>
|
||||||
<div class="img-relative-box">
|
<div class="img-relative-box">
|
||||||
<img #modalImg [src]="selectedZoomImage()" (load)="onImageLoad(modalImg, modalOverlay, selectedZoomAction())" alt="Zoomed Screenshot" />
|
<img #modalImg [src]="selectedZoomImage()" (load)="onImageLoad(modalImg, modalOverlay, selectedZoomAction())" alt="放大截图" />
|
||||||
<div #modalOverlay class="overlay-layer"></div>
|
<div #modalOverlay class="overlay-layer"></div>
|
||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { UiLabelPipe } from '../../utils/ui-label.pipe';
|
||||||
/**
|
/**
|
||||||
* Copyright 2026 Google LLC
|
* Copyright 2026 Google LLC
|
||||||
*
|
*
|
||||||
@@ -34,22 +35,22 @@ import {
|
|||||||
} from '../../utils/run-info.util';
|
} from '../../utils/run-info.util';
|
||||||
|
|
||||||
export const PLANNING_LOADER_PHRASES: string[] = [
|
export const PLANNING_LOADER_PHRASES: string[] = [
|
||||||
'Planning next step...',
|
"正在规划下一步…",
|
||||||
'Analyzing screen coordinates...',
|
"正在分析屏幕坐标…",
|
||||||
'Consulting neural network...',
|
"正在请求模型…",
|
||||||
'Formulating tactical action plan...',
|
"正在制定操作计划…",
|
||||||
'Deciphering UI state & elements...',
|
"正在识别界面状态与元素…",
|
||||||
'Synthesizing decision pathways...',
|
"正在评估执行路径…",
|
||||||
'Calibrating next move...',
|
"正在确认下一步操作…",
|
||||||
'Aligning logical vectors...',
|
"正在整理推理结果…",
|
||||||
'Evaluating optimal sub-goals...',
|
"正在评估子目标…",
|
||||||
'Gathering sensory inputs...',
|
"正在收集屏幕信息…",
|
||||||
'Computing next interaction...',
|
"正在分析下一步交互…",
|
||||||
'Optimizing execution strategy...',
|
"正在调整执行策略…",
|
||||||
'Simulating probable outcomes...',
|
"正在预测操作结果…",
|
||||||
'Summoning AI intuition...',
|
"正在分析任务…",
|
||||||
'Brewing the next command...',
|
"正在准备下一条指令…",
|
||||||
'Strategizing tactical moves...'
|
"正在规划操作步骤…"
|
||||||
];
|
];
|
||||||
|
|
||||||
export interface StartupWorkItem extends StartupProgressEvent {
|
export interface StartupWorkItem extends StartupProgressEvent {
|
||||||
@@ -71,12 +72,12 @@ const STARTUP_WORK_STAGES: StartupWorkStage[] = [
|
|||||||
{
|
{
|
||||||
started: 'device_check',
|
started: 'device_check',
|
||||||
completed: 'device_ready',
|
completed: 'device_ready',
|
||||||
completedMessage: 'Android device connected'
|
completedMessage: "Android 设备已连接"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
started: 'uiautomator',
|
started: 'uiautomator',
|
||||||
completed: 'uiautomator_ready',
|
completed: 'uiautomator_ready',
|
||||||
completedMessage: 'UI hierarchy service is ready',
|
completedMessage: "界面层级服务已就绪",
|
||||||
// First task on a device installs / upgrades the accessibility helper
|
// First task on a device installs / upgrades the accessibility helper
|
||||||
// (a few seconds): say so instead of a generic "connecting".
|
// (a few seconds): say so instead of a generic "connecting".
|
||||||
liveDetailStages: ['helper_install', 'helper_upgrade'],
|
liveDetailStages: ['helper_install', 'helper_upgrade'],
|
||||||
@@ -86,7 +87,7 @@ const STARTUP_WORK_STAGES: StartupWorkStage[] = [
|
|||||||
{
|
{
|
||||||
started: 'environment',
|
started: 'environment',
|
||||||
completed: 'environment_ready',
|
completed: 'environment_ready',
|
||||||
completedMessage: 'Device environment is ready'
|
completedMessage: "设备环境已就绪"
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
|
|
||||||
@@ -233,7 +234,7 @@ export type { MarkdownSegment, MarkdownLine, NoteMilestone, ParsedNote, StreamRe
|
|||||||
@Component({
|
@Component({
|
||||||
selector: 'app-agent-stream',
|
selector: 'app-agent-stream',
|
||||||
standalone: true,
|
standalone: true,
|
||||||
imports: [CommonModule, FormsModule, OverlayModule],
|
imports: [UiLabelPipe, CommonModule, FormsModule, OverlayModule],
|
||||||
templateUrl: './agent-stream.component.html',
|
templateUrl: './agent-stream.component.html',
|
||||||
styleUrl: './agent-stream.component.scss',
|
styleUrl: './agent-stream.component.scss',
|
||||||
changeDetection: ChangeDetectionStrategy.OnPush
|
changeDetection: ChangeDetectionStrategy.OnPush
|
||||||
@@ -558,7 +559,7 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
onCleanup(() => clearInterval(clockInterval));
|
onCleanup(() => clearInterval(clockInterval));
|
||||||
});
|
});
|
||||||
|
|
||||||
// The retry clock only needs to tick while something on screen is timing
|
// The retry clock only needs to tick while something 在屏幕上 is timing
|
||||||
// (startup progress, LLM retry countdowns). An idle page stays quiet.
|
// (startup progress, LLM retry countdowns). An idle page stays quiet.
|
||||||
this.zone.runOutsideAngular(() => {
|
this.zone.runOutsideAngular(() => {
|
||||||
const retryClockInterval = setInterval(() => {
|
const retryClockInterval = setInterval(() => {
|
||||||
@@ -712,7 +713,7 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
this.scheduledCollapses.clear();
|
this.scheduledCollapses.clear();
|
||||||
this.collapsedStreams.set(new Set<string>());
|
this.collapsedStreams.set(new Set<string>());
|
||||||
this.isUserAtBottom = true;
|
this.isUserAtBottom = true;
|
||||||
|
|
||||||
setTimeout(() => {
|
setTimeout(() => {
|
||||||
const container = document.querySelector('.stream-logs-content');
|
const container = document.querySelector('.stream-logs-content');
|
||||||
if (container) {
|
if (container) {
|
||||||
@@ -929,12 +930,12 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
|
|
||||||
public deleteTask(sessionId: string, event?: Event): void {
|
public deleteTask(sessionId: string, event?: Event): void {
|
||||||
if (event) event.stopPropagation();
|
if (event) event.stopPropagation();
|
||||||
if (!confirm(`Are you sure you want to delete this task? This cannot be undone.`)) {
|
if (!confirm("确定删除此任务?此操作无法撤销。")) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
this.agentService.deleteSession(sessionId).subscribe({
|
this.agentService.deleteSession(sessionId).subscribe({
|
||||||
error: (err) => {
|
error: (err) => {
|
||||||
console.error(`Failed to delete session ${sessionId}:`, err);
|
console.error(`删除会话失败:${sessionId}:`, err);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
@@ -1516,12 +1517,12 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
}
|
}
|
||||||
|
|
||||||
public getVideoAnalysisLabel(tool: any): string {
|
public getVideoAnalysisLabel(tool: any): string {
|
||||||
return getVideoAnalysisView(tool)?.title || 'Analyzing screen recording';
|
return getVideoAnalysisView(tool)?.title || "正在分析屏幕录像";
|
||||||
}
|
}
|
||||||
|
|
||||||
public getVideoAnalysisRangeLabel(tool: any): string {
|
public getVideoAnalysisRangeLabel(tool: any): string {
|
||||||
const range = getVideoAnalysisView(tool)?.requestedRange;
|
const range = getVideoAnalysisView(tool)?.requestedRange;
|
||||||
if (!range) return 'Screen recording';
|
if (!range) return "屏幕录像";
|
||||||
return `${formatVideoTime(range.start)}–${formatVideoTime(range.end)}`;
|
return `${formatVideoTime(range.start)}–${formatVideoTime(range.end)}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1530,11 +1531,11 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
if (!view || view.totalCount <= 1) return '';
|
if (!view || view.totalCount <= 1) return '';
|
||||||
if (view.outcome === 'running' || view.outcome === 'recovering') {
|
if (view.outcome === 'running' || view.outcome === 'recovering') {
|
||||||
return view.completedCount > 0
|
return view.completedCount > 0
|
||||||
? `${view.completedCount}/${view.totalCount} segments saved`
|
? `${view.completedCount}/${view.totalCount} 段已保存`
|
||||||
: '';
|
: '';
|
||||||
}
|
}
|
||||||
if (view.outcome === 'partial') {
|
if (view.outcome === 'partial') {
|
||||||
return `${view.completedCount}/${view.totalCount} segments saved`;
|
return `${view.completedCount}/${view.totalCount} 段已保存`;
|
||||||
}
|
}
|
||||||
return '';
|
return '';
|
||||||
}
|
}
|
||||||
@@ -1554,23 +1555,23 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
|
|
||||||
public getScreenRecordingButtonTitle(): string {
|
public getScreenRecordingButtonTitle(): string {
|
||||||
if (this.agentService.isCurrentSessionRunning()) {
|
if (this.agentService.isCurrentSessionRunning()) {
|
||||||
return 'Task is currently running (Recording screen)';
|
return "任务正在运行(录屏中)";
|
||||||
}
|
}
|
||||||
if (this.agentService.currentSessionRecordingStatus() === 'processing') {
|
if (this.agentService.currentSessionRecordingStatus() === 'processing') {
|
||||||
return 'Preparing screen recording...';
|
return "正在准备屏幕录像…";
|
||||||
}
|
}
|
||||||
if (this.agentService.currentSessionVideoUrl()) {
|
if (this.agentService.currentSessionVideoUrl()) {
|
||||||
return 'Play Screen Recording Video';
|
return "播放屏幕录像";
|
||||||
}
|
}
|
||||||
if (this.agentService.hasCurrentSessionStepFrames()) {
|
if (this.agentService.hasCurrentSessionStepFrames()) {
|
||||||
return this.agentService.currentSessionRecordingStatus() === 'failed'
|
return this.agentService.currentSessionRecordingStatus() === 'failed'
|
||||||
? 'Video recording failed — Click to replay step-by-step screenshots'
|
? "录屏失败,点击逐步回放截图"
|
||||||
: 'Play step-by-step screenshots replay';
|
: "逐步回放截图";
|
||||||
}
|
}
|
||||||
if (this.agentService.currentSessionRecordingStatus() === 'failed') {
|
if (this.agentService.currentSessionRecordingStatus() === 'failed') {
|
||||||
return 'Screen recording generation failed';
|
return "屏幕录像生成失败";
|
||||||
}
|
}
|
||||||
return 'Screen Recording';
|
return "屏幕录像";
|
||||||
}
|
}
|
||||||
|
|
||||||
public isDeviceActionTool(tool: any): boolean {
|
public isDeviceActionTool(tool: any): boolean {
|
||||||
@@ -1639,11 +1640,11 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
public getCheckerPhaseLabel(block: any): string {
|
public getCheckerPhaseLabel(block: any): string {
|
||||||
switch (block?.data?.phase) {
|
switch (block?.data?.phase) {
|
||||||
case 'final':
|
case 'final':
|
||||||
return 'Final check';
|
return "最终检查";
|
||||||
case 'outcome':
|
case 'outcome':
|
||||||
return 'Result';
|
return "结果";
|
||||||
default:
|
default:
|
||||||
return 'Check';
|
return "勾选";
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1660,8 +1661,8 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
|
|
||||||
/** How the item is judged, in plain words (verify = must pass, assert = recorded test result). */
|
/** How the item is judged, in plain words (verify = must pass, assert = recorded test result). */
|
||||||
public getCheckMethodLabel(item: any, block: any): string {
|
public getCheckMethodLabel(item: any, block: any): string {
|
||||||
const parts: string[] = [item?.kind === 'assert' ? 'Test assertion' : 'Must pass'];
|
const parts: string[] = [item?.kind === 'assert' ? "测试断言" : "必须通过"];
|
||||||
if (item?.when === 'at_end' && block?.data?.phase !== 'final') parts.push('at the end');
|
if (item?.when === 'at_end' && block?.data?.phase !== 'final') parts.push("结束时");
|
||||||
return parts.join(' · ');
|
return parts.join(' · ');
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1680,15 +1681,15 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
public getVerdictStatusText(status: string): string {
|
public getVerdictStatusText(status: string): string {
|
||||||
switch (status) {
|
switch (status) {
|
||||||
case 'passed':
|
case 'passed':
|
||||||
return 'Passed';
|
return "已通过";
|
||||||
case 'failed':
|
case 'failed':
|
||||||
return 'Failed';
|
return "失败";
|
||||||
case 'inconclusive':
|
case 'inconclusive':
|
||||||
return 'Could not be confirmed';
|
return "无法确认";
|
||||||
case 'superseded':
|
case 'superseded':
|
||||||
return 'Superseded by a newer check';
|
return "已被较新的检查替代";
|
||||||
case 'unchecked':
|
case 'unchecked':
|
||||||
return 'Not checked';
|
return "未检查";
|
||||||
default:
|
default:
|
||||||
return status || '';
|
return status || '';
|
||||||
}
|
}
|
||||||
@@ -1706,9 +1707,9 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
if (d.phase === 'outcome') return Array.isArray(d.last_findings) ? d.last_findings : [];
|
if (d.phase === 'outcome') return Array.isArray(d.last_findings) ? d.last_findings : [];
|
||||||
const notes: string[] = [];
|
const notes: string[] = [];
|
||||||
if (Array.isArray(d.unmet_subgoals)) {
|
if (Array.isArray(d.unmet_subgoals)) {
|
||||||
for (const text of d.unmet_subgoals) notes.push(`Not finished yet: ${text}`);
|
for (const text of d.unmet_subgoals) notes.push(`尚未完成:${text}`);
|
||||||
}
|
}
|
||||||
if (d.reverted) notes.push('The step was sent back to be redone.');
|
if (d.reverted) notes.push("此步骤已退回重新执行。");
|
||||||
if (d.error) notes.push(String(d.error));
|
if (d.error) notes.push(String(d.error));
|
||||||
return notes;
|
return notes;
|
||||||
}
|
}
|
||||||
@@ -1760,32 +1761,32 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
const d = block?.data || {};
|
const d = block?.data || {};
|
||||||
if (d.phase === 'outcome') {
|
if (d.phase === 'outcome') {
|
||||||
const t = d.tests || {};
|
const t = d.tests || {};
|
||||||
const label = d.task_status === 'completed' ? 'Goal completed'
|
const label = d.task_status === 'completed' ? "目标已完成"
|
||||||
: (d.task_status === 'blocked' ? 'Blocked' : 'Partially completed');
|
: (d.task_status === 'blocked' ? "受阻" : "部分完成");
|
||||||
const counts = [
|
const counts = [
|
||||||
[t.passed, 'passed'],
|
[t.passed, 'passed'],
|
||||||
[t.failed, 'failed'],
|
[t.failed, 'failed'],
|
||||||
[t.inconclusive, 'inconclusive'],
|
[t.inconclusive, 'inconclusive'],
|
||||||
[t.unchecked, 'not checked']
|
[t.unchecked, "未检查"]
|
||||||
]
|
]
|
||||||
.filter(([n]) => Number(n) > 0)
|
.filter(([n]) => Number(n) > 0)
|
||||||
.map(([n, word]) => `${n} ${word}`);
|
.map(([n, word]) => `${n} ${word}`);
|
||||||
return counts.length > 0 ? `${label} · ${counts.join(' · ')}` : label;
|
return counts.length > 0 ? `${label} · ${counts.join(' · ')}` : label;
|
||||||
}
|
}
|
||||||
if (d.isCompleted === false) return 'Checking…';
|
if (d.isCompleted === false) return "正在检查…";
|
||||||
switch (d.status) {
|
switch (d.status) {
|
||||||
case 'superseded':
|
case 'superseded':
|
||||||
return 'Superseded';
|
return "已替代";
|
||||||
case 'unchecked':
|
case 'unchecked':
|
||||||
return 'Not checked';
|
return "未检查";
|
||||||
case 'error':
|
case 'error':
|
||||||
return 'No verdict';
|
return "暂无结论";
|
||||||
}
|
}
|
||||||
const verdicts = this.getCheckerVerdicts(block);
|
const verdicts = this.getCheckerVerdicts(block);
|
||||||
const failed = verdicts.filter((v) => v.status === 'failed').length;
|
const failed = verdicts.filter((v) => v.status === 'failed').length;
|
||||||
if (failed > 0) return `${failed} failed`;
|
if (failed > 0) return `${failed} failed`;
|
||||||
if (verdicts.some((v) => v.status === 'inconclusive')) return 'Inconclusive';
|
if (verdicts.some((v) => v.status === 'inconclusive')) return "无法确定";
|
||||||
return verdicts.length > 0 ? 'Passed' : 'Done';
|
return verdicts.length > 0 ? "已通过" : "完成";
|
||||||
}
|
}
|
||||||
|
|
||||||
public getVerdictIcon(status: string): string {
|
public getVerdictIcon(status: string): string {
|
||||||
@@ -1854,7 +1855,7 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
}
|
}
|
||||||
|
|
||||||
public getBackendSwitchMessage(tool: any): string {
|
public getBackendSwitchMessage(tool: any): string {
|
||||||
return String(tool?.payload?.message || 'UI hierarchy source changed');
|
return String(tool?.payload?.message || "界面层级来源已变更");
|
||||||
}
|
}
|
||||||
|
|
||||||
public resumePausedTask(event: Event): void {
|
public resumePausedTask(event: Event): void {
|
||||||
@@ -1863,12 +1864,12 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
}
|
}
|
||||||
|
|
||||||
public getLLMErrorText(tool: any): string {
|
public getLLMErrorText(tool: any): string {
|
||||||
const noDetails = 'The AI provider did not return error details after the request failed.';
|
const noDetails = "模型请求失败,接口未返回错误详情。";
|
||||||
if (!tool) return noDetails;
|
if (!tool) return noDetails;
|
||||||
const rawError = tool.payload?.error || tool.error;
|
const rawError = tool.payload?.error || tool.error;
|
||||||
if (!rawError) return noDetails;
|
if (!rawError) return noDetails;
|
||||||
const cleaned = this.cleanErrorMessage(rawError);
|
const cleaned = this.cleanErrorMessage(rawError);
|
||||||
return cleaned === 'Unknown error' ? noDetails : cleaned;
|
return cleaned === "未知错误" ? noDetails : cleaned;
|
||||||
}
|
}
|
||||||
|
|
||||||
public isLLMRetry(tool: any): boolean {
|
public isLLMRetry(tool: any): boolean {
|
||||||
@@ -1945,7 +1946,7 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
|
|
||||||
public getToolCallName(action: any): string {
|
public getToolCallName(action: any): string {
|
||||||
const act = this.getActionObject(action);
|
const act = this.getActionObject(action);
|
||||||
if (!act) return 'Tool';
|
if (!act) return "工具";
|
||||||
const label = this.getToolDisplayLabel(act);
|
const label = this.getToolDisplayLabel(act);
|
||||||
const key = this.getToolKey(act);
|
const key = this.getToolKey(act);
|
||||||
return key ? `${label}: ${key}` : label;
|
return key ? `${label}: ${key}` : label;
|
||||||
@@ -2051,13 +2052,13 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
public getStatusLabel(status: string): string {
|
public getStatusLabel(status: string): string {
|
||||||
switch (status) {
|
switch (status) {
|
||||||
case 'idle':
|
case 'idle':
|
||||||
return 'Idle';
|
return "空闲";
|
||||||
case 'running':
|
case 'running':
|
||||||
return 'Running';
|
return "执行中";
|
||||||
case 'completed':
|
case 'completed':
|
||||||
return 'Completed';
|
return "已完成";
|
||||||
case 'offline':
|
case 'offline':
|
||||||
return 'Offline';
|
return "离线";
|
||||||
default:
|
default:
|
||||||
return status.toUpperCase();
|
return status.toUpperCase();
|
||||||
}
|
}
|
||||||
@@ -2085,7 +2086,7 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
public onToolKeyClick(key: string): void {
|
public onToolKeyClick(key: string): void {
|
||||||
this.agentService.selectedNoteKey.set(key);
|
this.agentService.selectedNoteKey.set(key);
|
||||||
this.agentService.activeTab.set('notes');
|
this.agentService.activeTab.set('notes');
|
||||||
|
|
||||||
const sessionId = this.agentService.currentSessionId();
|
const sessionId = this.agentService.currentSessionId();
|
||||||
if (sessionId) {
|
if (sessionId) {
|
||||||
this.agentService.fetchNotes(sessionId);
|
this.agentService.fetchNotes(sessionId);
|
||||||
@@ -2162,16 +2163,16 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
}
|
}
|
||||||
|
|
||||||
public getArchitectureTooltip(model?: ModelInfo | null): string {
|
public getArchitectureTooltip(model?: ModelInfo | null): string {
|
||||||
if (!model) return 'Agent Architecture: ARTEMIS Flash (Reactive Fast Loop)';
|
if (!model) return "执行模式:ARTEMIS Flash(快速响应)";
|
||||||
const name = this.getModelDisplayName(model.name);
|
const name = this.getModelDisplayName(model.name);
|
||||||
const isPro = name.toLowerCase().includes('pro');
|
const isPro = name.toLowerCase().includes('pro');
|
||||||
const archDesc = isPro
|
const archDesc = isPro
|
||||||
? 'ARTEMIS Pro (Multi-Agent Cognitive State Graph)'
|
? "ARTEMIS Pro(多智能体规划)"
|
||||||
: 'ARTEMIS Flash (Reactive Fast Loop)';
|
: "ARTEMIS Flash(快速响应)";
|
||||||
if (model.id) {
|
if (model.id) {
|
||||||
return `Agent Architecture: ${archDesc} · LLM: ${model.id} (${model.provider || 'google'})`;
|
return `智能体架构:${archDesc} · LLM: ${model.id} (${model.provider || 'google'})`;
|
||||||
}
|
}
|
||||||
return `Agent Architecture: ${archDesc}`;
|
return `智能体架构:${archDesc}`;
|
||||||
}
|
}
|
||||||
|
|
||||||
public formatTokenCount(tokens?: number): string {
|
public formatTokenCount(tokens?: number): string {
|
||||||
@@ -2181,9 +2182,9 @@ export class AgentStreamComponent implements AfterViewInit {
|
|||||||
public getTokenTooltip(phase: PhaseBlock): string {
|
public getTokenTooltip(phase: PhaseBlock): string {
|
||||||
if (!phase.tokens) return '';
|
if (!phase.tokens) return '';
|
||||||
if (phase.promptTokens && phase.completionTokens) {
|
if (phase.promptTokens && phase.completionTokens) {
|
||||||
return `Consumed: ${phase.tokens.toLocaleString()} tokens (${phase.promptTokens.toLocaleString()} in / ${phase.completionTokens.toLocaleString()} out)`;
|
return `已使用:${phase.tokens.toLocaleString()} tokens (${phase.promptTokens.toLocaleString()} 输入 / ${phase.completionTokens.toLocaleString()} 输出)`;
|
||||||
}
|
}
|
||||||
return `Consumed: ${phase.tokens.toLocaleString()} tokens`;
|
return `已使用:${phase.tokens.toLocaleString()} tokens`;
|
||||||
}
|
}
|
||||||
|
|
||||||
public isCurrentSessionActive(): boolean {
|
public isCurrentSessionActive(): boolean {
|
||||||
|
|||||||
@@ -25,7 +25,7 @@ limitations under the License.
|
|||||||
(click)="agentService.activeTab.set('tasks')"
|
(click)="agentService.activeTab.set('tasks')"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined tab-icon">list_alt</span>
|
<span class="material-symbols-outlined tab-icon">list_alt</span>
|
||||||
<span class="tab-text">Task Queue</span>
|
<span class="tab-text">任务队列</span>
|
||||||
</button>
|
</button>
|
||||||
<button
|
<button
|
||||||
class="tab-selector-btn"
|
class="tab-selector-btn"
|
||||||
@@ -33,7 +33,7 @@ limitations under the License.
|
|||||||
(click)="agentService.activeTab.set('notes')"
|
(click)="agentService.activeTab.set('notes')"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined tab-icon">description</span>
|
<span class="material-symbols-outlined tab-icon">description</span>
|
||||||
<span class="tab-text">Notes & Plans</span>
|
<span class="tab-text">笔记与计划</span>
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -42,8 +42,8 @@ limitations under the License.
|
|||||||
<button
|
<button
|
||||||
class="btn-clear-history"
|
class="btn-clear-history"
|
||||||
(click)="clearHistory()"
|
(click)="clearHistory()"
|
||||||
title="Clear all tasks and history"
|
title="清空所有任务与历史"
|
||||||
aria-label="Clear all tasks and history"
|
aria-label="清空所有任务与历史"
|
||||||
[disabled]="isSubmitting()"
|
[disabled]="isSubmitting()"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined">delete</span>
|
<span class="material-symbols-outlined">delete</span>
|
||||||
@@ -62,7 +62,8 @@ limitations under the License.
|
|||||||
</div>
|
</div>
|
||||||
@if (activeQueue().length === 0) {
|
@if (activeQueue().length === 0) {
|
||||||
<div class="empty-section-placeholder">
|
<div class="empty-section-placeholder">
|
||||||
No active or queued tasks.
|
|
||||||
|
没有正在执行或排队的任务。
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@for (session of activeQueue(); track trackSession($index, session)) {
|
@for (session of activeQueue(); track trackSession($index, session)) {
|
||||||
@@ -77,7 +78,7 @@ limitations under the License.
|
|||||||
@if (getTaskStatus(session) === 'running') {
|
@if (getTaskStatus(session) === 'running') {
|
||||||
<span class="status-dot"></span>
|
<span class="status-dot"></span>
|
||||||
}
|
}
|
||||||
{{ getTaskStatus(session) | uppercase }}
|
{{ getTaskStatus(session) | uiLabel }}
|
||||||
</span>
|
</span>
|
||||||
<div class="task-card-header-actions">
|
<div class="task-card-header-actions">
|
||||||
@if (getDeviceSerial(session)) {
|
@if (getDeviceSerial(session)) {
|
||||||
@@ -95,8 +96,8 @@ limitations under the License.
|
|||||||
type="button"
|
type="button"
|
||||||
class="btn-stop-card"
|
class="btn-stop-card"
|
||||||
(click)="stopTask(session.session_id, $event)"
|
(click)="stopTask(session.session_id, $event)"
|
||||||
title="Stop this task"
|
title="停止此任务"
|
||||||
aria-label="Stop this task"
|
aria-label="停止此任务"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined">stop_circle</span>
|
<span class="material-symbols-outlined">stop_circle</span>
|
||||||
</button>
|
</button>
|
||||||
@@ -106,8 +107,8 @@ limitations under the License.
|
|||||||
type="button"
|
type="button"
|
||||||
class="btn-delete-card"
|
class="btn-delete-card"
|
||||||
(click)="deleteTask(session.session_id, $event)"
|
(click)="deleteTask(session.session_id, $event)"
|
||||||
title="Delete this task"
|
title="删除此任务"
|
||||||
aria-label="Delete this task"
|
aria-label="删除此任务"
|
||||||
[disabled]="isSubmitting()"
|
[disabled]="isSubmitting()"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined">delete</span>
|
<span class="material-symbols-outlined">delete</span>
|
||||||
@@ -116,7 +117,7 @@ limitations under the License.
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<div class="task-goal" title="{{ session.initial_goal }}">
|
<div class="task-goal" title="{{ session.initial_goal }}">
|
||||||
{{ session.initial_goal || 'Unnamed Task' }}
|
{{ session.initial_goal || '未命名任务' }}
|
||||||
</div>
|
</div>
|
||||||
<div class="task-card-footer">
|
<div class="task-card-footer">
|
||||||
<span class="task-id" [title]="session.session_id">ID: {{ session.session_id }}</span>
|
<span class="task-id" [title]="session.session_id">ID: {{ session.session_id }}</span>
|
||||||
@@ -132,7 +133,8 @@ limitations under the License.
|
|||||||
<div class="history-list-wrapper">
|
<div class="history-list-wrapper">
|
||||||
@if (historyTasks().length === 0) {
|
@if (historyTasks().length === 0) {
|
||||||
<div class="empty-section-placeholder">
|
<div class="empty-section-placeholder">
|
||||||
No past tasks.
|
|
||||||
|
暂无历史任务。
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@for (session of historyTasks(); track trackSession($index, session)) {
|
@for (session of historyTasks(); track trackSession($index, session)) {
|
||||||
@@ -147,7 +149,7 @@ limitations under the License.
|
|||||||
@if (getTaskStatus(session) === 'running') {
|
@if (getTaskStatus(session) === 'running') {
|
||||||
<span class="status-dot"></span>
|
<span class="status-dot"></span>
|
||||||
}
|
}
|
||||||
{{ getTaskStatus(session) | uppercase }}
|
{{ getTaskStatus(session) | uiLabel }}
|
||||||
</span>
|
</span>
|
||||||
<div class="task-card-header-actions">
|
<div class="task-card-header-actions">
|
||||||
@if (getDeviceSerial(session)) {
|
@if (getDeviceSerial(session)) {
|
||||||
@@ -165,8 +167,8 @@ limitations under the License.
|
|||||||
type="button"
|
type="button"
|
||||||
class="btn-delete-card"
|
class="btn-delete-card"
|
||||||
(click)="deleteTask(session.session_id, $event)"
|
(click)="deleteTask(session.session_id, $event)"
|
||||||
title="Delete this task"
|
title="删除此任务"
|
||||||
aria-label="Delete this task"
|
aria-label="删除此任务"
|
||||||
[disabled]="isSubmitting()"
|
[disabled]="isSubmitting()"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined">delete</span>
|
<span class="material-symbols-outlined">delete</span>
|
||||||
@@ -175,7 +177,7 @@ limitations under the License.
|
|||||||
</div>
|
</div>
|
||||||
</div>
|
</div>
|
||||||
<div class="task-goal" title="{{ session.initial_goal }}">
|
<div class="task-goal" title="{{ session.initial_goal }}">
|
||||||
{{ session.initial_goal || 'Unnamed Task' }}
|
{{ session.initial_goal || '未命名任务' }}
|
||||||
</div>
|
</div>
|
||||||
<div class="task-card-footer">
|
<div class="task-card-footer">
|
||||||
<span class="task-id" [title]="session.session_id">ID: {{ session.session_id }}</span>
|
<span class="task-id" [title]="session.session_id">ID: {{ session.session_id }}</span>
|
||||||
@@ -212,8 +214,8 @@ limitations under the License.
|
|||||||
<div class="empty-notes-icon">
|
<div class="empty-notes-icon">
|
||||||
<span class="material-symbols-outlined">description</span>
|
<span class="material-symbols-outlined">description</span>
|
||||||
</div>
|
</div>
|
||||||
<h4 class="empty-notes-title">No Notes & Plans</h4>
|
<h4 class="empty-notes-title">暂无笔记与计划</h4>
|
||||||
<p class="empty-notes-subtitle">No notes or structured plans were recorded for this session.</p>
|
<p class="empty-notes-subtitle">此会话尚未记录笔记或结构化计划。</p>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
<!-- Notes Content Viewer (Clean card blocks matching the second screenshot style) -->
|
<!-- Notes Content Viewer (Clean card blocks matching the second screenshot style) -->
|
||||||
@@ -377,4 +379,3 @@ limitations under the License.
|
|||||||
}
|
}
|
||||||
</main>
|
</main>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
@@ -1,3 +1,4 @@
|
|||||||
|
import { UiLabelPipe } from '../../utils/ui-label.pipe';
|
||||||
/**
|
/**
|
||||||
* Copyright 2026 Google LLC
|
* Copyright 2026 Google LLC
|
||||||
*
|
*
|
||||||
@@ -27,7 +28,7 @@ export type { MarkdownSegment, MarkdownLine, NoteMilestone, ParsedNote };
|
|||||||
@Component({
|
@Component({
|
||||||
selector: 'app-chat-interface',
|
selector: 'app-chat-interface',
|
||||||
standalone: true,
|
standalone: true,
|
||||||
imports: [CommonModule, FormsModule],
|
imports: [UiLabelPipe, CommonModule, FormsModule],
|
||||||
templateUrl: './chat-interface.component.html',
|
templateUrl: './chat-interface.component.html',
|
||||||
styleUrl: './chat-interface.component.scss',
|
styleUrl: './chat-interface.component.scss',
|
||||||
changeDetection: ChangeDetectionStrategy.OnPush
|
changeDetection: ChangeDetectionStrategy.OnPush
|
||||||
@@ -95,7 +96,7 @@ export class ChatInterfaceComponent {
|
|||||||
error: (err) => {
|
error: (err) => {
|
||||||
console.error('Failed to submit task:', err);
|
console.error('Failed to submit task:', err);
|
||||||
this.isSubmitting.set(false);
|
this.isSubmitting.set(false);
|
||||||
this.errorMessage.set(err.error?.detail || 'The runner is busy. Please wait for the current task to finish.');
|
this.errorMessage.set(err.error?.detail || "执行器忙碌,请等待当前任务结束。");
|
||||||
// Auto-dismiss error banner after 5 seconds
|
// Auto-dismiss error banner after 5 seconds
|
||||||
setTimeout(() => {
|
setTimeout(() => {
|
||||||
this.errorMessage.set(null);
|
this.errorMessage.set(null);
|
||||||
@@ -133,7 +134,7 @@ export class ChatInterfaceComponent {
|
|||||||
* Clear all database data/history to start fresh
|
* Clear all database data/history to start fresh
|
||||||
*/
|
*/
|
||||||
public clearHistory(): void {
|
public clearHistory(): void {
|
||||||
if (!confirm('Are you sure you want to clear all tasks and history? This cannot be undone.')) {
|
if (!confirm("确定清空所有任务和历史记录?此操作无法撤销。")) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
this.isSubmitting.set(true);
|
this.isSubmitting.set(true);
|
||||||
@@ -145,7 +146,7 @@ export class ChatInterfaceComponent {
|
|||||||
error: (err: any) => {
|
error: (err: any) => {
|
||||||
console.error('Failed to clear history:', err);
|
console.error('Failed to clear history:', err);
|
||||||
this.isSubmitting.set(false);
|
this.isSubmitting.set(false);
|
||||||
this.errorMessage.set(err.error?.detail || 'Failed to clear history.');
|
this.errorMessage.set(err.error?.detail || "清空历史记录失败。");
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
@@ -155,7 +156,7 @@ export class ChatInterfaceComponent {
|
|||||||
*/
|
*/
|
||||||
public deleteTask(sessionId: string, event: MouseEvent): void {
|
public deleteTask(sessionId: string, event: MouseEvent): void {
|
||||||
event.stopPropagation();
|
event.stopPropagation();
|
||||||
if (!confirm(`Are you sure you want to delete this task? This cannot be undone.`)) {
|
if (!confirm("确定删除此任务?此操作无法撤销。")) {
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
this.isSubmitting.set(true);
|
this.isSubmitting.set(true);
|
||||||
@@ -165,9 +166,9 @@ export class ChatInterfaceComponent {
|
|||||||
this.isSubmitting.set(false);
|
this.isSubmitting.set(false);
|
||||||
},
|
},
|
||||||
error: (err: any) => {
|
error: (err: any) => {
|
||||||
console.error(`Failed to delete task ${sessionId}:`, err);
|
console.error(`删除任务失败:${sessionId}:`, err);
|
||||||
this.isSubmitting.set(false);
|
this.isSubmitting.set(false);
|
||||||
this.errorMessage.set(err.error?.detail || 'Failed to delete task.');
|
this.errorMessage.set(err.error?.detail || "删除任务失败。");
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|||||||
+39
-37
@@ -37,7 +37,7 @@ limitations under the License.
|
|||||||
{{ agentService.recordingPlaybackStatus() === 'live' ? 'sensors' : (agentService.playerMode() === 'steps' ? 'slideshow' : (agentService.recordingPlaybackStatus() === 'processing' ? 'progress_activity' : 'smart_display')) }}
|
{{ agentService.recordingPlaybackStatus() === 'live' ? 'sensors' : (agentService.playerMode() === 'steps' ? 'slideshow' : (agentService.recordingPlaybackStatus() === 'processing' ? 'progress_activity' : 'smart_display')) }}
|
||||||
</span>
|
</span>
|
||||||
<span class="header-title" [title]="agentService.activeVideoTitle()">
|
<span class="header-title" [title]="agentService.activeVideoTitle()">
|
||||||
{{ agentService.recordingPlaybackStatus() === 'live' ? 'Live Device Screen' : (agentService.playerMode() === 'steps' ? ('Step Replay (' + (totalStepFrames() > 0 ? (activeStepIndex() + 1) + '/' + totalStepFrames() : '0') + ')') : (agentService.recordingPlaybackStatus() === 'processing' ? 'Preparing Recording' : (agentService.activeVideoTitle() || 'Screen Recording'))) }}
|
{{ agentService.recordingPlaybackStatus() === 'live' ? '设备实时画面' : (agentService.playerMode() === 'steps' ? ('步骤回放(' + (totalStepFrames() > 0 ? (activeStepIndex() + 1) + '/' + totalStepFrames() : '0') + ')') : (agentService.recordingPlaybackStatus() === 'processing' ? '正在准备录像' : (agentService.activeVideoTitle() || '屏幕录像'))) }}
|
||||||
</span>
|
</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="header-actions" (mousedown)="$event.stopPropagation()">
|
<div class="header-actions" (mousedown)="$event.stopPropagation()">
|
||||||
@@ -48,19 +48,19 @@ limitations under the License.
|
|||||||
class="mode-btn"
|
class="mode-btn"
|
||||||
[class.active]="agentService.playerMode() === 'video'"
|
[class.active]="agentService.playerMode() === 'video'"
|
||||||
(click)="agentService.setPlayerMode('video')"
|
(click)="agentService.setPlayerMode('video')"
|
||||||
title="Video Recording"
|
title="视频录制"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined mode-icon">videocam</span>
|
<span class="material-symbols-outlined mode-icon">videocam</span>
|
||||||
<span>Video</span>
|
<span>视频</span>
|
||||||
</button>
|
</button>
|
||||||
<button
|
<button
|
||||||
class="mode-btn"
|
class="mode-btn"
|
||||||
[class.active]="agentService.playerMode() === 'steps'"
|
[class.active]="agentService.playerMode() === 'steps'"
|
||||||
(click)="agentService.setPlayerMode('steps')"
|
(click)="agentService.setPlayerMode('steps')"
|
||||||
title="Step Screenshot Replay"
|
title="步骤截图回放"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined mode-icon">slideshow</span>
|
<span class="material-symbols-outlined mode-icon">slideshow</span>
|
||||||
<span>Steps</span>
|
<span>步骤</span>
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -68,8 +68,8 @@ limitations under the License.
|
|||||||
<button
|
<button
|
||||||
class="header-btn"
|
class="header-btn"
|
||||||
(click)="toggleMinimize()"
|
(click)="toggleMinimize()"
|
||||||
[title]="agentService.isVideoMinimized() ? 'Restore' : 'Minimize'"
|
[title]="agentService.isVideoMinimized() ? 'Restore' : '最小化'"
|
||||||
aria-label="Minimize"
|
aria-label="最小化"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined">{{ agentService.isVideoMinimized() ? 'expand_less' : 'remove' }}</span>
|
<span class="material-symbols-outlined">{{ agentService.isVideoMinimized() ? 'expand_less' : 'remove' }}</span>
|
||||||
</button>
|
</button>
|
||||||
@@ -79,7 +79,7 @@ limitations under the License.
|
|||||||
class="header-btn"
|
class="header-btn"
|
||||||
(click)="toggleTheater()"
|
(click)="toggleTheater()"
|
||||||
[title]="isTheaterMode() ? 'Exit Theater' : 'Theater Mode'"
|
[title]="isTheaterMode() ? 'Exit Theater' : 'Theater Mode'"
|
||||||
aria-label="Theater"
|
aria-label="影院模式"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined">{{ isTheaterMode() ? 'close_fullscreen' : 'open_in_full' }}</span>
|
<span class="material-symbols-outlined">{{ isTheaterMode() ? 'close_fullscreen' : 'open_in_full' }}</span>
|
||||||
</button>
|
</button>
|
||||||
@@ -88,8 +88,8 @@ limitations under the License.
|
|||||||
<button
|
<button
|
||||||
class="header-btn close-btn"
|
class="header-btn close-btn"
|
||||||
(click)="agentService.closeVideoPlayer()"
|
(click)="agentService.closeVideoPlayer()"
|
||||||
title="Close"
|
title="关闭"
|
||||||
aria-label="Close"
|
aria-label="关闭"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined">close</span>
|
<span class="material-symbols-outlined">close</span>
|
||||||
</button>
|
</button>
|
||||||
@@ -106,14 +106,14 @@ limitations under the License.
|
|||||||
class="live-screen-img"
|
class="live-screen-img"
|
||||||
[src]="liveStreamUrl()"
|
[src]="liveStreamUrl()"
|
||||||
(error)="onLiveStreamError()"
|
(error)="onLiveStreamError()"
|
||||||
alt="Live Device Screen"
|
alt="设备实时画面"
|
||||||
/>
|
/>
|
||||||
<div class="live-status-overlay">
|
<div class="live-status-overlay">
|
||||||
<div class="live-badge">
|
<div class="live-badge">
|
||||||
<span class="pulse-dot"></span>
|
<span class="pulse-dot"></span>
|
||||||
<span>LIVE</span>
|
<span>实时</span>
|
||||||
</div>
|
</div>
|
||||||
<button class="live-refresh-btn" (click)="retryLiveStream()" title="Refresh Live Stream">
|
<button class="live-refresh-btn" (click)="retryLiveStream()" title="刷新实时画面">
|
||||||
<span class="material-symbols-outlined">refresh</span>
|
<span class="material-symbols-outlined">refresh</span>
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
@@ -124,10 +124,10 @@ limitations under the License.
|
|||||||
<div class="fallback-icon-wrap">
|
<div class="fallback-icon-wrap">
|
||||||
<span class="material-symbols-outlined fallback-icon">phonelink_off</span>
|
<span class="material-symbols-outlined fallback-icon">phonelink_off</span>
|
||||||
</div>
|
</div>
|
||||||
<h4 class="fallback-title">Device Stream Connecting</h4>
|
<h4 class="fallback-title">正在连接设备画面</h4>
|
||||||
<p class="fallback-desc">Waiting for ADB device screen stream...</p>
|
<p class="fallback-desc">正在等待 ADB 设备画面…</p>
|
||||||
<button class="retry-btn" (click)="retryLiveStream()">
|
<button class="retry-btn" (click)="retryLiveStream()">
|
||||||
<span class="material-symbols-outlined">refresh</span> Reconnect
|
<span class="material-symbols-outlined">refresh</span> 重新连接
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -138,7 +138,7 @@ limitations under the License.
|
|||||||
@if (agentService.playerMode() === 'video' && (agentService.recordingPlaybackStatus() === 'processing' || agentService.isVideoLoading())) {
|
@if (agentService.playerMode() === 'video' && (agentService.recordingPlaybackStatus() === 'processing' || agentService.isVideoLoading())) {
|
||||||
<div class="video-loading-state">
|
<div class="video-loading-state">
|
||||||
<span class="material-symbols-outlined spinning-icon">progress_activity</span>
|
<span class="material-symbols-outlined spinning-icon">progress_activity</span>
|
||||||
<span>{{ agentService.recordingPlaybackMessage() || 'Preparing screen recording...' }}</span>
|
<span>{{ agentService.recordingPlaybackMessage() || '正在准备屏幕录像…' }}</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
<!-- Option A: STEP REPLAY VIEW (ACTIVE IN STEP MODE) -->
|
<!-- Option A: STEP REPLAY VIEW (ACTIVE IN STEP MODE) -->
|
||||||
@@ -148,7 +148,7 @@ limitations under the License.
|
|||||||
@if (agentService.recordingPlaybackStatus() === 'failed' || videoLoadError()) {
|
@if (agentService.recordingPlaybackStatus() === 'failed' || videoLoadError()) {
|
||||||
<div class="step-replay-notice">
|
<div class="step-replay-notice">
|
||||||
<span class="material-symbols-outlined notice-icon">info</span>
|
<span class="material-symbols-outlined notice-icon">info</span>
|
||||||
<span>Screen recording unavailable. Replaying step-by-step screenshots.</span>
|
<span>录像不可用,正在逐步回放截图。</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
<!-- Step Image Viewport with Action Coordinates Overlay -->
|
<!-- Step Image Viewport with Action Coordinates Overlay -->
|
||||||
@@ -161,14 +161,14 @@ limitations under the License.
|
|||||||
@if (isStepImageLoading()) {
|
@if (isStepImageLoading()) {
|
||||||
<div class="step-image-loading-state">
|
<div class="step-image-loading-state">
|
||||||
<span class="material-symbols-outlined spinning-icon">progress_activity</span>
|
<span class="material-symbols-outlined spinning-icon">progress_activity</span>
|
||||||
<span>Loading screenshot...</span>
|
<span>正在加载截图…</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
<!-- Error / Missing Image Fallback -->
|
<!-- Error / Missing Image Fallback -->
|
||||||
@if (stepImageError() && !isStepImageLoading()) {
|
@if (stepImageError() && !isStepImageLoading()) {
|
||||||
<div class="step-image-error-state">
|
<div class="step-image-error-state">
|
||||||
<span class="material-symbols-outlined broken-icon">broken_image</span>
|
<span class="material-symbols-outlined broken-icon">broken_image</span>
|
||||||
<span>Screenshot preview unavailable for this step</span>
|
<span>此步骤暂无可用截图</span>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
<!-- Device Screen Frame & Action Coordinates Overlay -->
|
<!-- Device Screen Frame & Action Coordinates Overlay -->
|
||||||
@@ -182,7 +182,7 @@ limitations under the License.
|
|||||||
(load)="onStepImageLoad()"
|
(load)="onStepImageLoad()"
|
||||||
(error)="onStepImageError()"
|
(error)="onStepImageError()"
|
||||||
(click)="openImageInNewTab()"
|
(click)="openImageInNewTab()"
|
||||||
title="Click to view image in new tab"
|
title="点击在新标签页查看图片"
|
||||||
/>
|
/>
|
||||||
<!-- Action Visual Coordinates Overlay (Touch point, swipe arrow, sequence path) -->
|
<!-- Action Visual Coordinates Overlay (Touch point, swipe arrow, sequence path) -->
|
||||||
<div
|
<div
|
||||||
@@ -200,7 +200,7 @@ limitations under the License.
|
|||||||
(click)="$event.stopPropagation()"
|
(click)="$event.stopPropagation()"
|
||||||
>
|
>
|
||||||
<div class="badge-content-left">
|
<div class="badge-content-left">
|
||||||
<span class="badge-step-no">Step {{ frame.stepNumber }}</span>
|
<span class="badge-step-no">步骤 {{ frame.stepNumber }}</span>
|
||||||
<span class="badge-action-desc" [title]="frame.actionText || frame.title">
|
<span class="badge-action-desc" [title]="frame.actionText || frame.title">
|
||||||
@if (frame.action) {
|
@if (frame.action) {
|
||||||
<span class="material-symbols-outlined action-type-icon">
|
<span class="material-symbols-outlined action-type-icon">
|
||||||
@@ -218,17 +218,19 @@ limitations under the License.
|
|||||||
class="toggle-btn"
|
class="toggle-btn"
|
||||||
[class.active]="viewMode() === 'pre'"
|
[class.active]="viewMode() === 'pre'"
|
||||||
(click)="setStepViewMode('pre', $event)"
|
(click)="setStepViewMode('pre', $event)"
|
||||||
title="View screen before action (with touch coordinates)"
|
title="查看操作前画面(含点击坐标)"
|
||||||
>
|
>
|
||||||
Pre
|
|
||||||
|
操作前
|
||||||
</button>
|
</button>
|
||||||
<button
|
<button
|
||||||
class="toggle-btn"
|
class="toggle-btn"
|
||||||
[class.active]="viewMode() === 'post'"
|
[class.active]="viewMode() === 'post'"
|
||||||
(click)="setStepViewMode('post', $event)"
|
(click)="setStepViewMode('post', $event)"
|
||||||
title="View resulting screen after action executed"
|
title="查看操作后的屏幕"
|
||||||
>
|
>
|
||||||
Post
|
|
||||||
|
操作后
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
@@ -269,7 +271,7 @@ limitations under the License.
|
|||||||
[class.passed]="i < activeStepIndex()"
|
[class.passed]="i < activeStepIndex()"
|
||||||
[style.left.%]="(i / (totalStepFrames() - 1)) * 100"
|
[style.left.%]="(i / (totalStepFrames() - 1)) * 100"
|
||||||
(click)="seekStepFrame(i); $event.stopPropagation()"
|
(click)="seekStepFrame(i); $event.stopPropagation()"
|
||||||
[title]="'Step ' + (i + 1) + ': ' + (f.actionText || f.title)"
|
[title]="'步骤 ' + (i + 1) + ': ' + (f.actionText || f.title)"
|
||||||
></span>
|
></span>
|
||||||
}
|
}
|
||||||
</div>
|
</div>
|
||||||
@@ -279,24 +281,24 @@ limitations under the License.
|
|||||||
<div class="controls-row">
|
<div class="controls-row">
|
||||||
<div class="controls-left">
|
<div class="controls-left">
|
||||||
<!-- Play / Pause Button -->
|
<!-- Play / Pause Button -->
|
||||||
<button class="ctrl-btn play-btn" (click)="toggleStepPlay()" [title]="isStepPlaying() ? 'Pause' : 'Play Slideshow'">
|
<button class="ctrl-btn play-btn" (click)="toggleStepPlay()" [title]="isStepPlaying() ? '暂停' : '播放截图'">
|
||||||
<span class="material-symbols-outlined">{{ isStepPlaying() ? 'pause' : 'play_arrow' }}</span>
|
<span class="material-symbols-outlined">{{ isStepPlaying() ? 'pause' : 'play_arrow' }}</span>
|
||||||
</button>
|
</button>
|
||||||
<!-- Previous Frame Button -->
|
<!-- Previous Frame Button -->
|
||||||
<button class="ctrl-btn" (click)="prevStepFrame()" [disabled]="activeStepIndex() <= 0" title="Previous Step">
|
<button class="ctrl-btn" (click)="prevStepFrame()" [disabled]="activeStepIndex() <= 0" title="上一步">
|
||||||
<span class="material-symbols-outlined">skip_previous</span>
|
<span class="material-symbols-outlined">skip_previous</span>
|
||||||
</button>
|
</button>
|
||||||
<!-- Next Frame Button -->
|
<!-- Next Frame Button -->
|
||||||
<button class="ctrl-btn" (click)="nextStepFrame()" [disabled]="activeStepIndex() >= totalStepFrames() - 1" title="Next Step">
|
<button class="ctrl-btn" (click)="nextStepFrame()" [disabled]="activeStepIndex() >= totalStepFrames() - 1" title="下一步">
|
||||||
<span class="material-symbols-outlined">skip_next</span>
|
<span class="material-symbols-outlined">skip_next</span>
|
||||||
</button>
|
</button>
|
||||||
<!-- Step Replay / Reset -->
|
<!-- Step Replay / Reset -->
|
||||||
<button class="ctrl-btn" (click)="restartStep()" title="Replay from start">
|
<button class="ctrl-btn" (click)="restartStep()" title="从头回放">
|
||||||
<span class="material-symbols-outlined">replay</span>
|
<span class="material-symbols-outlined">replay</span>
|
||||||
</button>
|
</button>
|
||||||
<!-- Step Counter Display -->
|
<!-- Step Counter Display -->
|
||||||
<span class="time-display">
|
<span class="time-display">
|
||||||
Step {{ activeStepIndex() + 1 }} / {{ totalStepFrames() }}
|
步骤 {{ activeStepIndex() + 1 }} / {{ totalStepFrames() }}
|
||||||
</span>
|
</span>
|
||||||
</div>
|
</div>
|
||||||
<div class="controls-right">
|
<div class="controls-right">
|
||||||
@@ -318,7 +320,7 @@ limitations under the License.
|
|||||||
<span class="material-symbols-outlined">repeat</span>
|
<span class="material-symbols-outlined">repeat</span>
|
||||||
</button>
|
</button>
|
||||||
<!-- Open in new tab -->
|
<!-- Open in new tab -->
|
||||||
<button class="ctrl-btn" (click)="openImageInNewTab()" title="Open image in new tab">
|
<button class="ctrl-btn" (click)="openImageInNewTab()" title="在新标签页打开图片">
|
||||||
<span class="material-symbols-outlined">open_in_new</span>
|
<span class="material-symbols-outlined">open_in_new</span>
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
@@ -364,11 +366,11 @@ limitations under the License.
|
|||||||
<div class="controls-row">
|
<div class="controls-row">
|
||||||
<div class="controls-left">
|
<div class="controls-left">
|
||||||
<!-- Play / Pause Button -->
|
<!-- Play / Pause Button -->
|
||||||
<button class="ctrl-btn play-btn" (click)="togglePlay()" [title]="isPlaying() ? 'Pause' : 'Play'">
|
<button class="ctrl-btn play-btn" (click)="togglePlay()" [title]="isPlaying() ? '暂停' : '播放'">
|
||||||
<span class="material-symbols-outlined">{{ isPlaying() ? 'pause' : 'play_arrow' }}</span>
|
<span class="material-symbols-outlined">{{ isPlaying() ? 'pause' : 'play_arrow' }}</span>
|
||||||
</button>
|
</button>
|
||||||
<!-- Replay / Reset -->
|
<!-- Replay / Reset -->
|
||||||
<button class="ctrl-btn" (click)="restart()" title="Replay from start">
|
<button class="ctrl-btn" (click)="restart()" title="从头回放">
|
||||||
<span class="material-symbols-outlined">replay</span>
|
<span class="material-symbols-outlined">replay</span>
|
||||||
</button>
|
</button>
|
||||||
<!-- Time Display -->
|
<!-- Time Display -->
|
||||||
@@ -399,7 +401,7 @@ limitations under the License.
|
|||||||
<span class="material-symbols-outlined">{{ isMuted() ? 'volume_off' : 'volume_up' }}</span>
|
<span class="material-symbols-outlined">{{ isMuted() ? 'volume_off' : 'volume_up' }}</span>
|
||||||
</button>
|
</button>
|
||||||
<!-- Open in new tab -->
|
<!-- Open in new tab -->
|
||||||
<button class="ctrl-btn" (click)="openInNewTab()" title="Open video in new tab">
|
<button class="ctrl-btn" (click)="openInNewTab()" title="在新标签页打开视频">
|
||||||
<span class="material-symbols-outlined">open_in_new</span>
|
<span class="material-symbols-outlined">open_in_new</span>
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
@@ -421,7 +423,7 @@ limitations under the License.
|
|||||||
</p>
|
</p>
|
||||||
@if (agentService.recordingPlaybackStatus() === 'failed' || videoLoadError()) {
|
@if (agentService.recordingPlaybackStatus() === 'failed' || videoLoadError()) {
|
||||||
<button class="retry-btn" (click)="videoLoadError.set(false); agentService.retryVideoRecording()">
|
<button class="retry-btn" (click)="videoLoadError.set(false); agentService.retryVideoRecording()">
|
||||||
<span class="material-symbols-outlined">refresh</span> Retry
|
<span class="material-symbols-outlined">refresh</span> 重试
|
||||||
</button>
|
</button>
|
||||||
}
|
}
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
+13
-13
@@ -18,19 +18,19 @@ limitations under the License.
|
|||||||
<!-- Header with Session Selector -->
|
<!-- Header with Session Selector -->
|
||||||
<header class="stream-header">
|
<header class="stream-header">
|
||||||
<div class="header-title">
|
<div class="header-title">
|
||||||
<h2>👣 Real-time Agent Stream (Legacy Card View)</h2>
|
<h2>👣 智能体实时记录(经典卡片视图)</h2>
|
||||||
<p class="subtitle">Select a session to monitor raw agent activity</p>
|
<p class="subtitle">选择会话,查看智能体执行记录</p>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<div class="session-selector-wrapper">
|
<div class="session-selector-wrapper">
|
||||||
<label for="session-select">Active Session:</label>
|
<label for="session-select">当前会话:</label>
|
||||||
<select
|
<select
|
||||||
id="session-select"
|
id="session-select"
|
||||||
[value]="agentService.currentSessionId() || ''"
|
[value]="agentService.currentSessionId() || ''"
|
||||||
(change)="onSessionChange($event)"
|
(change)="onSessionChange($event)"
|
||||||
class="session-dropdown"
|
class="session-dropdown"
|
||||||
>
|
>
|
||||||
<option value="" disabled selected>-- Select a Session --</option>
|
<option value="" disabled selected>-- 选择会话 --</option>
|
||||||
@for (session of agentService.sessions(); track session) {
|
@for (session of agentService.sessions(); track session) {
|
||||||
<option
|
<option
|
||||||
[value]="session.session_id"
|
[value]="session.session_id"
|
||||||
@@ -39,7 +39,7 @@ limitations under the License.
|
|||||||
</option>
|
</option>
|
||||||
}
|
}
|
||||||
</select>
|
</select>
|
||||||
<button class="btn-refresh" (click)="agentService.fetchSessions()">🔄 Refresh</button>
|
<button class="btn-refresh" (click)="agentService.fetchSessions()">🔄 刷新</button>
|
||||||
</div>
|
</div>
|
||||||
</header>
|
</header>
|
||||||
|
|
||||||
@@ -48,8 +48,8 @@ limitations under the License.
|
|||||||
@if (agentService.sessionLogs().length === 0) {
|
@if (agentService.sessionLogs().length === 0) {
|
||||||
<div class="empty-state">
|
<div class="empty-state">
|
||||||
<div class="empty-icon">📡</div>
|
<div class="empty-icon">📡</div>
|
||||||
<h3>No Stream Activity Selected</h3>
|
<h3>尚未选择执行记录</h3>
|
||||||
<p>Please select a running session from the dropdown above to view real-time log payloads.</p>
|
<p>请在上方下拉列表选择正在运行的会话,查看实时日志。</p>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -73,28 +73,28 @@ limitations under the License.
|
|||||||
<!-- Step Summary (Message to User) -->
|
<!-- Step Summary (Message to User) -->
|
||||||
@if (log.data.summary) {
|
@if (log.data.summary) {
|
||||||
<div class="step-summary-section">
|
<div class="step-summary-section">
|
||||||
<div class="section-title">💬 Message to User / Summary:</div>
|
<div class="section-title">💬 用户消息 / 摘要:</div>
|
||||||
<div class="summary-bubble">{{ log.data.summary }}</div>
|
<div class="summary-bubble">{{ log.data.summary }}</div>
|
||||||
</div>
|
</div>
|
||||||
}
|
}
|
||||||
<!-- Model thinking process (collapsible) -->
|
<!-- Model thinking process (collapsible) -->
|
||||||
@if (log.data.operator_native_thinking) {
|
@if (log.data.operator_native_thinking) {
|
||||||
<details class="thinking-details">
|
<details class="thinking-details">
|
||||||
<summary class="details-summary">🧠 Model Thinking Process (Click to expand)</summary>
|
<summary class="details-summary">🧠 模型思考过程(点击展开)</summary>
|
||||||
<div class="details-content thinking-text">{{ log.data.operator_native_thinking }}</div>
|
<div class="details-content thinking-text">{{ log.data.operator_native_thinking }}</div>
|
||||||
</details>
|
</details>
|
||||||
}
|
}
|
||||||
<!-- Raw thinking process (collapsible) -->
|
<!-- Raw thinking process (collapsible) -->
|
||||||
@if (log.data.operator_raw_thinking) {
|
@if (log.data.operator_raw_thinking) {
|
||||||
<details class="planning-details">
|
<details class="planning-details">
|
||||||
<summary class="details-summary">📝 Raw Plan / Thoughts (Click to expand)</summary>
|
<summary class="details-summary">📝 原始计划 / 思考(点击展开)</summary>
|
||||||
<div class="details-content planning-text">{{ log.data.operator_raw_thinking }}</div>
|
<div class="details-content planning-text">{{ log.data.operator_raw_thinking }}</div>
|
||||||
</details>
|
</details>
|
||||||
}
|
}
|
||||||
<!-- Tool Call (collapsible) -->
|
<!-- Tool Call (collapsible) -->
|
||||||
@if (log.data.action_taken) {
|
@if (log.data.action_taken) {
|
||||||
<details class="tool-details" [open]="isShortContent(log.data.action_taken)">
|
<details class="tool-details" [open]="isShortContent(log.data.action_taken)">
|
||||||
<summary class="details-summary">🛠️ Tool Call Details (Click to expand)</summary>
|
<summary class="details-summary">🛠️ 工具调用详情(点击展开)</summary>
|
||||||
<div class="details-content">
|
<div class="details-content">
|
||||||
<pre class="json-code"><code>{{ formatJson(log.data.action_taken) }}</code></pre>
|
<pre class="json-code"><code>{{ formatJson(log.data.action_taken) }}</code></pre>
|
||||||
</div>
|
</div>
|
||||||
@@ -103,7 +103,7 @@ limitations under the License.
|
|||||||
<!-- Tool Call Result (collapsible) -->
|
<!-- Tool Call Result (collapsible) -->
|
||||||
@if (log.data.last_execution_result) {
|
@if (log.data.last_execution_result) {
|
||||||
<details class="result-details">
|
<details class="result-details">
|
||||||
<summary class="details-summary">✓ Execution Result (Click to expand)</summary>
|
<summary class="details-summary">✓ 执行结果(点击展开)</summary>
|
||||||
<div class="details-content">
|
<div class="details-content">
|
||||||
<pre class="json-code"><code>{{ formatJson(log.data.last_execution_result) }}</code></pre>
|
<pre class="json-code"><code>{{ formatJson(log.data.last_execution_result) }}</code></pre>
|
||||||
</div>
|
</div>
|
||||||
@@ -115,7 +115,7 @@ limitations under the License.
|
|||||||
@if (!isStepEvent(log.type)) {
|
@if (!isStepEvent(log.type)) {
|
||||||
<div class="log-card-body raw-body">
|
<div class="log-card-body raw-body">
|
||||||
<details class="raw-details" [open]="isShortContent(log.data)">
|
<details class="raw-details" [open]="isShortContent(log.data)">
|
||||||
<summary class="details-summary">📄 Raw Payload Data (Click to expand)</summary>
|
<summary class="details-summary">📄 原始数据(点击展开)</summary>
|
||||||
<div class="details-content">
|
<div class="details-content">
|
||||||
<pre class="json-code"><code>{{ formatJson(log.data) }}</code></pre>
|
<pre class="json-code"><code>{{ formatJson(log.data) }}</code></pre>
|
||||||
</div>
|
</div>
|
||||||
|
|||||||
@@ -0,0 +1,54 @@
|
|||||||
|
import { Component, inject, OnInit, signal, output } from '@angular/core';
|
||||||
|
import { FormsModule } from '@angular/forms';
|
||||||
|
import { HttpClient } from '@angular/common/http';
|
||||||
|
import { finalize } from 'rxjs';
|
||||||
|
|
||||||
|
interface Capabilities {
|
||||||
|
video_enabled: boolean;
|
||||||
|
outputter_enabled: boolean;
|
||||||
|
step_summarizer_enabled: boolean;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Component({
|
||||||
|
selector: 'app-capability-settings', standalone: true, imports: [FormsModule],
|
||||||
|
template: `
|
||||||
|
<section aria-label="功能开关">
|
||||||
|
<h3>功能开关</h3>
|
||||||
|
<p>按需启用,保存后对新任务生效。</p>
|
||||||
|
<label><input type="checkbox" role="switch" [(ngModel)]="config.video_enabled" [disabled]="busy() || loading() || !loaded()" />
|
||||||
|
<span>视频分析与录屏<small>录制设备画面,并允许智能体分析录像;关闭可减少资源占用。</small></span></label>
|
||||||
|
<label><input type="checkbox" role="switch" [(ngModel)]="config.outputter_enabled" [disabled]="busy() || loading() || !loaded()" />
|
||||||
|
<span>最终报告生成<small>根据执行记录生成任务结果报告。</small></span></label>
|
||||||
|
<label><input type="checkbox" role="switch" [(ngModel)]="config.step_summarizer_enabled" [disabled]="busy() || loading() || !loaded()" />
|
||||||
|
<span>步骤摘要压缩<small>Flash 模式下在后台生成摘要,减少长任务的上下文占用。</small></span></label>
|
||||||
|
<button type="button" (click)="save()" [disabled]="busy() || loading() || !loaded()">{{ busy() ? '正在保存…' : '保存功能设置' }}</button>
|
||||||
|
@if (message()) { <p role="status">{{ message() }}</p> }
|
||||||
|
@if (error()) { <p role="alert" class="error">{{ error() }}</p> }
|
||||||
|
</section>
|
||||||
|
`,
|
||||||
|
styles: [`section { margin-top: 24px; padding-top: 20px; border-top: 1px solid #64748b55; }
|
||||||
|
h3 { margin: 0 0 8px; } p, small { color: #64748b; line-height: 1.6; }
|
||||||
|
label { display: flex; gap: 12px; align-items: flex-start; padding: 14px 0; cursor: pointer; }
|
||||||
|
input { width: 20px; height: 20px; accent-color: #0284c7; flex-shrink: 0; }
|
||||||
|
small { display: block; margin-top: 4px; } button { padding: 10px 18px; border-radius: 10px; border: 1px solid #38bdf877; background: #0284c7; color: white; font: inherit; cursor: pointer; }
|
||||||
|
button:disabled { opacity: .5; cursor: default; } .error { color: #be123c; }`]
|
||||||
|
})
|
||||||
|
export class CapabilitySettingsComponent implements OnInit {
|
||||||
|
changed = output<Capabilities>();
|
||||||
|
private http = inject(HttpClient);
|
||||||
|
config: Capabilities = { video_enabled: true, outputter_enabled: true, step_summarizer_enabled: true };
|
||||||
|
busy = signal(false); loading = signal(true); loaded = signal(false); message = signal(''); error = signal('');
|
||||||
|
ngOnInit(): void {
|
||||||
|
this.http.get<Capabilities>('/api/system/capability-settings').pipe(finalize(() => this.loading.set(false))).subscribe({
|
||||||
|
next: config => { this.config = config; this.loaded.set(true); this.changed.emit(config); }, error: () => this.error.set('功能配置加载失败,请刷新页面重试。')
|
||||||
|
});
|
||||||
|
}
|
||||||
|
save(): void {
|
||||||
|
if (this.busy() || this.loading() || !this.loaded()) return;
|
||||||
|
this.busy.set(true); this.message.set(''); this.error.set('');
|
||||||
|
this.http.post<Capabilities>('/api/system/capability-settings', this.config).pipe(finalize(() => this.busy.set(false))).subscribe({
|
||||||
|
next: config => { this.config = config; this.changed.emit(config); this.message.set('功能设置已保存,新任务将使用此配置。'); },
|
||||||
|
error: () => this.error.set('保存失败,请检查服务器连接和 .env 文件权限。')
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,73 @@
|
|||||||
|
import { TestBed } from '@angular/core/testing';
|
||||||
|
import { provideHttpClient } from '@angular/common/http';
|
||||||
|
import { HttpTestingController, provideHttpClientTesting } from '@angular/common/http/testing';
|
||||||
|
import { ModelSettingsComponent } from './model-settings.component';
|
||||||
|
import { CapabilitySettingsComponent } from './capability-settings.component';
|
||||||
|
|
||||||
|
describe('持久化模型与功能设置', () => {
|
||||||
|
let http: HttpTestingController;
|
||||||
|
const config = { base_url: 'https://example.test/v1', model: 'mimo-v2.6-flash',
|
||||||
|
api_key_configured: true, timeout_seconds: 120, max_retries: 0, env_path: '/app/.env' };
|
||||||
|
beforeEach(() => {
|
||||||
|
TestBed.configureTestingModule({ providers: [provideHttpClient(), provideHttpClientTesting()] });
|
||||||
|
http = TestBed.inject(HttpTestingController);
|
||||||
|
});
|
||||||
|
afterEach(() => http.verify());
|
||||||
|
|
||||||
|
it('加载配置后保留密钥,保存新模型且不发送空密钥', () => {
|
||||||
|
const fixture = TestBed.createComponent(ModelSettingsComponent);
|
||||||
|
fixture.detectChanges();
|
||||||
|
http.expectOne('/api/system/model-settings').flush(config);
|
||||||
|
const component = fixture.componentInstance;
|
||||||
|
expect(component.apiKey).toBe('');
|
||||||
|
expect(component.keyConfigured()).toBeTrue();
|
||||||
|
component.model = 'another-model';
|
||||||
|
component.save();
|
||||||
|
const request = http.expectOne('/api/system/model-settings');
|
||||||
|
expect(request.request.method).toBe('POST');
|
||||||
|
expect(request.request.body.model).toBe('another-model');
|
||||||
|
expect(request.request.body.api_key).toBeUndefined();
|
||||||
|
request.flush({ ...config, model: 'another-model' });
|
||||||
|
expect(component.busy()).toBeNull();
|
||||||
|
expect(component.message()).toContain('保存');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('连接测试使用当前输入而不写入保存接口', () => {
|
||||||
|
const fixture = TestBed.createComponent(ModelSettingsComponent);
|
||||||
|
fixture.detectChanges();
|
||||||
|
http.expectOne('/api/system/model-settings').flush(config);
|
||||||
|
fixture.componentInstance.test();
|
||||||
|
const request = http.expectOne('/api/system/model-settings/test');
|
||||||
|
expect(request.request.body.model).toBe('mimo-v2.6-flash');
|
||||||
|
request.flush({ message: 'Responses 连接测试成功' });
|
||||||
|
http.expectNone('/api/system/model-settings');
|
||||||
|
expect(fixture.componentInstance.message()).toContain('成功');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('加载失败时保持功能开关不可保存,避免覆盖现有设置', () => {
|
||||||
|
const fixture = TestBed.createComponent(CapabilitySettingsComponent);
|
||||||
|
fixture.detectChanges();
|
||||||
|
http.expectOne('/api/system/capability-settings').flush({}, { status: 500, statusText: 'Error' });
|
||||||
|
expect(fixture.componentInstance.error()).toContain('加载失败');
|
||||||
|
fixture.componentInstance.save();
|
||||||
|
http.expectNone('/api/system/capability-settings');
|
||||||
|
});
|
||||||
|
|
||||||
|
it('保存关闭视频分析的配置,重新打开时恢复关闭状态', () => {
|
||||||
|
const capabilities = { video_enabled: true, outputter_enabled: true, step_summarizer_enabled: false };
|
||||||
|
const fixture = TestBed.createComponent(CapabilitySettingsComponent);
|
||||||
|
fixture.detectChanges();
|
||||||
|
http.expectOne('/api/system/capability-settings').flush(capabilities);
|
||||||
|
fixture.componentInstance.config.video_enabled = false;
|
||||||
|
fixture.componentInstance.save();
|
||||||
|
const request = http.expectOne('/api/system/capability-settings');
|
||||||
|
expect(request.request.body.video_enabled).toBeFalse();
|
||||||
|
const saved = { ...capabilities, video_enabled: false };
|
||||||
|
request.flush(saved);
|
||||||
|
fixture.destroy();
|
||||||
|
const reopened = TestBed.createComponent(CapabilitySettingsComponent);
|
||||||
|
reopened.detectChanges();
|
||||||
|
http.expectOne('/api/system/capability-settings').flush(saved);
|
||||||
|
expect(reopened.componentInstance.config.video_enabled).toBeFalse();
|
||||||
|
});
|
||||||
|
});
|
||||||
@@ -0,0 +1,108 @@
|
|||||||
|
import { Component, OnInit, inject, signal, output } from '@angular/core';
|
||||||
|
import { FormsModule } from '@angular/forms';
|
||||||
|
import { HttpClient } from '@angular/common/http';
|
||||||
|
import { finalize } from 'rxjs';
|
||||||
|
|
||||||
|
interface ModelSettings {
|
||||||
|
base_url: string;
|
||||||
|
model: string;
|
||||||
|
api_key_configured: boolean;
|
||||||
|
timeout_seconds: number;
|
||||||
|
max_retries: number;
|
||||||
|
env_path: string;
|
||||||
|
message?: string;
|
||||||
|
}
|
||||||
|
|
||||||
|
@Component({
|
||||||
|
selector: 'app-model-settings',
|
||||||
|
standalone: true,
|
||||||
|
imports: [FormsModule],
|
||||||
|
template: `
|
||||||
|
<form class="model-settings" (ngSubmit)="save()">
|
||||||
|
<p>统一使用 OpenAI Responses。保存后新任务立即生效,重启后自动加载。</p>
|
||||||
|
<label for="model-base-url">接口地址</label>
|
||||||
|
<input id="model-base-url" name="baseUrl" type="url" [(ngModel)]="baseUrl" required
|
||||||
|
placeholder="https://api.openai.com/v1" [disabled]="busy() || loading()" />
|
||||||
|
<small>填写接口根地址,通常包含 /v1,无需添加 /responses。</small>
|
||||||
|
<label for="model-name">模型名称</label>
|
||||||
|
<input id="model-name" name="model" [(ngModel)]="model" required placeholder="输入接口支持的模型名"
|
||||||
|
[disabled]="busy() || loading()" />
|
||||||
|
<label for="model-key">API Key <span>{{ keyConfigured() ? '(已配置,留空保留)' : '(尚未配置)' }}</span></label>
|
||||||
|
<input id="model-key" name="apiKey" type="password" [(ngModel)]="apiKey" autocomplete="new-password"
|
||||||
|
placeholder="输入新密钥" [disabled]="busy() || loading()" />
|
||||||
|
<div class="number-fields">
|
||||||
|
<label>请求超时(秒)<input name="timeout" type="number" min="1" max="3600" required
|
||||||
|
[(ngModel)]="timeout" [disabled]="busy() || loading()" /></label>
|
||||||
|
<label>接口重试次数<input name="retries" type="number" min="0" max="10" required
|
||||||
|
[(ngModel)]="retries" [disabled]="busy() || loading()" /></label>
|
||||||
|
</div>
|
||||||
|
<div class="actions">
|
||||||
|
<button type="submit" [disabled]="busy() || loading() || !valid()">{{ busy() === 'save' ? '正在保存…' : '保存配置' }}</button>
|
||||||
|
<button type="button" (click)="test()" [disabled]="busy() || loading() || !valid()">
|
||||||
|
{{ busy() === 'test' ? '正在测试…' : '测试连接' }}
|
||||||
|
</button>
|
||||||
|
</div>
|
||||||
|
@if (loading()) { <p role="status">正在加载配置…</p> }
|
||||||
|
@if (message()) { <p role="status" class="success">{{ message() }}</p> }
|
||||||
|
@if (error()) { <p role="alert" class="error">{{ error() }}</p> }
|
||||||
|
@if (envPath()) { <small>保存位置:{{ envPath() }}</small> }
|
||||||
|
</form>
|
||||||
|
`,
|
||||||
|
styles: [`
|
||||||
|
.model-settings { display: grid; gap: 10px; color: #1e293b; }
|
||||||
|
p, small { margin: 0; line-height: 1.6; color: #64748b; }
|
||||||
|
label { font-weight: 600; } label span { font-weight: 400; font-size: 12px; }
|
||||||
|
input { box-sizing: border-box; width: 100%; padding: 12px; border-radius: 10px; border: 1px solid #64748b66;
|
||||||
|
background: #f8fafc; color: inherit; font: inherit; }
|
||||||
|
input:focus { outline: 2px solid #38bdf8; }
|
||||||
|
.number-fields { display: grid; grid-template-columns: 1fr 1fr; gap: 16px; margin-top: 8px; }
|
||||||
|
.number-fields input { margin-top: 8px; }
|
||||||
|
.actions { display: flex; gap: 12px; margin-top: 8px; }
|
||||||
|
button { padding: 11px 20px; border-radius: 10px; border: 1px solid #38bdf877; background: #0284c7; color: white; cursor: pointer; font: inherit; }
|
||||||
|
button:disabled { opacity: .5; cursor: default; } .success { color: #047857; } .error { color: #be123c; }
|
||||||
|
small { overflow-wrap: anywhere; } @media(max-width: 480px) { .number-fields { grid-template-columns: 1fr; } }
|
||||||
|
`]
|
||||||
|
})
|
||||||
|
export class ModelSettingsComponent implements OnInit {
|
||||||
|
private http = inject(HttpClient);
|
||||||
|
saved = output<void>();
|
||||||
|
baseUrl = ''; model = ''; apiKey = ''; timeout = 120; retries = 0;
|
||||||
|
busy = signal<'save' | 'test' | null>(null);
|
||||||
|
loading = signal(true); keyConfigured = signal(false); envPath = signal('');
|
||||||
|
message = signal(''); error = signal('');
|
||||||
|
|
||||||
|
ngOnInit(): void {
|
||||||
|
this.http.get<ModelSettings>('/api/system/model-settings').pipe(finalize(() => this.loading.set(false))).subscribe({
|
||||||
|
next: config => this.apply(config),
|
||||||
|
error: () => this.error.set('配置加载失败,请刷新页面重试。')
|
||||||
|
});
|
||||||
|
}
|
||||||
|
valid(): boolean {
|
||||||
|
return !!this.baseUrl.trim() && !!this.model.trim() && (!!this.apiKey.trim() || this.keyConfigured())
|
||||||
|
&& this.timeout > 0 && this.timeout <= 3600 && Number.isInteger(this.retries) && this.retries >= 0 && this.retries <= 10;
|
||||||
|
}
|
||||||
|
private payload() {
|
||||||
|
return { base_url: this.baseUrl.trim(), model: this.model.trim(), api_key: this.apiKey.trim() || undefined,
|
||||||
|
timeout_seconds: this.timeout, max_retries: this.retries };
|
||||||
|
}
|
||||||
|
private apply(config: ModelSettings): void {
|
||||||
|
this.baseUrl = config.base_url; this.model = config.model; this.timeout = config.timeout_seconds;
|
||||||
|
this.retries = config.max_retries; this.keyConfigured.set(config.api_key_configured); this.envPath.set(config.env_path);
|
||||||
|
}
|
||||||
|
save(): void {
|
||||||
|
if (this.busy() || !this.valid()) return;
|
||||||
|
this.busy.set('save'); this.message.set(''); this.error.set('');
|
||||||
|
this.http.post<ModelSettings>('/api/system/model-settings', this.payload()).pipe(finalize(() => this.busy.set(null))).subscribe({
|
||||||
|
next: config => { this.apply(config); this.apiKey = ''; this.message.set(config.message || '配置已保存。'); this.saved.emit(); },
|
||||||
|
error: err => this.error.set(typeof err.error?.detail === 'string' ? err.error.detail : '保存失败,请检查配置内容。')
|
||||||
|
});
|
||||||
|
}
|
||||||
|
test(): void {
|
||||||
|
if (this.busy() || !this.valid()) return;
|
||||||
|
this.busy.set('test'); this.message.set(''); this.error.set('');
|
||||||
|
this.http.post<{ message: string }>('/api/system/model-settings/test', this.payload()).pipe(finalize(() => this.busy.set(null))).subscribe({
|
||||||
|
next: result => this.message.set(result.message),
|
||||||
|
error: err => this.error.set(typeof err.error?.detail === 'string' ? err.error.detail : '连接测试失败,请检查配置。')
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -23,25 +23,25 @@ import { RouterLink, RouterLinkActive } from '@angular/router';
|
|||||||
standalone: true,
|
standalone: true,
|
||||||
imports: [RouterLink, RouterLinkActive],
|
imports: [RouterLink, RouterLinkActive],
|
||||||
template: `
|
template: `
|
||||||
<nav class="floating-nav-switcher" aria-label="Main Navigation">
|
<nav class="floating-nav-switcher" aria-label="主导航">
|
||||||
<a
|
<a
|
||||||
routerLink="/"
|
routerLink="/"
|
||||||
routerLinkActive="active"
|
routerLinkActive="active"
|
||||||
[routerLinkActiveOptions]="{exact: true}"
|
[routerLinkActiveOptions]="{exact: true}"
|
||||||
class="nav-tab-btn"
|
class="nav-tab-btn"
|
||||||
title="Return to Home Launcher to start a new task"
|
title="返回首页开始新任务"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined tab-icon">add_task</span>
|
<span class="material-symbols-outlined tab-icon">add_task</span>
|
||||||
<span class="tab-label">New / Home</span>
|
<span class="tab-label">新任务 / 首页</span>
|
||||||
</a>
|
</a>
|
||||||
<a
|
<a
|
||||||
routerLink="/workspace"
|
routerLink="/workspace"
|
||||||
routerLinkActive="active"
|
routerLinkActive="active"
|
||||||
class="nav-tab-btn"
|
class="nav-tab-btn"
|
||||||
title="Open Workspace"
|
title="打开工作区"
|
||||||
>
|
>
|
||||||
<span class="material-symbols-outlined tab-icon">space_dashboard</span>
|
<span class="material-symbols-outlined tab-icon">space_dashboard</span>
|
||||||
<span class="tab-label">Workspace</span>
|
<span class="tab-label">工作区</span>
|
||||||
</a>
|
</a>
|
||||||
</nav>
|
</nav>
|
||||||
`,
|
`,
|
||||||
|
|||||||
@@ -47,29 +47,29 @@ export interface SmartSuggestion {
|
|||||||
*/
|
*/
|
||||||
export const APP_REGISTRY: Record<string, AppReference> = {
|
export const APP_REGISTRY: Record<string, AppReference> = {
|
||||||
// Google Suite & System
|
// Google Suite & System
|
||||||
'com.google.android.apps.maps': { name: 'Maps', icon: 'explore', pkg: 'com.google.android.apps.maps', category: 'navigation' },
|
'com.google.android.apps.maps': { name: "地图", icon: 'explore', pkg: 'com.google.android.apps.maps', category: 'navigation' },
|
||||||
'com.google.android.gm': { name: 'Gmail', icon: 'mail', pkg: 'com.google.android.gm', category: 'productivity' },
|
'com.google.android.gm': { name: 'Gmail', icon: 'mail', pkg: 'com.google.android.gm', category: 'productivity' },
|
||||||
'com.android.chrome': { name: 'Chrome', icon: 'public', pkg: 'com.android.chrome', category: 'browser' },
|
'com.android.chrome': { name: 'Chrome', icon: 'public', pkg: 'com.android.chrome', category: 'browser' },
|
||||||
'com.google.android.youtube': { name: 'YouTube', icon: 'smart_display', pkg: 'com.google.android.youtube', category: 'entertainment' },
|
'com.google.android.youtube': { name: 'YouTube', icon: 'smart_display', pkg: 'com.google.android.youtube', category: 'entertainment' },
|
||||||
'com.android.settings': { name: 'Settings', icon: 'settings', pkg: 'com.android.settings', category: 'system' },
|
'com.android.settings': { name: "设置", icon: 'settings', pkg: 'com.android.settings', category: 'system' },
|
||||||
'com.google.android.deskclock': { name: 'Clock', icon: 'timer', pkg: 'com.google.android.deskclock', category: 'utility' },
|
'com.google.android.deskclock': { name: "时钟", icon: 'timer', pkg: 'com.google.android.deskclock', category: 'utility' },
|
||||||
'com.android.deskclock': { name: 'Clock', icon: 'timer', pkg: 'com.android.deskclock', category: 'utility' },
|
'com.android.deskclock': { name: "时钟", icon: 'timer', pkg: 'com.android.deskclock', category: 'utility' },
|
||||||
'com.google.android.calculator': { name: 'Calculator', icon: 'calculate', pkg: 'com.google.android.calculator', category: 'utility' },
|
'com.google.android.calculator': { name: "计算器", icon: 'calculate', pkg: 'com.google.android.calculator', category: 'utility' },
|
||||||
'com.android.calculator2': { name: 'Calculator', icon: 'calculate', pkg: 'com.android.calculator2', category: 'utility' },
|
'com.android.calculator2': { name: "计算器", icon: 'calculate', pkg: 'com.android.calculator2', category: 'utility' },
|
||||||
'com.google.android.apps.photos': { name: 'Photos', icon: 'photo_library', pkg: 'com.google.android.apps.photos', category: 'media' },
|
'com.google.android.apps.photos': { name: "相册", icon: 'photo_library', pkg: 'com.google.android.apps.photos', category: 'media' },
|
||||||
'com.google.android.calendar': { name: 'Calendar', icon: 'calendar_month', pkg: 'com.google.android.calendar', category: 'productivity' },
|
'com.google.android.calendar': { name: "日历", icon: 'calendar_month', pkg: 'com.google.android.calendar', category: 'productivity' },
|
||||||
'com.google.android.keep': { name: 'Keep Notes', icon: 'note_alt', pkg: 'com.google.android.keep', category: 'productivity' },
|
'com.google.android.keep': { name: "Keep 笔记", icon: 'note_alt', pkg: 'com.google.android.keep', category: 'productivity' },
|
||||||
'com.android.vending': { name: 'Play Store', icon: 'storefront', pkg: 'com.android.vending', category: 'tools' },
|
'com.android.vending': { name: "应用商店", icon: 'storefront', pkg: 'com.android.vending', category: 'tools' },
|
||||||
'com.google.android.apps.messaging': { name: 'Messages', icon: 'chat', pkg: 'com.google.android.apps.messaging', category: 'communication' },
|
'com.google.android.apps.messaging': { name: "短信", icon: 'chat', pkg: 'com.google.android.apps.messaging', category: 'communication' },
|
||||||
|
|
||||||
// Popular Ecosystem Apps
|
// Popular Ecosystem Apps
|
||||||
'com.tencent.mm': { name: 'WeChat', icon: 'forum', pkg: 'com.tencent.mm', category: 'social' },
|
'com.tencent.mm': { name: "微信", icon: 'forum', pkg: 'com.tencent.mm', category: 'social' },
|
||||||
'com.xingin.xhs': { name: 'Xiaohongshu', icon: 'auto_stories', pkg: 'com.xingin.xhs', category: 'social' },
|
'com.xingin.xhs': { name: "小红书", icon: 'auto_stories', pkg: 'com.xingin.xhs', category: 'social' },
|
||||||
'com.sankuai.meituan': { name: 'Meituan', icon: 'restaurant', pkg: 'com.sankuai.meituan', category: 'lifestyle' },
|
'com.sankuai.meituan': { name: "美团", icon: 'restaurant', pkg: 'com.sankuai.meituan', category: 'lifestyle' },
|
||||||
'com.dianping.v1': { name: 'Dianping', icon: 'star', pkg: 'com.dianping.v1', category: 'lifestyle' },
|
'com.dianping.v1': { name: "大众点评", icon: 'star', pkg: 'com.dianping.v1', category: 'lifestyle' },
|
||||||
'tv.danmaku.bili': { name: 'Bilibili', icon: 'video_library', pkg: 'tv.danmaku.bili', category: 'entertainment' },
|
'tv.danmaku.bili': { name: "哔哩哔哩", icon: 'video_library', pkg: 'tv.danmaku.bili', category: 'entertainment' },
|
||||||
'com.eg.android.AlipayGphone': { name: 'Alipay', icon: 'account_balance_wallet', pkg: 'com.eg.android.AlipayGphone', category: 'finance' },
|
'com.eg.android.AlipayGphone': { name: "支付宝", icon: 'account_balance_wallet', pkg: 'com.eg.android.AlipayGphone', category: 'finance' },
|
||||||
'com.netease.cloudmusic': { name: 'NetEase Music', icon: 'headphones', pkg: 'com.netease.cloudmusic', category: 'entertainment' },
|
'com.netease.cloudmusic': { name: "网易云音乐", icon: 'headphones', pkg: 'com.netease.cloudmusic', category: 'entertainment' },
|
||||||
'com.spotify.music': { name: 'Spotify', icon: 'music_note', pkg: 'com.spotify.music', category: 'entertainment' }
|
'com.spotify.music': { name: 'Spotify', icon: 'music_note', pkg: 'com.spotify.music', category: 'entertainment' }
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -80,27 +80,27 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 1. Google Maps
|
// 1. Google Maps
|
||||||
{
|
{
|
||||||
id: 'maps_coffee',
|
id: 'maps_coffee',
|
||||||
title: 'Find Specialty Coffee',
|
title: "寻找精品咖啡",
|
||||||
description: 'Search nearby top-rated cafes in Google Maps',
|
description: "在 Google 地图查找附近高评分咖啡馆",
|
||||||
goal: 'Open Google Maps, search for top-rated specialty coffee shops nearby, and view the top result details.',
|
goal: "打开 Google 地图,搜索附近评分最高的精品咖啡店,查看首个结果详情。",
|
||||||
profile: 'flash',
|
profile: 'flash',
|
||||||
category: 'flash',
|
category: 'flash',
|
||||||
tag: 'Maps',
|
tag: "地图",
|
||||||
apps: [{ name: 'Maps', icon: 'explore', pkg: 'com.google.android.apps.maps' }],
|
apps: [{ name: "地图", icon: 'explore', pkg: 'com.google.android.apps.maps' }],
|
||||||
requiredPackages: ['com.google.android.apps.maps'],
|
requiredPackages: ['com.google.android.apps.maps'],
|
||||||
priority: 95
|
priority: 95
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: 'pro_commute_share',
|
id: 'pro_commute_share',
|
||||||
title: 'Commute ETA & Message Draft',
|
title: "通勤时间与短信草稿",
|
||||||
description: 'Check transit time on Maps and draft arrival ETA in Messages',
|
description: "查询地图通勤时间,并起草预计到达时间短信",
|
||||||
goal: 'Open Google Maps to check commute time to the International Airport, calculate arrival time, then open Messages and draft an ETA text message.',
|
goal: "打开 Google 地图查询前往国际机场的通勤时间,计算到达时间,再打开短信应用起草预计到达时间短信。",
|
||||||
profile: 'pro',
|
profile: 'pro',
|
||||||
category: 'cross_app',
|
category: 'cross_app',
|
||||||
tag: 'Maps + Messages',
|
tag: "地图 + 短信",
|
||||||
apps: [
|
apps: [
|
||||||
{ name: 'Maps', icon: 'explore', pkg: 'com.google.android.apps.maps' },
|
{ name: "地图", icon: 'explore', pkg: 'com.google.android.apps.maps' },
|
||||||
{ name: 'Messages', icon: 'chat', pkg: 'com.google.android.apps.messaging' }
|
{ name: "短信", icon: 'chat', pkg: 'com.google.android.apps.messaging' }
|
||||||
],
|
],
|
||||||
requiredPackages: ['com.google.android.apps.maps', 'com.google.android.apps.messaging'],
|
requiredPackages: ['com.google.android.apps.maps', 'com.google.android.apps.messaging'],
|
||||||
matchMode: 'all',
|
matchMode: 'all',
|
||||||
@@ -110,9 +110,9 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 2. Gmail
|
// 2. Gmail
|
||||||
{
|
{
|
||||||
id: 'gmail_receipts',
|
id: 'gmail_receipts',
|
||||||
title: 'Search Order Receipts',
|
title: "查找订单回执",
|
||||||
description: 'Find recent flight or delivery confirmation emails in Gmail',
|
description: "在 Gmail 查找近期航班或快递确认邮件",
|
||||||
goal: 'Open Gmail and search for recent flight or package delivery confirmation emails.',
|
goal: "打开 Gmail,搜索近期航班或包裹投递确认邮件。",
|
||||||
profile: 'flash',
|
profile: 'flash',
|
||||||
category: 'flash',
|
category: 'flash',
|
||||||
tag: 'Gmail',
|
tag: 'Gmail',
|
||||||
@@ -122,15 +122,15 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: 'pro_email_to_calendar',
|
id: 'pro_email_to_calendar',
|
||||||
title: 'Email Itinerary to Calendar',
|
title: "将邮件行程加入日历",
|
||||||
description: 'Extract flight or event dates from Gmail and schedule in Calendar',
|
description: "提取 Gmail 中的行程日期并创建日历事件",
|
||||||
goal: 'Open Gmail to find the latest event invitation or itinerary, extract dates and location, then open Google Calendar and create a corresponding calendar event.',
|
goal: "打开 Gmail 查找最新活动邀请或行程,提取日期和地点,再打开 Google 日历创建对应事件。",
|
||||||
profile: 'pro',
|
profile: 'pro',
|
||||||
category: 'cross_app',
|
category: 'cross_app',
|
||||||
tag: 'Gmail + Calendar',
|
tag: "Gmail + 日历",
|
||||||
apps: [
|
apps: [
|
||||||
{ name: 'Gmail', icon: 'mail', pkg: 'com.google.android.gm' },
|
{ name: 'Gmail', icon: 'mail', pkg: 'com.google.android.gm' },
|
||||||
{ name: 'Calendar', icon: 'calendar_month', pkg: 'com.google.android.calendar' }
|
{ name: "日历", icon: 'calendar_month', pkg: 'com.google.android.calendar' }
|
||||||
],
|
],
|
||||||
requiredPackages: ['com.google.android.gm'],
|
requiredPackages: ['com.google.android.gm'],
|
||||||
priority: 94
|
priority: 94
|
||||||
@@ -139,9 +139,9 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 3. Chrome
|
// 3. Chrome
|
||||||
{
|
{
|
||||||
id: 'chrome_research',
|
id: 'chrome_research',
|
||||||
title: 'Search AI News Breakthroughs',
|
title: "搜索 AI 最新进展",
|
||||||
description: 'Search latest multimodal AI developments in Chrome browser',
|
description: "在 Chrome 搜索多模态 AI 最新动态",
|
||||||
goal: 'Open Chrome browser and search for latest breakthroughs in multimodal mobile AI agents.',
|
goal: "打开 Chrome,搜索多模态移动 AI 智能体的最新进展。",
|
||||||
profile: 'flash',
|
profile: 'flash',
|
||||||
category: 'flash',
|
category: 'flash',
|
||||||
tag: 'Chrome',
|
tag: 'Chrome',
|
||||||
@@ -151,15 +151,15 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: 'pro_research_keep',
|
id: 'pro_research_keep',
|
||||||
title: 'Product Research & Notes Note',
|
title: "产品对比与笔记",
|
||||||
description: 'Compare top 3 headphones on Chrome and record comparison in Keep',
|
description: "在 Chrome 对比三款热门耳机,并记入 Keep",
|
||||||
goal: 'Open Chrome, research top 3 noise-cancelling headphones comparing price and battery life, then write a structured comparison summary note in Keep Notes.',
|
goal: "打开 Chrome,调研三款热门降噪耳机,对比价格和续航,再在 Keep 中写一份结构化对比笔记。",
|
||||||
profile: 'pro',
|
profile: 'pro',
|
||||||
category: 'pro',
|
category: 'pro',
|
||||||
tag: 'Chrome + Keep',
|
tag: "Chrome + Keep 笔记",
|
||||||
apps: [
|
apps: [
|
||||||
{ name: 'Chrome', icon: 'public', pkg: 'com.android.chrome' },
|
{ name: 'Chrome', icon: 'public', pkg: 'com.android.chrome' },
|
||||||
{ name: 'Keep Notes', icon: 'note_alt', pkg: 'com.google.android.keep' }
|
{ name: "Keep 笔记", icon: 'note_alt', pkg: 'com.google.android.keep' }
|
||||||
],
|
],
|
||||||
requiredPackages: ['com.android.chrome'],
|
requiredPackages: ['com.android.chrome'],
|
||||||
priority: 91
|
priority: 91
|
||||||
@@ -168,9 +168,9 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 4. YouTube
|
// 4. YouTube
|
||||||
{
|
{
|
||||||
id: 'youtube_lofi',
|
id: 'youtube_lofi',
|
||||||
title: 'Play Lo-Fi Music Radio',
|
title: "播放 Lo-Fi 音乐电台",
|
||||||
description: 'Search and play a Lo-Fi hip hop live stream on YouTube',
|
description: "在 YouTube 搜索并播放 Lo-Fi 嘻哈直播",
|
||||||
goal: 'Open YouTube, search for "Lofi hip hop beats relaxing radio" and tap on the live stream.',
|
goal: "打开 YouTube,搜索“Lofi hip hop beats relaxing radio”,播放对应直播。",
|
||||||
profile: 'flash',
|
profile: 'flash',
|
||||||
category: 'flash',
|
category: 'flash',
|
||||||
tag: 'YouTube',
|
tag: 'YouTube',
|
||||||
@@ -182,25 +182,25 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 5. Settings
|
// 5. Settings
|
||||||
{
|
{
|
||||||
id: 'settings_display_wifi',
|
id: 'settings_display_wifi',
|
||||||
title: 'Dark Mode & Wi-Fi Check',
|
title: "深色模式与 Wi-Fi 检查",
|
||||||
description: 'Toggle dark theme and verify network connection in Settings',
|
description: "在设置中启用深色模式并检查网络连接",
|
||||||
goal: 'Open Settings app, navigate to Display settings, ensure Dark theme is enabled, and check Wi-Fi connection status.',
|
goal: "打开设置,进入显示设置,确认深色模式已开启,再检查 Wi-Fi 连接状态。",
|
||||||
profile: 'flash',
|
profile: 'flash',
|
||||||
category: 'flash',
|
category: 'flash',
|
||||||
tag: 'Settings',
|
tag: "设置",
|
||||||
apps: [{ name: 'Settings', icon: 'settings', pkg: 'com.android.settings' }],
|
apps: [{ name: "设置", icon: 'settings', pkg: 'com.android.settings' }],
|
||||||
requiredPackages: ['com.android.settings'],
|
requiredPackages: ['com.android.settings'],
|
||||||
priority: 87
|
priority: 87
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: 'pro_settings_qa',
|
id: 'pro_settings_qa',
|
||||||
title: 'Subsystem Health & Crash Probe',
|
title: "系统功能与崩溃检查",
|
||||||
description: 'Traverse Settings submenus to verify screens and check for crash dialogs',
|
description: "遍历设置子菜单,检查页面与崩溃弹窗",
|
||||||
goal: 'Explore Settings submenus (Network, Connected devices, Apps, Battery, Storage), verify each screen loads properly without ANR or crash dialogs, and summarize results.',
|
goal: "遍历设置中的网络、已连接设备、应用、电池和存储菜单,确认各页面正常加载且没有无响应或崩溃弹窗,汇总检查结果。",
|
||||||
profile: 'pro',
|
profile: 'pro',
|
||||||
category: 'monitor',
|
category: 'monitor',
|
||||||
tag: 'Settings QA',
|
tag: "设置功能测试",
|
||||||
apps: [{ name: 'Settings', icon: 'settings', pkg: 'com.android.settings' }],
|
apps: [{ name: "设置", icon: 'settings', pkg: 'com.android.settings' }],
|
||||||
requiredPackages: ['com.android.settings'],
|
requiredPackages: ['com.android.settings'],
|
||||||
priority: 93
|
priority: 93
|
||||||
},
|
},
|
||||||
@@ -208,13 +208,13 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 6. Clock
|
// 6. Clock
|
||||||
{
|
{
|
||||||
id: 'clock_timer',
|
id: 'clock_timer',
|
||||||
title: '25-Min Pomodoro Timer',
|
title: "25 分钟番茄钟",
|
||||||
description: 'Start a 25-minute focus countdown timer in Clock app',
|
description: "在时钟应用启动 25 分钟专注计时",
|
||||||
goal: 'Open Clock app, switch to Timer tab, set 25 minutes and start the countdown timer.',
|
goal: "打开时钟,切换至计时器,设置 25 分钟并开始倒计时。",
|
||||||
profile: 'flash',
|
profile: 'flash',
|
||||||
category: 'flash',
|
category: 'flash',
|
||||||
tag: 'Clock',
|
tag: "时钟",
|
||||||
apps: [{ name: 'Clock', icon: 'timer', pkg: 'com.google.android.deskclock' }],
|
apps: [{ name: "时钟", icon: 'timer', pkg: 'com.google.android.deskclock' }],
|
||||||
requiredPackages: ['com.google.android.deskclock', 'com.android.deskclock'],
|
requiredPackages: ['com.google.android.deskclock', 'com.android.deskclock'],
|
||||||
priority: 86
|
priority: 86
|
||||||
},
|
},
|
||||||
@@ -222,13 +222,13 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 7. Calculator
|
// 7. Calculator
|
||||||
{
|
{
|
||||||
id: 'calc_gratuity',
|
id: 'calc_gratuity',
|
||||||
title: 'Split Bill & Calculate Tip',
|
title: "分摊账单与小费",
|
||||||
description: 'Calculate 18% gratuity on $186.40 for 3 people in Calculator',
|
description: "计算 186.40 美元账单加 18% 小费后的三人均摊金额",
|
||||||
goal: 'Open Calculator and calculate 18% tip on a bill of $186.40, then divide by 3 people.',
|
goal: "打开计算器,为 186.40 美元账单计算 18% 小费,再计算三人均摊金额。",
|
||||||
profile: 'flash',
|
profile: 'flash',
|
||||||
category: 'flash',
|
category: 'flash',
|
||||||
tag: 'Calculator',
|
tag: "计算器",
|
||||||
apps: [{ name: 'Calculator', icon: 'calculate', pkg: 'com.google.android.calculator' }],
|
apps: [{ name: "计算器", icon: 'calculate', pkg: 'com.google.android.calculator' }],
|
||||||
requiredPackages: ['com.google.android.calculator', 'com.android.calculator2'],
|
requiredPackages: ['com.google.android.calculator', 'com.android.calculator2'],
|
||||||
priority: 84
|
priority: 84
|
||||||
},
|
},
|
||||||
@@ -236,13 +236,13 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 8. Photos
|
// 8. Photos
|
||||||
{
|
{
|
||||||
id: 'photos_inspect',
|
id: 'photos_inspect',
|
||||||
title: 'Inspect Recent Screenshot',
|
title: "查看最新截图",
|
||||||
description: 'Open Google Photos and review the latest screenshot taken',
|
description: "打开 Google 相册查看最新截图",
|
||||||
goal: 'Open Google Photos and view the most recent screenshot in the screenshots album.',
|
goal: "打开 Google 相册,查看截图相册中的最新截图。",
|
||||||
profile: 'flash',
|
profile: 'flash',
|
||||||
category: 'flash',
|
category: 'flash',
|
||||||
tag: 'Photos',
|
tag: "相册",
|
||||||
apps: [{ name: 'Photos', icon: 'photo_library', pkg: 'com.google.android.apps.photos' }],
|
apps: [{ name: "相册", icon: 'photo_library', pkg: 'com.google.android.apps.photos' }],
|
||||||
requiredPackages: ['com.google.android.apps.photos'],
|
requiredPackages: ['com.google.android.apps.photos'],
|
||||||
priority: 82
|
priority: 82
|
||||||
},
|
},
|
||||||
@@ -250,27 +250,27 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 9. WeChat
|
// 9. WeChat
|
||||||
{
|
{
|
||||||
id: 'wechat_browse',
|
id: 'wechat_browse',
|
||||||
title: 'Check WeChat Messages',
|
title: "查看微信消息",
|
||||||
description: 'Open WeChat and view top recent chat conversations',
|
description: "打开微信查看最近聊天",
|
||||||
goal: 'Open WeChat and view the top recent chat messages.',
|
goal: "打开微信,查看最近的聊天消息。",
|
||||||
profile: 'flash',
|
profile: 'flash',
|
||||||
category: 'flash',
|
category: 'flash',
|
||||||
tag: 'WeChat',
|
tag: "微信",
|
||||||
apps: [{ name: 'WeChat', icon: 'forum', pkg: 'com.tencent.mm' }],
|
apps: [{ name: "微信", icon: 'forum', pkg: 'com.tencent.mm' }],
|
||||||
requiredPackages: ['com.tencent.mm'],
|
requiredPackages: ['com.tencent.mm'],
|
||||||
priority: 89
|
priority: 89
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: 'pro_wechat_to_calendar',
|
id: 'pro_wechat_to_calendar',
|
||||||
title: 'WeChat Notice to Calendar',
|
title: "将微信通知加入日历",
|
||||||
description: 'Extract meeting notice from WeChat chat and add to Calendar',
|
description: "提取微信会议通知并加入日历",
|
||||||
goal: 'Open WeChat, locate the latest meeting announcement or event message in the top chat, extract the time and topic, then open Calendar and schedule an event.',
|
goal: "打开微信,在最近聊天中查找最新会议或活动通知,提取时间和主题,再打开日历创建事件。",
|
||||||
profile: 'pro',
|
profile: 'pro',
|
||||||
category: 'cross_app',
|
category: 'cross_app',
|
||||||
tag: 'WeChat + Calendar',
|
tag: "微信 + 日历",
|
||||||
apps: [
|
apps: [
|
||||||
{ name: 'WeChat', icon: 'forum', pkg: 'com.tencent.mm' },
|
{ name: "微信", icon: 'forum', pkg: 'com.tencent.mm' },
|
||||||
{ name: 'Calendar', icon: 'calendar_month', pkg: 'com.google.android.calendar' }
|
{ name: "日历", icon: 'calendar_month', pkg: 'com.google.android.calendar' }
|
||||||
],
|
],
|
||||||
requiredPackages: ['com.tencent.mm'],
|
requiredPackages: ['com.tencent.mm'],
|
||||||
priority: 93
|
priority: 93
|
||||||
@@ -279,13 +279,13 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 10. Xiaohongshu
|
// 10. Xiaohongshu
|
||||||
{
|
{
|
||||||
id: 'xhs_coffee_guide',
|
id: 'xhs_coffee_guide',
|
||||||
title: 'RED Cafe Guide Search',
|
title: "小红书咖啡店攻略",
|
||||||
description: 'Search trending specialty cafe reviews on Xiaohongshu',
|
description: "在小红书搜索热门精品咖啡店评测",
|
||||||
goal: 'Open Xiaohongshu, search for top-rated specialty coffee shops, and view the top post.',
|
goal: "打开小红书,搜索高评分精品咖啡店,查看首篇笔记。",
|
||||||
profile: 'flash',
|
profile: 'flash',
|
||||||
category: 'flash',
|
category: 'flash',
|
||||||
tag: 'Xiaohongshu',
|
tag: "小红书",
|
||||||
apps: [{ name: 'Xiaohongshu', icon: 'auto_stories', pkg: 'com.xingin.xhs' }],
|
apps: [{ name: "小红书", icon: 'auto_stories', pkg: 'com.xingin.xhs' }],
|
||||||
requiredPackages: ['com.xingin.xhs'],
|
requiredPackages: ['com.xingin.xhs'],
|
||||||
priority: 87
|
priority: 87
|
||||||
},
|
},
|
||||||
@@ -293,13 +293,13 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 11. Meituan / Dianping
|
// 11. Meituan / Dianping
|
||||||
{
|
{
|
||||||
id: 'meituan_ramen_search',
|
id: 'meituan_ramen_search',
|
||||||
title: 'Meituan Food Search',
|
title: "美团美食搜索",
|
||||||
description: 'Search top-rated Ramen nearby on Meituan or Dianping',
|
description: "在美团或大众点评搜索附近高评分拉面店",
|
||||||
goal: 'Open Meituan or Dianping, search for top-rated Ramen nearby, and view top restaurant rating.',
|
goal: "打开美团或大众点评,搜索附近高评分拉面店,查看首家餐厅评分。",
|
||||||
profile: 'flash',
|
profile: 'flash',
|
||||||
category: 'flash',
|
category: 'flash',
|
||||||
tag: 'Meituan',
|
tag: "美团",
|
||||||
apps: [{ name: 'Meituan', icon: 'restaurant', pkg: 'com.sankuai.meituan' }],
|
apps: [{ name: "美团", icon: 'restaurant', pkg: 'com.sankuai.meituan' }],
|
||||||
requiredPackages: ['com.sankuai.meituan', 'com.dianping.v1'],
|
requiredPackages: ['com.sankuai.meituan', 'com.dianping.v1'],
|
||||||
priority: 86
|
priority: 86
|
||||||
},
|
},
|
||||||
@@ -307,13 +307,13 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 12. Bilibili
|
// 12. Bilibili
|
||||||
{
|
{
|
||||||
id: 'bilibili_stream',
|
id: 'bilibili_stream',
|
||||||
title: 'Bilibili Tech Video',
|
title: "哔哩哔哩科技视频",
|
||||||
description: 'Search and play an AI Agent tutorial video on Bilibili',
|
description: "在哔哩哔哩搜索并播放 AI 智能体教程",
|
||||||
goal: 'Open Bilibili, search for "AI Agent Architecture", and play the top matching video.',
|
goal: "打开哔哩哔哩,搜索“AI Agent Architecture”,播放最匹配的视频。",
|
||||||
profile: 'flash',
|
profile: 'flash',
|
||||||
category: 'flash',
|
category: 'flash',
|
||||||
tag: 'Bilibili',
|
tag: "哔哩哔哩",
|
||||||
apps: [{ name: 'Bilibili', icon: 'video_library', pkg: 'tv.danmaku.bili' }],
|
apps: [{ name: "哔哩哔哩", icon: 'video_library', pkg: 'tv.danmaku.bili' }],
|
||||||
requiredPackages: ['tv.danmaku.bili'],
|
requiredPackages: ['tv.danmaku.bili'],
|
||||||
priority: 85
|
priority: 85
|
||||||
},
|
},
|
||||||
@@ -321,13 +321,13 @@ export const SMART_TASK_LIBRARY: SmartSuggestion[] = [
|
|||||||
// 13. Play Store
|
// 13. Play Store
|
||||||
{
|
{
|
||||||
id: 'pro_playstore_review',
|
id: 'pro_playstore_review',
|
||||||
title: 'Play Store App Review Study',
|
title: "应用商店评价调研",
|
||||||
description: 'Compare top task management apps and ratings on Google Play',
|
description: "对比 Google Play 上的任务管理应用与评价",
|
||||||
goal: 'Open Google Play Store, search for top rated task management apps, compare ratings and latest user reviews of the top 2 candidates, and record recommendations.',
|
goal: "打开 Google Play,搜索高评分任务管理应用,对比前两款应用的评分和最新评论,记录推荐结果。",
|
||||||
profile: 'pro',
|
profile: 'pro',
|
||||||
category: 'pro',
|
category: 'pro',
|
||||||
tag: 'Play Store',
|
tag: "应用商店",
|
||||||
apps: [{ name: 'Play Store', icon: 'storefront', pkg: 'com.android.vending' }],
|
apps: [{ name: "应用商店", icon: 'storefront', pkg: 'com.android.vending' }],
|
||||||
requiredPackages: ['com.android.vending'],
|
requiredPackages: ['com.android.vending'],
|
||||||
priority: 88
|
priority: 88
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -44,89 +44,89 @@ export interface TuningLevel<TId extends string = string> {
|
|||||||
export const VERIFICATION_LEVELS: readonly TuningLevel<VerificationLevelId>[] = [
|
export const VERIFICATION_LEVELS: readonly TuningLevel<VerificationLevelId>[] = [
|
||||||
{
|
{
|
||||||
id: 'off',
|
id: 'off',
|
||||||
label: 'Off',
|
label: "关闭",
|
||||||
tagline: 'No checking. The run ends as soon as the task looks done.',
|
tagline: "不检查结果,任务看似完成时即结束。",
|
||||||
latency: 'no extra time',
|
latency: "不增加耗时",
|
||||||
runs: [
|
runs: [
|
||||||
'Each step is treated as finished the moment it is carried out.',
|
"每个步骤执行后即视为完成。",
|
||||||
'You get the full action trace, but no pass / fail verdict.'
|
"保留完整操作记录,不给出通过或失败结论。"
|
||||||
],
|
],
|
||||||
skips: ['Nothing is double-checked and nothing is retried.'],
|
skips: ["不复查,也不重试。"],
|
||||||
bestFor: 'Quick tries and demos, when you only want to watch what happens.'
|
bestFor: "适合快速试用和演示,仅观察执行过程。"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: 'final',
|
id: 'final',
|
||||||
label: 'At the end',
|
label: "结束时检查",
|
||||||
tagline: 'One check of the finished result against your goal. This is the default.',
|
tagline: "任务结束后按目标检查一次结果,默认使用此模式。",
|
||||||
latency: 'adds about 20–60 s at the end',
|
latency: "结束时额外耗时约 20–60 秒",
|
||||||
runs: [
|
runs: [
|
||||||
'When the task finishes, the final screen, the step history and the device state are compared with what you asked for.',
|
"任务完成后,将最终屏幕、操作历史和设备状态与目标进行对照。",
|
||||||
'If the result does not match, the task goes back and tries to fix it, up to 3 times.'
|
"若结果不符合目标,将返回修正,最多尝试 3 次。"
|
||||||
],
|
],
|
||||||
skips: ['Nothing is checked while the task is still running.'],
|
skips: ["任务执行期间不进行检查。"],
|
||||||
bestFor: 'Everyday tasks: an honest pass / fail without slowing the run down.'
|
bestFor: "适合日常任务,在结束时给出通过或失败结论。"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: 'checkpoints',
|
id: 'checkpoints',
|
||||||
label: 'Every step',
|
label: "逐步检查",
|
||||||
tagline: 'Each step is checked as soon as it is done, plus the final check.',
|
tagline: "每步完成后立即检查,结束时再做最终检查。",
|
||||||
latency: 'a short check after each step, done in the background',
|
latency: "每步结束后在后台进行简短检查",
|
||||||
runs: [
|
runs: [
|
||||||
'Every step is checked right after it completes, using the screenshots from that moment.',
|
"每步完成后,立即使用当时的截图进行检查。",
|
||||||
'If a step went wrong, it gets fixed before moving on (up to 2 tries per step).',
|
"若某步出错,会先修正再继续,每步最多重试 2 次。",
|
||||||
'A failed test condition is written down and the task keeps going.',
|
"记录失败的测试条件,然后继续执行任务。",
|
||||||
'The final check still runs at the end.'
|
"结束时仍会进行最终检查。"
|
||||||
],
|
],
|
||||||
bestFor: 'Long tasks where one early mistake would spoil everything after it.'
|
bestFor: "适合前期错误会影响后续结果的长流程任务。"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: 'strict',
|
id: 'strict',
|
||||||
label: 'Strict',
|
label: "严格检查",
|
||||||
tagline: 'Every step is checked, with more retries. The first failed test stops the run.',
|
tagline: "逐步检查并增加重试次数,首次测试失败即停止。",
|
||||||
latency: 'slowest: more checks and more retries',
|
latency: "耗时最长:更多检查与重试",
|
||||||
runs: [
|
runs: [
|
||||||
'Each check takes longer and gets more attempts: 4 fixes per step and 5 at the end.',
|
"增加检查时间与修正次数:每步最多 4 次,结束时最多 5 次。",
|
||||||
'The first failed test condition stops the run immediately, with the evidence attached.'
|
"首次测试条件失败时立即停止,并附上证据。"
|
||||||
],
|
],
|
||||||
bestFor: 'Release checks and regression runs, where a wrong pass is never acceptable.'
|
bestFor: "适合发布验收和回归测试,严格避免误判通过。"
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
|
|
||||||
export const EXPLORER_MODES: readonly TuningLevel<ExplorerModeId>[] = [
|
export const EXPLORER_MODES: readonly TuningLevel<ExplorerModeId>[] = [
|
||||||
{
|
{
|
||||||
id: 'flash',
|
id: 'flash',
|
||||||
label: 'Quick glance',
|
label: "快速识别",
|
||||||
tagline: 'Finds buttons and text on the screen in a single look.',
|
tagline: "通过一次识别查找屏幕按钮和文本。",
|
||||||
latency: '1 look per search',
|
latency: "每次搜索识别 1 次",
|
||||||
runs: [
|
runs: [
|
||||||
'Something on screen is asked for by name, icon or colour and its position comes back straight away.',
|
"根据名称、图标或颜色查找屏幕元素,直接返回位置。",
|
||||||
'Several things can be looked up at once.'
|
"支持一次查找多个元素。"
|
||||||
],
|
],
|
||||||
skips: ['No zooming in and no second try.'],
|
skips: ["不放大,也不再次识别。"],
|
||||||
bestFor: 'Ordinary apps with clearly labelled buttons, icons and text.'
|
bestFor: "适合按钮、图标和文字清晰的普通应用。"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: 'pro',
|
id: 'pro',
|
||||||
label: 'Second look',
|
label: "多次识别",
|
||||||
tagline: 'Takes up to 3 looks, thinking in between, before answering.',
|
tagline: "最多识别 3 次,结合推理后返回结果。",
|
||||||
latency: 'up to 3 looks per search',
|
latency: "每次搜索最多识别 3 次",
|
||||||
runs: [
|
runs: [
|
||||||
'The screen layout is read first, then the picture is searched.',
|
"先读取界面布局,再搜索画面。",
|
||||||
'If the first try misses, a different approach is tried within the 3 looks.'
|
"首次未找到时,会在 3 次预算内尝试其他方式。"
|
||||||
],
|
],
|
||||||
skips: ['Still no zooming into small areas, to keep searches short.'],
|
skips: ["不放大小区域,以缩短搜索时间。"],
|
||||||
bestFor: 'Things described by where they are ("the switch next to Wi-Fi") or with unclear labels.'
|
bestFor: "适合按位置描述的元素(如“Wi-Fi 旁的开关”)或标签不清晰的目标。"
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
id: 'ultra',
|
id: 'ultra',
|
||||||
label: 'Close-up',
|
label: "精细识别",
|
||||||
tagline: 'Zooms into parts of the screen and takes up to 8 looks.',
|
tagline: "放大屏幕局部,最多识别 8 次。",
|
||||||
latency: 'up to 8 looks per search (slowest)',
|
latency: "每次搜索最多识别 8 次(耗时最长)",
|
||||||
runs: [
|
runs: [
|
||||||
'Parts of the screen can be cropped and magnified to read tiny text and crowded layouts piece by piece.',
|
"可裁剪和放大屏幕局部,逐一识别小字与密集布局。",
|
||||||
'Later looks reuse the earlier ones, so they cost less time than they sound.'
|
"后续识别会复用之前的信息,减少重复处理。"
|
||||||
],
|
],
|
||||||
bestFor: 'Crowded screens, tiny targets, charts and drawings, and checks where exact placement matters.'
|
bestFor: "适合密集界面、微小目标、图表和需要精确定位的检查。"
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
|
|
||||||
|
|||||||
@@ -57,7 +57,7 @@ export interface LLMStreamEventData {
|
|||||||
}
|
}
|
||||||
|
|
||||||
export const DEFAULT_STREAM_RESET_MESSAGE =
|
export const DEFAULT_STREAM_RESET_MESSAGE =
|
||||||
'A request error occurred during output generation, typically caused by lower API priority. Retrying automatically...';
|
"生成过程中请求出错,正在自动重试…";
|
||||||
|
|
||||||
export interface StreamResetNotice {
|
export interface StreamResetNotice {
|
||||||
id: string;
|
id: string;
|
||||||
@@ -243,4 +243,3 @@ export interface StepReplayFrame {
|
|||||||
phaseId?: string;
|
phaseId?: string;
|
||||||
summary?: string;
|
summary?: string;
|
||||||
}
|
}
|
||||||
|
|
||||||
File diff suppressed because it is too large.
Load diff
@@ -1,3 +1,5 @@
|
|||||||
|
import { CapabilitySettingsComponent } from '../../components/model-settings/capability-settings.component';
|
||||||
|
import { ModelSettingsComponent } from '../../components/model-settings/model-settings.component';
|
||||||
/**
|
/**
|
||||||
* Copyright 2026 Google LLC
|
* Copyright 2026 Google LLC
|
||||||
*
|
*
|
||||||
@@ -14,9 +16,9 @@
|
|||||||
* limitations under the License.
|
* limitations under the License.
|
||||||
*/
|
*/
|
||||||
|
|
||||||
import { Component, signal, computed, effect, inject, OnInit, OnDestroy, ChangeDetectionStrategy } from '@angular/core';
|
import { Component, signal, computed, inject, OnInit, OnDestroy, ChangeDetectionStrategy } from '@angular/core';
|
||||||
import { FormsModule } from '@angular/forms';
|
import { FormsModule } from '@angular/forms';
|
||||||
import { Router, RouterLink } from '@angular/router';
|
import { Router } from '@angular/router';
|
||||||
import { AgentService } from '../../services/agent.service';
|
import { AgentService } from '../../services/agent.service';
|
||||||
import { SystemService } from '../../services/system.service';
|
import { SystemService } from '../../services/system.service';
|
||||||
import {
|
import {
|
||||||
@@ -103,7 +105,7 @@ type AdbGuideTab = 'emulator' | 'usb' | 'wifi' | 'remote';
|
|||||||
@Component({
|
@Component({
|
||||||
selector: 'app-home',
|
selector: 'app-home',
|
||||||
standalone: true,
|
standalone: true,
|
||||||
imports: [FormsModule],
|
imports: [FormsModule, ModelSettingsComponent, CapabilitySettingsComponent],
|
||||||
templateUrl: './home.component.html',
|
templateUrl: './home.component.html',
|
||||||
changeDetection: ChangeDetectionStrategy.Eager,
|
changeDetection: ChangeDetectionStrategy.Eager,
|
||||||
styleUrl: './home.component.scss'
|
styleUrl: './home.component.scss'
|
||||||
@@ -121,32 +123,6 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
public activeAdbGuideTab = signal<AdbGuideTab>('emulator');
|
public activeAdbGuideTab = signal<AdbGuideTab>('emulator');
|
||||||
public emulatorSetupMode = signal<'studio' | 'cli'>('studio');
|
public emulatorSetupMode = signal<'studio' | 'cli'>('studio');
|
||||||
|
|
||||||
// Interactive guide tab for LLM / OCR credentials: 'gemini' | 'ocr'
|
|
||||||
public modelSetupMode = signal<'gemini' | 'custom'>('gemini');
|
|
||||||
public showOcrConfig = signal<boolean>(false);
|
|
||||||
public showFullConfigFile = signal<boolean>(false);
|
|
||||||
|
|
||||||
// Model & Environment configuration from backend
|
|
||||||
public modelConfigEnv = computed(() => this.systemService.modelConfigEnv());
|
|
||||||
|
|
||||||
// Google Gemini API Key State
|
|
||||||
public geminiKeyInput = signal<string>('');
|
|
||||||
public showGeminiKey = signal<boolean>(false);
|
|
||||||
public isSavingGeminiKey = signal<boolean>(false);
|
|
||||||
public isTestingGeminiKey = signal<boolean>(false);
|
|
||||||
public geminiSaveMessage = signal<string | null>(null);
|
|
||||||
public geminiSaveError = signal<string | null>(null);
|
|
||||||
public isGeminiKeyEdited = signal<boolean>(false);
|
|
||||||
|
|
||||||
// Vision OCR API Key State
|
|
||||||
public ocrKeyInput = signal<string>('');
|
|
||||||
public showOcrKey = signal<boolean>(false);
|
|
||||||
public isSavingOcrKey = signal<boolean>(false);
|
|
||||||
public isTestingOcrKey = signal<boolean>(false);
|
|
||||||
public ocrSaveMessage = signal<string | null>(null);
|
|
||||||
public ocrSaveError = signal<string | null>(null);
|
|
||||||
public isOcrKeyEdited = signal<boolean>(false);
|
|
||||||
|
|
||||||
// Clipboard copy state tracker for interactive feedback
|
// Clipboard copy state tracker for interactive feedback
|
||||||
public copiedId = signal<string | null>(null);
|
public copiedId = signal<string | null>(null);
|
||||||
|
|
||||||
@@ -198,7 +174,6 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
|
|
||||||
// Pro Mode Outputter & Structured Output Configuration
|
// Pro Mode Outputter & Structured Output Configuration
|
||||||
public expectedOutput = signal<string>('');
|
public expectedOutput = signal<string>('');
|
||||||
public enableOutputter = signal<boolean>(true);
|
|
||||||
public showOutputterDrawer = signal<boolean>(false);
|
public showOutputterDrawer = signal<boolean>(false);
|
||||||
|
|
||||||
// Slider indexes map to the API ids in VERIFICATION_LEVELS / EXPLORER_MODES.
|
// Slider indexes map to the API ids in VERIFICATION_LEVELS / EXPLORER_MODES.
|
||||||
@@ -238,8 +213,8 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
return [
|
return [
|
||||||
{
|
{
|
||||||
kind: 'verify',
|
kind: 'verify',
|
||||||
name: 'Result check',
|
name: "结果检查",
|
||||||
ends: ['Off', 'Strict'],
|
ends: ["关闭", "严格检查"],
|
||||||
ladder: VERIFICATION_LEVELS,
|
ladder: VERIFICATION_LEVELS,
|
||||||
index: vi,
|
index: vi,
|
||||||
level: VERIFICATION_LEVELS[vi],
|
level: VERIFICATION_LEVELS[vi],
|
||||||
@@ -247,8 +222,8 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
},
|
},
|
||||||
{
|
{
|
||||||
kind: 'explore',
|
kind: 'explore',
|
||||||
name: 'Screen reading',
|
name: "屏幕识别",
|
||||||
ends: ['Faster', 'Sharper'],
|
ends: ["更快", "更精细"],
|
||||||
ladder: EXPLORER_MODES,
|
ladder: EXPLORER_MODES,
|
||||||
index: ei,
|
index: ei,
|
||||||
level: EXPLORER_MODES[ei],
|
level: EXPLORER_MODES[ei],
|
||||||
@@ -352,13 +327,13 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
public emulatorLaunchState = computed(() => this.systemService.emulatorLaunchState());
|
public emulatorLaunchState = computed(() => this.systemService.emulatorLaunchState());
|
||||||
public isEmulatorLaunching = computed(() => this.systemService.isEmulatorLaunching());
|
public isEmulatorLaunching = computed(() => this.systemService.isEmulatorLaunching());
|
||||||
public showLaunchLogs = signal<boolean>(false);
|
public showLaunchLogs = signal<boolean>(false);
|
||||||
|
|
||||||
// Probes
|
// Probes
|
||||||
public pythonProbe = computed(() => this.systemService.pythonProbe());
|
public pythonProbe = computed(() => this.systemService.pythonProbe());
|
||||||
public configProbe = computed(() => this.systemService.configProbe());
|
public configProbe = computed(() => this.systemService.configProbe());
|
||||||
public adbProbe = computed(() => this.systemService.adbProbe());
|
public adbProbe = computed(() => this.systemService.adbProbe());
|
||||||
public llmProbe = computed(() => this.systemService.llmProbe());
|
public llmProbe = computed(() => this.systemService.llmProbe());
|
||||||
public geminiProbe = computed(() => this.systemService.geminiProbe());
|
public openaiProbe = computed(() => this.systemService.openaiProbe());
|
||||||
public ocrProbe = computed(() => this.systemService.ocrProbe());
|
public ocrProbe = computed(() => this.systemService.ocrProbe());
|
||||||
public toolchainProbe = computed(() => this.systemService.toolchainProbe());
|
public toolchainProbe = computed(() => this.systemService.toolchainProbe());
|
||||||
|
|
||||||
@@ -426,16 +401,16 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
|
|
||||||
public emuHypervisorTitle = computed(() => {
|
public emuHypervisorTitle = computed(() => {
|
||||||
const os = this.effectiveOs();
|
const os = this.effectiveOs();
|
||||||
if (os === 'windows') return 'Enable Windows Hypervisor Platform (WHPX)';
|
if (os === 'windows') return "启用 Windows 虚拟机监控程序平台(WHPX)";
|
||||||
if (os === 'darwin') return 'Verify macOS Hypervisor / Install Tools';
|
if (os === 'darwin') return "检查 macOS 虚拟化并安装工具";
|
||||||
return 'Enable KVM Hardware Acceleration (Linux)';
|
return "启用 KVM 硬件加速(Linux)";
|
||||||
});
|
});
|
||||||
|
|
||||||
public emuHypervisorDesc = computed(() => {
|
public emuHypervisorDesc = computed(() => {
|
||||||
const os = this.effectiveOs();
|
const os = this.effectiveOs();
|
||||||
if (os === 'windows') return 'Enable Windows Hypervisor Platform in PowerShell (Run as Administrator):';
|
if (os === 'windows') return "以管理员身份打开 PowerShell,启用 Windows 虚拟机监控程序平台:";
|
||||||
if (os === 'darwin') return 'macOS uses native Hypervisor.framework. Install SDK tools via Homebrew (or Studio):';
|
if (os === 'darwin') return "macOS 使用系统 Hypervisor.framework。通过 Homebrew 或 Android Studio 安装 SDK 工具:";
|
||||||
return 'Ensure virtualization permissions are granted to your user account:';
|
return "确保当前用户拥有虚拟化权限:";
|
||||||
});
|
});
|
||||||
|
|
||||||
public emuHypervisorCmd = computed(() => {
|
public emuHypervisorCmd = computed(() => {
|
||||||
@@ -467,27 +442,6 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
return meta?.['configured'] === true;
|
return meta?.['configured'] === true;
|
||||||
});
|
});
|
||||||
|
|
||||||
public currentApiKey = computed<string>(() => this.systemService.currentApiKey());
|
|
||||||
public apiKeysMap = computed<Record<string, string>>(() => this.systemService.apiKeysMap());
|
|
||||||
|
|
||||||
public savedGeminiKey = computed<string>(() => {
|
|
||||||
const keys = this.apiKeysMap();
|
|
||||||
return keys['google'] || this.currentApiKey() || '';
|
|
||||||
});
|
|
||||||
|
|
||||||
public isGeminiModified = computed<boolean>(() => {
|
|
||||||
return this.geminiKeyInput().trim() !== this.savedGeminiKey().trim();
|
|
||||||
});
|
|
||||||
|
|
||||||
public savedOcrKey = computed<string>(() => {
|
|
||||||
const keys = this.apiKeysMap();
|
|
||||||
return keys['ocr'] || '';
|
|
||||||
});
|
|
||||||
|
|
||||||
public isOcrModified = computed<boolean>(() => {
|
|
||||||
return this.ocrKeyInput().trim() !== this.savedOcrKey().trim();
|
|
||||||
});
|
|
||||||
|
|
||||||
// Rich Smart Suggestions Library (Device-Aware, Flash vs Pro Tailored)
|
// Rich Smart Suggestions Library (Device-Aware, Flash vs Pro Tailored)
|
||||||
public readonly allSuggestions = this.taskRecService.allTasks;
|
public readonly allSuggestions = this.taskRecService.allTasks;
|
||||||
|
|
||||||
@@ -520,21 +474,6 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
this.systemService.fetchReadiness().subscribe();
|
this.systemService.fetchReadiness().subscribe();
|
||||||
};
|
};
|
||||||
|
|
||||||
constructor() {
|
|
||||||
effect(() => {
|
|
||||||
const keys = this.systemService.apiKeysMap();
|
|
||||||
const current = this.systemService.currentApiKey();
|
|
||||||
const googleKey = keys['google'] || current || '';
|
|
||||||
const ocrKey = keys['ocr'] || '';
|
|
||||||
if (!this.isGeminiKeyEdited()) {
|
|
||||||
this.geminiKeyInput.set(googleKey);
|
|
||||||
}
|
|
||||||
if (!this.isOcrKeyEdited()) {
|
|
||||||
this.ocrKeyInput.set(ocrKey);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
ngOnInit(): void {
|
ngOnInit(): void {
|
||||||
|
|
||||||
this.loadProTuningDefaults();
|
this.loadProTuningDefaults();
|
||||||
@@ -581,234 +520,15 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
this.emulatorSetupMode.set(mode);
|
this.emulatorSetupMode.set(mode);
|
||||||
}
|
}
|
||||||
|
|
||||||
public setModelSetupMode(mode: 'gemini' | 'custom'): void {
|
|
||||||
this.modelSetupMode.set(mode);
|
|
||||||
if (mode === 'custom') {
|
|
||||||
this.systemService.setSkipCredentialsCheck(true);
|
|
||||||
this.systemService.fetchModelConfigEnv().subscribe();
|
|
||||||
} else {
|
|
||||||
this.systemService.setSkipCredentialsCheck(false);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
public toggleFullConfigFile(): void {
|
|
||||||
this.showFullConfigFile.update(v => !v);
|
|
||||||
}
|
|
||||||
|
|
||||||
public toggleGeminiKeyVisibility(): void {
|
|
||||||
this.showGeminiKey.update(v => !v);
|
|
||||||
}
|
|
||||||
|
|
||||||
public toggleOcrKeyVisibility(): void {
|
|
||||||
this.showOcrKey.update(v => !v);
|
|
||||||
}
|
|
||||||
|
|
||||||
public toggleOcrConfig(): void {
|
|
||||||
this.showOcrConfig.update(v => !v);
|
|
||||||
}
|
|
||||||
|
|
||||||
public onGeminiKeyChange(val: string): void {
|
|
||||||
this.geminiKeyInput.set(val);
|
|
||||||
this.isGeminiKeyEdited.set(true);
|
|
||||||
this.geminiSaveError.set(null);
|
|
||||||
this.geminiSaveMessage.set(null);
|
|
||||||
}
|
|
||||||
|
|
||||||
public onOcrKeyChange(val: string): void {
|
|
||||||
this.ocrKeyInput.set(val);
|
|
||||||
this.isOcrKeyEdited.set(true);
|
|
||||||
this.ocrSaveError.set(null);
|
|
||||||
this.ocrSaveMessage.set(null);
|
|
||||||
}
|
|
||||||
|
|
||||||
public saveGeminiKey(): void {
|
|
||||||
const key = this.geminiKeyInput().trim();
|
|
||||||
if (!key) return;
|
|
||||||
this.isSavingGeminiKey.set(true);
|
|
||||||
this.geminiSaveError.set(null);
|
|
||||||
this.geminiSaveMessage.set(null);
|
|
||||||
|
|
||||||
this.systemService.updateApiKey('google', key, true).subscribe({
|
|
||||||
next: (res) => {
|
|
||||||
this.isSavingGeminiKey.set(false);
|
|
||||||
this.isGeminiKeyEdited.set(false);
|
|
||||||
this.geminiSaveMessage.set(res?.message || '✓ Gemini API key verified & saved successfully.');
|
|
||||||
setTimeout(() => this.geminiSaveMessage.set(null), 5000);
|
|
||||||
},
|
|
||||||
error: (err) => {
|
|
||||||
this.isSavingGeminiKey.set(false);
|
|
||||||
this.geminiSaveError.set(err?.error?.detail || err?.message || 'Failed to update Gemini API key.');
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
public clearGeminiKey(): void {
|
|
||||||
this.geminiKeyInput.set('');
|
|
||||||
this.isGeminiKeyEdited.set(false);
|
|
||||||
this.geminiSaveError.set(null);
|
|
||||||
this.geminiSaveMessage.set(null);
|
|
||||||
|
|
||||||
if (this.savedGeminiKey().trim()) {
|
|
||||||
this.isSavingGeminiKey.set(true);
|
|
||||||
this.systemService.updateApiKey('google', '', true).subscribe({
|
|
||||||
next: (res) => {
|
|
||||||
this.isSavingGeminiKey.set(false);
|
|
||||||
this.geminiSaveMessage.set(res?.message || '✓ Gemini API key cleared.');
|
|
||||||
setTimeout(() => this.geminiSaveMessage.set(null), 5000);
|
|
||||||
},
|
|
||||||
error: (err) => {
|
|
||||||
this.isSavingGeminiKey.set(false);
|
|
||||||
this.geminiSaveError.set(err?.error?.detail || err?.message || 'Failed to clear Gemini API key.');
|
|
||||||
}
|
|
||||||
});
|
|
||||||
} else {
|
|
||||||
this.geminiSaveMessage.set('✓ Gemini API key cleared.');
|
|
||||||
setTimeout(() => this.geminiSaveMessage.set(null), 3000);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
public saveOcrKey(): void {
|
|
||||||
const key = this.ocrKeyInput().trim();
|
|
||||||
if (!key) return;
|
|
||||||
this.isSavingOcrKey.set(true);
|
|
||||||
this.ocrSaveError.set(null);
|
|
||||||
this.ocrSaveMessage.set(null);
|
|
||||||
|
|
||||||
this.systemService.updateApiKey('ocr', key, true).subscribe({
|
|
||||||
next: (res) => {
|
|
||||||
this.isSavingOcrKey.set(false);
|
|
||||||
this.isOcrKeyEdited.set(false);
|
|
||||||
this.ocrSaveMessage.set(res?.message || '✓ Vision OCR API key verified & saved.');
|
|
||||||
setTimeout(() => this.ocrSaveMessage.set(null), 5000);
|
|
||||||
},
|
|
||||||
error: (err) => {
|
|
||||||
this.isSavingOcrKey.set(false);
|
|
||||||
this.ocrSaveError.set(err?.error?.detail || err?.message || 'Failed to update Vision OCR key.');
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
public clearOcrKey(): void {
|
|
||||||
this.ocrKeyInput.set('');
|
|
||||||
this.isOcrKeyEdited.set(false);
|
|
||||||
this.ocrSaveError.set(null);
|
|
||||||
this.ocrSaveMessage.set(null);
|
|
||||||
|
|
||||||
if (this.savedOcrKey().trim()) {
|
|
||||||
this.isSavingOcrKey.set(true);
|
|
||||||
this.systemService.updateApiKey('ocr', '', true).subscribe({
|
|
||||||
next: (res) => {
|
|
||||||
this.isSavingOcrKey.set(false);
|
|
||||||
this.ocrSaveMessage.set(res?.message || '✓ Vision OCR API key cleared.');
|
|
||||||
setTimeout(() => this.ocrSaveMessage.set(null), 5000);
|
|
||||||
},
|
|
||||||
error: (err) => {
|
|
||||||
this.isSavingOcrKey.set(false);
|
|
||||||
this.ocrSaveError.set(err?.error?.detail || err?.message || 'Failed to clear Vision OCR key.');
|
|
||||||
}
|
|
||||||
});
|
|
||||||
} else {
|
|
||||||
this.ocrSaveMessage.set('✓ Vision OCR API key cleared.');
|
|
||||||
setTimeout(() => this.ocrSaveMessage.set(null), 3000);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
public testGeminiKey(): void {
|
|
||||||
const key = this.geminiKeyInput().trim();
|
|
||||||
if (!key) return;
|
|
||||||
this.isTestingGeminiKey.set(true);
|
|
||||||
this.geminiSaveError.set(null);
|
|
||||||
this.geminiSaveMessage.set(null);
|
|
||||||
|
|
||||||
this.systemService.testApiKey('google', key).subscribe({
|
|
||||||
next: (res) => {
|
|
||||||
this.isTestingGeminiKey.set(false);
|
|
||||||
if (res?.valid) {
|
|
||||||
this.geminiSaveMessage.set(res?.message || '✓ Gemini API key is valid!');
|
|
||||||
} else {
|
|
||||||
this.geminiSaveError.set(res?.message || 'Gemini API key verification failed.');
|
|
||||||
}
|
|
||||||
setTimeout(() => this.geminiSaveMessage.set(null), 5000);
|
|
||||||
},
|
|
||||||
error: (err) => {
|
|
||||||
this.isTestingGeminiKey.set(false);
|
|
||||||
this.geminiSaveError.set(err?.error?.detail || err?.message || 'Gemini API key test failed.');
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
public testOcrKey(): void {
|
|
||||||
const key = this.ocrKeyInput().trim();
|
|
||||||
if (!key) return;
|
|
||||||
this.isTestingOcrKey.set(true);
|
|
||||||
this.ocrSaveError.set(null);
|
|
||||||
this.ocrSaveMessage.set(null);
|
|
||||||
|
|
||||||
this.systemService.testApiKey('ocr', key).subscribe({
|
|
||||||
next: (res) => {
|
|
||||||
this.isTestingOcrKey.set(false);
|
|
||||||
if (res?.valid) {
|
|
||||||
this.ocrSaveMessage.set(res?.message || '✓ Vision OCR API key is valid!');
|
|
||||||
} else {
|
|
||||||
this.ocrSaveError.set(res?.message || 'Vision OCR API key verification failed.');
|
|
||||||
}
|
|
||||||
setTimeout(() => this.ocrSaveMessage.set(null), 5000);
|
|
||||||
},
|
|
||||||
error: (err) => {
|
|
||||||
this.isTestingOcrKey.set(false);
|
|
||||||
this.ocrSaveError.set(err?.error?.detail || err?.message || 'Vision OCR API key test failed.');
|
|
||||||
}
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
public getProviderDisplayName(tab: string): string {
|
|
||||||
switch (tab) {
|
|
||||||
case 'gemini': return 'Gemini';
|
|
||||||
case 'ocr': return 'Vision OCR';
|
|
||||||
default: return tab;
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
public getProviderEnvVar(tab: string): string {
|
|
||||||
switch (tab) {
|
|
||||||
case 'gemini': return 'GEMINI_API_KEY';
|
|
||||||
case 'ocr': return 'GOOGLE_VISION_API_KEY';
|
|
||||||
default: return 'API_KEY';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
public getProviderHint(tab: string): string {
|
|
||||||
switch (tab) {
|
|
||||||
case 'gemini':
|
|
||||||
return 'For a quick start, Google Gemini provides a free API key. Artemis also supports other models (OpenAI, Claude, OpenRouter, etc.)—you can configure your own API keys directly in .env or your environment.';
|
|
||||||
case 'ocr':
|
|
||||||
return 'Google Cloud Vision API key for on-screen OCR text detection and UI grounding.';
|
|
||||||
default:
|
|
||||||
return 'Configure your API key or use environment definitions.';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
public skipCredentialsCheck(): void {
|
public skipCredentialsCheck(): void {
|
||||||
this.systemService.skipCredentialsCheck();
|
this.systemService.skipCredentialsCheck();
|
||||||
}
|
}
|
||||||
|
|
||||||
public getApiKeyPlaceholder(tab: string): string {
|
|
||||||
switch (tab) {
|
|
||||||
case 'gemini': return 'Enter Gemini API Key (e.g. AIzaSy...)';
|
|
||||||
case 'ocr': return 'Enter Vision OCR API Key (e.g. AIzaSy...)';
|
|
||||||
default: return 'Enter API Key...';
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
public isTabProviderActive(tab: string): boolean {
|
|
||||||
if (tab === 'ocr') {
|
|
||||||
return this.isOcrConfigured();
|
|
||||||
}
|
|
||||||
const targetProvider = tab === 'gemini' ? 'google' : tab;
|
|
||||||
const providers = this.configuredLlmProviders();
|
|
||||||
return providers.some(p => p.provider === targetProvider);
|
|
||||||
}
|
|
||||||
|
|
||||||
public refreshReadiness(): void {
|
public refreshReadiness(): void {
|
||||||
this.isRefreshingDiagnostics.set(true);
|
this.isRefreshingDiagnostics.set(true);
|
||||||
this.systemService.fetchReadiness(false, true).subscribe({
|
this.systemService.fetchReadiness(false, true).subscribe({
|
||||||
@@ -829,12 +549,12 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
this.systemService.restartAdb().subscribe({
|
this.systemService.restartAdb().subscribe({
|
||||||
next: (res) => {
|
next: (res) => {
|
||||||
this.adbRestartFeedback.set(
|
this.adbRestartFeedback.set(
|
||||||
res?.restart_result?.skipped ? 'Devices Refreshed ✓' : 'ADB Refreshed ✓'
|
res?.restart_result?.skipped ? "设备已刷新 ✓" : "ADB 已刷新 ✓"
|
||||||
);
|
);
|
||||||
setTimeout(() => this.adbRestartFeedback.set(null), 2500);
|
setTimeout(() => this.adbRestartFeedback.set(null), 2500);
|
||||||
},
|
},
|
||||||
error: () => {
|
error: () => {
|
||||||
this.adbRestartFeedback.set('Restart Failed');
|
this.adbRestartFeedback.set("重启失败");
|
||||||
setTimeout(() => this.adbRestartFeedback.set(null), 3000);
|
setTimeout(() => this.adbRestartFeedback.set(null), 3000);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
@@ -842,7 +562,7 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
|
|
||||||
public connectWifiDevice(): void {
|
public connectWifiDevice(): void {
|
||||||
if (this.isRemoteAdbServer()) {
|
if (this.isRemoteAdbServer()) {
|
||||||
this.wifiConnectError.set('Switch to local ADB before connecting a Wireless ADB device.');
|
this.wifiConnectError.set("连接无线设备前,请先切换至本地 ADB。");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const host = this.wifiHost().trim();
|
const host = this.wifiHost().trim();
|
||||||
@@ -850,7 +570,7 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
const port = parseInt(portStr, 10) || 5555;
|
const port = parseInt(portStr, 10) || 5555;
|
||||||
|
|
||||||
if (!host) {
|
if (!host) {
|
||||||
this.wifiConnectError.set('Please enter a valid IP address.');
|
this.wifiConnectError.set("请输入有效的 IP 地址。");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -863,15 +583,15 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
this.isConnectingWifi.set(false);
|
this.isConnectingWifi.set(false);
|
||||||
const cr = res?.connect_result;
|
const cr = res?.connect_result;
|
||||||
if (cr?.success) {
|
if (cr?.success) {
|
||||||
this.wifiConnectMessage.set(`Connected to ${host}:${port}!`);
|
this.wifiConnectMessage.set(`已连接至 ${host}:${port}!`);
|
||||||
setTimeout(() => this.wifiConnectMessage.set(null), 4000);
|
setTimeout(() => this.wifiConnectMessage.set(null), 4000);
|
||||||
} else {
|
} else {
|
||||||
this.wifiConnectError.set(cr?.message || 'Connection failed. Please check phone IP & Wi-Fi.');
|
this.wifiConnectError.set(cr?.message || "连接失败,请检查手机 IP 地址和 Wi-Fi。");
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
error: (err) => {
|
error: (err) => {
|
||||||
this.isConnectingWifi.set(false);
|
this.isConnectingWifi.set(false);
|
||||||
this.wifiConnectError.set(err?.error?.detail || 'Failed to connect. Please check adb connection.');
|
this.wifiConnectError.set(err?.error?.detail || "连接失败,请检查 ADB 连接。");
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
@@ -881,11 +601,11 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
const port = Number(this.remoteAdbPort().trim());
|
const port = Number(this.remoteAdbPort().trim());
|
||||||
|
|
||||||
if (!host) {
|
if (!host) {
|
||||||
this.remoteAdbError.set('Enter the host name or IP address of the ADB server.');
|
this.remoteAdbError.set("请输入 ADB 服务器的主机名或 IP 地址。");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
if (!Number.isInteger(port) || port < 1 || port > 65535) {
|
if (!Number.isInteger(port) || port < 1 || port > 65535) {
|
||||||
this.remoteAdbError.set('Enter a port between 1 and 65535.');
|
this.remoteAdbError.set("请输入 1–65535 之间的端口。");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -910,7 +630,7 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
error: error => {
|
error: error => {
|
||||||
this.isConnectingRemoteAdb.set(false);
|
this.isConnectingRemoteAdb.set(false);
|
||||||
this.remoteAdbError.set(
|
this.remoteAdbError.set(
|
||||||
error?.error?.detail || 'Unable to test the ADB server endpoint.'
|
error?.error?.detail || "无法测试 ADB 服务器连接。"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
@@ -919,7 +639,7 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
public activateRemoteAdbServer(): void {
|
public activateRemoteAdbServer(): void {
|
||||||
const tested = this.remoteAdbProbeResult();
|
const tested = this.remoteAdbProbeResult();
|
||||||
if (!tested?.success) {
|
if (!tested?.success) {
|
||||||
this.remoteAdbError.set('Test the endpoint before using it.');
|
this.remoteAdbError.set("请先测试服务器连接。");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -944,7 +664,7 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
error: error => {
|
error: error => {
|
||||||
this.isActivatingRemoteAdb.set(false);
|
this.isActivatingRemoteAdb.set(false);
|
||||||
this.remoteAdbError.set(
|
this.remoteAdbError.set(
|
||||||
error?.error?.detail || 'Unable to use the ADB server endpoint.'
|
error?.error?.detail || "无法使用此 ADB 服务器。"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
@@ -979,7 +699,7 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
error: error => {
|
error: error => {
|
||||||
this.isSwitchingToLocalAdb.set(false);
|
this.isSwitchingToLocalAdb.set(false);
|
||||||
this.remoteAdbError.set(
|
this.remoteAdbError.set(
|
||||||
error?.error?.detail || 'Unable to switch back to the local ADB server.'
|
error?.error?.detail || "无法切换回本地 ADB 服务器。"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
@@ -1069,12 +789,12 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
public runTask(): void {
|
public runTask(): void {
|
||||||
const goal = this.taskGoal().trim();
|
const goal = this.taskGoal().trim();
|
||||||
if (!goal) {
|
if (!goal) {
|
||||||
this.errorMessage.set('Please enter a task goal before running.');
|
this.errorMessage.set("请先输入任务目标。");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
if (!this.isReady()) {
|
if (!this.isReady()) {
|
||||||
this.errorMessage.set('System prerequisites are not satisfied. Please review System Setup first.');
|
this.errorMessage.set("运行条件尚未满足,请先检查系统设置。");
|
||||||
this.activeTab.set('diagnostics');
|
this.activeTab.set('diagnostics');
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -1089,7 +809,7 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
this.selectedProfile() === 'pro' && this.expectedOutput().trim()
|
this.selectedProfile() === 'pro' && this.expectedOutput().trim()
|
||||||
? this.expectedOutput().trim()
|
? this.expectedOutput().trim()
|
||||||
: undefined,
|
: undefined,
|
||||||
this.selectedProfile() === 'pro' ? this.enableOutputter() : undefined,
|
undefined,
|
||||||
this.selectedProfile() === 'pro'
|
this.selectedProfile() === 'pro'
|
||||||
? { verificationLevel: this.verificationLevel().id, explorerMode: this.explorerMode().id }
|
? { verificationLevel: this.verificationLevel().id, explorerMode: this.explorerMode().id }
|
||||||
: undefined
|
: undefined
|
||||||
@@ -1103,7 +823,7 @@ export class HomeComponent implements OnInit, OnDestroy {
|
|||||||
console.error('Failed to submit task from home page:', err);
|
console.error('Failed to submit task from home page:', err);
|
||||||
this.isSubmitting.set(false);
|
this.isSubmitting.set(false);
|
||||||
this.errorMessage.set(
|
this.errorMessage.set(
|
||||||
err?.error?.detail || 'Failed to submit task. Please check server connection.'
|
err?.error?.detail || "提交任务失败,请检查服务器连接。"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|||||||
@@ -27,7 +27,7 @@ limitations under the License.
|
|||||||
<div class="floating-dock-toast error">
|
<div class="floating-dock-toast error">
|
||||||
<span class="material-symbols-outlined toast-icon">error</span>
|
<span class="material-symbols-outlined toast-icon">error</span>
|
||||||
<span class="toast-text">{{ errorMessage() }}</span>
|
<span class="toast-text">{{ errorMessage() }}</span>
|
||||||
<button type="button" class="btn-toast-close" (click)="errorMessage.set(null)" aria-label="Close error message">
|
<button type="button" class="btn-toast-close" (click)="errorMessage.set(null)" aria-label="关闭错误消息">
|
||||||
<span class="material-symbols-outlined">close</span>
|
<span class="material-symbols-outlined">close</span>
|
||||||
</button>
|
</button>
|
||||||
</div>
|
</div>
|
||||||
@@ -89,7 +89,7 @@ limitations under the License.
|
|||||||
|
|
||||||
<!-- Dormant / Capsule Peek View (Visible when mouse is away and input idle) -->
|
<!-- Dormant / Capsule Peek View (Visible when mouse is away and input idle) -->
|
||||||
<div class="capsule-peek-view">
|
<div class="capsule-peek-view">
|
||||||
<span class="capsule-placeholder">Start a new task...</span>
|
<span class="capsule-placeholder">开始新任务…</span>
|
||||||
</div>
|
</div>
|
||||||
|
|
||||||
<!-- Expanded Full View Content (Visible on hover, focus, or text) -->
|
<!-- Expanded Full View Content (Visible on hover, focus, or text) -->
|
||||||
@@ -104,7 +104,7 @@ limitations under the License.
|
|||||||
(blur)="onInputBlur()"
|
(blur)="onInputBlur()"
|
||||||
rows="1"
|
rows="1"
|
||||||
[disabled]="isSubmitting()"
|
[disabled]="isSubmitting()"
|
||||||
[placeholder]="selectedProfile() === 'pro'
|
[placeholder]="selectedProfile() === 'pro'
|
||||||
? 'Pro: plans first, checks results and recovers from blocked steps. For long or complex tasks'
|
? 'Pro: plans first, checks results and recovers from blocked steps. For long or complex tasks'
|
||||||
: 'Flash: fast single-look loop for everyday app tasks'"
|
: 'Flash: fast single-look loop for everyday app tasks'"
|
||||||
class="dock-textarea"
|
class="dock-textarea"
|
||||||
@@ -115,7 +115,7 @@ limitations under the License.
|
|||||||
<!-- Clear Button (visible when text is entered) -->
|
<!-- Clear Button (visible when text is entered) -->
|
||||||
@if (taskInput.trim() && !isSubmitting()) {
|
@if (taskInput.trim() && !isSubmitting()) {
|
||||||
<button type="button" class="dock-circle-btn btn-clear"
|
<button type="button" class="dock-circle-btn btn-clear"
|
||||||
(click)="clearInput($event)" title="Clear input text" aria-label="Clear input text">
|
(click)="clearInput($event)" title="清空输入" aria-label="清空输入">
|
||||||
<span class="material-symbols-outlined">close</span>
|
<span class="material-symbols-outlined">close</span>
|
||||||
</button>
|
</button>
|
||||||
}
|
}
|
||||||
@@ -137,7 +137,7 @@ limitations under the License.
|
|||||||
[disabled]="!taskInput.trim() || isSubmitting()"
|
[disabled]="!taskInput.trim() || isSubmitting()"
|
||||||
(click)="submitTask()"
|
(click)="submitTask()"
|
||||||
[title]="isSubmitting() ? 'Starting task...' : 'Start task (Enter)'"
|
[title]="isSubmitting() ? 'Starting task...' : 'Start task (Enter)'"
|
||||||
aria-label="Start task"
|
aria-label="开始任务"
|
||||||
>
|
>
|
||||||
@if (isSubmitting()) {
|
@if (isSubmitting()) {
|
||||||
<span class="material-symbols-outlined spinning-icon">sync</span>
|
<span class="material-symbols-outlined spinning-icon">sync</span>
|
||||||
@@ -152,16 +152,16 @@ limitations under the License.
|
|||||||
<div class="card-footer-row">
|
<div class="card-footer-row">
|
||||||
<div class="footer-left-group">
|
<div class="footer-left-group">
|
||||||
<!-- Segmented Architecture Toggle (Flash / Pro) -->
|
<!-- Segmented Architecture Toggle (Flash / Pro) -->
|
||||||
<div class="model-profile-toggle" role="group" aria-label="Select Agent Architecture">
|
<div class="model-profile-toggle" role="group" aria-label="选择执行模式">
|
||||||
<button type="button" class="profile-toggle-pill" [class.active]="selectedProfile() === 'flash'"
|
<button type="button" class="profile-toggle-pill" [class.active]="selectedProfile() === 'flash'"
|
||||||
(mousedown)="$event.preventDefault()" (click)="setProfile('flash', $event)"
|
(mousedown)="$event.preventDefault()" (click)="setProfile('flash', $event)"
|
||||||
title="Flash: fast single-look loop for everyday app tasks (3–5s/step)">
|
title="Flash:快速执行日常应用任务(每步 3–5 秒)">
|
||||||
<span class="material-symbols-outlined pill-icon icon-flash">bolt</span>
|
<span class="material-symbols-outlined pill-icon icon-flash">bolt</span>
|
||||||
<span class="pill-label">Flash</span>
|
<span class="pill-label">Flash</span>
|
||||||
</button>
|
</button>
|
||||||
<button type="button" class="profile-toggle-pill" [class.active]="selectedProfile() === 'pro'"
|
<button type="button" class="profile-toggle-pill" [class.active]="selectedProfile() === 'pro'"
|
||||||
(mousedown)="$event.preventDefault()" (click)="setProfile('pro', $event)"
|
(mousedown)="$event.preventDefault()" (click)="setProfile('pro', $event)"
|
||||||
title="Pro: plans first, checks results and recovers from blocked steps. For long or complex tasks (15–40s/step)">
|
title="Pro:先规划、检查结果并恢复受阻步骤,适合长流程或复杂任务(每步 15–40 秒)">
|
||||||
<span class="material-symbols-outlined pill-icon icon-pro">psychology</span>
|
<span class="material-symbols-outlined pill-icon icon-pro">psychology</span>
|
||||||
<span class="pill-label">Pro</span>
|
<span class="pill-label">Pro</span>
|
||||||
</button>
|
</button>
|
||||||
|
|||||||
@@ -112,13 +112,13 @@ export class WorkspaceComponent implements OnInit {
|
|||||||
const truncated = session.initial_goal.length > 45
|
const truncated = session.initial_goal.length > 45
|
||||||
? session.initial_goal.substring(0, 42) + '...'
|
? session.initial_goal.substring(0, 42) + '...'
|
||||||
: session.initial_goal;
|
: session.initial_goal;
|
||||||
return `Stop current task: "${truncated}"`;
|
return `停止当前任务:"${truncated}"`;
|
||||||
}
|
}
|
||||||
const curId = this.agentService.currentSessionId();
|
const curId = this.agentService.currentSessionId();
|
||||||
if (curId) {
|
if (curId) {
|
||||||
return `Stop current task (${curId})`;
|
return `停止当前任务(${curId})`;
|
||||||
}
|
}
|
||||||
return 'Stop current running task';
|
return "停止当前任务";
|
||||||
});
|
});
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -244,7 +244,7 @@ export class WorkspaceComponent implements OnInit {
|
|||||||
error: (err) => {
|
error: (err) => {
|
||||||
console.error('Failed to submit task:', err);
|
console.error('Failed to submit task:', err);
|
||||||
this.isSubmitting.set(false);
|
this.isSubmitting.set(false);
|
||||||
this.errorMessage.set(err.error?.detail || 'The runner is busy. Please wait for current task to finish.');
|
this.errorMessage.set(err.error?.detail || "执行器忙碌,请等待当前任务结束。");
|
||||||
setTimeout(() => {
|
setTimeout(() => {
|
||||||
this.errorMessage.set(null);
|
this.errorMessage.set(null);
|
||||||
}, 5000);
|
}, 5000);
|
||||||
|
|||||||
@@ -356,8 +356,8 @@ describe('AgentService task cancellation and active session tracking', () => {
|
|||||||
|
|
||||||
// 2. Completed session selected -> false
|
// 2. Completed session selected -> false
|
||||||
service.rawSessions.set([
|
service.rawSessions.set([
|
||||||
{ session_id: 'sess-completed', status: 'completed', initial_goal: 'Done', start_time: 100 },
|
{ session_id: 'sess-completed', status: 'completed', initial_goal: '完成', start_time: 100 },
|
||||||
{ session_id: 'sess-running', status: 'running', initial_goal: 'Running', start_time: 101 }
|
{ session_id: 'sess-running', status: 'running', initial_goal: '执行中', start_time: 101 }
|
||||||
]);
|
]);
|
||||||
service.currentSessionId.set('sess-completed');
|
service.currentSessionId.set('sess-completed');
|
||||||
expect(service.isCurrentSessionRunning()).toBeFalse();
|
expect(service.isCurrentSessionRunning()).toBeFalse();
|
||||||
@@ -368,7 +368,7 @@ describe('AgentService task cancellation and active session tracking', () => {
|
|||||||
|
|
||||||
// 4. Paused session selected -> true
|
// 4. Paused session selected -> true
|
||||||
service.rawSessions.set([
|
service.rawSessions.set([
|
||||||
{ session_id: 'sess-paused', status: 'paused', initial_goal: 'Paused', start_time: 102 }
|
{ session_id: 'sess-paused', status: 'paused', initial_goal: '已暂停', start_time: 102 }
|
||||||
]);
|
]);
|
||||||
service.currentSessionId.set('sess-paused');
|
service.currentSessionId.set('sess-paused');
|
||||||
expect(service.isCurrentSessionRunning()).toBeTrue();
|
expect(service.isCurrentSessionRunning()).toBeTrue();
|
||||||
|
|||||||
@@ -425,7 +425,7 @@ export class AgentService {
|
|||||||
return new Observable((obs) => {
|
return new Observable((obs) => {
|
||||||
const submittedEvent: StartupProgressEvent = {
|
const submittedEvent: StartupProgressEvent = {
|
||||||
stage: 'submitting',
|
stage: 'submitting',
|
||||||
message: 'Submitting the task',
|
message: "正在提交任务",
|
||||||
timestamp: Date.now() / 1000
|
timestamp: Date.now() / 1000
|
||||||
};
|
};
|
||||||
this.pendingStartupProgress.set([submittedEvent]);
|
this.pendingStartupProgress.set([submittedEvent]);
|
||||||
@@ -655,7 +655,7 @@ export class AgentService {
|
|||||||
obs.complete();
|
obs.complete();
|
||||||
},
|
},
|
||||||
error: (err) => {
|
error: (err) => {
|
||||||
console.error(`Failed to delete session ${sessionId}:`, err);
|
console.error(`删除会话失败:${sessionId}:`, err);
|
||||||
this.fetchSessions();
|
this.fetchSessions();
|
||||||
this.fetchStatus();
|
this.fetchStatus();
|
||||||
obs.error(err);
|
obs.error(err);
|
||||||
@@ -828,7 +828,7 @@ export class AgentService {
|
|||||||
this.activeVideoSegments.set([]);
|
this.activeVideoSegments.set([]);
|
||||||
this.recordingPlaybackStatus.set('failed');
|
this.recordingPlaybackStatus.set('failed');
|
||||||
this.recordingPlaybackMessage.set(
|
this.recordingPlaybackMessage.set(
|
||||||
parsedData?.error || 'Recording finalization failed.'
|
parsedData?.error || "录屏收尾失败。"
|
||||||
);
|
);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -929,9 +929,9 @@ export class AgentService {
|
|||||||
const attempt = Number(parsedData.attempt || 0);
|
const attempt = Number(parsedData.attempt || 0);
|
||||||
const max = Number(parsedData.max_retries || 0);
|
const max = Number(parsedData.max_retries || 0);
|
||||||
const delay = Number(parsedData.delay || 0);
|
const delay = Number(parsedData.delay || 0);
|
||||||
const attemptText = attempt && max ? ` (Attempt ${attempt}/${max})` : '';
|
const attemptText = attempt && max ? `(第 ${attempt}/${max})` : '';
|
||||||
const delayText = delay > 0 ? `; retrying in ${delay.toFixed(2).replace(/\.00$/, '')}s` : '';
|
const delayText = delay > 0 ? `;重试等待 ${delay.toFixed(2).replace(/\.00$/, '')}s` : '';
|
||||||
this.retryMessage.set(`AI service is temporarily busy${attemptText}${delayText}...`);
|
this.retryMessage.set(`AI 服务暂时繁忙${attemptText}${delayText}...`);
|
||||||
if (curId) {
|
if (curId) {
|
||||||
this.appendLiveLLMRetryTrace(parsedData, curId);
|
this.appendLiveLLMRetryTrace(parsedData, curId);
|
||||||
}
|
}
|
||||||
@@ -945,7 +945,7 @@ export class AgentService {
|
|||||||
if (curId) {
|
if (curId) {
|
||||||
this.runningSessionId.set(curId);
|
this.runningSessionId.set(curId);
|
||||||
this.setSessionStatus(curId, 'paused');
|
this.setSessionStatus(curId, 'paused');
|
||||||
const pauseError = parsedData.error || 'AI call failed';
|
const pauseError = parsedData.error || "模型调用失败";
|
||||||
this.pausedError.set(pauseError);
|
this.pausedError.set(pauseError);
|
||||||
this.appendPausedErrorCard(
|
this.appendPausedErrorCard(
|
||||||
pauseError,
|
pauseError,
|
||||||
@@ -1522,7 +1522,7 @@ export class AgentService {
|
|||||||
this.isRetrying.set(false);
|
this.isRetrying.set(false);
|
||||||
const pauseError = data.paused_error
|
const pauseError = data.paused_error
|
||||||
|| this.pausedError()
|
|| this.pausedError()
|
||||||
|| 'AI model request failed. The task is paused.';
|
|| "模型请求失败,任务已暂停。";
|
||||||
this.pausedError.set(pauseError);
|
this.pausedError.set(pauseError);
|
||||||
if (data.session_id && this.currentSessionId() === data.session_id) {
|
if (data.session_id && this.currentSessionId() === data.session_id) {
|
||||||
this.appendPausedErrorCard(pauseError, data.session_id);
|
this.appendPausedErrorCard(pauseError, data.session_id);
|
||||||
@@ -1563,7 +1563,7 @@ export class AgentService {
|
|||||||
this.lastActiveTasksSignature = activeTasksSignature;
|
this.lastActiveTasksSignature = activeTasksSignature;
|
||||||
this.activeTasks.set(data.active_tasks || []);
|
this.activeTasks.set(data.active_tasks || []);
|
||||||
}
|
}
|
||||||
|
|
||||||
if (oldStatus !== data.status || oldRunningSessionId !== data.session_id) {
|
if (oldStatus !== data.status || oldRunningSessionId !== data.session_id) {
|
||||||
this.fetchSessions();
|
this.fetchSessions();
|
||||||
}
|
}
|
||||||
@@ -1677,8 +1677,8 @@ export class AgentService {
|
|||||||
attempt_id: attemptId,
|
attempt_id: attemptId,
|
||||||
checkpoint_id: checkpointId,
|
checkpoint_id: checkpointId,
|
||||||
subgoal_text: String(rec.subgoal_text || (checkpointId === 'final'
|
subgoal_text: String(rec.subgoal_text || (checkpointId === 'final'
|
||||||
? "Final review against the user's original goal"
|
? "根据用户原始目标进行最终检查"
|
||||||
: `Subgoal ${checkpointId.slice(0, 8)}`)),
|
: `子目标 ${checkpointId.slice(0, 8)}`)),
|
||||||
anchor_step_id: rec.anchor_step_id ?? null,
|
anchor_step_id: rec.anchor_step_id ?? null,
|
||||||
trace_id: rec.trace_id ?? null,
|
trace_id: rec.trace_id ?? null,
|
||||||
status: 'done',
|
status: 'done',
|
||||||
@@ -1749,7 +1749,7 @@ export class AgentService {
|
|||||||
next: (res) => {
|
next: (res) => {
|
||||||
if (res && res.notes) {
|
if (res && res.notes) {
|
||||||
this.currentNotes.set(res.notes);
|
this.currentNotes.set(res.notes);
|
||||||
|
|
||||||
// Default to task_plan.md if available, otherwise first note
|
// Default to task_plan.md if available, otherwise first note
|
||||||
const keys = Object.keys(res.notes);
|
const keys = Object.keys(res.notes);
|
||||||
if (keys.length > 0) {
|
if (keys.length > 0) {
|
||||||
@@ -1783,7 +1783,7 @@ export class AgentService {
|
|||||||
const targetSessionId = sessionId || this.currentSessionId();
|
const targetSessionId = sessionId || this.currentSessionId();
|
||||||
const session = this.sessions().find(s => s.session_id === targetSessionId);
|
const session = this.sessions().find(s => s.session_id === targetSessionId);
|
||||||
const targetUrl = videoUrl || session?.video_url || null;
|
const targetUrl = videoUrl || session?.video_url || null;
|
||||||
const goalTitle = title || session?.initial_goal || (targetSessionId ? `Task: ${targetSessionId.slice(0, 8)}...` : 'Screen Recording');
|
const goalTitle = title || session?.initial_goal || (targetSessionId ? `任务:${targetSessionId.slice(0, 8)}...` : "屏幕录像");
|
||||||
|
|
||||||
this.activeVideoTitle.set(goalTitle);
|
this.activeVideoTitle.set(goalTitle);
|
||||||
this.isVideoWindowOpen.set(true);
|
this.isVideoWindowOpen.set(true);
|
||||||
@@ -1818,7 +1818,7 @@ export class AgentService {
|
|||||||
this.videoWaitStartedAt = Date.now();
|
this.videoWaitStartedAt = Date.now();
|
||||||
this.isVideoLoading.set(true);
|
this.isVideoLoading.set(true);
|
||||||
this.recordingPlaybackStatus.set('processing');
|
this.recordingPlaybackStatus.set('processing');
|
||||||
this.recordingPlaybackMessage.set('Loading screen recording...');
|
this.recordingPlaybackMessage.set("正在加载屏幕录像…");
|
||||||
if (targetUrl) {
|
if (targetUrl) {
|
||||||
this.playerMode?.set('video');
|
this.playerMode?.set('video');
|
||||||
} else if (this.hasCurrentSessionStepFrames?.()) {
|
} else if (this.hasCurrentSessionStepFrames?.()) {
|
||||||
@@ -1870,7 +1870,7 @@ export class AgentService {
|
|||||||
this.shouldAutoplayVideo.set(true);
|
this.shouldAutoplayVideo.set(true);
|
||||||
this.isVideoLoading.set(true);
|
this.isVideoLoading.set(true);
|
||||||
this.recordingPlaybackStatus.set('processing');
|
this.recordingPlaybackStatus.set('processing');
|
||||||
this.recordingPlaybackMessage.set('Finalizing screen recording...');
|
this.recordingPlaybackMessage.set("正在完成录屏处理…");
|
||||||
this.requestSessionVideo(sessionId, this.videoRequestGeneration);
|
this.requestSessionVideo(sessionId, this.videoRequestGeneration);
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1901,7 +1901,7 @@ export class AgentService {
|
|||||||
this.activeVideoSegments.set([]);
|
this.activeVideoSegments.set([]);
|
||||||
this.isVideoLoading.set(true);
|
this.isVideoLoading.set(true);
|
||||||
this.recordingPlaybackStatus.set('processing');
|
this.recordingPlaybackStatus.set('processing');
|
||||||
this.recordingPlaybackMessage.set('Finalizing screen recording...');
|
this.recordingPlaybackMessage.set("正在完成录屏处理…");
|
||||||
this.scheduleVideoRetry(sessionId, generation, res.retry_after_ms);
|
this.scheduleVideoRetry(sessionId, generation, res.retry_after_ms);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -1913,8 +1913,8 @@ export class AgentService {
|
|||||||
this.recordingPlaybackStatus.set(status === 'failed' ? 'failed' : 'unavailable');
|
this.recordingPlaybackStatus.set(status === 'failed' ? 'failed' : 'unavailable');
|
||||||
this.recordingPlaybackMessage.set(
|
this.recordingPlaybackMessage.set(
|
||||||
res.message || (status === 'failed'
|
res.message || (status === 'failed'
|
||||||
? 'Recording finalization failed.'
|
? "录屏收尾失败。"
|
||||||
: 'No screen recording is available for this task.')
|
: "此任务暂无可用录像。")
|
||||||
);
|
);
|
||||||
if (this.hasCurrentSessionStepFrames?.()) {
|
if (this.hasCurrentSessionStepFrames?.()) {
|
||||||
this.playerMode?.set('steps');
|
this.playerMode?.set('steps');
|
||||||
@@ -1930,7 +1930,7 @@ export class AgentService {
|
|||||||
}
|
}
|
||||||
this.isVideoLoading.set(false);
|
this.isVideoLoading.set(false);
|
||||||
this.recordingPlaybackStatus.set('failed');
|
this.recordingPlaybackStatus.set('failed');
|
||||||
this.recordingPlaybackMessage.set('Unable to load the screen recording.');
|
this.recordingPlaybackMessage.set("无法加载屏幕录像。");
|
||||||
if (this.hasCurrentSessionStepFrames?.()) {
|
if (this.hasCurrentSessionStepFrames?.()) {
|
||||||
this.playerMode?.set('steps');
|
this.playerMode?.set('steps');
|
||||||
}
|
}
|
||||||
@@ -1943,7 +1943,7 @@ export class AgentService {
|
|||||||
if (Date.now() - this.videoWaitStartedAt > 120_000) {
|
if (Date.now() - this.videoWaitStartedAt > 120_000) {
|
||||||
this.isVideoLoading.set(false);
|
this.isVideoLoading.set(false);
|
||||||
this.recordingPlaybackStatus.set('failed');
|
this.recordingPlaybackStatus.set('failed');
|
||||||
this.recordingPlaybackMessage.set('Recording finalization timed out. You can retry.');
|
this.recordingPlaybackMessage.set("录屏处理超时,可以重试。");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
const delay = Math.max(500, Math.min(3000, retryAfterMs));
|
const delay = Math.max(500, Math.min(3000, retryAfterMs));
|
||||||
|
|||||||
@@ -56,8 +56,8 @@ export class SystemService {
|
|||||||
public configProbe = computed(() => this.probes().find(p => p.id === 'system_config') || null);
|
public configProbe = computed(() => this.probes().find(p => p.id === 'system_config') || null);
|
||||||
public toolchainProbe = computed(() => this.probes().find(p => p.id === 'toolchain') || null);
|
public toolchainProbe = computed(() => this.probes().find(p => p.id === 'toolchain') || null);
|
||||||
public adbProbe = computed(() => this.probes().find(p => p.id === 'android_adb') || null);
|
public adbProbe = computed(() => this.probes().find(p => p.id === 'android_adb') || null);
|
||||||
public llmProbe = computed(() => this.probes().find(p => p.id === 'gemini_api_key' || p.id === 'llm_api_key') || null);
|
public llmProbe = computed(() => this.probes().find(p => p.id === 'openai_api_key' || p.id === 'llm_api_key') || null);
|
||||||
public geminiProbe = computed(() => this.llmProbe());
|
public openaiProbe = computed(() => this.llmProbe());
|
||||||
public ocrProbe = computed(() => this.probes().find(p => p.id === 'vision_ocr_key' || p.id === 'ocr_api_key') || null);
|
public ocrProbe = computed(() => this.probes().find(p => p.id === 'vision_ocr_key' || p.id === 'ocr_api_key') || null);
|
||||||
|
|
||||||
// Grouped readiness helpers for the 3-step onboarding flow
|
// Grouped readiness helpers for the 3-step onboarding flow
|
||||||
@@ -324,7 +324,7 @@ export class SystemService {
|
|||||||
status: 'starting',
|
status: 'starting',
|
||||||
pid: null,
|
pid: null,
|
||||||
serial: null,
|
serial: null,
|
||||||
stage_message: 'Spawning emulator process...',
|
stage_message: "正在启动模拟器进程…",
|
||||||
progress_percent: 15,
|
progress_percent: 15,
|
||||||
started_at: Date.now() / 1000,
|
started_at: Date.now() / 1000,
|
||||||
elapsed_seconds: 0,
|
elapsed_seconds: 0,
|
||||||
@@ -346,13 +346,13 @@ export class SystemService {
|
|||||||
error: (err) => {
|
error: (err) => {
|
||||||
console.error('Failed to launch emulator:', err);
|
console.error('Failed to launch emulator:', err);
|
||||||
this.launchingAvd.set(null);
|
this.launchingAvd.set(null);
|
||||||
const errorMsg = err?.error?.detail || err?.message || 'Failed to start emulator process.';
|
const errorMsg = err?.error?.detail || err?.message || "启动模拟器进程失败。";
|
||||||
this.emulatorLaunchState.set({
|
this.emulatorLaunchState.set({
|
||||||
avd_name: avdName,
|
avd_name: avdName,
|
||||||
status: 'failed',
|
status: 'failed',
|
||||||
pid: null,
|
pid: null,
|
||||||
serial: null,
|
serial: null,
|
||||||
stage_message: 'Failed to initiate launch.',
|
stage_message: "启动失败。",
|
||||||
progress_percent: 0,
|
progress_percent: 0,
|
||||||
started_at: null,
|
started_at: null,
|
||||||
elapsed_seconds: 0,
|
elapsed_seconds: 0,
|
||||||
@@ -607,4 +607,3 @@ export interface ModelConfigEnvResponse {
|
|||||||
description: string;
|
description: string;
|
||||||
}>;
|
}>;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -250,7 +250,7 @@ describe('extractStepReplayFrames', () => {
|
|||||||
expect(frames.length).toBe(2);
|
expect(frames.length).toBe(2);
|
||||||
expect(frames[0].stepNumber).toBe(1);
|
expect(frames[0].stepNumber).toBe(1);
|
||||||
expect(frames[0].action?.action).toBe('click');
|
expect(frames[0].action?.action).toBe('click');
|
||||||
expect(frames[0].actionText).toContain('Tapping Element');
|
expect(frames[0].actionText).toContain('点击元素');
|
||||||
expect(frames[1].stepNumber).toBe(2);
|
expect(frames[1].stepNumber).toBe(2);
|
||||||
expect(frames[1].title).toContain('Report Task Status');
|
expect(frames[1].title).toContain('Report Task Status');
|
||||||
});
|
});
|
||||||
@@ -288,7 +288,7 @@ describe('extractStepReplayFrames', () => {
|
|||||||
step_number: 1,
|
step_number: 1,
|
||||||
pre_image_name: 'img1',
|
pre_image_name: 'img1',
|
||||||
post_image_name: 'img2',
|
post_image_name: 'img2',
|
||||||
action_taken: { action: 'manage_app', app_name: 'Maps' },
|
action_taken: { action: 'manage_app', app_name: '地图' },
|
||||||
timestamp: 100
|
timestamp: 100
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
@@ -344,20 +344,20 @@ describe('extractStepReplayFrames', () => {
|
|||||||
// Frame 1 MUST start at Step 1, NOT Step 4!
|
// Frame 1 MUST start at Step 1, NOT Step 4!
|
||||||
expect(frames[0].stepNumber).toBe(1);
|
expect(frames[0].stepNumber).toBe(1);
|
||||||
expect(frames[0].index).toBe(0);
|
expect(frames[0].index).toBe(0);
|
||||||
expect(frames[0].title).toContain('Step 1');
|
expect(frames[0].title).toContain('步骤 1');
|
||||||
expect(frames[0].summary).toBe('In Step 1, I launched Google Maps');
|
expect(frames[0].summary).toBe('In Step 1, I launched Google Maps');
|
||||||
expect(frames[0].preImageUrl).toBe('/images/img1');
|
expect(frames[0].preImageUrl).toBe('/images/img1');
|
||||||
|
|
||||||
// Frame 2 MUST be Step 2
|
// Frame 2 MUST be Step 2
|
||||||
expect(frames[1].stepNumber).toBe(2);
|
expect(frames[1].stepNumber).toBe(2);
|
||||||
expect(frames[1].index).toBe(1);
|
expect(frames[1].index).toBe(1);
|
||||||
expect(frames[1].title).toContain('Step 2');
|
expect(frames[1].title).toContain('步骤 2');
|
||||||
expect(frames[1].summary).toBe('In Step 2, I typed Coffee');
|
expect(frames[1].summary).toBe('In Step 2, I typed Coffee');
|
||||||
|
|
||||||
// Frame 3 MUST be Step 3
|
// Frame 3 MUST be Step 3
|
||||||
expect(frames[2].stepNumber).toBe(3);
|
expect(frames[2].stepNumber).toBe(3);
|
||||||
expect(frames[2].index).toBe(2);
|
expect(frames[2].index).toBe(2);
|
||||||
expect(frames[2].title).toContain('Step 3');
|
expect(frames[2].title).toContain('步骤 3');
|
||||||
expect(frames[2].summary).toBe('In Step 3, I pressed ENTER');
|
expect(frames[2].summary).toBe('In Step 3, I pressed ENTER');
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -419,9 +419,9 @@ describe('getActionTargetText self-described targets', () => {
|
|||||||
it('prefers observed target_text over target_description when both are present', () => {
|
it('prefers observed target_text over target_description when both are present', () => {
|
||||||
expect(getActionTargetText({
|
expect(getActionTargetText({
|
||||||
action: 'click',
|
action: 'click',
|
||||||
target_text: 'Play',
|
target_text: '播放',
|
||||||
target_description: 'play button'
|
target_description: 'play button'
|
||||||
})).toBe('Play');
|
})).toBe('播放');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('chains per-point target_descriptions for click_sequence with the same arrow as coordinates', () => {
|
it('chains per-point target_descriptions for click_sequence with the same arrow as coordinates', () => {
|
||||||
@@ -464,7 +464,7 @@ describe('isActionFailed / getActionErrorMessage with Validator attempts', () =>
|
|||||||
const action = { action: 'click', coordinates: [500, 900] };
|
const action = { action: 'click', coordinates: [500, 900] };
|
||||||
|
|
||||||
it('treats a retry that ended in Dispatched as not failed', () => {
|
it('treats a retry that ended in Dispatched as not failed', () => {
|
||||||
const stepData = stepWithAttempts(['Execution error: x', 'Dispatched']);
|
const stepData = stepWithAttempts(['Execution error: x', '已发送']);
|
||||||
expect(isActionFailed(action, stepData)).toBeFalse();
|
expect(isActionFailed(action, stepData)).toBeFalse();
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -67,7 +67,7 @@ export function isAndroidAction(action: any): boolean {
|
|||||||
name = (act.name || act.action || '').toLowerCase();
|
name = (act.name || act.action || '').toLowerCase();
|
||||||
}
|
}
|
||||||
return [
|
return [
|
||||||
'tap', 'click', 'click_sequence', 'input', 'input_text', 'swipe', 'scroll', 'press_key',
|
'tap', 'click', 'click_sequence', 'input', 'input_text', 'swipe', 'scroll', 'press_key',
|
||||||
'press_home', 'press_back', 'launch_app', 'manage_app', 'open_app',
|
'press_home', 'press_back', 'launch_app', 'manage_app', 'open_app',
|
||||||
'focus_and_input_text', 'focus_and_clear_text', 'long_press', 'long_press_on',
|
'focus_and_input_text', 'focus_and_clear_text', 'long_press', 'long_press_on',
|
||||||
'wait_for_delay', 'wait', 'drag', 'drag_and_drop'
|
'wait_for_delay', 'wait', 'drag', 'drag_and_drop'
|
||||||
@@ -165,60 +165,60 @@ export function getActionIcon(action: any): string {
|
|||||||
*/
|
*/
|
||||||
export function getActionTitle(action: any): string {
|
export function getActionTitle(action: any): string {
|
||||||
const act = getActionObject(action);
|
const act = getActionObject(action);
|
||||||
if (!act) return 'Action';
|
if (!act) return "操作";
|
||||||
const name = (act.name || act.action || '').toLowerCase();
|
const name = (act.name || act.action || '').toLowerCase();
|
||||||
switch (name) {
|
switch (name) {
|
||||||
case 'tap':
|
case 'tap':
|
||||||
case 'click':
|
case 'click':
|
||||||
case 'tap_element':
|
case 'tap_element':
|
||||||
case 'click_element':
|
case 'click_element':
|
||||||
return 'Tapping Element';
|
return "点击元素";
|
||||||
case 'input':
|
case 'input':
|
||||||
case 'input_text':
|
case 'input_text':
|
||||||
case 'focus_and_input_text':
|
case 'focus_and_input_text':
|
||||||
return 'Entering Text';
|
return "输入文本";
|
||||||
case 'focus_and_clear_text':
|
case 'focus_and_clear_text':
|
||||||
case 'clear_text':
|
case 'clear_text':
|
||||||
return 'Clearing Text';
|
return "清空文本";
|
||||||
case 'swipe':
|
case 'swipe':
|
||||||
case 'scroll': {
|
case 'scroll': {
|
||||||
const actObj = getActionObject(action);
|
const actObj = getActionObject(action);
|
||||||
const args = actObj?.args && typeof actObj.args === 'object' ? actObj.args : {};
|
const args = actObj?.args && typeof actObj.args === 'object' ? actObj.args : {};
|
||||||
const dir = actObj?.direction || actObj?.gesture || args.direction || args.gesture || (typeof args.action === 'string' ? args.action : '') || (typeof actObj?.action === 'string' && actObj.action !== name ? actObj.action : '');
|
const dir = actObj?.direction || actObj?.gesture || args.direction || args.gesture || (typeof args.action === 'string' ? args.action : '') || (typeof actObj?.action === 'string' && actObj.action !== name ? actObj.action : '');
|
||||||
if (dir && isPureDirectionString(dir)) {
|
if (dir && isPureDirectionString(dir)) {
|
||||||
return `Swiping Screen (${String(dir).toUpperCase()})`;
|
return `滑动屏幕(${String(dir).toUpperCase()})`;
|
||||||
}
|
}
|
||||||
return 'Swiping Screen';
|
return "滑动屏幕";
|
||||||
}
|
}
|
||||||
case 'drag':
|
case 'drag':
|
||||||
case 'drag_and_drop':
|
case 'drag_and_drop':
|
||||||
return 'Dragging Screen';
|
return "拖动屏幕";
|
||||||
case 'press_key':
|
case 'press_key':
|
||||||
case 'press_home':
|
case 'press_home':
|
||||||
case 'press_back':
|
case 'press_back':
|
||||||
return 'Pressing Hardware Key';
|
return "按下实体按键";
|
||||||
case 'launch_app':
|
case 'launch_app':
|
||||||
case 'open_app':
|
case 'open_app':
|
||||||
return 'Launching Application';
|
return "启动应用";
|
||||||
case 'stop_app':
|
case 'stop_app':
|
||||||
case 'close_app':
|
case 'close_app':
|
||||||
return 'Stopping Application';
|
return "停止应用";
|
||||||
case 'manage_app': {
|
case 'manage_app': {
|
||||||
const actObj = getActionObject(action);
|
const actObj = getActionObject(action);
|
||||||
const actStr = (actObj?.action || actObj?.args?.action || '').toLowerCase();
|
const actStr = (actObj?.action || actObj?.args?.action || '').toLowerCase();
|
||||||
if (actStr === 'launch') return 'Launching Application';
|
if (actStr === 'launch') return "启动应用";
|
||||||
if (actStr === 'stop' || actStr === 'close') return 'Stopping Application';
|
if (actStr === 'stop' || actStr === 'close') return "停止应用";
|
||||||
return 'Managing Application';
|
return "管理应用";
|
||||||
}
|
}
|
||||||
case 'wait_for_delay':
|
case 'wait_for_delay':
|
||||||
case 'delay':
|
case 'delay':
|
||||||
case 'wait':
|
case 'wait':
|
||||||
return 'Waiting for Delay';
|
return "等待";
|
||||||
case 'long_press':
|
case 'long_press':
|
||||||
case 'long_press_on':
|
case 'long_press_on':
|
||||||
return 'Long Pressing Element';
|
return "长按元素";
|
||||||
case 'click_sequence':
|
case 'click_sequence':
|
||||||
return 'Clicking Sequence';
|
return "连续点击";
|
||||||
default:
|
default:
|
||||||
return name.replace(/_/g, ' ').replace(/\b\w/g, (c: string) => c.toUpperCase());
|
return name.replace(/_/g, ' ').replace(/\b\w/g, (c: string) => c.toUpperCase());
|
||||||
}
|
}
|
||||||
@@ -264,27 +264,27 @@ export function getActionTargetText(action: any): string {
|
|||||||
*/
|
*/
|
||||||
export function getActionInputLabel(action: any): string {
|
export function getActionInputLabel(action: any): string {
|
||||||
const act = getActionObject(action);
|
const act = getActionObject(action);
|
||||||
if (!act) return 'Input';
|
if (!act) return "输入";
|
||||||
const name = (act.name || act.action || '').toLowerCase();
|
const name = (act.name || act.action || '').toLowerCase();
|
||||||
if (name.includes('delay') || name.includes('wait')) {
|
if (name.includes('delay') || name.includes('wait')) {
|
||||||
return 'Duration';
|
return "耗时";
|
||||||
}
|
}
|
||||||
if (name === 'swipe' || name === 'scroll' || name === 'drag' || name === 'drag_and_drop') {
|
if (name === 'swipe' || name === 'scroll' || name === 'drag' || name === 'drag_and_drop') {
|
||||||
const actObj = getActionObject(action);
|
const actObj = getActionObject(action);
|
||||||
const args = actObj?.args && typeof actObj.args === 'object' ? actObj.args : {};
|
const args = actObj?.args && typeof actObj.args === 'object' ? actObj.args : {};
|
||||||
const dir = actObj?.direction || actObj?.gesture || args.direction || args.gesture || (typeof args.action === 'string' ? args.action : '') || (typeof actObj?.action === 'string' && actObj.action !== name ? actObj.action : '');
|
const dir = actObj?.direction || actObj?.gesture || args.direction || args.gesture || (typeof args.action === 'string' ? args.action : '') || (typeof actObj?.action === 'string' && actObj.action !== name ? actObj.action : '');
|
||||||
if (dir && isPureDirectionString(dir)) {
|
if (dir && isPureDirectionString(dir)) {
|
||||||
return 'Direction';
|
return "方向";
|
||||||
}
|
}
|
||||||
return 'Input';
|
return "输入";
|
||||||
}
|
}
|
||||||
if (name === 'press_key' || name.includes('key')) {
|
if (name === 'press_key' || name.includes('key')) {
|
||||||
return 'Key';
|
return "按键";
|
||||||
}
|
}
|
||||||
if (name === 'input_text' || name.includes('input')) {
|
if (name === 'input_text' || name.includes('input')) {
|
||||||
return 'Input Text';
|
return "输入文本";
|
||||||
}
|
}
|
||||||
return 'Input';
|
return "输入";
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -376,12 +376,12 @@ export function getActionCoords(action: any): string {
|
|||||||
*/
|
*/
|
||||||
function attemptsFailed(attempts: any): boolean {
|
function attemptsFailed(attempts: any): boolean {
|
||||||
if (!Array.isArray(attempts) || attempts.length === 0) return false;
|
if (!Array.isArray(attempts) || attempts.length === 0) return false;
|
||||||
return String(attempts[attempts.length - 1]) !== 'Dispatched';
|
return String(attempts[attempts.length - 1]) !== "已发送";
|
||||||
}
|
}
|
||||||
|
|
||||||
function lastFailedAttempt(attempts: any): string | null {
|
function lastFailedAttempt(attempts: any): string | null {
|
||||||
if (!Array.isArray(attempts)) return null;
|
if (!Array.isArray(attempts)) return null;
|
||||||
const failing = attempts.filter((a: any) => String(a) !== 'Dispatched');
|
const failing = attempts.filter((a: any) => String(a) !== "已发送");
|
||||||
return failing.length > 0 ? String(failing[failing.length - 1]) : null;
|
return failing.length > 0 ? String(failing[failing.length - 1]) : null;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -450,7 +450,7 @@ export function getActionErrorMessage(action: any, stepData?: any): string {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return 'Action Failed';
|
return "操作失败";
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -531,7 +531,7 @@ export function extractActionExtraParams(action: any, cache?: WeakMap<any, Actio
|
|||||||
// Explicitly check duration if available and not already formatted
|
// Explicitly check duration if available and not already formatted
|
||||||
const dur = mergedObj.duration || mergedObj.duration_ms;
|
const dur = mergedObj.duration || mergedObj.duration_ms;
|
||||||
if (dur !== undefined && dur !== null && dur !== '') {
|
if (dur !== undefined && dur !== null && dur !== '') {
|
||||||
result.push({ key: 'Duration', value: typeof dur === 'number' ? `${dur}ms` : String(dur) });
|
result.push({ key: "耗时", value: typeof dur === 'number' ? `${dur}ms` : String(dur) });
|
||||||
}
|
}
|
||||||
|
|
||||||
for (const [k, v] of Object.entries(mergedObj)) {
|
for (const [k, v] of Object.entries(mergedObj)) {
|
||||||
@@ -547,7 +547,7 @@ export function extractActionExtraParams(action: any, cache?: WeakMap<any, Actio
|
|||||||
}
|
}
|
||||||
|
|
||||||
let prettyKey = k.replace(/_/g, ' ').replace(/\b\w/g, c => c.toUpperCase());
|
let prettyKey = k.replace(/_/g, ' ').replace(/\b\w/g, c => c.toUpperCase());
|
||||||
if (lowerK === 'time_in_ms' || lowerK === 'delay_ms') prettyKey = 'Delay';
|
if (lowerK === 'time_in_ms' || lowerK === 'delay_ms') prettyKey = "延迟";
|
||||||
|
|
||||||
result.push({ key: prettyKey, value: valStr });
|
result.push({ key: prettyKey, value: valStr });
|
||||||
}
|
}
|
||||||
@@ -981,7 +981,7 @@ export function extractStepReplayFrames(logsOrSteps: any[]): StepReplayFrame[] {
|
|||||||
for (let i = 0; i < visualCandidates.length; i++) {
|
for (let i = 0; i < visualCandidates.length; i++) {
|
||||||
const { stepData, act, preUrl, postUrl, primaryImg } = visualCandidates[i];
|
const { stepData, act, preUrl, postUrl, primaryImg } = visualCandidates[i];
|
||||||
const stepNum = i + 1; // 1-indexed sequential frame number (1, 2, 3...)
|
const stepNum = i + 1; // 1-indexed sequential frame number (1, 2, 3...)
|
||||||
const title = act ? (getActionTitle(act) || 'Action') : `Step ${stepNum}`;
|
const title = act ? (getActionTitle(act) || "操作") : `步骤 ${stepNum}`;
|
||||||
const coords = act ? getActionCoords(act) : '';
|
const coords = act ? getActionCoords(act) : '';
|
||||||
const targetText = act ? getActionTargetText(act) : '';
|
const targetText = act ? getActionTargetText(act) : '';
|
||||||
const actionDesc = coords ? `${title} (${coords})` : (targetText ? `${title} (${targetText})` : title);
|
const actionDesc = coords ? `${title} (${coords})` : (targetText ? `${title} (${targetText})` : title);
|
||||||
@@ -995,7 +995,7 @@ export function extractStepReplayFrames(logsOrSteps: any[]): StepReplayFrame[] {
|
|||||||
stepNumber: stepNum,
|
stepNumber: stepNum,
|
||||||
rawStepNumber: rawStepNum,
|
rawStepNumber: rawStepNum,
|
||||||
stepId: String(stepData.step_id || `step-${stepNum}`),
|
stepId: String(stepData.step_id || `step-${stepNum}`),
|
||||||
title: `Step ${stepNum}: ${title}`,
|
title: `步骤 ${stepNum}: ${title}`,
|
||||||
actionText: actionDesc,
|
actionText: actionDesc,
|
||||||
action: act,
|
action: act,
|
||||||
actionType: act?.action || act?.name || 'action',
|
actionType: act?.action || act?.name || 'action',
|
||||||
|
|||||||
@@ -228,9 +228,9 @@ export function drawActionCoordinatesOnOverlay(
|
|||||||
// 2. Check separate start and end coordinate objects
|
// 2. Check separate start and end coordinate objects
|
||||||
const normStart = extractNumbersFromCoordinateValue(act.normalized_start_coordinates || args.normalized_start_coordinates);
|
const normStart = extractNumbersFromCoordinateValue(act.normalized_start_coordinates || args.normalized_start_coordinates);
|
||||||
const normEnd = extractNumbersFromCoordinateValue(act.normalized_end_coordinates || args.normalized_end_coordinates);
|
const normEnd = extractNumbersFromCoordinateValue(act.normalized_end_coordinates || args.normalized_end_coordinates);
|
||||||
const startCoords = normStart || extractNumbersFromCoordinateValue(act.start_coordinates || act.start_point || act.start || act.from ||
|
const startCoords = normStart || extractNumbersFromCoordinateValue(act.start_coordinates || act.start_point || act.start || act.from ||
|
||||||
args.start_coordinates || args.start_point || args.start || args.from);
|
args.start_coordinates || args.start_point || args.start || args.from);
|
||||||
const endCoords = normEnd || extractNumbersFromCoordinateValue(act.end_coordinates || act.end_point || act.end || act.to ||
|
const endCoords = normEnd || extractNumbersFromCoordinateValue(act.end_coordinates || act.end_point || act.end || act.to ||
|
||||||
args.end_coordinates || args.end_point || args.end || args.to);
|
args.end_coordinates || args.end_point || args.end || args.to);
|
||||||
|
|
||||||
if (startCoords && endCoords && startCoords.length >= 2 && endCoords.length >= 2) {
|
if (startCoords && endCoords && startCoords.length >= 2 && endCoords.length >= 2) {
|
||||||
@@ -320,12 +320,12 @@ export function drawActionCoordinatesOnOverlay(
|
|||||||
* Helper to create and position an animated red dot point marker with edge-aware label
|
* Helper to create and position an animated red dot point marker with edge-aware label
|
||||||
*/
|
*/
|
||||||
export function createPointMarker(
|
export function createPointMarker(
|
||||||
x: number,
|
x: number,
|
||||||
y: number,
|
y: number,
|
||||||
actionType: string,
|
actionType: string,
|
||||||
naturalWidth: number,
|
naturalWidth: number,
|
||||||
naturalHeight: number,
|
naturalHeight: number,
|
||||||
overlay: HTMLElement,
|
overlay: HTMLElement,
|
||||||
isNormalized: boolean = false,
|
isNormalized: boolean = false,
|
||||||
sequenceIndex?: number,
|
sequenceIndex?: number,
|
||||||
totalSequence?: number
|
totalSequence?: number
|
||||||
@@ -387,7 +387,7 @@ export function createPointMarker(
|
|||||||
|
|
||||||
const label = document.createElement('div');
|
const label = document.createElement('div');
|
||||||
if (sequenceIndex !== undefined) {
|
if (sequenceIndex !== undefined) {
|
||||||
label.innerText = totalSequence && totalSequence > 1 ? `#${sequenceIndex} TAP` : actionType.toUpperCase();
|
label.innerText = totalSequence && totalSequence > 1 ? `#${sequenceIndex} 点击` : actionType.toUpperCase();
|
||||||
} else {
|
} else {
|
||||||
label.innerText = actionType.toUpperCase();
|
label.innerText = actionType.toUpperCase();
|
||||||
}
|
}
|
||||||
@@ -527,13 +527,13 @@ export function createSequenceConnector(
|
|||||||
* Helper to create and position an SVG arrow, high-contrast dashed line, and badge for swipes/drags
|
* Helper to create and position an SVG arrow, high-contrast dashed line, and badge for swipes/drags
|
||||||
*/
|
*/
|
||||||
export function createLineMarker(
|
export function createLineMarker(
|
||||||
x1: number,
|
x1: number,
|
||||||
y1: number,
|
y1: number,
|
||||||
x2: number,
|
x2: number,
|
||||||
y2: number,
|
y2: number,
|
||||||
naturalWidth: number,
|
naturalWidth: number,
|
||||||
naturalHeight: number,
|
naturalHeight: number,
|
||||||
overlay: HTMLElement,
|
overlay: HTMLElement,
|
||||||
isNormalized: boolean = false,
|
isNormalized: boolean = false,
|
||||||
actionType: string = 'swipe'
|
actionType: string = 'swipe'
|
||||||
): void {
|
): void {
|
||||||
|
|||||||
@@ -199,7 +199,7 @@ export function extractCheckerResult(text: string): CheckerResult | null {
|
|||||||
if (parsed && typeof parsed === 'object' && 'success' in parsed) {
|
if (parsed && typeof parsed === 'object' && 'success' in parsed) {
|
||||||
return {
|
return {
|
||||||
success: Boolean(parsed.success),
|
success: Boolean(parsed.success),
|
||||||
reason: parsed.reason || 'No reason provided.'
|
reason: parsed.reason || "未提供原因。"
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
} catch {
|
} catch {
|
||||||
|
|||||||
@@ -62,10 +62,10 @@ describe('run-info.util', () => {
|
|||||||
|
|
||||||
describe('tuningLabel', () => {
|
describe('tuningLabel', () => {
|
||||||
it('maps ids to the launcher labels and defaults unknown ids', () => {
|
it('maps ids to the launcher labels and defaults unknown ids', () => {
|
||||||
expect(tuningLabel('verify', 'checkpoints')).toBe('Every step');
|
expect(tuningLabel('verify', 'checkpoints')).toBe('逐步检查');
|
||||||
expect(tuningLabel('verify', 'nonsense')).toBe('At the end');
|
expect(tuningLabel('verify', 'nonsense')).toBe('结束时检查');
|
||||||
expect(tuningLabel('explore', 'ultra')).toBe('Close-up');
|
expect(tuningLabel('explore', 'ultra')).toBe('精细识别');
|
||||||
expect(tuningLabel('explore', null)).toBe('Quick glance');
|
expect(tuningLabel('explore', null)).toBe('快速识别');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
|
|||||||
@@ -507,10 +507,10 @@ describe('stream aggregator single source of truth (Option A)', () => {
|
|||||||
it('does not duplicate action_taken when generic_tools already contains the action trace', () => {
|
it('does not duplicate action_taken when generic_tools already contains the action trace', () => {
|
||||||
const events = getSortedStepEvents({
|
const events = getSortedStepEvents({
|
||||||
timestamp: 100,
|
timestamp: 100,
|
||||||
action_taken: [{ action: 'launch_app', app_name: 'Maps', timestamp: 102 }],
|
action_taken: [{ action: 'launch_app', app_name: '地图', timestamp: 102 }],
|
||||||
generic_tools: [
|
generic_tools: [
|
||||||
{ trace_id: 'tool-note', type: 'tool', name: 'update_note', timestamp: 101 },
|
{ trace_id: 'tool-note', type: 'tool', name: 'update_note', timestamp: 101 },
|
||||||
{ trace_id: 'act-launch', type: 'action', name: 'launch_app', timestamp: 102, payload: { args: { action: { action: 'launch_app', app_name: 'Maps' } } } }
|
{ trace_id: 'act-launch', type: 'action', name: 'launch_app', timestamp: 102, payload: { args: { action: { action: 'launch_app', app_name: '地图' } } } }
|
||||||
]
|
]
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -565,7 +565,7 @@ describe('stream aggregator single source of truth (Option A)', () => {
|
|||||||
{ trace_id: 't-adb-1-sub', parent_trace_id: 't-adb-1', name: 'run_adb_command', type: 'tool', timestamp: 1788457631.5, status: 'success' },
|
{ trace_id: 't-adb-1-sub', parent_trace_id: 't-adb-1', name: 'run_adb_command', type: 'tool', timestamp: 1788457631.5, status: 'success' },
|
||||||
{ trace_id: 't-adb-2', parent_trace_id: 't-op', name: 'run_adb_command', type: 'tool', timestamp: 1788457632, status: 'success', payload: { args: { CommandLine: 'am start -a android.intent.action.SENDTO ...' } } },
|
{ trace_id: 't-adb-2', parent_trace_id: 't-op', name: 'run_adb_command', type: 'tool', timestamp: 1788457632, status: 'success', payload: { args: { CommandLine: 'am start -a android.intent.action.SENDTO ...' } } },
|
||||||
{ trace_id: 't-adb-2-sub', parent_trace_id: 't-adb-2', name: 'run_adb_command', type: 'tool', timestamp: 1788457632.5, status: 'success' },
|
{ trace_id: 't-adb-2-sub', parent_trace_id: 't-adb-2', name: 'run_adb_command', type: 'tool', timestamp: 1788457632.5, status: 'success' },
|
||||||
{ trace_id: 't-exp', parent_trace_id: 't-op', name: 'ask_explorer', type: 'tool', timestamp: 1788457633, status: 'success', payload: { args: { query: 'any text on screen' } } },
|
{ trace_id: 't-exp', parent_trace_id: 't-op', name: 'ask_explorer', type: 'tool', timestamp: 1788457633, status: 'success', payload: { args: { query: 'any text 在屏幕上' } } },
|
||||||
{ trace_id: 't-exp-sub', parent_trace_id: 't-exp', name: 'ask_explorer', type: 'tool', timestamp: 1788457633.5, status: 'success' },
|
{ trace_id: 't-exp-sub', parent_trace_id: 't-exp', name: 'ask_explorer', type: 'tool', timestamp: 1788457633.5, status: 'success' },
|
||||||
{ trace_id: 't-wait', parent_trace_id: 't-val', name: 'wait_for_delay', type: 'action', timestamp: 1788457635, status: 'success', payload: { args: { action: { action: 'wait_for_delay', time_in_ms: 1000 } } } },
|
{ trace_id: 't-wait', parent_trace_id: 't-val', name: 'wait_for_delay', type: 'action', timestamp: 1788457635, status: 'success', payload: { args: { action: { action: 'wait_for_delay', time_in_ms: 1000 } } } },
|
||||||
{ trace_id: 't-safety', parent_trace_id: 't-wait', name: 'safety_net_pixel_validation', type: 'tool', timestamp: 1788457636, status: 'success' }
|
{ trace_id: 't-safety', parent_trace_id: 't-wait', name: 'safety_net_pixel_validation', type: 'tool', timestamp: 1788457636, status: 'success' }
|
||||||
@@ -597,7 +597,7 @@ describe('stream aggregator single source of truth (Option A)', () => {
|
|||||||
text: 'Here is the aborted plan...',
|
text: 'Here is the aborted plan...',
|
||||||
isCompleted: false,
|
isCompleted: false,
|
||||||
isReset: true,
|
isReset: true,
|
||||||
resetMessage: 'A request error occurred during output generation, typically caused by lower API priority. Retrying automatically...'
|
resetMessage: '生成过程中请求出错,正在自动重试…'
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
];
|
];
|
||||||
@@ -605,7 +605,7 @@ describe('stream aggregator single source of truth (Option A)', () => {
|
|||||||
const initialBlocks = consolidateLogsToBlocks(resetLogs);
|
const initialBlocks = consolidateLogsToBlocks(resetLogs);
|
||||||
expect(initialBlocks.length).toBe(1);
|
expect(initialBlocks.length).toBe(1);
|
||||||
expect(initialBlocks[0].data.isReset).toBeTrue();
|
expect(initialBlocks[0].data.isReset).toBeTrue();
|
||||||
expect(initialBlocks[0].data.resetMessage).toBe('A request error occurred during output generation, typically caused by lower API priority. Retrying automatically...');
|
expect(initialBlocks[0].data.resetMessage).toBe('生成过程中请求出错,正在自动重试…');
|
||||||
|
|
||||||
// getSortedStepEvents retains the reset state and message
|
// getSortedStepEvents retains the reset state and message
|
||||||
const resetEvents = getSortedStepEvents(initialBlocks[0].data);
|
const resetEvents = getSortedStepEvents(initialBlocks[0].data);
|
||||||
@@ -637,4 +637,3 @@ describe('stream aggregator single source of truth (Option A)', () => {
|
|||||||
expect(updatedBlocks[0].data.isCompleted).toBeTrue();
|
expect(updatedBlocks[0].data.isCompleted).toBeTrue();
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -750,13 +750,13 @@ export function formatTokenCount(tokens?: number): string {
|
|||||||
export function groupBlocksToPhases(blocks: StepBlock[], sessionStartTime: number): PhaseBlock[] {
|
export function groupBlocksToPhases(blocks: StepBlock[], sessionStartTime: number): PhaseBlock[] {
|
||||||
if (!blocks || blocks.length === 0) return [];
|
if (!blocks || blocks.length === 0) return [];
|
||||||
const phases: PhaseBlock[] = [];
|
const phases: PhaseBlock[] = [];
|
||||||
|
|
||||||
// Base start timestamp in ms
|
// Base start timestamp in ms
|
||||||
let previousEndTime = sessionStartTime ? sessionStartTime * 1000 : 0;
|
let previousEndTime = sessionStartTime ? sessionStartTime * 1000 : 0;
|
||||||
|
|
||||||
blocks.forEach((block, index) => {
|
blocks.forEach((block, index) => {
|
||||||
const blockTime = block.timestamp ? new Date(block.timestamp).getTime() : Date.now();
|
const blockTime = block.timestamp ? new Date(block.timestamp).getTime() : Date.now();
|
||||||
|
|
||||||
if (previousEndTime === 0 && blockTime > 0) {
|
if (previousEndTime === 0 && blockTime > 0) {
|
||||||
previousEndTime = blockTime;
|
previousEndTime = blockTime;
|
||||||
}
|
}
|
||||||
@@ -791,7 +791,7 @@ export function groupBlocksToPhases(blocks: StepBlock[], sessionStartTime: numbe
|
|||||||
* Returns all actions and tools inside a step card, sorted in true chronological sequence by timestamp
|
* Returns all actions and tools inside a step card, sorted in true chronological sequence by timestamp
|
||||||
*/
|
*/
|
||||||
export function getSortedStepEvents(
|
export function getSortedStepEvents(
|
||||||
stepData: any,
|
stepData: any,
|
||||||
cache?: WeakMap<any, { signature: string; events: StepEvent[] }>
|
cache?: WeakMap<any, { signature: string; events: StepEvent[] }>
|
||||||
): StepEvent[] {
|
): StepEvent[] {
|
||||||
if (!stepData || typeof stepData !== 'object') return [];
|
if (!stepData || typeof stepData !== 'object') return [];
|
||||||
@@ -940,20 +940,20 @@ export function getSortedStepEvents(
|
|||||||
* Compile human-readable session summary from logs
|
* Compile human-readable session summary from logs
|
||||||
*/
|
*/
|
||||||
export function compileSessionSummary(logs: any[]): string {
|
export function compileSessionSummary(logs: any[]): string {
|
||||||
if (!logs || logs.length === 0) return 'No logs available.';
|
if (!logs || logs.length === 0) return "暂无日志。";
|
||||||
|
|
||||||
// Check for goal completed / checker response / cancellation / report_task_status
|
// Check for goal completed / checker response / cancellation / report_task_status
|
||||||
for (let i = logs.length - 1; i >= 0; i--) {
|
for (let i = logs.length - 1; i >= 0; i--) {
|
||||||
const log = logs[i];
|
const log = logs[i];
|
||||||
if (log.type === 'session_ended') {
|
if (log.type === 'session_ended') {
|
||||||
if (log.data?.status === 'cancelled' || log.data?.was_stopped_manually) {
|
if (log.data?.status === 'cancelled' || log.data?.was_stopped_manually) {
|
||||||
return 'Task stopped manually.';
|
return "任务已手动停止。";
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (log.type === 'llm_stream' && log.data?.text) {
|
if (log.type === 'llm_stream' && log.data?.text) {
|
||||||
const text = log.data.text;
|
const text = log.data.text;
|
||||||
if (text.includes('```json') && text.includes('"success"')) {
|
if (text.includes('```json') && text.includes('"success"')) {
|
||||||
return 'Execution completed and verified by checker.';
|
return "任务已执行完成,并通过检查。";
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
if (log.type === 'step_recorded' || log.type === 'step_updated') {
|
if (log.type === 'step_recorded' || log.type === 'step_updated') {
|
||||||
@@ -963,12 +963,12 @@ export function compileSessionSummary(logs: any[]): string {
|
|||||||
if (exp) return exp;
|
if (exp) return exp;
|
||||||
}
|
}
|
||||||
if (log.data?.status === 'completed') {
|
if (log.data?.status === 'completed') {
|
||||||
return log.data.message || 'Task completed successfully.';
|
return log.data.message || "任务已成功完成。";
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return 'Execution session in progress or ended.';
|
return "会话正在执行或已结束。";
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -986,8 +986,8 @@ export function computeSessionDuration(sessionStartTime: number, logs: any[]): n
|
|||||||
* Check if the planning loader should be visible (while running, waiting for next step/action and not actively typing a stream)
|
* Check if the planning loader should be visible (while running, waiting for next step/action and not actively typing a stream)
|
||||||
*/
|
*/
|
||||||
export function checkPlanningLoader(
|
export function checkPlanningLoader(
|
||||||
logs: any[],
|
logs: any[],
|
||||||
isRunning: boolean,
|
isRunning: boolean,
|
||||||
isCurrentRunningSession: boolean = true
|
isCurrentRunningSession: boolean = true
|
||||||
): boolean {
|
): boolean {
|
||||||
if (!isRunning || !isCurrentRunningSession) return false;
|
if (!isRunning || !isCurrentRunningSession) return false;
|
||||||
|
|||||||
@@ -84,10 +84,10 @@ describe('compress_history timeline line', () => {
|
|||||||
describe('phase (args.phase from the backend)', () => {
|
describe('phase (args.phase from the backend)', () => {
|
||||||
it('reads the declared phase and maps it to one short plain-language label', () => {
|
it('reads the declared phase and maps it to one short plain-language label', () => {
|
||||||
const cases: Array<[string, string, string]> = [
|
const cases: Array<[string, string, string]> = [
|
||||||
['running', 'summarizing', 'Summarizing this stretch'],
|
['running', 'summarizing', '正在总结此段记录'],
|
||||||
['running', 'ready', 'Summary ready; kept in reserve until the context fills up'],
|
['running', 'ready', '摘要已就绪,上下文接近容量上限时替换原文'],
|
||||||
['success', 'applied', 'Replaced this stretch with its summary'],
|
['success', 'applied', '已使用摘要替换此段记录'],
|
||||||
['failed', 'failed', 'Summary failed; full record kept']
|
['failed', 'failed', '摘要失败,保留完整记录']
|
||||||
];
|
];
|
||||||
for (const [status, phase, label] of cases) {
|
for (const [status, phase, label] of cases) {
|
||||||
const tool = trace(status, { start_step: 1, end_step: 4, phase });
|
const tool = trace(status, { start_step: 1, end_step: 4, phase });
|
||||||
@@ -129,7 +129,7 @@ describe('cleanErrorMessage', () => {
|
|||||||
});
|
});
|
||||||
|
|
||||||
it('does not present an empty LLM wrapper label as an error reason', () => {
|
it('does not present an empty LLM wrapper label as an error reason', () => {
|
||||||
expect(cleanErrorMessage('LLM Error:')).toBe('Unknown error');
|
expect(cleanErrorMessage('LLM Error:')).toBe('未知错误');
|
||||||
});
|
});
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -222,7 +222,7 @@ describe('video analysis timeline formatting', () => {
|
|||||||
|
|
||||||
expect(view?.outcome).toBe('complete');
|
expect(view?.outcome).toBe('complete');
|
||||||
expect(view?.reuse).toBe('full');
|
expect(view?.reuse).toBe('full');
|
||||||
expect(view?.title).toBe('Reused video analysis');
|
expect(view?.title).toBe('已复用视频分析');
|
||||||
expect(view?.requestedRange).toEqual({ start: 0, end: 42 });
|
expect(view?.requestedRange).toEqual({ start: 0, end: 42 });
|
||||||
});
|
});
|
||||||
|
|
||||||
@@ -259,7 +259,7 @@ describe('video analysis timeline formatting', () => {
|
|||||||
expect(result[0].trace_id).toBe('video-analysis-video-agent-1');
|
expect(result[0].trace_id).toBe('video-analysis-video-agent-1');
|
||||||
expect(result[0].payload.result.outcome).toBe('partial');
|
expect(result[0].payload.result.outcome).toBe('partial');
|
||||||
expect(result[0].payload.result.requested_range).toEqual({ start: 0, end: 60 });
|
expect(result[0].payload.result.requested_range).toEqual({ start: 0, end: 60 });
|
||||||
expect(getVideoAnalysisView(result[0])?.title).toBe('Video analysis partially completed');
|
expect(getVideoAnalysisView(result[0])?.title).toBe('视频分析部分完成');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('keeps video analyses from independent parent executions separate', () => {
|
it('keeps video analyses from independent parent executions separate', () => {
|
||||||
@@ -333,8 +333,8 @@ describe('getToolTargetText self-described targets', () => {
|
|||||||
it('prefers observed target_text over target_description', () => {
|
it('prefers observed target_text over target_description', () => {
|
||||||
expect(getToolTargetText({
|
expect(getToolTargetText({
|
||||||
name: 'click',
|
name: 'click',
|
||||||
payload: { args: { target_text: 'Play', target_description: 'play button' } }
|
payload: { args: { target_text: '播放', target_description: 'play button' } }
|
||||||
})).toBe('Play');
|
})).toBe('播放');
|
||||||
});
|
});
|
||||||
|
|
||||||
it('chains click_sequence target_descriptions with the coordinate arrow', () => {
|
it('chains click_sequence target_descriptions with the coordinate arrow', () => {
|
||||||
|
|||||||
@@ -138,7 +138,7 @@ function parseVideoResultText(text: string): Partial<VideoAnalysisView> {
|
|||||||
return { outcome: 'failed', summary: value };
|
return { outcome: 'failed', summary: value };
|
||||||
}
|
}
|
||||||
if (value.includes('Analysis is already in progress in another video agent')) {
|
if (value.includes('Analysis is already in progress in another video agent')) {
|
||||||
return { outcome: 'waiting', summary: 'Another video agent is already analyzing this evidence.' };
|
return { outcome: 'waiting', summary: "另一个视频智能体正在分析此段内容。" };
|
||||||
}
|
}
|
||||||
return { outcome: 'complete', summary: value };
|
return { outcome: 'complete', summary: value };
|
||||||
}
|
}
|
||||||
@@ -175,14 +175,14 @@ export function getVideoAnalysisView(tool: any): VideoAnalysisView | null {
|
|||||||
const completedCount = Number(structured.completed_count ?? completedRanges.length ?? 0);
|
const completedCount = Number(structured.completed_count ?? completedRanges.length ?? 0);
|
||||||
const totalCount = Number(structured.total_count ?? (completedRanges.length + failedRanges.length));
|
const totalCount = Number(structured.total_count ?? (completedRanges.length + failedRanges.length));
|
||||||
const titleByOutcome: Record<VideoAnalysisOutcome, string> = {
|
const titleByOutcome: Record<VideoAnalysisOutcome, string> = {
|
||||||
running: 'Analyzing screen recording',
|
running: "正在分析屏幕录像",
|
||||||
recovering: 'Analyzing unfinished recording segment',
|
recovering: "正在分析尚未结束的录像片段",
|
||||||
waiting: 'Waiting for existing video analysis',
|
waiting: "正在等待已有的视频分析",
|
||||||
complete: structured.reuse === 'full' || parsed.reuse === 'full'
|
complete: structured.reuse === 'full' || parsed.reuse === 'full'
|
||||||
? 'Reused video analysis'
|
? "已复用视频分析"
|
||||||
: 'Analyzed screen recording',
|
: "已分析屏幕录像",
|
||||||
partial: 'Video analysis partially completed',
|
partial: "视频分析部分完成",
|
||||||
failed: 'Video analysis returned no result'
|
failed: "视频分析未返回结果"
|
||||||
};
|
};
|
||||||
|
|
||||||
return {
|
return {
|
||||||
@@ -236,7 +236,7 @@ export function getVideoToolTarget(tool: any): string | null {
|
|||||||
* Canonical list of mobile device automation and ADB command tool names.
|
* Canonical list of mobile device automation and ADB command tool names.
|
||||||
*/
|
*/
|
||||||
export const DEVICE_ACTION_TOOL_NAMES = new Set([
|
export const DEVICE_ACTION_TOOL_NAMES = new Set([
|
||||||
'click', 'click_sequence', 'long_press', 'long_press_on', 'input', 'input_text',
|
'click', 'click_sequence', 'long_press', 'long_press_on', 'input', 'input_text',
|
||||||
'swipe', 'scroll', 'press_key', 'press_home', 'press_back', 'launch_app', 'manage_app', 'open_app',
|
'swipe', 'scroll', 'press_key', 'press_home', 'press_back', 'launch_app', 'manage_app', 'open_app',
|
||||||
'focus_and_input_text', 'focus_and_clear_text', 'tap', 'wait_for_delay', 'wait',
|
'focus_and_input_text', 'focus_and_clear_text', 'tap', 'wait_for_delay', 'wait',
|
||||||
'run_adb_command', 'run_short_adb_command'
|
'run_adb_command', 'run_short_adb_command'
|
||||||
@@ -558,9 +558,9 @@ export function getToolDisplayLabel(tool: any, isFirstSaveNote: boolean = false)
|
|||||||
case 'manage_app':
|
case 'manage_app':
|
||||||
case 'launch_app': {
|
case 'launch_app': {
|
||||||
const rawApp = args.app_name || args.package_name || args.app || '';
|
const rawApp = args.app_name || args.package_name || args.app || '';
|
||||||
const app = rawApp ? (rawApp.charAt(0).toUpperCase() + rawApp.slice(1)) : 'Application';
|
const app = rawApp ? (rawApp.charAt(0).toUpperCase() + rawApp.slice(1)) : "应用";
|
||||||
const rawAction = args.action ? String(args.action).toLowerCase() : '';
|
const rawAction = args.action ? String(args.action).toLowerCase() : '';
|
||||||
const verb = rawAction === 'launch' ? 'Launching' : (rawAction === 'stop' || rawAction === 'close' ? 'Stopping' : 'Managing');
|
const verb = rawAction === 'launch' ? "启动中" : (rawAction === 'stop' || rawAction === 'close' ? "停止中" : "管理中");
|
||||||
return `${verb} "${app}"`;
|
return `${verb} "${app}"`;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -568,100 +568,100 @@ export function getToolDisplayLabel(tool: any, isFirstSaveNote: boolean = false)
|
|||||||
case 'wait_delay':
|
case 'wait_delay':
|
||||||
case 'wait': {
|
case 'wait': {
|
||||||
const delay = args.delay_seconds || args.seconds || args.delay || args.duration;
|
const delay = args.delay_seconds || args.seconds || args.delay || args.duration;
|
||||||
return delay ? `Waiting for ${delay} second${Number(delay) > 1 ? 's' : ''}...` : 'Waiting for delay...';
|
return delay ? `等待 ${delay} second${Number(delay) > 1 ? 's' : ''}...` : "正在等待…";
|
||||||
}
|
}
|
||||||
case 'wait_for_text': {
|
case 'wait_for_text': {
|
||||||
const text = args.text || args.target_text || '';
|
const text = args.text || args.target_text || '';
|
||||||
return text ? `Waiting for text "${text}" to appear on screen` : 'Waiting for text on screen';
|
return text ? `等待文本 "${text}" 出现在屏幕上` : "正在等待屏幕出现文本";
|
||||||
}
|
}
|
||||||
|
|
||||||
case 'input_text':
|
case 'input_text':
|
||||||
case 'input': {
|
case 'input': {
|
||||||
const text = args.text || args.input_text || '';
|
const text = args.text || args.input_text || '';
|
||||||
return text ? `Entering text "${text}" into field` : 'Entering text into input field';
|
return text ? `输入文本 "${text}" 至输入框` : "正在向输入框输入文本";
|
||||||
}
|
}
|
||||||
case 'focus_and_input_text': {
|
case 'focus_and_input_text': {
|
||||||
const text = args.text || args.input_text || '';
|
const text = args.text || args.input_text || '';
|
||||||
return text ? `Focusing field and typing "${text}"` : 'Focusing field and entering text';
|
return text ? `聚焦输入框并输入 "${text}"` : "正在聚焦输入框并输入文本";
|
||||||
}
|
}
|
||||||
case 'focus_and_clear_text':
|
case 'focus_and_clear_text':
|
||||||
return 'Focusing and clearing field text';
|
return "正在聚焦输入框并清空文本";
|
||||||
|
|
||||||
case 'click':
|
case 'click':
|
||||||
case 'tap': {
|
case 'tap': {
|
||||||
const target = args.target_text || args.text || args.query || '';
|
const target = args.target_text || args.text || args.query || '';
|
||||||
return target ? `Tapping on "${target}"` : 'Tapping on screen element';
|
return target ? `点击 "${target}"` : "正在点击屏幕元素";
|
||||||
}
|
}
|
||||||
case 'click_sequence':
|
case 'click_sequence':
|
||||||
return 'Executing click sequence';
|
return "正在执行连续点击";
|
||||||
case 'long_press': {
|
case 'long_press': {
|
||||||
const target = args.target_text || args.text || '';
|
const target = args.target_text || args.text || '';
|
||||||
return target ? `Long pressing on "${target}"` : 'Long pressing screen element';
|
return target ? `长按 "${target}"` : "正在长按屏幕元素";
|
||||||
}
|
}
|
||||||
case 'swipe': {
|
case 'swipe': {
|
||||||
const dir = args.action || args.direction || '';
|
const dir = args.action || args.direction || '';
|
||||||
return dir ? `Swiping ${String(dir).toUpperCase()} on screen` : 'Swiping screen';
|
return dir ? `滑动 ${String(dir).toUpperCase()} 在屏幕上` : "正在滑动屏幕";
|
||||||
}
|
}
|
||||||
case 'press_key': {
|
case 'press_key': {
|
||||||
const key = args.key || args.keycode || '';
|
const key = args.key || args.keycode || '';
|
||||||
return key ? `Pressing key ${String(key).toUpperCase()}` : 'Pressing hardware key';
|
return key ? `按键 ${String(key).toUpperCase()}` : "正在按下实体按键";
|
||||||
}
|
}
|
||||||
|
|
||||||
case 'save_note':
|
case 'save_note':
|
||||||
return isFirstSaveNote ? 'Creating note' : 'Saving note';
|
return isFirstSaveNote ? "正在创建笔记" : "正在保存笔记";
|
||||||
case 'read_note':
|
case 'read_note':
|
||||||
return 'Reading note';
|
return "正在读取笔记";
|
||||||
case 'list_notes':
|
case 'list_notes':
|
||||||
return 'Browsing all saved notes';
|
return "正在浏览已保存笔记";
|
||||||
case 'update_note':
|
case 'update_note':
|
||||||
return 'Updating note';
|
return "正在更新笔记";
|
||||||
case 'append_note':
|
case 'append_note':
|
||||||
return 'Updating note';
|
return "正在更新笔记";
|
||||||
|
|
||||||
case 'object_detection': {
|
case 'object_detection': {
|
||||||
const q = Array.isArray(args.queries) ? args.queries.join(', ') : (args.queries || '');
|
const q = Array.isArray(args.queries) ? args.queries.join(', ') : (args.queries || '');
|
||||||
return q ? `Locating on screen: "${q}"` : 'Locating elements on screen';
|
return q ? `Locating 在屏幕上: "${q}"` : "正在定位屏幕元素";
|
||||||
}
|
}
|
||||||
case 'ask_explorer': {
|
case 'ask_explorer': {
|
||||||
const query = args.query || args.prompt || '';
|
const query = args.query || args.prompt || '';
|
||||||
return query ? `Searching on screen: "${query}"` : 'Searching on screen';
|
return query ? `Searching 在屏幕上: "${query}"` : "正在搜索屏幕";
|
||||||
}
|
}
|
||||||
case 'report_failure_analysis': {
|
case 'report_failure_analysis': {
|
||||||
const reason = args.reason || args.analysis || '';
|
const reason = args.reason || args.analysis || '';
|
||||||
return reason ? `Investigating issue: ${reason}` : 'Investigating execution issue';
|
return reason ? `排查问题:${reason}` : "正在排查执行问题";
|
||||||
}
|
}
|
||||||
case 'run_adb_command':
|
case 'run_adb_command':
|
||||||
case 'run_short_adb_command': {
|
case 'run_short_adb_command': {
|
||||||
const cmd = args.command || args.cmd || '';
|
const cmd = args.command || args.cmd || '';
|
||||||
return cmd ? `Running command: ${cmd}` : 'Running system command';
|
return cmd ? `运行命令:${cmd}` : "正在运行系统命令";
|
||||||
}
|
}
|
||||||
case 'search_logs':
|
case 'search_logs':
|
||||||
case 'read_logs': {
|
case 'read_logs': {
|
||||||
const q = args.query || args.filter || '';
|
const q = args.query || args.filter || '';
|
||||||
return q ? `Searching logs for "${q}"` : 'Analyzing system logs';
|
return q ? `搜索日志:"${q}"` : "正在分析系统日志";
|
||||||
}
|
}
|
||||||
case 'log_analyzer':
|
case 'log_analyzer':
|
||||||
case 'output_analyzer':
|
case 'output_analyzer':
|
||||||
return 'Analyzing logs';
|
return "正在分析日志";
|
||||||
case 'diagnoser':
|
case 'diagnoser':
|
||||||
case 'diagnose':
|
case 'diagnose':
|
||||||
return 'Diagnosing issue';
|
return "正在诊断问题";
|
||||||
case 'video_analyzer':
|
case 'video_analyzer':
|
||||||
case 'video_analyzer_pure':
|
case 'video_analyzer_pure':
|
||||||
return 'Analyzing screen recording';
|
return "正在分析屏幕录像";
|
||||||
case 'extract_segment_metadata': {
|
case 'extract_segment_metadata': {
|
||||||
const start = args.start_time !== undefined ? `${args.start_time}s` : '';
|
const start = args.start_time !== undefined ? `${args.start_time}s` : '';
|
||||||
const end = args.end_time !== undefined ? `${args.end_time}s` : '';
|
const end = args.end_time !== undefined ? `${args.end_time}s` : '';
|
||||||
const range = (start && end) ? ` (${start} - ${end})` : (start ? ` (from ${start})` : '');
|
const range = (start && end) ? ` (${start} - ${end})` : (start ? ` (from ${start})` : '');
|
||||||
return `Cropping screen recording segment${range}`;
|
return `截取录屏片段${range}`;
|
||||||
}
|
}
|
||||||
case 'spawn_sub_agent': {
|
case 'spawn_sub_agent': {
|
||||||
const q = args.specific_query || args.query || args.prompt || '';
|
const q = args.specific_query || args.query || args.prompt || '';
|
||||||
return q ? `Analyzing recording with sub-agent: "${q}"` : 'Analyzing recording with sub-agent';
|
return q ? `分析录屏:"${q}"` : "正在调用子智能体分析录像";
|
||||||
}
|
}
|
||||||
case 'analyze_audio_only': {
|
case 'analyze_audio_only': {
|
||||||
const q = args.specific_query || args.query || '';
|
const q = args.specific_query || args.query || '';
|
||||||
return q ? `Analyzing audio track: "${q}"` : 'Analyzing recording audio track';
|
return q ? `Analyzing audio track: "${q}"` : "正在分析录像音轨";
|
||||||
}
|
}
|
||||||
case 'search_history': {
|
case 'search_history': {
|
||||||
const q = args.query || '';
|
const q = args.query || '';
|
||||||
@@ -676,30 +676,30 @@ export function getToolDisplayLabel(tool: any, isFirstSaveNote: boolean = false)
|
|||||||
if (n !== undefined && n !== '' && end !== undefined && end !== null && end !== '' && String(end) !== String(n)) {
|
if (n !== undefined && n !== '' && end !== undefined && end !== null && end !== '' && String(end) !== String(n)) {
|
||||||
return `Reviewing steps ${n}–${end}`;
|
return `Reviewing steps ${n}–${end}`;
|
||||||
}
|
}
|
||||||
return n !== undefined && n !== '' ? `Reviewing step ${n}` : 'Reviewing step details';
|
return n !== undefined && n !== '' ? `Reviewing step ${n}` : "正在查看步骤详情";
|
||||||
}
|
}
|
||||||
case 'get_step_screenshot': {
|
case 'get_step_screenshot': {
|
||||||
const n = args.step_number;
|
const n = args.step_number;
|
||||||
const variant = String(args.which || '').toLowerCase();
|
const variant = String(args.which || '').toLowerCase();
|
||||||
if (variant === 'overlay') {
|
if (variant === 'overlay') {
|
||||||
return n !== undefined && n !== '' ? `Looking at where step ${n}'s action landed` : 'Looking at where an action landed';
|
return n !== undefined && n !== '' ? `Looking at where step ${n}'s action landed` : "正在查看操作位置";
|
||||||
}
|
}
|
||||||
const which = variant === 'post' ? 'after' : 'before';
|
const which = variant === 'post' ? 'after' : 'before';
|
||||||
return n !== undefined && n !== '' ? `Looking at the screen ${which} step ${n}` : 'Looking at a step screenshot';
|
return n !== undefined && n !== '' ? `Looking at the screen ${which} step ${n}` : "正在查看步骤截图";
|
||||||
}
|
}
|
||||||
case 'probe_device': {
|
case 'probe_device': {
|
||||||
const kind = args.kind ? String(args.kind).replace(/_/g, ' ') : '';
|
const kind = args.kind ? String(args.kind).replace(/_/g, ' ') : '';
|
||||||
return kind ? `Reading ${kind} from the device` : 'Reading device state';
|
return kind ? `Reading ${kind} from the device` : "正在读取设备状态";
|
||||||
}
|
}
|
||||||
case 'outputter':
|
case 'outputter':
|
||||||
case 'output_synthesis':
|
case 'output_synthesis':
|
||||||
return 'Synthesizing output report';
|
return "正在生成输出报告";
|
||||||
case 'web_search': {
|
case 'web_search': {
|
||||||
const q = args.query || '';
|
const q = args.query || '';
|
||||||
return q ? `Searching web for "${q}"` : 'Searching the web';
|
return q ? `Searching web for "${q}"` : "正在搜索网页";
|
||||||
}
|
}
|
||||||
case 'read_url':
|
case 'read_url':
|
||||||
return 'Fetching web page';
|
return "正在读取网页";
|
||||||
case 'compress_history':
|
case 'compress_history':
|
||||||
return getCompressionLabel(tool);
|
return getCompressionLabel(tool);
|
||||||
|
|
||||||
@@ -723,7 +723,7 @@ export function getCompressionLabel(tool: any): string {
|
|||||||
const hasRange = Number.isFinite(start) && Number.isFinite(end) && start > 0 && end > 0;
|
const hasRange = Number.isFinite(start) && Number.isFinite(end) && start > 0 && end > 0;
|
||||||
const range = hasRange
|
const range = hasRange
|
||||||
? (start === end ? `step ${start}` : `steps ${start}–${end}`)
|
? (start === end ? `step ${start}` : `steps ${start}–${end}`)
|
||||||
: 'earlier steps';
|
: "之前的步骤";
|
||||||
const rangeCapitalized = range.charAt(0).toUpperCase() + range.slice(1);
|
const rangeCapitalized = range.charAt(0).toUpperCase() + range.slice(1);
|
||||||
const status = String(tool?.status || '').toLowerCase();
|
const status = String(tool?.status || '').toLowerCase();
|
||||||
const phase = getCompressionPhase(tool);
|
const phase = getCompressionPhase(tool);
|
||||||
@@ -802,13 +802,13 @@ export function getCompressionPhase(tool: any): CompressionPhase {
|
|||||||
export function getCompressionPhaseLabel(tool: any): string {
|
export function getCompressionPhaseLabel(tool: any): string {
|
||||||
switch (getCompressionPhase(tool)) {
|
switch (getCompressionPhase(tool)) {
|
||||||
case 'ready':
|
case 'ready':
|
||||||
return 'Summary ready; kept in reserve until the context fills up';
|
return "摘要已就绪,上下文接近容量上限时替换原文";
|
||||||
case 'applied':
|
case 'applied':
|
||||||
return 'Replaced this stretch with its summary';
|
return "已使用摘要替换此段记录";
|
||||||
case 'failed':
|
case 'failed':
|
||||||
return 'Summary failed; full record kept';
|
return "摘要失败,保留完整记录";
|
||||||
default:
|
default:
|
||||||
return 'Summarizing this stretch';
|
return "正在总结此段记录";
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -898,76 +898,76 @@ export function getToolIcon(tool: any): string {
|
|||||||
* Get formatted title for a tool call card
|
* Get formatted title for a tool call card
|
||||||
*/
|
*/
|
||||||
export function getToolTitle(tool: any): string {
|
export function getToolTitle(tool: any): string {
|
||||||
if (!tool || !tool.name) return 'Tool Call';
|
if (!tool || !tool.name) return "工具调用";
|
||||||
const cleanName = tool.name.replace(/^(_)?exec_/, '');
|
const cleanName = tool.name.replace(/^(_)?exec_/, '');
|
||||||
const name = cleanName.toLowerCase();
|
const name = cleanName.toLowerCase();
|
||||||
switch (name) {
|
switch (name) {
|
||||||
case 'click':
|
case 'click':
|
||||||
case 'tap':
|
case 'tap':
|
||||||
return 'Tapping Element';
|
return "点击元素";
|
||||||
case 'click_sequence':
|
case 'click_sequence':
|
||||||
return 'Executing Click Sequence';
|
return "执行连续点击";
|
||||||
case 'long_press':
|
case 'long_press':
|
||||||
return 'Long Pressing Element';
|
return "长按元素";
|
||||||
case 'input_text':
|
case 'input_text':
|
||||||
case 'input':
|
case 'input':
|
||||||
return 'Entering Text';
|
return "输入文本";
|
||||||
case 'swipe':
|
case 'swipe':
|
||||||
case 'scroll': {
|
case 'scroll': {
|
||||||
const args = getToolArgs(tool);
|
const args = getToolArgs(tool);
|
||||||
const dir = args.direction || args.gesture || (typeof args.action === 'string' ? args.action : '');
|
const dir = args.direction || args.gesture || (typeof args.action === 'string' ? args.action : '');
|
||||||
if (dir && isPureDirectionString(dir)) return `Swiping Screen (${String(dir).toUpperCase()})`;
|
if (dir && isPureDirectionString(dir)) return `滑动屏幕(${String(dir).toUpperCase()})`;
|
||||||
return 'Swiping Screen';
|
return "滑动屏幕";
|
||||||
}
|
}
|
||||||
case 'drag':
|
case 'drag':
|
||||||
case 'drag_and_drop':
|
case 'drag_and_drop':
|
||||||
return 'Dragging Screen';
|
return "拖动屏幕";
|
||||||
case 'press_key':
|
case 'press_key':
|
||||||
return 'Pressing Hardware Key';
|
return "按下实体按键";
|
||||||
case 'manage_app':
|
case 'manage_app':
|
||||||
case 'launch_app': {
|
case 'launch_app': {
|
||||||
const args = getToolArgs(tool);
|
const args = getToolArgs(tool);
|
||||||
const rawAction = args.action ? String(args.action).toLowerCase() : '';
|
const rawAction = args.action ? String(args.action).toLowerCase() : '';
|
||||||
if (rawAction === 'launch') return 'Launching Application';
|
if (rawAction === 'launch') return "启动应用";
|
||||||
if (rawAction === 'stop' || rawAction === 'close') return 'Stopping Application';
|
if (rawAction === 'stop' || rawAction === 'close') return "停止应用";
|
||||||
return 'Managing Application';
|
return "管理应用";
|
||||||
}
|
}
|
||||||
case 'wait_for_delay':
|
case 'wait_for_delay':
|
||||||
case 'wait_delay':
|
case 'wait_delay':
|
||||||
return 'Waiting for Delay';
|
return "等待";
|
||||||
case 'wait_for_text':
|
case 'wait_for_text':
|
||||||
return 'Waiting for Text';
|
return "等待文本";
|
||||||
case 'object_detection':
|
case 'object_detection':
|
||||||
return 'Locating Elements';
|
return "定位元素";
|
||||||
case 'ask_explorer':
|
case 'ask_explorer':
|
||||||
return 'Searching on Screen';
|
return "搜索屏幕";
|
||||||
case 'report_failure_analysis':
|
case 'report_failure_analysis':
|
||||||
return 'Investigating Issue';
|
return "排查问题";
|
||||||
case 'run_adb_command':
|
case 'run_adb_command':
|
||||||
case 'run_short_adb_command':
|
case 'run_short_adb_command':
|
||||||
return 'Running System Command';
|
return "运行系统命令";
|
||||||
case 'web_search':
|
case 'web_search':
|
||||||
return 'Web Search';
|
return "网页搜索";
|
||||||
case 'read_url':
|
case 'read_url':
|
||||||
return 'Fetching Web Page';
|
return "读取网页";
|
||||||
case 'search_logs':
|
case 'search_logs':
|
||||||
case 'read_logs':
|
case 'read_logs':
|
||||||
return 'Searching Logs';
|
return "搜索日志";
|
||||||
case 'log_analyzer':
|
case 'log_analyzer':
|
||||||
case 'output_analyzer':
|
case 'output_analyzer':
|
||||||
return 'Analyzing Logs';
|
return "分析日志";
|
||||||
case 'diagnoser':
|
case 'diagnoser':
|
||||||
case 'diagnose':
|
case 'diagnose':
|
||||||
return 'Diagnosing Issue';
|
return "诊断问题";
|
||||||
case 'video_analyzer':
|
case 'video_analyzer':
|
||||||
case 'video_analyzer_pure':
|
case 'video_analyzer_pure':
|
||||||
return 'Analyzing Screen Recording';
|
return "分析屏幕录像";
|
||||||
case 'extract_segment_metadata':
|
case 'extract_segment_metadata':
|
||||||
return 'Cropping Screen Recording';
|
return "裁剪屏幕录像";
|
||||||
case 'spawn_sub_agent':
|
case 'spawn_sub_agent':
|
||||||
return 'Delegating Video Analysis';
|
return "委派视频分析";
|
||||||
case 'analyze_audio_only':
|
case 'analyze_audio_only':
|
||||||
return 'Analyzing Audio Track';
|
return "分析音轨";
|
||||||
default:
|
default:
|
||||||
return cleanName.replace(/_/g, ' ').replace(/\b\w/g, (c: string) => c.toUpperCase());
|
return cleanName.replace(/_/g, ' ').replace(/\b\w/g, (c: string) => c.toUpperCase());
|
||||||
}
|
}
|
||||||
@@ -1046,26 +1046,26 @@ export function getToolTargetText(tool: any): string {
|
|||||||
* Get input label for tools
|
* Get input label for tools
|
||||||
*/
|
*/
|
||||||
export function getToolInputLabel(tool: any): string {
|
export function getToolInputLabel(tool: any): string {
|
||||||
if (!tool || !tool.name) return 'Input';
|
if (!tool || !tool.name) return "输入";
|
||||||
const name = tool.name.toLowerCase().replace(/^(_)?exec_/, '');
|
const name = tool.name.toLowerCase().replace(/^(_)?exec_/, '');
|
||||||
if (name === 'wait_for_delay' || name === 'wait_delay' || name === 'delay' || name === 'wait') {
|
if (name === 'wait_for_delay' || name === 'wait_delay' || name === 'delay' || name === 'wait') {
|
||||||
return 'Duration';
|
return "耗时";
|
||||||
}
|
}
|
||||||
if (name === 'swipe' || name === 'scroll' || name === 'drag' || name === 'drag_and_drop') {
|
if (name === 'swipe' || name === 'scroll' || name === 'drag' || name === 'drag_and_drop') {
|
||||||
const args = getToolArgs(tool);
|
const args = getToolArgs(tool);
|
||||||
const dir = args.direction || args.gesture || (typeof args.action === 'string' ? args.action : '');
|
const dir = args.direction || args.gesture || (typeof args.action === 'string' ? args.action : '');
|
||||||
if (dir && isPureDirectionString(dir)) {
|
if (dir && isPureDirectionString(dir)) {
|
||||||
return 'Direction';
|
return "方向";
|
||||||
}
|
}
|
||||||
return 'Input';
|
return "输入";
|
||||||
}
|
}
|
||||||
if (name === 'press_key' || name === 'press_home' || name === 'press_back') {
|
if (name === 'press_key' || name === 'press_home' || name === 'press_back') {
|
||||||
return 'Key';
|
return "按键";
|
||||||
}
|
}
|
||||||
if (name === 'input_text' || name === 'input') {
|
if (name === 'input_text' || name === 'input') {
|
||||||
return 'Input Text';
|
return "输入文本";
|
||||||
}
|
}
|
||||||
return 'Input';
|
return "输入";
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -1220,14 +1220,14 @@ export function isToolFailed(tool: any): boolean {
|
|||||||
* Get error message for a failed tool
|
* Get error message for a failed tool
|
||||||
*/
|
*/
|
||||||
export function getToolErrorMessage(tool: any): string {
|
export function getToolErrorMessage(tool: any): string {
|
||||||
if (!tool) return 'Tool Failed';
|
if (!tool) return "工具失败";
|
||||||
const args = getToolArgs(tool);
|
const args = getToolArgs(tool);
|
||||||
if (args.status === 'cannot_fix') return 'Status: cannot_fix';
|
if (args.status === 'cannot_fix') return 'Status: cannot_fix';
|
||||||
if (args.error || args.message || args.failure_reason) {
|
if (args.error || args.message || args.failure_reason) {
|
||||||
return args.error || args.message || args.failure_reason;
|
return args.error || args.message || args.failure_reason;
|
||||||
}
|
}
|
||||||
if (tool.error || tool.message) return tool.error || tool.message;
|
if (tool.error || tool.message) return tool.error || tool.message;
|
||||||
return 'Action Failed';
|
return "操作失败";
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -1286,7 +1286,7 @@ export function extractToolExtraParams(toolData: any, cache?: WeakMap<any, Actio
|
|||||||
}
|
}
|
||||||
|
|
||||||
let prettyKey = k.replace(/_/g, ' ').replace(/\b\w/g, c => c.toUpperCase());
|
let prettyKey = k.replace(/_/g, ' ').replace(/\b\w/g, c => c.toUpperCase());
|
||||||
if (lowerK === 'time_in_ms' || lowerK === 'delay_ms') prettyKey = 'Delay';
|
if (lowerK === 'time_in_ms' || lowerK === 'delay_ms') prettyKey = "延迟";
|
||||||
|
|
||||||
result.push({ key: prettyKey, value: valStr });
|
result.push({ key: prettyKey, value: valStr });
|
||||||
}
|
}
|
||||||
@@ -1363,7 +1363,7 @@ export function cleanErrorMessage(rawError: any): string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const errorStr = String(rawError).trim();
|
const errorStr = String(rawError).trim();
|
||||||
if (!errorStr) return 'Unknown error';
|
if (!errorStr) return "未知错误";
|
||||||
|
|
||||||
// 1. Try regex extraction for "message": "..."
|
// 1. Try regex extraction for "message": "..."
|
||||||
const doubleQuoteMsgMatch = errorStr.match(/"message"\s*:\s*"((?:[^"\\]|\\.)*)"/i);
|
const doubleQuoteMsgMatch = errorStr.match(/"message"\s*:\s*"((?:[^"\\]|\\.)*)"/i);
|
||||||
@@ -1415,5 +1415,5 @@ export function cleanErrorMessage(rawError: any): string {
|
|||||||
.replace(/^(?:LLM\s+(?:Request\s+)?Error\s*:\s*)+/i, '')
|
.replace(/^(?:LLM\s+(?:Request\s+)?Error\s*:\s*)+/i, '')
|
||||||
.trim();
|
.trim();
|
||||||
|
|
||||||
return fallback || 'Unknown error';
|
return fallback || "未知错误";
|
||||||
}
|
}
|
||||||
@@ -0,0 +1,17 @@
|
|||||||
|
import { Pipe, PipeTransform } from '@angular/core';
|
||||||
|
|
||||||
|
const LABELS: Record<string, string> = {
|
||||||
|
running: '执行中', queued: '排队中', pending: '等待中', completed: '已完成',
|
||||||
|
success: '成功', succeeded: '成功', failed: '失败', error: '错误', paused: '已暂停',
|
||||||
|
cancelled: '已取消', canceled: '已取消', stopped: '已停止', unknown: '未知',
|
||||||
|
idle: '空闲', starting: '启动中', launching: '启动中', ready: '就绪',
|
||||||
|
pass: '通过', fail: '失败', warn: '警告', warning: '警告', blocked: '受阻',
|
||||||
|
done: '完成', active: '当前', waiting: '等待中', processing: '处理中',
|
||||||
|
};
|
||||||
|
|
||||||
|
@Pipe({ name: 'uiLabel', standalone: true })
|
||||||
|
export class UiLabelPipe implements PipeTransform {
|
||||||
|
transform(value: string | null | undefined): string {
|
||||||
|
return LABELS[(value || '').toLowerCase()] || value || '';
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -15,7 +15,7 @@
|
|||||||
limitations under the License.
|
limitations under the License.
|
||||||
-->
|
-->
|
||||||
|
|
||||||
<html lang="en">
|
<html lang="zh-CN">
|
||||||
<head>
|
<head>
|
||||||
<meta charset="utf-8">
|
<meta charset="utf-8">
|
||||||
<title>artemis</title>
|
<title>artemis</title>
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
{
|
{
|
||||||
"system": "You are a precise, evidence-driven verification expert (Checker). You audit declared check items against recorded execution evidence and issue verdicts. You never operate the device, never modify any state, and never negotiate with the executing agent — you only observe, probe read-only state where allowed, and judge.",
|
"system": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。\n\nYou are a precise, evidence-driven verification expert (Checker). You audit declared check items against recorded execution evidence and issue verdicts. You never operate the device, never modify any state, and never negotiate with the executing agent — you only observe, probe read-only state where allowed, and judge.",
|
||||||
"base_rules": "# Verdict Rules (always apply)\n- Quote every check item verbatim in `item_text` — never paraphrase.\n- Every verdict MUST carry concrete evidence: a specific probe output, a specific note excerpt, or a specific on-screen/screenshot observation. A `failed` verdict without concrete evidence is invalid and will be downgraded.\n- `inconclusive` means the evidence needed for a judgment was never captured or is genuinely ambiguous. When evidence was never collected, output `inconclusive` and state explicitly that the evidence is missing — do NOT output `failed` for lack of evidence.\n- Judge only what the check item declares. Do not invent additional criteria and do not relax the declared ones.",
|
"base_rules": "# Verdict Rules (always apply)\n- Quote every check item verbatim in `item_text` — never paraphrase.\n- Every verdict MUST carry concrete evidence: a specific probe output, a specific note excerpt, or a specific on-screen/screenshot observation. A `failed` verdict without concrete evidence is invalid and will be downgraded.\n- `inconclusive` means the evidence needed for a judgment was never captured or is genuinely ambiguous. When evidence was never collected, output `inconclusive` and state explicitly that the evidence is missing — do NOT output `failed` for lack of evidence.\n- Judge only what the check item declares. Do not invent additional criteria and do not relax the declared ones.",
|
||||||
"verify_semantics": "# `verify` items (completion acceptance criteria)\n- A `verify` item asks: was the anchored subgoal actually accomplished to this standard?\n- A `failed` verify verdict can send its milestone back for repair, so it must include a `suggestion`: a concrete, actionable hint telling the executor what is missing or wrong.\n- The release decision (whether execution proceeds) is computed outside of you and is fail-open on errors; your job is only the truthful verdict value.",
|
"verify_semantics": "# `verify` items (completion acceptance criteria)\n- A `verify` item asks: was the anchored subgoal actually accomplished to this standard?\n- A `failed` verify verdict can send its milestone back for repair, so it must include a `suggestion`: a concrete, actionable hint telling the executor what is missing or wrong.\n- The release decision (whether execution proceeds) is computed outside of you and is fail-open on errors; your job is only the truthful verdict value.",
|
||||||
"assert_semantics": "# `assert` items (test assertions)\n- An `assert` item is a test expectation, not an instruction. A failed assertion is a legitimate, valuable test result: record it truthfully with evidence. Nothing will (or should) try to repair it.\n- Never mark an assert `failed` merely because evidence is absent — absence of captured evidence is `inconclusive` with an explicit note that the evidence is missing.\n- Precondition first: before judging an assert, confirm from the execution history (step replay, notes, anchor evidence) that the action(s) meant to trigger the asserted behavior were actually and correctly performed by the executor (the toggle was really tapped, the intended screen was really reached).\n- If that precondition is not established — the executor never performed the trigger, performed the wrong action, or the evidence cannot show it — output `inconclusive` and state explicitly that the precondition failed / the triggering action is not confirmed as executed. Never output `failed` here: a failed assert is attributed to the device under test, and an execution problem must not be reported as a device defect.\n- Output `failed` only when the precondition is established AND the device state, probe output, or recorded evidence still contradicts the expectation. The evidence must cite the concrete step(s) that performed the trigger plus the probe output / observation that contradicts the expectation, so the verdict reads as a device-defect diagnosis.\n- `suggestion` is not needed for assert verdicts.",
|
"assert_semantics": "# `assert` items (test assertions)\n- An `assert` item is a test expectation, not an instruction. A failed assertion is a legitimate, valuable test result: record it truthfully with evidence. Nothing will (or should) try to repair it.\n- Never mark an assert `failed` merely because evidence is absent — absence of captured evidence is `inconclusive` with an explicit note that the evidence is missing.\n- Precondition first: before judging an assert, confirm from the execution history (step replay, notes, anchor evidence) that the action(s) meant to trigger the asserted behavior were actually and correctly performed by the executor (the toggle was really tapped, the intended screen was really reached).\n- If that precondition is not established — the executor never performed the trigger, performed the wrong action, or the evidence cannot show it — output `inconclusive` and state explicitly that the precondition failed / the triggering action is not confirmed as executed. Never output `failed` here: a failed assert is attributed to the device under test, and an execution problem must not be reported as a device defect.\n- Output `failed` only when the precondition is established AND the device state, probe output, or recorded evidence still contradicts the expectation. The evidence must cite the concrete step(s) that performed the trigger plus the probe output / observation that contradicts the expectation, so the verdict reads as a device-defect diagnosis.\n- `suggestion` is not needed for assert verdicts.",
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"system_prompt": [
|
"system_prompt": [
|
||||||
"# Diagnoser",
|
"# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。\n\n# Diagnoser",
|
||||||
"",
|
"",
|
||||||
"## Objective",
|
"## Objective",
|
||||||
"You are an expert AI diagnoser. Your objective is to pinpoint the root cause of mobile execution failures by cross-referencing visual evidence (videos/screenshots) with system telemetry (logs). Precisely answer the `Diagnostic Query` based on the provided context.",
|
"You are an expert AI diagnoser. Your objective is to pinpoint the root cause of mobile execution failures by cross-referencing visual evidence (videos/screenshots) with system telemetry (logs). Precisely answer the `Diagnostic Query` based on the provided context.",
|
||||||
|
|||||||
@@ -22,6 +22,8 @@ import uuid
|
|||||||
|
|
||||||
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage, ToolMessage
|
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage, ToolMessage
|
||||||
|
|
||||||
|
from artemis.llm.structured import content_to_text
|
||||||
|
|
||||||
from artemis.core.tool_failure import is_tool_failure
|
from artemis.core.tool_failure import is_tool_failure
|
||||||
from artemis.context import ArtemisContext
|
from artemis.context import ArtemisContext
|
||||||
from artemis.data_engine.trace import (
|
from artemis.data_engine.trace import (
|
||||||
@@ -31,7 +33,7 @@ from artemis.data_engine.trace import (
|
|||||||
trace_langchain_tool,
|
trace_langchain_tool,
|
||||||
)
|
)
|
||||||
from artemis.graph.state import State
|
from artemis.graph.state import State
|
||||||
from artemis.llm.google import usage_from_message
|
from artemis.llm.usage import usage_from_message
|
||||||
from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
|
from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
|
||||||
from artemis.tools.command_tool import get_run_short_adb_command_tool
|
from artemis.tools.command_tool import get_run_short_adb_command_tool
|
||||||
from artemis.tools.diagnoser_submit_answer_tool import get_submit_answer_tool
|
from artemis.tools.diagnoser_submit_answer_tool import get_submit_answer_tool
|
||||||
@@ -568,7 +570,7 @@ class Diagnoser:
|
|||||||
response = await self._invoke_model(llm, current_messages, context_components)
|
response = await self._invoke_model(llm, current_messages, context_components)
|
||||||
|
|
||||||
if response.content:
|
if response.content:
|
||||||
outcome = response.content
|
outcome = content_to_text(response.content)
|
||||||
|
|
||||||
if not response.tool_calls:
|
if not response.tool_calls:
|
||||||
break
|
break
|
||||||
|
|||||||
@@ -23,7 +23,9 @@ Explorer tier the user configured. Tier behavior lives in
|
|||||||
ASK_EXPLORER_TOOL_NAME = "ask_explorer"
|
ASK_EXPLORER_TOOL_NAME = "ask_explorer"
|
||||||
|
|
||||||
#: Model used when the LLM configuration carries no Explorer entry.
|
#: Model used when the LLM configuration carries no Explorer entry.
|
||||||
DEFAULT_EXPLORER_MODEL = "gemini-3.8-flash"
|
from artemis.config.settings import settings
|
||||||
|
|
||||||
|
DEFAULT_EXPLORER_MODEL = settings.OPENAI_MODEL
|
||||||
|
|
||||||
ASK_EXPLORER_DESCRIPTION = (
|
ASK_EXPLORER_DESCRIPTION = (
|
||||||
"[EXPLORER] Ask the UI Explorer to locate elements on the current screen"
|
"[EXPLORER] Ask the UI Explorer to locate elements on the current screen"
|
||||||
|
|||||||
@@ -6,7 +6,7 @@
|
|||||||
// detect_objects, get_ocr_list, inspect_region, ask_image_processor.
|
// detect_objects, get_ocr_list, inspect_region, ask_image_processor.
|
||||||
// submit_answer is always available.
|
// submit_answer is always available.
|
||||||
{
|
{
|
||||||
"IDENTITY": "You are the agentic UI Explorer: given a phone screenshot and a query, you locate the matching UI element(s) and return their normalized [nx, ny] coordinates on a 0-1000 scale.",
|
"IDENTITY": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。\n\nYou are the agentic UI Explorer: given a phone screenshot and a query, you locate the matching UI element(s) and return their normalized [nx, ny] coordinates on a 0-1000 scale.",
|
||||||
"OPERATING PRINCIPLES": [
|
"OPERATING PRINCIPLES": [
|
||||||
"**Confidence-Driven Cost-Benefit Strategy**: Maximize localization accuracy while minimizing turns and tool calls. Every turn costs latency; stop exploring as soon as further calls would yield negligible accuracy gains.",
|
"**Confidence-Driven Cost-Benefit Strategy**: Maximize localization accuracy while minimizing turns and tool calls. Every turn costs latency; stop exploring as soon as further calls would yield negligible accuracy gains.",
|
||||||
"**Direct Submit (High Confidence)**: If the initial annotated screenshot and the marked UI element list already make the target coordinates obvious, call `submit_answer` immediately in the first turn.",
|
"**Direct Submit (High Confidence)**: If the initial annotated screenshot and the marked UI element list already make the target coordinates obvious, call `submit_answer` immediately in the first turn.",
|
||||||
|
|||||||
@@ -17,19 +17,16 @@
|
|||||||
The :class:`Explorer` class combines the following modules:
|
The :class:`Explorer` class combines the following modules:
|
||||||
|
|
||||||
- ``tiers``: engine, tools and budgets for flash / pro / ultra.
|
- ``tiers``: engine, tools and budgets for flash / pro / ultra.
|
||||||
- ``tool_declarations``: static tool schemas (universal + native).
|
- ``tool_declarations``: Responses function tool schemas.
|
||||||
- ``perception_tools``: the ``exec_*`` perception/vision tool method group.
|
- ``perception_tools``: the ``exec_*`` perception/vision tool method group.
|
||||||
- ``universal_runner``: the LangChain-based ``_run_universal`` loop.
|
- ``universal_runner``: the LangChain-based ``_run_universal`` loop.
|
||||||
- ``run_setup``: flash mode and the pre-loop setup phases of ``run``.
|
- ``run_setup``: flash mode and the pre-loop setup phases of ``run``.
|
||||||
- ``native_runner``: the native Gemini reasoning loop of ``run``.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import json
|
import json
|
||||||
import os
|
import os
|
||||||
from typing import Any, Literal
|
from typing import Any, Literal
|
||||||
|
|
||||||
from google import genai
|
|
||||||
from google.genai import types
|
|
||||||
|
|
||||||
from artemis.agents.explorer.constants import DEFAULT_EXPLORER_MODEL
|
from artemis.agents.explorer.constants import DEFAULT_EXPLORER_MODEL
|
||||||
from artemis.agents.explorer.geometry import (
|
from artemis.agents.explorer.geometry import (
|
||||||
@@ -38,18 +35,16 @@ from artemis.agents.explorer.geometry import (
|
|||||||
norm_to_pixel,
|
norm_to_pixel,
|
||||||
pixel_to_norm,
|
pixel_to_norm,
|
||||||
)
|
)
|
||||||
from artemis.agents.explorer.native_runner import NativeRunnerMixin
|
|
||||||
from artemis.agents.explorer.perception_tools import PerceptionToolsMixin
|
from artemis.agents.explorer.perception_tools import PerceptionToolsMixin
|
||||||
from artemis.agents.explorer.run_setup import RunSetupMixin
|
from artemis.agents.explorer.run_setup import RunSetupMixin
|
||||||
from artemis.agents.explorer.screen_index import normalize_text, ScreenElement, ScreenIndex
|
from artemis.agents.explorer.screen_index import normalize_text, ScreenElement, ScreenIndex
|
||||||
from artemis.agents.explorer.tiers import ExplorerTier, get_tier
|
from artemis.agents.explorer.tiers import ExplorerTier, get_tier
|
||||||
from artemis.agents.explorer.tool_declarations import NATIVE_EXPLORER_TOOL_DECLARATIONS
|
from artemis.agents.explorer.tool_declarations import UNIVERSAL_EXPLORER_TOOLS
|
||||||
from artemis.agents.explorer.universal_runner import UniversalRunnerMixin
|
from artemis.agents.explorer.universal_runner import UniversalRunnerMixin
|
||||||
from artemis.config import settings
|
from artemis.config import settings
|
||||||
from artemis.context import ArtemisContext
|
from artemis.context import ArtemisContext
|
||||||
from artemis.data_engine.trace import trace
|
from artemis.data_engine.trace import trace
|
||||||
from artemis.graph.state import State
|
from artemis.graph.state import State
|
||||||
from artemis.llm.google import is_gemini_model, strip_provider_prefix
|
|
||||||
from artemis.utils.logger import get_logger
|
from artemis.utils.logger import get_logger
|
||||||
from artemis.utils.ocr_api import is_ocr_configured
|
from artemis.utils.ocr_api import is_ocr_configured
|
||||||
|
|
||||||
@@ -57,28 +52,25 @@ logger = get_logger(__name__)
|
|||||||
|
|
||||||
__all__ = ["Explorer"]
|
__all__ = ["Explorer"]
|
||||||
|
|
||||||
#: Environment override for Gemini explicit context caching ("true" / "false").
|
|
||||||
CACHING_ENV_VAR = "ARTEMIS_EXPLORER_CACHING"
|
|
||||||
|
|
||||||
#: How far outside its labeled element (as a fraction of the screen, per side)
|
#: How far outside its labeled element (as a fraction of the screen, per side)
|
||||||
#: a candidate's point may fall and still inherit that element's bounds.
|
#: a candidate's point may fall and still inherit that element's bounds.
|
||||||
LABEL_TOLERANCE_RATIO = 0.02
|
LABEL_TOLERANCE_RATIO = 0.02
|
||||||
|
|
||||||
|
|
||||||
class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, NativeRunnerMixin):
|
class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin):
|
||||||
"""Locates UI elements on a screenshot for the ``ask_explorer`` tool.
|
"""Locates UI elements on a screenshot for the ``ask_explorer`` tool.
|
||||||
|
|
||||||
The tier (flash / pro / ultra) is applied per ``run`` call from
|
The tier (flash / pro / ultra) is applied per ``run`` call from
|
||||||
:data:`artemis.agents.explorer.tiers.EXPLORER_TIERS`; it decides the
|
:data:`artemis.agents.explorer.tiers.EXPLORER_TIERS`; it decides the
|
||||||
engine, the turn budget, the exposed perception tools and the caching
|
engine, the turn budget, the exposed perception tools and the caching
|
||||||
default. Construction is side-effect free: no network client is created
|
default. Construction is side-effect free: no network client is created
|
||||||
until the native Gemini branch actually needs one.
|
until the Responses execution loop needs one.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
DENYLIST_TEMPLATE = (
|
DENYLIST_TEMPLATE = (
|
||||||
"\n# TOOL DENYLIST\n- The following tools are denylisted and cannot be used: {tools}\n"
|
"\n# TOOL DENYLIST\n- The following tools are denylisted and cannot be used: {tools}\n"
|
||||||
)
|
)
|
||||||
TOOLS = NATIVE_EXPLORER_TOOL_DECLARATIONS
|
TOOLS = UNIVERSAL_EXPLORER_TOOLS
|
||||||
|
|
||||||
def __init__(self, ctx: ArtemisContext):
|
def __init__(self, ctx: ArtemisContext):
|
||||||
self.ctx = ctx
|
self.ctx = ctx
|
||||||
@@ -107,7 +99,6 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
|
|||||||
self.denylisted_tools = self._compute_denylist(None)
|
self.denylisted_tools = self._compute_denylist(None)
|
||||||
|
|
||||||
self.model_name = self._resolve_model_name()
|
self.model_name = self._resolve_model_name()
|
||||||
self.use_native_gemini = self._detect_native_engine()
|
|
||||||
|
|
||||||
# ------------------------------------------------------------------ #
|
# ------------------------------------------------------------------ #
|
||||||
# Engine selection (no side effects)
|
# Engine selection (no side effects)
|
||||||
@@ -119,37 +110,7 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
|
|||||||
llm_cfg = getattr(llm_config, "explorer", None) if llm_config else None
|
llm_cfg = getattr(llm_config, "explorer", None) if llm_config else None
|
||||||
model = getattr(llm_cfg, "model", None) if llm_cfg else None
|
model = getattr(llm_cfg, "model", None) if llm_cfg else None
|
||||||
model_name = model if isinstance(model, str) and model else DEFAULT_EXPLORER_MODEL
|
model_name = model if isinstance(model, str) and model else DEFAULT_EXPLORER_MODEL
|
||||||
return strip_provider_prefix(model_name)
|
return model_name
|
||||||
|
|
||||||
def _detect_native_engine(self) -> bool:
|
|
||||||
"""True when the native google-genai SDK should drive the reasoning loop.
|
|
||||||
|
|
||||||
A Gemini model alone is not enough: without a Google API key (or a
|
|
||||||
client already shared on the context) the SDK cannot be constructed,
|
|
||||||
so such configurations run through the universal LangChain engine.
|
|
||||||
"""
|
|
||||||
if not is_gemini_model(self.model_name):
|
|
||||||
return False
|
|
||||||
if getattr(self.ctx, "_genai_client", None) is not None:
|
|
||||||
return True
|
|
||||||
api_key = getattr(settings, "GOOGLE_API_KEY", None)
|
|
||||||
return bool(api_key and api_key.get_secret_value())
|
|
||||||
|
|
||||||
def _ensure_genai_client(self):
|
|
||||||
"""Returns the context-level GenAI client, creating it on first use.
|
|
||||||
|
|
||||||
Only the native branch calls this, so universal configurations never
|
|
||||||
touch ``genai`` (and never fail on a missing Google key). The client
|
|
||||||
is stored on the context for connection pooling across agents.
|
|
||||||
"""
|
|
||||||
client = getattr(self.ctx, "_genai_client", None)
|
|
||||||
if client is None:
|
|
||||||
logger.info("Initializing GenAI client on context for connection pooling (Explorer)")
|
|
||||||
api_key = getattr(settings, "GOOGLE_API_KEY", None)
|
|
||||||
client = genai.Client(api_key=api_key.get_secret_value() if api_key else None)
|
|
||||||
self.ctx._genai_client = client
|
|
||||||
self.client = client
|
|
||||||
return client
|
|
||||||
|
|
||||||
# ------------------------------------------------------------------ #
|
# ------------------------------------------------------------------ #
|
||||||
# Tier application
|
# Tier application
|
||||||
@@ -180,32 +141,6 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
|
|||||||
self.max_turns = tier.max_turns
|
self.max_turns = tier.max_turns
|
||||||
self.denylisted_tools = self._compute_denylist(tier)
|
self.denylisted_tools = self._compute_denylist(tier)
|
||||||
|
|
||||||
def _resolve_caching(self, tier: ExplorerTier, enable_caching: bool | None) -> bool:
|
|
||||||
"""Resolves Gemini explicit context caching for this run.
|
|
||||||
|
|
||||||
Precedence: explicit argument, ``ARTEMIS_EXPLORER_CACHING`` env
|
|
||||||
override, the agent configuration, the execution setup, the global
|
|
||||||
settings, then the tier default. Configuration values are only
|
|
||||||
trusted when they are real booleans so a mocked or malformed config
|
|
||||||
(``None``, sentinel objects) falls through instead of forcing a value.
|
|
||||||
"""
|
|
||||||
if isinstance(enable_caching, bool):
|
|
||||||
return enable_caching
|
|
||||||
env_value = os.getenv(CACHING_ENV_VAR, "").strip().lower()
|
|
||||||
if env_value in ("true", "false"):
|
|
||||||
return env_value == "true"
|
|
||||||
for source in (
|
|
||||||
getattr(self.ctx, "agent_config", None),
|
|
||||||
getattr(self.ctx, "execution_setup", None),
|
|
||||||
):
|
|
||||||
value = getattr(getattr(source, "explorer", None), "caching", None)
|
|
||||||
if isinstance(value, bool):
|
|
||||||
return value
|
|
||||||
value = getattr(settings, "EXPLORER_CACHING", None)
|
|
||||||
if isinstance(value, bool):
|
|
||||||
return value
|
|
||||||
return tier.caching
|
|
||||||
|
|
||||||
def _hidden_tool_names(self) -> set[str]:
|
def _hidden_tool_names(self) -> set[str]:
|
||||||
"""Denylisted tools plus OCR when it is unconfigured, evaluated at call time.
|
"""Denylisted tools plus OCR when it is unconfigured, evaluated at call time.
|
||||||
|
|
||||||
@@ -218,34 +153,14 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
|
|||||||
hidden.add("get_ocr_list")
|
hidden.add("get_ocr_list")
|
||||||
return hidden
|
return hidden
|
||||||
|
|
||||||
def get_exposed_tools(self, only_submit: bool = False) -> list[types.FunctionDeclaration]:
|
def get_exposed_tools(self, only_submit: bool = False) -> list[dict[str, Any]]:
|
||||||
"""Native tool declarations the model may see for the current tier."""
|
"""Native tool declarations the model may see for the current tier."""
|
||||||
hidden = self._hidden_tool_names()
|
hidden = self._hidden_tool_names()
|
||||||
tools = [tool for tool in self.TOOLS if tool.name not in hidden]
|
tools = [tool for tool in self.TOOLS if tool["function"]["name"] not in hidden]
|
||||||
if only_submit:
|
if only_submit:
|
||||||
tools = [tool for tool in tools if tool.name == "submit_answer"]
|
tools = [tool for tool in tools if tool["function"]["name"] == "submit_answer"]
|
||||||
return tools
|
return tools
|
||||||
|
|
||||||
def _prune_historical_images(self, contents, keep_last=1):
|
|
||||||
image_parts = []
|
|
||||||
for content in contents:
|
|
||||||
for part in content.parts:
|
|
||||||
if getattr(part, "inline_data", None) or getattr(part, "file_data", None):
|
|
||||||
image_parts.append(part)
|
|
||||||
if len(image_parts) <= keep_last:
|
|
||||||
return
|
|
||||||
to_keep_ids = {id(p) for p in image_parts[-keep_last:]}
|
|
||||||
for content in contents:
|
|
||||||
content.parts = [
|
|
||||||
types.Part.from_text(
|
|
||||||
text=("[Image pruned to maintain visual focus on latest state]")
|
|
||||||
)
|
|
||||||
if (getattr(p, "inline_data", None) or getattr(p, "file_data", None))
|
|
||||||
and id(p) not in to_keep_ids
|
|
||||||
else p
|
|
||||||
for p in content.parts
|
|
||||||
]
|
|
||||||
|
|
||||||
# ------------------------------------------------------------------ #
|
# ------------------------------------------------------------------ #
|
||||||
# Label bounds and the structural pre-pass
|
# Label bounds and the structural pre-pass
|
||||||
# ------------------------------------------------------------------ #
|
# ------------------------------------------------------------------ #
|
||||||
@@ -434,30 +349,13 @@ class Explorer(PerceptionToolsMixin, UniversalRunnerMixin, RunSetupMixin, Native
|
|||||||
fused_xml, screenshot_path, minimal_list, self.image_name
|
fused_xml, screenshot_path, minimal_list, self.image_name
|
||||||
)
|
)
|
||||||
|
|
||||||
if not self.use_native_gemini:
|
return await self._run_universal(
|
||||||
return await self._run_universal(
|
|
||||||
query=query,
|
|
||||||
context_feedback=context_feedback,
|
|
||||||
screenshot_path=img_to_read,
|
|
||||||
state=state,
|
|
||||||
minimal_list=minimal_list,
|
|
||||||
prompt_template=prompt_template,
|
|
||||||
max_turns=self.max_turns,
|
|
||||||
)
|
|
||||||
|
|
||||||
client = self._ensure_genai_client()
|
|
||||||
model_name, temperature, thinking_level, _fallback_model = self._resolve_model_params()
|
|
||||||
return await self._run_native(
|
|
||||||
client=client,
|
|
||||||
query=query,
|
query=query,
|
||||||
context_feedback=context_feedback,
|
context_feedback=context_feedback,
|
||||||
|
screenshot_path=img_to_read,
|
||||||
|
state=state,
|
||||||
minimal_list=minimal_list,
|
minimal_list=minimal_list,
|
||||||
img_to_read=img_to_read,
|
|
||||||
enable_caching=self._resolve_caching(tier, enable_caching),
|
|
||||||
prompt_template=prompt_template,
|
prompt_template=prompt_template,
|
||||||
model_name=model_name,
|
|
||||||
temperature=temperature,
|
|
||||||
thinking_level=thinking_level,
|
|
||||||
max_turns=self.max_turns,
|
max_turns=self.max_turns,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -1,878 +0,0 @@
|
|||||||
# Copyright 2026 Google LLC
|
|
||||||
#
|
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
||||||
# you may not use this file except in compliance with the License.
|
|
||||||
# You may obtain a copy of the License at
|
|
||||||
#
|
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
#
|
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
# See the License for the specific language governing permissions and
|
|
||||||
# limitations under the License.
|
|
||||||
|
|
||||||
"""Native Gemini execution loop of ``Explorer.run``.
|
|
||||||
|
|
||||||
Split out of ``artemis.agents.explorer.explorer``: the native-SDK reasoning
|
|
||||||
loop and its named phases (model invocation, submit_answer interception and
|
|
||||||
validation, tool dispatching, failure mapping, resource cleanup), packaged as
|
|
||||||
a mixin consumed by ``Explorer``. Every google-genai call goes through the
|
|
||||||
SDK's ``client.aio`` surface so uploads, cache management and generation
|
|
||||||
never block the event loop. This module also owns
|
|
||||||
:func:`_generate_content_with_reliability`, the shared reliability wrapper
|
|
||||||
for native google-genai model calls.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from collections.abc import Awaitable, Callable
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import time
|
|
||||||
from typing import TYPE_CHECKING, Any
|
|
||||||
|
|
||||||
from google.genai import types
|
|
||||||
|
|
||||||
from artemis.agents.explorer.geometry import is_valid_norm_point
|
|
||||||
from artemis.agents.explorer.tiers import SUBMIT_TOOL
|
|
||||||
from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
|
|
||||||
from artemis.data_engine.trace import TraceSpan
|
|
||||||
from artemis.llm.reliability import (
|
|
||||||
LLMExhaustedError,
|
|
||||||
LLMPermanentError,
|
|
||||||
classify_failure,
|
|
||||||
retry_policy_for,
|
|
||||||
)
|
|
||||||
from artemis.services.llm import _record_llm_event, _record_llm_retry
|
|
||||||
from artemis.utils.logger import get_logger
|
|
||||||
|
|
||||||
logger = get_logger(__name__)
|
|
||||||
|
|
||||||
#: Loads a local image as a ``Part`` (inline bytes or a File API reference).
|
|
||||||
ImagePartGetter = Callable[[str], Awaitable[types.Part]]
|
|
||||||
|
|
||||||
|
|
||||||
async def _generate_content_with_reliability(operation, *, label: str = "Explorer model call"):
|
|
||||||
"""Run one native google-genai model call under the shared reliability layer.
|
|
||||||
|
|
||||||
``operation`` must return a fresh awaitable on each call. Non-retryable
|
|
||||||
failures raise ``LLMPermanentError``; exhausted retries raise
|
|
||||||
``LLMExhaustedError`` according to the shared category policy.
|
|
||||||
"""
|
|
||||||
attempt = 0
|
|
||||||
while True:
|
|
||||||
try:
|
|
||||||
return await operation()
|
|
||||||
except Exception as call_err:
|
|
||||||
attempt += 1
|
|
||||||
failure = classify_failure(call_err)
|
|
||||||
if not failure.retryable:
|
|
||||||
logger.error(f"{label} permanently failed [{failure.category.value}]: {call_err}")
|
|
||||||
_record_llm_event(
|
|
||||||
"llm_gave_up",
|
|
||||||
{
|
|
||||||
"source": "explorer",
|
|
||||||
"error": str(call_err)[:1000],
|
|
||||||
"category": failure.category.value,
|
|
||||||
"retryable": False,
|
|
||||||
},
|
|
||||||
status="failed",
|
|
||||||
)
|
|
||||||
raise LLMPermanentError(
|
|
||||||
f"{label} failed [{failure.category.value}]: {call_err}",
|
|
||||||
failure=failure,
|
|
||||||
cause=call_err,
|
|
||||||
) from call_err
|
|
||||||
policy = retry_policy_for(failure.category)
|
|
||||||
if attempt >= policy.max_attempts:
|
|
||||||
logger.error(
|
|
||||||
f"{label} exhausted {attempt} attempt(s) [{failure.category.value}]: {call_err}"
|
|
||||||
)
|
|
||||||
_record_llm_event(
|
|
||||||
"llm_gave_up",
|
|
||||||
{
|
|
||||||
"source": "explorer",
|
|
||||||
"error": str(call_err)[:1000],
|
|
||||||
"category": failure.category.value,
|
|
||||||
"retryable": True,
|
|
||||||
"attempts": attempt,
|
|
||||||
},
|
|
||||||
status="failed",
|
|
||||||
)
|
|
||||||
raise LLMExhaustedError(
|
|
||||||
f"{label} exhausted {attempt} attempt(s)"
|
|
||||||
f" [{failure.category.value}]: {call_err}",
|
|
||||||
failure=failure,
|
|
||||||
cause=call_err,
|
|
||||||
) from call_err
|
|
||||||
delay = policy.delay_for(attempt)
|
|
||||||
logger.warning(
|
|
||||||
f"{label} failed [{failure.category.value}] on attempt"
|
|
||||||
f" {attempt}/{policy.max_attempts}: {call_err}."
|
|
||||||
f" Retrying in {delay:.2f}s..."
|
|
||||||
)
|
|
||||||
_record_llm_retry(
|
|
||||||
str(call_err),
|
|
||||||
delay,
|
|
||||||
attempt=attempt,
|
|
||||||
max_retries=policy.max_attempts,
|
|
||||||
source="explorer",
|
|
||||||
)
|
|
||||||
await asyncio.sleep(delay)
|
|
||||||
|
|
||||||
|
|
||||||
def _bare_tool_name(name: str) -> str:
|
|
||||||
"""Strips a namespace prefix (``ns:tool``) some models add to tool names."""
|
|
||||||
return name.split(":")[-1] if ":" in name else name
|
|
||||||
|
|
||||||
|
|
||||||
class NativeRunnerMixin:
|
|
||||||
"""Native Gemini SDK reasoning loop of :class:`Explorer`."""
|
|
||||||
|
|
||||||
# This mixin is composed into ``Explorer``. Declaring the attributes and
|
|
||||||
# sibling-mixin methods it consumes makes that host contract visible to
|
|
||||||
# static analysis without adding runtime shims or changing MRO behavior.
|
|
||||||
if TYPE_CHECKING:
|
|
||||||
from artemis.context import ArtemisContext
|
|
||||||
|
|
||||||
ctx: ArtemisContext
|
|
||||||
denylisted_tools: set[str]
|
|
||||||
turn_latencies: list[float]
|
|
||||||
turn_cached_tokens: list[int]
|
|
||||||
trace_history: list[dict[str, Any]]
|
|
||||||
|
|
||||||
def get_exposed_tools(
|
|
||||||
self, only_submit: bool = False
|
|
||||||
) -> list[types.FunctionDeclaration]: ...
|
|
||||||
|
|
||||||
def _prune_historical_images(self, contents: list, keep_last: int = 1) -> None: ...
|
|
||||||
|
|
||||||
def _enrich_candidates(self, candidates: list[Any]) -> list[Any]: ...
|
|
||||||
|
|
||||||
async def exec_ask_perception_tool(
|
|
||||||
self,
|
|
||||||
search_query: str,
|
|
||||||
nx: int,
|
|
||||||
ny: int,
|
|
||||||
detect_queries: list[str],
|
|
||||||
) -> dict[str, Any]: ...
|
|
||||||
|
|
||||||
async def exec_detect_objects(
|
|
||||||
self, queries: list[str], target_image_id: str = "img_0"
|
|
||||||
) -> dict[str, Any]: ...
|
|
||||||
|
|
||||||
async def exec_ask_image_processor(
|
|
||||||
self, instruction: str, target_image_id: str = "img_0"
|
|
||||||
) -> dict[str, Any]: ...
|
|
||||||
|
|
||||||
async def exec_get_ocr_list(self) -> dict[str, Any]: ...
|
|
||||||
|
|
||||||
async def exec_inspect_region(
|
|
||||||
self,
|
|
||||||
x_min: int,
|
|
||||||
y_min: int,
|
|
||||||
x_max: int,
|
|
||||||
y_max: int,
|
|
||||||
zoom_factor: float = 2.0,
|
|
||||||
) -> dict[str, Any]: ...
|
|
||||||
|
|
||||||
def _make_image_part_getter(
|
|
||||||
self, client, use_file_api: bool, uploaded_files: list
|
|
||||||
) -> ImagePartGetter:
|
|
||||||
"""Builds the image-part loader used for screenshots and tool images."""
|
|
||||||
|
|
||||||
async def get_image_part(file_path: str) -> types.Part:
|
|
||||||
if use_file_api:
|
|
||||||
file_ref = await client.aio.files.upload(file=file_path)
|
|
||||||
uploaded_files.append(file_ref)
|
|
||||||
return types.Part(
|
|
||||||
file_data=types.FileData(
|
|
||||||
file_uri=file_ref.uri,
|
|
||||||
mime_type=file_ref.mime_type or "image/jpeg",
|
|
||||||
)
|
|
||||||
)
|
|
||||||
with open(file_path, "rb") as f:
|
|
||||||
img_bytes = f.read()
|
|
||||||
return types.Part.from_bytes(data=img_bytes, mime_type="image/jpeg")
|
|
||||||
|
|
||||||
return get_image_part
|
|
||||||
|
|
||||||
def _build_initial_contents(
|
|
||||||
self, query: str, context_feedback: str, minimal_list: str, initial_part: types.Part
|
|
||||||
) -> list[types.Content]:
|
|
||||||
"""Builds the initial user content with the operator request and screenshot."""
|
|
||||||
return [
|
|
||||||
types.Content(
|
|
||||||
role="user",
|
|
||||||
parts=[
|
|
||||||
types.Part.from_text(
|
|
||||||
text=f"""Operator Request:
|
|
||||||
- Query: {query}
|
|
||||||
- Context Feedback: {context_feedback}
|
|
||||||
|
|
||||||
Initial marked UI elements list (corresponding to numbers ①, ②... in the image):
|
|
||||||
{minimal_list}
|
|
||||||
"""
|
|
||||||
),
|
|
||||||
initial_part,
|
|
||||||
],
|
|
||||||
)
|
|
||||||
]
|
|
||||||
|
|
||||||
async def _create_cache_resource(
|
|
||||||
self, client, model_name: str, prompt_template: str, contents: list
|
|
||||||
):
|
|
||||||
"""Creates the explicit cache resource; returns None on failure."""
|
|
||||||
try:
|
|
||||||
logger.info("Creating cache resource for Explorer...")
|
|
||||||
cached_content = await client.aio.caches.create(
|
|
||||||
model=model_name,
|
|
||||||
config=types.CreateCachedContentConfig(
|
|
||||||
contents=[contents[0]],
|
|
||||||
system_instruction=prompt_template,
|
|
||||||
tools=[types.Tool(function_declarations=self.get_exposed_tools())],
|
|
||||||
tool_config=types.ToolConfig(
|
|
||||||
function_calling_config=types.FunctionCallingConfig(
|
|
||||||
mode=types.FunctionCallingConfigMode.ANY
|
|
||||||
),
|
|
||||||
include_server_side_tool_invocations=True,
|
|
||||||
),
|
|
||||||
ttl="300s",
|
|
||||||
),
|
|
||||||
)
|
|
||||||
logger.info(f"Cache resource created successfully: {cached_content.name}")
|
|
||||||
return cached_content
|
|
||||||
except Exception as cache_err:
|
|
||||||
logger.error(f"Failed to create cache resource: {cache_err}")
|
|
||||||
return None
|
|
||||||
|
|
||||||
async def _run_native(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
client,
|
|
||||||
query: str,
|
|
||||||
context_feedback: str,
|
|
||||||
minimal_list: str,
|
|
||||||
img_to_read: str,
|
|
||||||
enable_caching: bool,
|
|
||||||
prompt_template: str,
|
|
||||||
model_name: str,
|
|
||||||
temperature,
|
|
||||||
thinking_level,
|
|
||||||
max_turns: int,
|
|
||||||
) -> str:
|
|
||||||
"""Runs the native Gemini path: cache setup, reasoning loop, cleanup."""
|
|
||||||
use_file_api = os.getenv("ARTEMIS_USE_FILE_API", "false").lower() == "true"
|
|
||||||
uploaded_files: list = []
|
|
||||||
get_image_part = self._make_image_part_getter(client, use_file_api, uploaded_files)
|
|
||||||
|
|
||||||
cached_content = None
|
|
||||||
try:
|
|
||||||
initial_part = await get_image_part(img_to_read)
|
|
||||||
contents = self._build_initial_contents(
|
|
||||||
query, context_feedback, minimal_list, initial_part
|
|
||||||
)
|
|
||||||
|
|
||||||
if enable_caching:
|
|
||||||
cached_content = await self._create_cache_resource(
|
|
||||||
client, model_name, prompt_template, contents
|
|
||||||
)
|
|
||||||
|
|
||||||
agent_outcome = await self._native_loop(
|
|
||||||
client=client,
|
|
||||||
contents=contents,
|
|
||||||
get_image_part=get_image_part,
|
|
||||||
cached_content=cached_content,
|
|
||||||
prompt_template=prompt_template,
|
|
||||||
model_name=model_name,
|
|
||||||
temperature=temperature,
|
|
||||||
thinking_level=thinking_level,
|
|
||||||
max_turns=max_turns,
|
|
||||||
)
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
agent_outcome = self._format_native_failure(e, model_name)
|
|
||||||
finally:
|
|
||||||
await self._cleanup_native_resources(client, cached_content, uploaded_files)
|
|
||||||
|
|
||||||
return agent_outcome
|
|
||||||
|
|
||||||
async def _native_loop(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
client,
|
|
||||||
contents: list,
|
|
||||||
get_image_part: ImagePartGetter,
|
|
||||||
cached_content,
|
|
||||||
prompt_template: str,
|
|
||||||
model_name: str,
|
|
||||||
temperature,
|
|
||||||
thinking_level,
|
|
||||||
max_turns: int,
|
|
||||||
) -> str:
|
|
||||||
"""Main native reasoning loop: one model turn per iteration."""
|
|
||||||
turn = 0
|
|
||||||
agent_outcome = ""
|
|
||||||
self.turn_latencies = []
|
|
||||||
self.turn_cached_tokens = []
|
|
||||||
self.trace_history = []
|
|
||||||
|
|
||||||
while turn < max_turns:
|
|
||||||
turn += 1
|
|
||||||
logger.info(f"Turn {turn}/{max_turns}: Invoking Native Gemini SDK for Explorer...")
|
|
||||||
|
|
||||||
self._prune_historical_images(contents, keep_last=1)
|
|
||||||
|
|
||||||
is_final_turn = turn == max_turns
|
|
||||||
if is_final_turn:
|
|
||||||
warning_msg = (
|
|
||||||
"\n[WARNING] This is your final iteration. You MUST"
|
|
||||||
" call 'submit_answer' to submit your final result. No"
|
|
||||||
" other tools are available."
|
|
||||||
)
|
|
||||||
contents.append(
|
|
||||||
types.Content(
|
|
||||||
role="user",
|
|
||||||
parts=[types.Part.from_text(text=warning_msg)],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
response, thinking_parts, text_parts = await self._invoke_native_model(
|
|
||||||
client=client,
|
|
||||||
contents=contents,
|
|
||||||
cached_content=cached_content,
|
|
||||||
is_final_turn=is_final_turn,
|
|
||||||
prompt_template=prompt_template,
|
|
||||||
model_name=model_name,
|
|
||||||
temperature=temperature,
|
|
||||||
thinking_level=thinking_level,
|
|
||||||
)
|
|
||||||
|
|
||||||
turn_record = {
|
|
||||||
"iteration": turn,
|
|
||||||
"thoughts": ("\n".join(thinking_parts) if thinking_parts else ""),
|
|
||||||
"tool_calls": [],
|
|
||||||
}
|
|
||||||
|
|
||||||
function_calls = response.function_calls
|
|
||||||
if not function_calls:
|
|
||||||
self._handle_missing_tool_calls(
|
|
||||||
contents, response, turn_record, text_parts, max_turns, turn
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
|
|
||||||
# Record the model's function call request in history
|
|
||||||
contents.append(self._model_content(response, function_calls))
|
|
||||||
|
|
||||||
tool_response_parts: list[types.Part] = []
|
|
||||||
|
|
||||||
# Intercept and validate submit_answer to manage the task lifecycle
|
|
||||||
submit_outcome = self._process_submit_answer(
|
|
||||||
function_calls, turn_record, tool_response_parts
|
|
||||||
)
|
|
||||||
if submit_outcome is not None:
|
|
||||||
agent_outcome = submit_outcome
|
|
||||||
break
|
|
||||||
|
|
||||||
await self._dispatch_tool_calls(
|
|
||||||
function_calls, get_image_part, turn_record, tool_response_parts
|
|
||||||
)
|
|
||||||
|
|
||||||
if tool_response_parts:
|
|
||||||
# Native Gemini SDK allows the 'user' role to return mixed parts
|
|
||||||
contents.append(types.Content(role="user", parts=tool_response_parts))
|
|
||||||
|
|
||||||
self.trace_history.append(turn_record)
|
|
||||||
|
|
||||||
if turn >= max_turns and not agent_outcome:
|
|
||||||
agent_outcome = (
|
|
||||||
"Error: Explorer reached maximum iterations without a conclusive answer."
|
|
||||||
)
|
|
||||||
|
|
||||||
return agent_outcome
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _model_content(response, function_calls: list) -> types.Content:
|
|
||||||
"""The model turn to record: the SDK's content when present, else a rebuild."""
|
|
||||||
candidates = getattr(response, "candidates", None)
|
|
||||||
if candidates:
|
|
||||||
content = getattr(candidates[0], "content", None)
|
|
||||||
if content:
|
|
||||||
return content
|
|
||||||
return types.Content(
|
|
||||||
role="model",
|
|
||||||
parts=[types.Part(function_call=fc) for fc in function_calls],
|
|
||||||
)
|
|
||||||
|
|
||||||
def _build_generate_config(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
cached_content,
|
|
||||||
is_final_turn: bool,
|
|
||||||
prompt_template: str,
|
|
||||||
temperature,
|
|
||||||
thinking_level,
|
|
||||||
) -> types.GenerateContentConfig:
|
|
||||||
"""Builds the per-turn generation config (cached vs full tool config)."""
|
|
||||||
thinking_config = (
|
|
||||||
types.ThinkingConfig(thinking_level=thinking_level) if thinking_level else None
|
|
||||||
)
|
|
||||||
if cached_content and not is_final_turn:
|
|
||||||
return types.GenerateContentConfig(
|
|
||||||
cached_content=cached_content.name,
|
|
||||||
temperature=temperature,
|
|
||||||
safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
|
|
||||||
thinking_config=thinking_config,
|
|
||||||
)
|
|
||||||
return types.GenerateContentConfig(
|
|
||||||
system_instruction=prompt_template,
|
|
||||||
temperature=temperature,
|
|
||||||
safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
|
|
||||||
tools=[
|
|
||||||
types.Tool(function_declarations=self.get_exposed_tools(only_submit=is_final_turn))
|
|
||||||
],
|
|
||||||
tool_config=types.ToolConfig(
|
|
||||||
function_calling_config=types.FunctionCallingConfig(
|
|
||||||
mode=types.FunctionCallingConfigMode.ANY
|
|
||||||
),
|
|
||||||
include_server_side_tool_invocations=True,
|
|
||||||
),
|
|
||||||
thinking_config=thinking_config,
|
|
||||||
)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _extract_response_texts(response) -> tuple[list[str], list[str]]:
|
|
||||||
"""Extracts model thoughts and plain text parts from a response."""
|
|
||||||
thinking_parts = []
|
|
||||||
text_parts = []
|
|
||||||
candidates = getattr(response, "candidates", None)
|
|
||||||
if not candidates and isinstance(response, dict):
|
|
||||||
candidates = response.get("candidates")
|
|
||||||
|
|
||||||
if candidates and len(candidates) > 0:
|
|
||||||
candidate = candidates[0]
|
|
||||||
content = getattr(candidate, "content", None)
|
|
||||||
if not content and isinstance(candidate, dict):
|
|
||||||
content = candidate.get("content")
|
|
||||||
|
|
||||||
parts = getattr(content, "parts", None)
|
|
||||||
if not parts and isinstance(content, dict):
|
|
||||||
parts = content.get("parts")
|
|
||||||
|
|
||||||
if parts:
|
|
||||||
for part in parts:
|
|
||||||
is_thought = False
|
|
||||||
part_text = None
|
|
||||||
if isinstance(part, dict):
|
|
||||||
is_thought = part.get("thought", False)
|
|
||||||
part_text = part.get("text", None)
|
|
||||||
else:
|
|
||||||
is_thought = getattr(part, "thought", False)
|
|
||||||
part_text = getattr(part, "text", None)
|
|
||||||
|
|
||||||
if is_thought and part_text:
|
|
||||||
thinking_parts.append(part_text)
|
|
||||||
elif part_text:
|
|
||||||
text_parts.append(part_text)
|
|
||||||
return thinking_parts, text_parts
|
|
||||||
|
|
||||||
async def _invoke_native_model(
|
|
||||||
self,
|
|
||||||
*,
|
|
||||||
client,
|
|
||||||
contents: list,
|
|
||||||
cached_content,
|
|
||||||
is_final_turn: bool,
|
|
||||||
prompt_template: str,
|
|
||||||
model_name: str,
|
|
||||||
temperature,
|
|
||||||
thinking_level,
|
|
||||||
):
|
|
||||||
"""Invokes the model for one turn under a trace span; returns response and texts."""
|
|
||||||
ctx = self.ctx
|
|
||||||
start_turn = time.perf_counter()
|
|
||||||
with TraceSpan(name="gemini_explorer_call", ctx=ctx) as span:
|
|
||||||
generate_config = self._build_generate_config(
|
|
||||||
cached_content=cached_content,
|
|
||||||
is_final_turn=is_final_turn,
|
|
||||||
prompt_template=prompt_template,
|
|
||||||
temperature=temperature,
|
|
||||||
thinking_level=thinking_level,
|
|
||||||
)
|
|
||||||
|
|
||||||
response = await _generate_content_with_reliability(
|
|
||||||
lambda: asyncio.wait_for(
|
|
||||||
client.aio.models.generate_content(
|
|
||||||
model=model_name,
|
|
||||||
contents=contents,
|
|
||||||
config=generate_config,
|
|
||||||
),
|
|
||||||
timeout=180,
|
|
||||||
),
|
|
||||||
)
|
|
||||||
end_turn = time.perf_counter()
|
|
||||||
turn_dur = end_turn - start_turn
|
|
||||||
self.turn_latencies.append(turn_dur)
|
|
||||||
|
|
||||||
cached_tokens = 0
|
|
||||||
if getattr(response, "usage_metadata", None) and getattr(
|
|
||||||
response.usage_metadata,
|
|
||||||
"cached_content_token_count",
|
|
||||||
None,
|
|
||||||
):
|
|
||||||
cached_tokens = response.usage_metadata.cached_content_token_count
|
|
||||||
self.turn_cached_tokens.append(cached_tokens)
|
|
||||||
span.result = (
|
|
||||||
f"Function calls: {len(response.function_calls)}"
|
|
||||||
if response.function_calls
|
|
||||||
else "Final answer"
|
|
||||||
)
|
|
||||||
# Extract and log model thoughts and text if present
|
|
||||||
thinking_parts, text_parts = self._extract_response_texts(response)
|
|
||||||
if thinking_parts:
|
|
||||||
span.payload["explorer_thought"] = "\n".join(thinking_parts)
|
|
||||||
if text_parts:
|
|
||||||
span.payload["explorer_text"] = "\n".join(text_parts)
|
|
||||||
|
|
||||||
return response, thinking_parts, text_parts
|
|
||||||
|
|
||||||
def _handle_missing_tool_calls(
|
|
||||||
self,
|
|
||||||
contents: list,
|
|
||||||
response,
|
|
||||||
turn_record: dict,
|
|
||||||
text_parts: list[str],
|
|
||||||
max_turns: int,
|
|
||||||
turn: int,
|
|
||||||
) -> None:
|
|
||||||
"""Handles a turn where the model produced plain text instead of tool calls.
|
|
||||||
|
|
||||||
Responses without candidates (safety blocks, empty replies) are
|
|
||||||
recorded as a synthetic model text part so the conversation history
|
|
||||||
stays well-formed for the retry.
|
|
||||||
"""
|
|
||||||
logger.warning(f"Explorer turn {turn}: Model hallucinated plain text. Forcing retry.")
|
|
||||||
model_text = "\n".join(text_parts) if text_parts else ""
|
|
||||||
turn_record["tool_calls"].append(
|
|
||||||
{
|
|
||||||
"name": "hallucinated_plain_text",
|
|
||||||
"args": {"text": model_text},
|
|
||||||
"response": {
|
|
||||||
"error": (
|
|
||||||
"Forced retry because model failed to call submit_answer or any other tool"
|
|
||||||
)
|
|
||||||
},
|
|
||||||
}
|
|
||||||
)
|
|
||||||
self.trace_history.append(turn_record)
|
|
||||||
|
|
||||||
candidates = getattr(response, "candidates", None)
|
|
||||||
model_content = getattr(candidates[0], "content", None) if candidates else None
|
|
||||||
if not model_content:
|
|
||||||
model_content = types.Content(
|
|
||||||
role="model",
|
|
||||||
parts=[types.Part.from_text(text=model_text or "(no response)")],
|
|
||||||
)
|
|
||||||
contents.append(model_content)
|
|
||||||
contents.append(
|
|
||||||
types.Content(
|
|
||||||
role="user",
|
|
||||||
parts=[
|
|
||||||
types.Part.from_text(
|
|
||||||
text=(
|
|
||||||
"You have not called 'submit_answer'"
|
|
||||||
" to submit your final answer yet. If"
|
|
||||||
" you are not certain about the final"
|
|
||||||
" answer, you can continue exploring"
|
|
||||||
f" {max_turns - turn} more"
|
|
||||||
" time(s)."
|
|
||||||
)
|
|
||||||
)
|
|
||||||
],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def _validate_submit_args(function_calls: list, args: dict, candidates: list) -> list[str]:
|
|
||||||
"""Validates a submit_answer call; returns the list of validation errors."""
|
|
||||||
errors = []
|
|
||||||
|
|
||||||
if len(function_calls) > 1:
|
|
||||||
errors.append(
|
|
||||||
"The tool 'submit_answer' MUST be called"
|
|
||||||
" individually in your final turn without any other"
|
|
||||||
" parallel tool calls. Please remove other tool"
|
|
||||||
" calls and re-submit."
|
|
||||||
)
|
|
||||||
|
|
||||||
fallback_message = args.get("fallback_message", "")
|
|
||||||
if not candidates and not fallback_message:
|
|
||||||
errors.append(
|
|
||||||
"The candidates list is empty. You must provide at"
|
|
||||||
" least one candidate matching the query, or"
|
|
||||||
" explain observations via fallback_message."
|
|
||||||
)
|
|
||||||
|
|
||||||
for i, cand in enumerate(candidates):
|
|
||||||
label = cand.get("label")
|
|
||||||
coords = cand.get("coords")
|
|
||||||
if not label:
|
|
||||||
errors.append(f"Candidate at index {i} is missing a 'label'.")
|
|
||||||
if not coords or not isinstance(coords, list) or len(coords) != 2:
|
|
||||||
errors.append(
|
|
||||||
f"Candidate '{label or i}' must have 'coords'"
|
|
||||||
" as an array of exactly 2 integers `[nx,"
|
|
||||||
" ny]`."
|
|
||||||
)
|
|
||||||
elif not is_valid_norm_point(coords):
|
|
||||||
try:
|
|
||||||
nx, ny = int(coords[0]), int(coords[1])
|
|
||||||
except (ValueError, TypeError):
|
|
||||||
errors.append(f"Candidate '{label or i}' coordinates are invalid integers.")
|
|
||||||
else:
|
|
||||||
errors.append(
|
|
||||||
f"Candidate '{label or i}' coordinates"
|
|
||||||
f" `[{nx}, {ny}]` must strictly be in"
|
|
||||||
" the `[0-1000]` normalized scale"
|
|
||||||
" range inclusive."
|
|
||||||
)
|
|
||||||
|
|
||||||
return errors
|
|
||||||
|
|
||||||
def _process_submit_answer(
|
|
||||||
self, function_calls: list, turn_record: dict, tool_response_parts: list
|
|
||||||
) -> str | None:
|
|
||||||
"""Intercepts submit_answer; returns the final outcome JSON when valid.
|
|
||||||
|
|
||||||
On validation failure the error is fed back as a tool response and
|
|
||||||
None is returned so the ReAct loop can continue and self-correct.
|
|
||||||
"""
|
|
||||||
submit_call = next(
|
|
||||||
(fc for fc in function_calls if _bare_tool_name(fc.name) == SUBMIT_TOOL),
|
|
||||||
None,
|
|
||||||
)
|
|
||||||
if not submit_call:
|
|
||||||
return None
|
|
||||||
|
|
||||||
args = submit_call.args or {}
|
|
||||||
candidates = args.get("candidates", [])
|
|
||||||
errors = self._validate_submit_args(function_calls, args, candidates)
|
|
||||||
|
|
||||||
if errors:
|
|
||||||
logger.warning(f"Explorer submit_answer validation failed: {errors}")
|
|
||||||
turn_record["tool_calls"].append(
|
|
||||||
{
|
|
||||||
"name": SUBMIT_TOOL,
|
|
||||||
"args": args,
|
|
||||||
"response": {"error": ("Validation Failed:\n" + "\n".join(errors))},
|
|
||||||
}
|
|
||||||
)
|
|
||||||
tool_response_parts.append(
|
|
||||||
types.Part.from_function_response(
|
|
||||||
name=SUBMIT_TOOL,
|
|
||||||
response={
|
|
||||||
"error": (
|
|
||||||
"Validation Failed:\n"
|
|
||||||
+ "\n".join(errors)
|
|
||||||
+ "\nPlease correct these formatting"
|
|
||||||
" errors and re-submit using"
|
|
||||||
" submit_answer."
|
|
||||||
)
|
|
||||||
},
|
|
||||||
)
|
|
||||||
)
|
|
||||||
# Allow the ReAct loop to continue so the model can self-correct
|
|
||||||
return None
|
|
||||||
|
|
||||||
logger.info("Explorer submit_answer validation passed successfully!")
|
|
||||||
# Candidates submitted by a registered label inherit that element's
|
|
||||||
# bounds; the rest of the model's arguments are passed through as is.
|
|
||||||
outcome_args = dict(args)
|
|
||||||
outcome_args["candidates"] = self._enrich_candidates(candidates)
|
|
||||||
agent_outcome = json.dumps(outcome_args, ensure_ascii=False)
|
|
||||||
turn_record["tool_calls"].append(
|
|
||||||
{"name": SUBMIT_TOOL, "args": args, "response": {"result": "success"}}
|
|
||||||
)
|
|
||||||
self.trace_history.append(turn_record)
|
|
||||||
return agent_outcome
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
async def _append_tool_response_with_images(
|
|
||||||
name: str,
|
|
||||||
res: dict,
|
|
||||||
get_image_part: ImagePartGetter,
|
|
||||||
tool_response_parts: list,
|
|
||||||
*,
|
|
||||||
result_key: str = "result",
|
|
||||||
) -> None:
|
|
||||||
"""Appends a tool result part plus every annotated image it produced.
|
|
||||||
|
|
||||||
Image loading failures are logged and skipped: the textual result is
|
|
||||||
still delivered, so one unreadable annotation cannot sink the turn.
|
|
||||||
"""
|
|
||||||
tool_response_parts.append(
|
|
||||||
types.Part.from_function_response(name=name, response={result_key: res.get("text")})
|
|
||||||
)
|
|
||||||
image_paths = [p for p in [res.get("image_path"), *(res.get("image_paths") or [])] if p]
|
|
||||||
for img_p in image_paths:
|
|
||||||
try:
|
|
||||||
tool_response_parts.append(await get_image_part(img_p))
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"Failed to load image response part for {img_p}: {e}")
|
|
||||||
|
|
||||||
async def _execute_native_tool(
|
|
||||||
self,
|
|
||||||
name: str,
|
|
||||||
args: dict,
|
|
||||||
get_image_part: ImagePartGetter,
|
|
||||||
tool_call_trace: dict,
|
|
||||||
tool_response_parts: list,
|
|
||||||
) -> None:
|
|
||||||
"""Executes one non-submit tool call and appends its response parts."""
|
|
||||||
if name == "ask_perception_tool":
|
|
||||||
search_query = args.get("search_query")
|
|
||||||
nx = args.get("nx")
|
|
||||||
ny = args.get("ny")
|
|
||||||
detect_queries = args.get("detect_queries")
|
|
||||||
res = await self.exec_ask_perception_tool(
|
|
||||||
search_query=search_query if isinstance(search_query, str) else "",
|
|
||||||
nx=nx if isinstance(nx, int) else -1,
|
|
||||||
ny=ny if isinstance(ny, int) else -1,
|
|
||||||
detect_queries=detect_queries if isinstance(detect_queries, list) else [],
|
|
||||||
)
|
|
||||||
tool_call_trace["response"] = res
|
|
||||||
await self._append_tool_response_with_images(
|
|
||||||
name, res, get_image_part, tool_response_parts, result_key="text"
|
|
||||||
)
|
|
||||||
elif name == "detect_objects":
|
|
||||||
res = await self.exec_detect_objects(
|
|
||||||
queries=args.get("queries") or [],
|
|
||||||
target_image_id=args.get("target_image_id", "img_0"),
|
|
||||||
)
|
|
||||||
tool_call_trace["response"] = res
|
|
||||||
await self._append_tool_response_with_images(
|
|
||||||
name, res, get_image_part, tool_response_parts
|
|
||||||
)
|
|
||||||
elif name == "ask_image_processor":
|
|
||||||
res = await self.exec_ask_image_processor(
|
|
||||||
instruction=args.get("instruction") or "",
|
|
||||||
target_image_id=args.get("target_image_id", "img_0"),
|
|
||||||
)
|
|
||||||
tool_call_trace["response"] = res
|
|
||||||
await self._append_tool_response_with_images(
|
|
||||||
name, res, get_image_part, tool_response_parts
|
|
||||||
)
|
|
||||||
elif name == "get_ocr_list":
|
|
||||||
res = await self.exec_get_ocr_list()
|
|
||||||
tool_call_trace["response"] = res
|
|
||||||
await self._append_tool_response_with_images(
|
|
||||||
name, res, get_image_part, tool_response_parts
|
|
||||||
)
|
|
||||||
elif name == "inspect_region":
|
|
||||||
res = await self.exec_inspect_region(
|
|
||||||
x_min=args["x_min"],
|
|
||||||
y_min=args["y_min"],
|
|
||||||
x_max=args["x_max"],
|
|
||||||
y_max=args["y_max"],
|
|
||||||
zoom_factor=args.get("zoom_factor", 2.0),
|
|
||||||
)
|
|
||||||
tool_call_trace["response"] = res
|
|
||||||
await self._append_tool_response_with_images(
|
|
||||||
name, res, get_image_part, tool_response_parts
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
tool_call_trace["response"] = {"error": f"Tool {name} not found"}
|
|
||||||
tool_response_parts.append(
|
|
||||||
types.Part.from_function_response(
|
|
||||||
name=name,
|
|
||||||
response={"error": f"Tool {name} not found"},
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
async def _dispatch_tool_calls(
|
|
||||||
self,
|
|
||||||
function_calls: list,
|
|
||||||
get_image_part: ImagePartGetter,
|
|
||||||
turn_record: dict,
|
|
||||||
tool_response_parts: list,
|
|
||||||
) -> None:
|
|
||||||
"""Dispatches all non-submit tool calls sequentially with tracing.
|
|
||||||
|
|
||||||
Denylisted names are refused here as well, even though they are
|
|
||||||
absent from the declarations, because models occasionally call tools
|
|
||||||
they were never offered.
|
|
||||||
"""
|
|
||||||
for fc in function_calls:
|
|
||||||
name = _bare_tool_name(fc.name)
|
|
||||||
if name == SUBMIT_TOOL:
|
|
||||||
continue
|
|
||||||
args = fc.args or {}
|
|
||||||
tool_call_trace = {"name": name, "args": args}
|
|
||||||
if name in self.denylisted_tools:
|
|
||||||
logger.warning(
|
|
||||||
f"Explorer attempted to call denylisted tool '{name}'. Blocking execution."
|
|
||||||
)
|
|
||||||
error = f"Tool '{name}' is denylisted and unavailable."
|
|
||||||
tool_call_trace["response"] = {"error": error}
|
|
||||||
turn_record["tool_calls"].append(tool_call_trace)
|
|
||||||
tool_response_parts.append(
|
|
||||||
types.Part.from_function_response(name=fc.name, response={"error": error})
|
|
||||||
)
|
|
||||||
continue
|
|
||||||
|
|
||||||
logger.info(f"Explorer executing tool '{name}' sequentially...")
|
|
||||||
|
|
||||||
try:
|
|
||||||
await self._execute_native_tool(
|
|
||||||
name, args, get_image_part, tool_call_trace, tool_response_parts
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Explorer tool {name} execution failed: {e}")
|
|
||||||
tool_call_trace["response"] = {"error": str(e)}
|
|
||||||
tool_response_parts.append(
|
|
||||||
types.Part.from_function_response(name=name, response={"error": str(e)})
|
|
||||||
)
|
|
||||||
finally:
|
|
||||||
turn_record["tool_calls"].append(tool_call_trace)
|
|
||||||
|
|
||||||
def _format_native_failure(self, e: Exception, model_name: str) -> str:
|
|
||||||
"""Maps a loop failure to the user-facing fallback outcome JSON."""
|
|
||||||
logger.error(f"Explorer execution loop failed: {e}")
|
|
||||||
|
|
||||||
err_msg = str(e)
|
|
||||||
if "preempted" in err_msg.lower():
|
|
||||||
clean_msg = (
|
|
||||||
"Model request was preempted by the server due to high"
|
|
||||||
" demand. Please try again later."
|
|
||||||
)
|
|
||||||
elif "overloaded" in err_msg.lower() or "503" in err_msg:
|
|
||||||
clean_msg = "Model service is currently overloaded. Please try again later."
|
|
||||||
elif "quota" in err_msg.lower() or "429" in err_msg:
|
|
||||||
clean_msg = "Model API quota exceeded or rate limited."
|
|
||||||
elif "thinkingconfig" in err_msg.lower():
|
|
||||||
clean_msg = (
|
|
||||||
"Model configuration error: ThinkingConfig is not"
|
|
||||||
f" supported for model {model_name}."
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
clean_msg = f"Explorer agent execution failed: {err_msg}"
|
|
||||||
|
|
||||||
return json.dumps(
|
|
||||||
{"candidates": [], "fallback_message": clean_msg},
|
|
||||||
ensure_ascii=False,
|
|
||||||
)
|
|
||||||
|
|
||||||
async def _cleanup_native_resources(self, client, cached_content, uploaded_files: list) -> None:
|
|
||||||
"""Deletes the cache resource and File API uploads created for this run."""
|
|
||||||
if cached_content:
|
|
||||||
try:
|
|
||||||
logger.info(f"Deleting cache resource: {cached_content.name}")
|
|
||||||
await client.aio.caches.delete(name=cached_content.name)
|
|
||||||
except Exception as cleanup_cache_err:
|
|
||||||
logger.warning(f"Failed to delete cache resource: {cleanup_cache_err}")
|
|
||||||
for file_ref in uploaded_files:
|
|
||||||
try:
|
|
||||||
await client.aio.files.delete(name=file_ref.name)
|
|
||||||
except Exception as cleanup_err:
|
|
||||||
logger.warning(f"Failed to delete uploaded file {file_ref.name}: {cleanup_err}")
|
|
||||||
logger.info(f"Explorer turn latencies: {self.turn_latencies}")
|
|
||||||
@@ -14,14 +14,9 @@
|
|||||||
|
|
||||||
"""Static tool declarations for the Explorer agent.
|
"""Static tool declarations for the Explorer agent.
|
||||||
|
|
||||||
Split out of ``artemis.agents.explorer.explorer`` as pure data:
|
OpenAI function schemas consumed by the Responses model adapter.
|
||||||
``UNIVERSAL_EXPLORER_TOOLS`` (OpenAI-style dict schemas for the universal
|
|
||||||
LangChain path) and ``NATIVE_EXPLORER_TOOL_DECLARATIONS`` (google-genai
|
|
||||||
``FunctionDeclaration`` objects for the native Gemini path).
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from google.genai import types
|
|
||||||
|
|
||||||
UNIVERSAL_EXPLORER_TOOLS = [
|
UNIVERSAL_EXPLORER_TOOLS = [
|
||||||
{
|
{
|
||||||
"type": "function",
|
"type": "function",
|
||||||
@@ -183,212 +178,3 @@ UNIVERSAL_EXPLORER_TOOLS = [
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
NATIVE_EXPLORER_TOOL_DECLARATIONS = [
|
|
||||||
types.FunctionDeclaration(
|
|
||||||
name="ask_perception_tool",
|
|
||||||
description=(
|
|
||||||
"[Perception] Concurrently executes and awaits three sub-tasks"
|
|
||||||
" in parallel, including UI-tree text search, coordinate audit,"
|
|
||||||
" and visual object detection on the screen.\n\nOutput"
|
|
||||||
" Format:\nReturns:'text' (str): A single string consolidating"
|
|
||||||
" results with 0-1000 normalized coordinates, including UI"
|
|
||||||
" search matches labeled [X],"
|
|
||||||
" coordinate search matches labeled [X], and visually detected"
|
|
||||||
" object targets labeled [D].\n(Annotated screenshots are also"
|
|
||||||
" added to the context containing corresponding colored label"
|
|
||||||
" dots)."
|
|
||||||
),
|
|
||||||
parameters=types.Schema(
|
|
||||||
type=types.Type.OBJECT,
|
|
||||||
properties={
|
|
||||||
"search_query": types.Schema(
|
|
||||||
type=types.Type.STRING,
|
|
||||||
description=("Required. Text query to search for in UI elements."),
|
|
||||||
),
|
|
||||||
"nx": types.Schema(
|
|
||||||
type=types.Type.INTEGER,
|
|
||||||
description=(
|
|
||||||
"Required. Normalized X coordinate (0-1000 range)"
|
|
||||||
" to audit for overlapping elements."
|
|
||||||
),
|
|
||||||
),
|
|
||||||
"ny": types.Schema(
|
|
||||||
type=types.Type.INTEGER,
|
|
||||||
description=(
|
|
||||||
"Required. Normalized Y coordinate (0-1000 range)"
|
|
||||||
" to audit for overlapping elements."
|
|
||||||
),
|
|
||||||
),
|
|
||||||
"detect_queries": types.Schema(
|
|
||||||
type=types.Type.ARRAY,
|
|
||||||
items=types.Schema(type=types.Type.STRING),
|
|
||||||
description=(
|
|
||||||
"Required. List of query strings to visually detect on the screen."
|
|
||||||
),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
required=["search_query", "nx", "ny", "detect_queries"],
|
|
||||||
),
|
|
||||||
),
|
|
||||||
types.FunctionDeclaration(
|
|
||||||
name="detect_objects",
|
|
||||||
description=(
|
|
||||||
"[Perception] Locates elements, shapes and more using a"
|
|
||||||
" VLM. Returns their normalized coordinates (in a [0, 1000]"
|
|
||||||
" scale) labeled with [D1], [D2] etc. in the text output, along"
|
|
||||||
" with an annotated image containing corresponding visual label"
|
|
||||||
" dots."
|
|
||||||
),
|
|
||||||
parameters=types.Schema(
|
|
||||||
type=types.Type.OBJECT,
|
|
||||||
properties={
|
|
||||||
"target_image_id": types.Schema(
|
|
||||||
type=types.Type.STRING,
|
|
||||||
description="The ID of the image to process.",
|
|
||||||
),
|
|
||||||
"queries": types.Schema(
|
|
||||||
type=types.Type.ARRAY,
|
|
||||||
items=types.Schema(type=types.Type.STRING),
|
|
||||||
description=("A list of single query strings to detect on the screen."),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
required=["queries", "target_image_id"],
|
|
||||||
),
|
|
||||||
),
|
|
||||||
types.FunctionDeclaration(
|
|
||||||
name="get_ocr_list",
|
|
||||||
description=(
|
|
||||||
"[Perception] Retrieves all text elements detected on the"
|
|
||||||
" original, unprocessed screen via OCR. Returns their"
|
|
||||||
" normalized coordinates (in a [0, 1000] scale) labeled with"
|
|
||||||
" [O1], [O2] etc. in the text output, along with an annotated"
|
|
||||||
" image containing corresponding visual label dots."
|
|
||||||
),
|
|
||||||
parameters=types.Schema(
|
|
||||||
type=types.Type.OBJECT,
|
|
||||||
properties={},
|
|
||||||
),
|
|
||||||
),
|
|
||||||
types.FunctionDeclaration(
|
|
||||||
name="ask_image_processor",
|
|
||||||
description=(
|
|
||||||
"[Transformation] A scripting tool for pixel-level image"
|
|
||||||
" processing. CAUTION: This tool is slow and expensive. Returns"
|
|
||||||
" a summary of the operations performed and new image IDs,"
|
|
||||||
" along with new labels and screen coordinates."
|
|
||||||
),
|
|
||||||
parameters=types.Schema(
|
|
||||||
type=types.Type.OBJECT,
|
|
||||||
properties={
|
|
||||||
"target_image_id": types.Schema(
|
|
||||||
type=types.Type.STRING,
|
|
||||||
description=(
|
|
||||||
"The ID of the image in the Image Pool to start"
|
|
||||||
" from (e.g. 'img_0', 'img_1')."
|
|
||||||
),
|
|
||||||
),
|
|
||||||
"instruction": types.Schema(
|
|
||||||
type=types.Type.STRING,
|
|
||||||
description=(
|
|
||||||
"1-2 precise commands describing a simple task."
|
|
||||||
" Include all necessary coordinates and other"
|
|
||||||
" parameters.\n"
|
|
||||||
),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
required=["instruction", "target_image_id"],
|
|
||||||
),
|
|
||||||
),
|
|
||||||
types.FunctionDeclaration(
|
|
||||||
name="submit_answer",
|
|
||||||
description=(
|
|
||||||
"[Finalization] Submits the final list of ranked candidate UI"
|
|
||||||
" elements matching the search query, or a fallback message if"
|
|
||||||
" no elements were found, to complete the search task. Fails"
|
|
||||||
" validation if: (1) candidates is empty and no"
|
|
||||||
" fallback_message is provided, (2) coordinates are not exactly"
|
|
||||||
" 2 integers [nx, ny], or (3) coordinates lie outside"
|
|
||||||
" normalized [0, 1000] range."
|
|
||||||
),
|
|
||||||
parameters=types.Schema(
|
|
||||||
type=types.Type.OBJECT,
|
|
||||||
properties={
|
|
||||||
"candidates": types.Schema(
|
|
||||||
type=types.Type.ARRAY,
|
|
||||||
items=types.Schema(
|
|
||||||
type=types.Type.OBJECT,
|
|
||||||
properties={
|
|
||||||
"label": types.Schema(
|
|
||||||
type=types.Type.STRING,
|
|
||||||
description=(
|
|
||||||
"Reference label (e.g. X1, O2, D3, or a number like 3)"
|
|
||||||
),
|
|
||||||
),
|
|
||||||
"coords": types.Schema(
|
|
||||||
type=types.Type.ARRAY,
|
|
||||||
items=types.Schema(type=types.Type.INTEGER),
|
|
||||||
description=("[nx, ny] coordinates in normalized 0-1000 scale"),
|
|
||||||
),
|
|
||||||
"description": types.Schema(
|
|
||||||
type=types.Type.STRING,
|
|
||||||
description=("Brief description of this candidate element"),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
required=["label", "coords"],
|
|
||||||
),
|
|
||||||
description=(
|
|
||||||
"A ranked list of candidate UI elements that match"
|
|
||||||
" the query, sorted from highest to lowest"
|
|
||||||
" confidence (maximum 10 candidates). Leave empty"
|
|
||||||
" if no matching elements were found on the screen."
|
|
||||||
),
|
|
||||||
),
|
|
||||||
"fallback_message": types.Schema(
|
|
||||||
type=types.Type.STRING,
|
|
||||||
description=(
|
|
||||||
"Optional fallback explanation if target elements"
|
|
||||||
" were not found or if there are special"
|
|
||||||
" observations."
|
|
||||||
),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
required=["candidates"],
|
|
||||||
),
|
|
||||||
),
|
|
||||||
types.FunctionDeclaration(
|
|
||||||
name="inspect_region",
|
|
||||||
description=(
|
|
||||||
"[Perception] Crops and resizes a bounding box region of the"
|
|
||||||
" original screenshot. Returns the cropped and resized image"
|
|
||||||
" directly, without any image ID or coordinate transform."
|
|
||||||
),
|
|
||||||
parameters=types.Schema(
|
|
||||||
type=types.Type.OBJECT,
|
|
||||||
properties={
|
|
||||||
"x_min": types.Schema(
|
|
||||||
type=types.Type.INTEGER,
|
|
||||||
description="Left X coordinate (0-1000).",
|
|
||||||
),
|
|
||||||
"y_min": types.Schema(
|
|
||||||
type=types.Type.INTEGER,
|
|
||||||
description="Top Y coordinate (0-1000).",
|
|
||||||
),
|
|
||||||
"x_max": types.Schema(
|
|
||||||
type=types.Type.INTEGER,
|
|
||||||
description="Right X coordinate (0-1000).",
|
|
||||||
),
|
|
||||||
"y_max": types.Schema(
|
|
||||||
type=types.Type.INTEGER,
|
|
||||||
description="Bottom Y coordinate (0-1000).",
|
|
||||||
),
|
|
||||||
"zoom_factor": types.Schema(
|
|
||||||
type=types.Type.NUMBER,
|
|
||||||
description=("Zoom factor to upscale the cropped region (1.0 to 4.0)."),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
required=["x_min", "y_min", "x_max", "y_max", "zoom_factor"],
|
|
||||||
),
|
|
||||||
),
|
|
||||||
]
|
|
||||||
@@ -30,6 +30,8 @@ from langchain_core.messages import (
|
|||||||
ToolMessage,
|
ToolMessage,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
from artemis.llm.structured import content_to_text
|
||||||
|
|
||||||
from artemis.agents.explorer.geometry import is_valid_norm_point
|
from artemis.agents.explorer.geometry import is_valid_norm_point
|
||||||
from artemis.agents.explorer.tiers import SUBMIT_TOOL
|
from artemis.agents.explorer.tiers import SUBMIT_TOOL
|
||||||
from artemis.agents.explorer.tool_declarations import UNIVERSAL_EXPLORER_TOOLS
|
from artemis.agents.explorer.tool_declarations import UNIVERSAL_EXPLORER_TOOLS
|
||||||
@@ -128,7 +130,8 @@ class UniversalRunnerMixin:
|
|||||||
return json.dumps(
|
return json.dumps(
|
||||||
{
|
{
|
||||||
"candidates": [],
|
"candidates": [],
|
||||||
"fallback_message": str(response.content) or "No candidates found.",
|
"fallback_message": content_to_text(response.content)
|
||||||
|
or "No candidates found.",
|
||||||
},
|
},
|
||||||
ensure_ascii=False,
|
ensure_ascii=False,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -1,3 +1,11 @@
|
|||||||
|
# 输出语言要求
|
||||||
|
所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。
|
||||||
|
保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
|
||||||
|
遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。
|
||||||
|
|
||||||
|
每次调用设备操作工具前,用一小段简体中文说明当前观察、即将执行的动作和目的;操作后仅在获得结果或新观察时说明实际变化。不要只在最终答复中使用中文。工具参数中的自由描述文本也使用简体中文,精确引用的界面文字仍保留原文。
|
||||||
|
|
||||||
|
|
||||||
# ROLE & OBJECTIVE
|
# ROLE & OBJECTIVE
|
||||||
You are an autonomous and highly efficient Android Device Execution Agent. Your goal is to accomplish the user's objective on the device.
|
You are an autonomous and highly efficient Android Device Execution Agent. Your goal is to accomplish the user's objective on the device.
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,17 @@
|
|||||||
|
# 输出语言要求
|
||||||
|
所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。
|
||||||
|
保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
|
||||||
|
遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。
|
||||||
|
|
||||||
|
|
||||||
# ROLE & OBJECTIVE
|
# ROLE & OBJECTIVE
|
||||||
You are the Step Summarizer for an Android UI automation agent. Your task is to synthesize the Before/After screenshots and physical action into a concise, high-information-density, strictly objective first-person historical memory.
|
You are the Step Summarizer for an Android UI automation agent. Your task is to synthesize the Before/After screenshots and physical action into a concise, high-information-density, strictly objective first-person historical memory.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
# PERSPECTIVE & FORMAT CONSTRAINTS
|
# PERSPECTIVE & FORMAT CONSTRAINTS
|
||||||
1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **"I"** (e.g., "In Step {{ step_number }}, I tapped...", "I swiped up on... and observed..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
|
1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **“我”** (e.g., "In Step {{ step_number }}, I tapped...", "I swiped up on... and observed..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
|
||||||
2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 sentences, 60–100 words).
|
2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 句,约 100–200 个汉字;保留关键原文证据,不为凑字数添加内容).
|
||||||
3. **No Lists or Formatting**: Do NOT use bullet points, numbered lists, markdown headers, bold labels, or line breaks in your output. Your output must NEVER contain `---` separators or section-marker lines — those belong to the input, not the summary.
|
3. **No Lists or Formatting**: Do NOT use bullet points, numbered lists, markdown headers, bold labels, or line breaks in your output. Your output must NEVER contain `---` separators or section-marker lines — those belong to the input, not the summary.
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -13,7 +19,7 @@ You are the Step Summarizer for an Android UI automation agent. Your task is to
|
|||||||
# CORE OBSERVATION RULES
|
# CORE OBSERVATION RULES
|
||||||
1. **Zero Subjective Validation (STRICT)**:
|
1. **Zero Subjective Validation (STRICT)**:
|
||||||
- Absolutely avoid declaring semantic goal achievement, subjective success, or final failure. In historical compression, subjective assumptions cause future turns to falsely believe a subgoal was definitively accomplished or permanently blocked.
|
- Absolutely avoid declaring semantic goal achievement, subjective success, or final failure. In historical compression, subjective assumptions cause future turns to falsely believe a subgoal was definitively accomplished or permanently blocked.
|
||||||
- **STRICTLY BANNED WORDS**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
|
- **STRICTLY BANNED WORDS (including Chinese equivalents such as “成功完成”、“已达成”、“失败” and “不可能”)**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
|
||||||
2. **Action & Visual Delta Structure**:
|
2. **Action & Visual Delta Structure**:
|
||||||
- **Target & Action**: Identify what control/button was targeted (referencing the red visual indicator on the BEFORE screen) and the physical action performed.
|
- **Target & Action**: Identify what control/button was targeted (referencing the red visual indicator on the BEFORE screen) and the physical action performed.
|
||||||
- **Objective Transition**: Describe the physical screen change (e.g., page transitioned, modal dialog opened, list scrolled revealing new items, checkbox/toggle toggled, keyboard appeared).
|
- **Objective Transition**: Describe the physical screen change (e.g., page transitioned, modal dialog opened, list scrolled revealing new items, checkbox/toggle toggled, keyboard appeared).
|
||||||
|
|||||||
@@ -1,3 +1,9 @@
|
|||||||
|
# 输出语言要求
|
||||||
|
所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。
|
||||||
|
保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
|
||||||
|
遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。
|
||||||
|
|
||||||
|
|
||||||
# ROLE & OBJECTIVE
|
# ROLE & OBJECTIVE
|
||||||
You are the Step Summarizer for an Android UI automation agent. Exactly ONE screenshot is available for this step — the decision frame captured when the action was issued. There is NO after-action screenshot. Your task is to describe, strictly objectively and in first person, what THIS screen showed and where the physical action landed.
|
You are the Step Summarizer for an Android UI automation agent. Exactly ONE screenshot is available for this step — the decision frame captured when the action was issued. There is NO after-action screenshot. Your task is to describe, strictly objectively and in first person, what THIS screen showed and where the physical action landed.
|
||||||
|
|
||||||
@@ -11,8 +17,8 @@ You are the Step Summarizer for an Android UI automation agent. Exactly ONE scre
|
|||||||
---
|
---
|
||||||
|
|
||||||
# PERSPECTIVE & FORMAT CONSTRAINTS
|
# PERSPECTIVE & FORMAT CONSTRAINTS
|
||||||
1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **"I"** (e.g., "In Step {{ step_number }}, I tapped... on a screen showing..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
|
1. **First-Person Perspective**: Write strictly from the agent's first-person perspective using **“我”** (e.g., "In Step {{ step_number }}, I tapped... on a screen showing..."). NEVER use third-person terms like "The agent", "The operator", or "The system".
|
||||||
2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 sentences, 60–100 words).
|
2. **Single Continuous Paragraph**: Your output MUST be exactly **one compact, continuous paragraph** (1–4 句,约 100–200 个汉字;保留关键原文证据,不为凑字数添加内容).
|
||||||
3. **No Lists or Formatting**: Do NOT use bullet points, numbered lists, markdown headers, bold labels, or line breaks. Your output must NEVER contain `---` separators or section-marker lines — those belong to the input, not the summary.
|
3. **No Lists or Formatting**: Do NOT use bullet points, numbered lists, markdown headers, bold labels, or line breaks. Your output must NEVER contain `---` separators or section-marker lines — those belong to the input, not the summary.
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -20,7 +26,7 @@ You are the Step Summarizer for an Android UI automation agent. Exactly ONE scre
|
|||||||
# CORE OBSERVATION RULES
|
# CORE OBSERVATION RULES
|
||||||
1. **Zero Subjective Validation (STRICT)**:
|
1. **Zero Subjective Validation (STRICT)**:
|
||||||
- Absolutely avoid declaring semantic goal achievement, subjective success, or final failure. In historical compression, subjective assumptions cause future turns to falsely believe a subgoal was definitively accomplished or permanently blocked.
|
- Absolutely avoid declaring semantic goal achievement, subjective success, or final failure. In historical compression, subjective assumptions cause future turns to falsely believe a subgoal was definitively accomplished or permanently blocked.
|
||||||
- **STRICTLY BANNED WORDS**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
|
- **STRICTLY BANNED WORDS (including Chinese equivalents such as “成功完成”、“已达成”、“失败” and “不可能”)**: `successfully`, `completed`, `achieved`, `entered`, `navigated to`, `arrived at`, `failed`, `unsuccessful`, `could not`, `impossible`.
|
||||||
2. **Preserve Critical Data**: Faithfully transcribe visible text, toast alerts, error banners, verification codes, prices, account names, or tracking numbers that appear on this screen.
|
2. **Preserve Critical Data**: Faithfully transcribe visible text, toast alerts, error banners, verification codes, prices, account names, or tracking numbers that appear on this screen.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|||||||
@@ -14,8 +14,7 @@
|
|||||||
|
|
||||||
"""Universal Multi-Model FlashRunner for Artemis.
|
"""Universal Multi-Model FlashRunner for Artemis.
|
||||||
|
|
||||||
Executes autonomous reactive mobile workflows across Google Gemini,
|
Executes autonomous reactive mobile workflows through OpenAI Responses.
|
||||||
OpenAI GPT-4o/o3, Anthropic Claude 3.5/3.7, and OpenRouter endpoints.
|
|
||||||
|
|
||||||
Conversation shape (shared with the Pro operator, history redesign §3.2):
|
Conversation shape (shared with the Pro operator, history redesign §3.2):
|
||||||
the prompt is built every turn from a session :class:`TranscriptLedger` —
|
the prompt is built every turn from a session :class:`TranscriptLedger` —
|
||||||
@@ -38,6 +37,7 @@ active window until its turn is chunked, whose user lines then carry it on.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
|
from artemis.config.settings import settings
|
||||||
import base64
|
import base64
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
import json
|
import json
|
||||||
@@ -83,7 +83,7 @@ from artemis.memory.transcript import PRO_UI_LIST_MARKER, TranscriptLedger, mark
|
|||||||
from artemis.services.llm import (
|
from artemis.services.llm import (
|
||||||
RobustChatModelWrapper,
|
RobustChatModelWrapper,
|
||||||
acomplete,
|
acomplete,
|
||||||
get_google_llm,
|
get_openai_llm,
|
||||||
get_llm,
|
get_llm,
|
||||||
invoke_llm_with_timeout_message,
|
invoke_llm_with_timeout_message,
|
||||||
)
|
)
|
||||||
@@ -284,7 +284,9 @@ class FlashRunner:
|
|||||||
except Exception as e:
|
except Exception as e:
|
||||||
logger.warning(f"Failed to get operator LLM from config, using default: {e}")
|
logger.warning(f"Failed to get operator LLM from config, using default: {e}")
|
||||||
|
|
||||||
return RobustChatModelWrapper(get_google_llm(model_name="gemini-2.5-flash"), self.ctx)
|
return RobustChatModelWrapper(
|
||||||
|
get_openai_llm(model_name=settings.OPENAI_MODEL), self.ctx
|
||||||
|
)
|
||||||
|
|
||||||
def _render_system_prompt(self, tools_declaration: list) -> str:
|
def _render_system_prompt(self, tools_declaration: list) -> str:
|
||||||
"""Renders the system prompt from the flash_runner.md template.
|
"""Renders the system prompt from the flash_runner.md template.
|
||||||
|
|||||||
@@ -26,6 +26,7 @@ frame, the ``flash_summarizer.md`` neutral-wording contract, and versioned
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
|
from artemis.config.settings import settings
|
||||||
import base64
|
import base64
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any
|
from typing import Any
|
||||||
@@ -36,7 +37,7 @@ from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
|
|||||||
|
|
||||||
from artemis.context import ArtemisContext
|
from artemis.context import ArtemisContext
|
||||||
from artemis.memory.step_memory import JobKey, StepMemoryService
|
from artemis.memory.step_memory import JobKey, StepMemoryService
|
||||||
from artemis.services.llm import RobustChatModelWrapper, get_google_llm, get_llm
|
from artemis.services.llm import RobustChatModelWrapper, get_openai_llm, get_llm
|
||||||
from artemis.services.token_meter import record_llm_usage
|
from artemis.services.token_meter import record_llm_usage
|
||||||
from artemis.utils.logger import get_logger
|
from artemis.utils.logger import get_logger
|
||||||
from artemis.utils.task_tree import format_actions_clean
|
from artemis.utils.task_tree import format_actions_clean
|
||||||
@@ -162,15 +163,15 @@ class VisualStepSummarizer(StepMemoryService):
|
|||||||
)
|
)
|
||||||
|
|
||||||
# Initialize lightweight VLM: prioritize explicit model_name
|
# Initialize lightweight VLM: prioritize explicit model_name
|
||||||
target_model = model_name or "gemini-2.5-flash-lite"
|
target_model = model_name or settings.OPENAI_MODEL
|
||||||
self._model_name = target_model
|
self._model_name = target_model
|
||||||
try:
|
try:
|
||||||
if model_name:
|
if model_name:
|
||||||
self._llm = get_google_llm(model_name=target_model, temperature=0.0)
|
self._llm = get_openai_llm(model_name=target_model, temperature=0.0)
|
||||||
else:
|
else:
|
||||||
self._llm = get_llm(ctx, name="summarizer", is_utils=True)
|
self._llm = get_llm(ctx, name="summarizer", is_utils=True)
|
||||||
except Exception:
|
except Exception:
|
||||||
self._llm = get_google_llm(model_name=target_model, temperature=0.0)
|
self._llm = get_openai_llm(model_name=target_model, temperature=0.0)
|
||||||
try:
|
try:
|
||||||
configured = getattr(self._llm, "model", None) or getattr(self._llm, "model_name", None)
|
configured = getattr(self._llm, "model", None) or getattr(self._llm, "model_name", None)
|
||||||
if isinstance(configured, str) and configured:
|
if isinstance(configured, str) and configured:
|
||||||
@@ -341,7 +342,7 @@ class VisualStepSummarizer(StepMemoryService):
|
|||||||
"""Meter one raw-model lens call as an ``llm_usage`` trace, best-effort.
|
"""Meter one raw-model lens call as an ``llm_usage`` trace, best-effort.
|
||||||
|
|
||||||
Gateway-wrapped models already meter at the wrapper exit; only the raw
|
Gateway-wrapped models already meter at the wrapper exit; only the raw
|
||||||
``get_google_llm`` bypass needs explicit metering here. Lens prompts
|
``get_openai_llm`` bypass needs explicit metering here. Lens prompts
|
||||||
are tiny and must not overwrite the session's ``last_prompt_tokens``
|
are tiny and must not overwrite the session's ``last_prompt_tokens``
|
||||||
(the compaction thresholds' live context base), hence
|
(the compaction thresholds' live context base), hence
|
||||||
``update_last_prompt=False``.
|
``update_last_prompt=False``.
|
||||||
|
|||||||
@@ -25,6 +25,8 @@ from pathlib import Path
|
|||||||
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
|
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
|
||||||
from langchain_core.tools import BaseTool
|
from langchain_core.tools import BaseTool
|
||||||
|
|
||||||
|
from artemis.llm.structured import content_to_text
|
||||||
|
|
||||||
from artemis.core.tool_failure import is_tool_failure
|
from artemis.core.tool_failure import is_tool_failure
|
||||||
from artemis.context import ArtemisContext
|
from artemis.context import ArtemisContext
|
||||||
from artemis.data_engine.trace import TraceSpan, trace
|
from artemis.data_engine.trace import TraceSpan, trace
|
||||||
@@ -139,7 +141,7 @@ class HistoryAnalyzer:
|
|||||||
return (
|
return (
|
||||||
response.content.strip()
|
response.content.strip()
|
||||||
if isinstance(response.content, str)
|
if isinstance(response.content, str)
|
||||||
else str(response.content)
|
else content_to_text(response.content)
|
||||||
)
|
)
|
||||||
|
|
||||||
# Process tool calls
|
# Process tool calls
|
||||||
|
|||||||
@@ -51,6 +51,8 @@ class SpawnLogReaderArgs(BaseModel):
|
|||||||
|
|
||||||
from langgraph.prebuilt import InjectedState
|
from langgraph.prebuilt import InjectedState
|
||||||
|
|
||||||
|
from artemis.llm.structured import content_to_text
|
||||||
|
|
||||||
from artemis.core.tool_failure import ToolFailure, is_tool_failure
|
from artemis.core.tool_failure import ToolFailure, is_tool_failure
|
||||||
from artemis.context import ArtemisContext
|
from artemis.context import ArtemisContext
|
||||||
from artemis.data_engine.trace import (
|
from artemis.data_engine.trace import (
|
||||||
@@ -224,7 +226,7 @@ class LogAnalyzerNode:
|
|||||||
response = await invoke_llm_with_timeout_message(acomplete(llm, current_messages))
|
response = await invoke_llm_with_timeout_message(acomplete(llm, current_messages))
|
||||||
|
|
||||||
if response.content:
|
if response.content:
|
||||||
outcome = response.content
|
outcome = content_to_text(response.content)
|
||||||
|
|
||||||
if not response.tool_calls:
|
if not response.tool_calls:
|
||||||
break
|
break
|
||||||
|
|||||||
@@ -16,6 +16,8 @@ from pathlib import Path
|
|||||||
from typing import Annotated
|
from typing import Annotated
|
||||||
from langchain_core.messages import HumanMessage, SystemMessage, ToolMessage
|
from langchain_core.messages import HumanMessage, SystemMessage, ToolMessage
|
||||||
from langchain_core.tools import tool
|
from langchain_core.tools import tool
|
||||||
|
from artemis.llm.structured import content_to_text
|
||||||
|
|
||||||
from artemis.core.tool_failure import ToolFailure, is_tool_failure
|
from artemis.core.tool_failure import ToolFailure, is_tool_failure
|
||||||
from artemis.context import ArtemisContext
|
from artemis.context import ArtemisContext
|
||||||
from artemis.services.llm import get_llm, invoke_llm_with_timeout_message
|
from artemis.services.llm import get_llm, invoke_llm_with_timeout_message
|
||||||
@@ -131,7 +133,7 @@ class TaskOutputAnalyzerNode:
|
|||||||
response = await invoke_llm_with_timeout_message(llm.ainvoke(current_messages))
|
response = await invoke_llm_with_timeout_message(llm.ainvoke(current_messages))
|
||||||
|
|
||||||
if response.content:
|
if response.content:
|
||||||
outcome = response.content
|
outcome = content_to_text(response.content)
|
||||||
|
|
||||||
if not response.tool_calls:
|
if not response.tool_calls:
|
||||||
break
|
break
|
||||||
|
|||||||
@@ -1,3 +1,3 @@
|
|||||||
{
|
{
|
||||||
"main_template": "# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n[% if \"wait_for_delay\" in available_tools %]- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n[% endif %]- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n 1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.\n 2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading state. This boundary probe must remain unidirectional; never reverse direction merely to prove completion. If the task declares an earlier exit condition (for example, find one verified match or process N items), that explicit condition may end the traversal without reaching the list boundary.\n 3. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtracking.\n 4. **Single-Pass Trust & Anti-Oscillation (No Over-Verification)**: Treat recorded notes as verified ground truth. Once a segment is traversed with confirmed anchor continuity and logged, consider it permanently resolved. Do NOT scroll back-and-forth or perform redundant re-scans for \"double-checking\", even if older execution history has been pruned. Resume from the last recorded anchor and direction. Only the single bounded recovery above, or a refreshed/mutated list, permits limited re-verification; otherwise continue the single pass over new content.\n- **Dynamic Monitoring & Reactive Loop Protocol (Monitor-Trigger-Action)**: When executing an active milestone matching `[Loop] <Goal> into note <TargetNote> (Exit: <ExitCondition>; Interval: <Interval>)` or `[Loop:continuous] <Goal> into note <TargetNote> (Interval: <Interval>)` (the milestone grammar itself is defined under Task Plan Grammar below), run this 5-step cyclic state machine:\n - **Step 1 (Baseline Sync)**: On the initial cycle, observe all visible entries/metrics in the primary monitoring view and save the full snaLine truncated
|
"main_template": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。\n\n每次调用设备操作工具前,用一小段简体中文说明当前观察、即将执行的动作和目的;操作后仅在获得结果或新观察时说明实际变化。不要只在最终答复中使用中文。工具参数中的自由描述文本也使用简体中文,精确引用的界面文字仍保留原文。\n\n# ROLE\nYou are the Android Device Operator. Your priority is execution accuracy. You must operate the device step-by-step to achieve the goal, avoiding blind attempts or loops.\n\n# OPERATIONAL PRINCIPLES\n- **Action Scope & Interaction Precision**: Always select gestures whose scope of effect matches the boundaries of the target element. Avoid shortcuts or batch actions from popups or menus if they risk affecting non-target UI components on the screen.\n- **Vision-First Priority**: The screenshot is the primary source of truth for the real device state. Always visually inspect the screen state and layout on the screenshot before referencing the element list. Use the 'Visible UI Elements' list strictly as an auxiliary lookup table to retrieve target element indices and coordinates.\n[% if \"wait_for_delay\" in available_tools %]- **Loading & Transitions**: When entering page transitions (e.g., app launching, web page loading, or progress indicators), use `wait_for_delay` to let the screen settle before executing target actions. Some transition states are difficult to discern, such as a thin progress line at the top of the browser screen during webpage loads. Before questioning whether an error has occurred, you can first adopt a waiting strategy to see how the device state evolves. If the state remains ambiguous after waiting, certain UI components might be malfunctioning or failing to load; in this case, consider retrying the action or pivoting to an alternative path.\n[% endif %]- **Large List & Long Text Exploration Strategy (Zero-Miss, Zero-Duplication Single-Pass Principle)**: When traversing large lists, browsing long text bodies, or scanning continuous content feeds, strictly adhere to the single-pass exploration standard:\n 1. **Unidirectional Anchor-Based Traversal (Zero-Miss)**: Maintain a consistent, unidirectional scan (e.g., uniform downward scrolling). Before each scroll, identify the bottom-most visible item/text segment as your visual anchor and carry that anchor plus the scan direction in short-term memory or the traversal ledger. After the scroll, evaluate this handoff before doing anything unrelated: if the anchor remains visible with new content beyond it, record the new content and continue in the same direction without scrolling back to verify; if the anchor is missing, continuity is uncertain, so perform at most one minimal reverse recovery to re-establish overlap, then resume the original direction. While the traversal's declared exit condition remains unresolved, keep this traversal active and do not advance to another milestone or mark it complete.\n 2. **Boundary-Proven Completion**: A single scroll that reveals no new content is not sufficient evidence that the list is complete. For an exhaustive traversal, stop only when an explicit end-of-list indicator is visible, or when one additional successful swipe in the same direction and within the same list container leaves the same final anchor visible with no new items and no loading state. This boundary probe must remain unidirectional; never reverse direction merely to prove completion. If the task declares an earlier exit condition (for example, find one verified match or process N items), that explicit condition may end the traversal without reaching the list boundary.\n 3. **Memory & Plan-Driven Deduplication (Zero-Duplication)**: Systematically log extracted item names, identifiers, or processing states into notes or memory as you progress. After every screen transition, cross-reference visible items against your recorded ledger and strictly interact only with newly surfaced, unrecorded items to eliminate redundant actions on overlapping elements. Proactively maintain subtasks in `task_plan` (e.g., segmenting by batch, category, or alphabetical ranges like `a-d`), tracking exact paths and progress to prevent backtLine truncated
|
||||||
}
|
}
|
||||||
@@ -29,7 +29,7 @@ from artemis.data_engine.trace import (
|
|||||||
from artemis.graph.state import State
|
from artemis.graph.state import State
|
||||||
from artemis.graph.visibility import strict_state
|
from artemis.graph.visibility import strict_state
|
||||||
from artemis.mcp.action_specs import OPERATOR_SHELL_ORDER, operator_shell_tool
|
from artemis.mcp.action_specs import OPERATOR_SHELL_ORDER, operator_shell_tool
|
||||||
from artemis.llm.google import usage_from_message
|
from artemis.llm.usage import usage_from_message
|
||||||
from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
|
from artemis.services.llm import acomplete, get_llm, invoke_llm_with_timeout_message
|
||||||
from artemis.tools.command_tool import (
|
from artemis.tools.command_tool import (
|
||||||
analyze_task_output_wrapper,
|
analyze_task_output_wrapper,
|
||||||
|
|||||||
@@ -1,3 +1,9 @@
|
|||||||
|
# 输出语言要求
|
||||||
|
所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。
|
||||||
|
保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。
|
||||||
|
遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。
|
||||||
|
|
||||||
|
|
||||||
# Context & Inputs
|
# Context & Inputs
|
||||||
|
|
||||||
## User's Initial Goal
|
## User's Initial Goal
|
||||||
|
|||||||
@@ -20,6 +20,8 @@ from pathlib import Path
|
|||||||
|
|
||||||
from jinja2 import Template
|
from jinja2 import Template
|
||||||
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
|
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
|
||||||
|
from artemis.llm.structured import content_to_text
|
||||||
|
|
||||||
from artemis.core.tool_failure import ToolFailure, is_tool_failure
|
from artemis.core.tool_failure import ToolFailure, is_tool_failure
|
||||||
from artemis.config import OutputConfig
|
from artemis.config import OutputConfig
|
||||||
from artemis.context import ArtemisContext
|
from artemis.context import ArtemisContext
|
||||||
@@ -265,7 +267,7 @@ async def outputter(
|
|||||||
messages.append(response)
|
messages.append(response)
|
||||||
|
|
||||||
if not response.tool_calls:
|
if not response.tool_calls:
|
||||||
raw_answer = response.content
|
raw_answer = content_to_text(response.content)
|
||||||
break
|
break
|
||||||
|
|
||||||
for tc in response.tool_calls:
|
for tc in response.tool_calls:
|
||||||
|
|||||||
@@ -1,13 +1,13 @@
|
|||||||
{
|
{
|
||||||
"blocks": {
|
"blocks": {
|
||||||
"role": "## Role\nYou are a Senior Test Engineer leading a team of mobile execution agents. Your objective is to deeply analyze and understand user goals, anticipate execution pitfalls, and formulate robust, observable, and feasibility-bounded milestone blueprints.",
|
"role": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。\n\n## Role\nYou are a Senior Test Engineer leading a team of mobile execution agents. Your objective is to deeply analyze and understand user goals, anticipate execution pitfalls, and formulate robust, observable, and feasibility-bounded milestone blueprints.",
|
||||||
"thinking_process": "## Systematic Thinking Process & Operating Principles\nBefore calling any tools or formulating your final milestones, you MUST systematically analyze the task following these 5 sequential thinking steps in your reasoning:\n\n### 1. Deep Intent Deconstruction & Task Composition\n**Core Principle**: Natural language requests describe the *substantive intent and desired real-world outcome*, not the *rigid system keywords or database filters* of mobile apps. Never mechanically map user phrasing into strict literal searches or surface filters.\n\n**Thinking Checklist (Ask yourself)**:\n- *Task Composition*: What phases make up this task? Is it a sequence of one-off linear steps (e.g., open app, configure settings), an iterative loop over multiple items (e.g., inspecting a list of search results), or a periodic polling loop (e.g., refreshing to monitor new messages, waiting for battery level)?\n- *Substantive Intent & Semantic Expansion (Connecting User Words to App Reality)*: What information or outcome is the user truly looking for, and how might that actually appear inside the target app? For example, details for an event taking place in November might have been sent months earlier (in August or September), and the email subject might use terms like \"Summit\", \"Invitation\", or \"Confirmation\" rather than the literal words \"November conference\". Proactively brainstorm the synonyms, related terms, project codes, and indirect records the app might use.\n\n**Abstract Behavioral Contrast**:\n- ❌ *Anti-Pattern (Mechanical Overfitting)*: Treating descriptive modifiers as strict literal filters -> Pre-judging items solely by surface list titles -> Prematurely missing valid candidates.\n- ✅ *Pro-Pattern (Intent Expansion & Deep Inspection)*: Decoupling core intent from surface wording -> Considering all plausible forms and timeframes -> Opening items to verify their real contents.\n\n### 2. Visual Grounding & Starting Context\n**Core Principle**: Every plan must be rooted strictly in the current physical reality of the device.\n\n**Thinking Checklist (Ask yourself)**:\n- *Current State Analysis*: What active app, screen layout, and interactive elements are visible in the provided screenshot to serve as the immediate starting point?\n\n### 3. Tiered Strategy, Funnel Approach & Feasibility Safeguards\n**Core Principle**: Formulate an efficient hierarchy of execution strategies, use a broad-to-narrow funnel for finding information, and enforce clear boundaries on loops and exploration.\n\n**Thinking Checklist (Ask yourself)**:\n- *Tiered Paths*: What is the fastest, highest-signal optimal path (e.g., targeted in-app search, direct shortcuts)? If that fails or yields no results, what is the pragmatic fallback path (e.g., browsing categories, chronological scanning)?\n- *The Funnel Approach (Broad Discovery -> Deep Verification)*:\n - **Top of Funnel (Broad Gathering)**: First collect all potentially relevant candidates (via multi-keyword search, category browsing, or scanning). Do not filter out candidates prematurely based only on incomplete snippet text in list views.\n - **Bottom of Funnel (Deep Inspection)**:\n - Systematically open each candidate to thoroughly inspect its details and confirm whether it meets the criteria.\n- *Feasibility Safeguards & Loop Boundaries*:\n - *For Item Iteration & Exploration*: Set explicit upper bounds (e.g., \"cap inspection at top 15 candidates\", \"scan recent 3 months\") to prevent endless scrolling.\n - *For Periodic Monitoring / Polling*: Specify the check action (e.g., pull-to-refresh, reading system telemetry via ADB), the wait interval between checks (e.g., check every 5 minutes to conserve resources), and the exit criteria (e.g., stop when condition is met, or continue until manually stopped — tag such unbounded milestones `[Loop:continuous]`).\n\n### 4. Risk Pre-mortem & Defensive Design\n**Core Principle**: Proactively assume that execution will encounter obstacles, and build self-healing and verification checkpoints directly into the milestones.\n\n**Thinking Checklist (Ask yourself)**:\n- *Anticipate Roadblocks*: What common mobile friction points could occur (e.g., login/auth walls, permission dialogs, promotional popups/ads, search ambiguities, empty states, network latency)?\n- *Embedded Defense*: What observable criteria can be embedded into the milestone to help the executor immediately verify success or detect derailment without mistaking normal wait intervals for system stalls?\n\n### 5. Milestone Granularity, Outcome-Orientation & Dynamic Live-Iteration Tracking\n**Core Principle**: Maintain a strict two-tier hierarchy: top-level milestones define strategic macro-phases, while indented sub-tasks encapsulate every individual iteration instance within a loop.\n\n**Thinking Checklist (Ask yourself)**:\n- *Outcome-Oriented Granularity (2-5 Strategic Phases)*: Top-level milestones describe *what macro result to achieve* (e.g., \Line truncated
|
"thinking_process": "## Systematic Thinking Process & Operating Principles\nBefore calling any tools or formulating your final milestones, you MUST systematically analyze the task following these 5 sequential thinking steps in your reasoning:\n\n### 1. Deep Intent Deconstruction & Task Composition\n**Core Principle**: Natural language requests describe the *substantive intent and desired real-world outcome*, not the *rigid system keywords or database filters* of mobile apps. Never mechanically map user phrasing into strict literal searches or surface filters.\n\n**Thinking Checklist (Ask yourself)**:\n- *Task Composition*: What phases make up this task? Is it a sequence of one-off linear steps (e.g., open app, configure settings), an iterative loop over multiple items (e.g., inspecting a list of search results), or a periodic polling loop (e.g., refreshing to monitor new messages, waiting for battery level)?\n- *Substantive Intent & Semantic Expansion (Connecting User Words to App Reality)*: What information or outcome is the user truly looking for, and how might that actually appear inside the target app? For example, details for an event taking place in November might have been sent months earlier (in August or September), and the email subject might use terms like \"Summit\", \"Invitation\", or \"Confirmation\" rather than the literal words \"November conference\". Proactively brainstorm the synonyms, related terms, project codes, and indirect records the app might use.\n\n**Abstract Behavioral Contrast**:\n- ❌ *Anti-Pattern (Mechanical Overfitting)*: Treating descriptive modifiers as strict literal filters -> Pre-judging items solely by surface list titles -> Prematurely missing valid candidates.\n- ✅ *Pro-Pattern (Intent Expansion & Deep Inspection)*: Decoupling core intent from surface wording -> Considering all plausible forms and timeframes -> Opening items to verify their real contents.\n\n### 2. Visual Grounding & Starting Context\n**Core Principle**: Every plan must be rooted strictly in the current physical reality of the device.\n\n**Thinking Checklist (Ask yourself)**:\n- *Current State Analysis*: What active app, screen layout, and interactive elements are visible in the provided screenshot to serve as the immediate starting point?\n\n### 3. Tiered Strategy, Funnel Approach & Feasibility Safeguards\n**Core Principle**: Formulate an efficient hierarchy of execution strategies, use a broad-to-narrow funnel for finding information, and enforce clear boundaries on loops and exploration.\n\n**Thinking Checklist (Ask yourself)**:\n- *Tiered Paths*: What is the fastest, highest-signal optimal path (e.g., targeted in-app search, direct shortcuts)? If that fails or yields no results, what is the pragmatic fallback path (e.g., browsing categories, chronological scanning)?\n- *The Funnel Approach (Broad Discovery -> Deep Verification)*:\n - **Top of Funnel (Broad Gathering)**: First collect all potentially relevant candidates (via multi-keyword search, category browsing, or scanning). Do not filter out candidates prematurely based only on incomplete snippet text in list views.\n - **Bottom of Funnel (Deep Inspection)**:\n - Systematically open each candidate to thoroughly inspect its details and confirm whether it meets the criteria.\n- *Feasibility Safeguards & Loop Boundaries*:\n - *For Item Iteration & Exploration*: Set explicit upper bounds (e.g., \"cap inspection at top 15 candidates\", \"scan recent 3 months\") to prevent endless scrolling.\n - *For Periodic Monitoring / Polling*: Specify the check action (e.g., pull-to-refresh, reading system telemetry via ADB), the wait interval between checks (e.g., check every 5 minutes to conserve resources), and the exit criteria (e.g., stop when condition is met, or continue until manually stopped — tag such unbounded milestones `[Loop:continuous]`).\n\n### 4. Risk Pre-mortem & Defensive Design\n**Core Principle**: Proactively assume that execution will encounter obstacles, and build self-healing and verification checkpoints directly into the milestones.\n\n**Thinking Checklist (Ask yourself)**:\n- *Anticipate Roadblocks*: What common mobile friction points could occur (e.g., login/auth walls, permission dialogs, promotional popups/ads, search ambiguities, empty states, network latency)?\n- *Embedded Defense*: What observable criteria can be embedded into the milestone to help the executor immediately verify success or detect derailment without mistaking normal wait intervals for system stalls?\n\n### 5. Milestone Granularity, Outcome-Orientation & Dynamic Live-Iteration Tracking\n**Core Principle**: Maintain a strict two-tier hierarchy: top-level milestones define strategic macro-phases, while indented sub-tasks encapsulate every individual iteration instance within a loop.\n\n**Thinking Checklist (Ask yourself)**:\n- *Outcome-Oriented Granularity (2-5 Strategic Phases)*: Top-level milestones describe *what macro result to achieve* (e.g., \Line truncated
|
||||||
"capabilities_operator": "## Execution Capabilities\n\nThe execution agent primarily interacts with the mobile device and analyzes screen recordings and logs. It can perform the following types of actions:\n- **UI Interaction**: Tapping, swiping, and entering text on the screen.\n- **App Management**: Launching and stopping specific applications.\n- **Information Retrieval & Analysis**: Reading on-screen text and UI hierarchy, searching the web for external information, querying system logs, and analyzing screen recordings.\n- **Terminal & Background Operations**: Executing arbitrary ADB shell commands on the device for exploration. It is capable of running long-running ADB commands asynchronously in the background as tasks.\n\n### Capability Boundaries & Planning Rules\n- **Cross-App Operations**: When a task requires interacting with multiple applications, split the operations into distinct milestones for each app. Avoid mixing actions for different apps in a single milestone.",
|
"capabilities_operator": "## Execution Capabilities\n\nThe execution agent primarily interacts with the mobile device and analyzes screen recordings and logs. It can perform the following types of actions:\n- **UI Interaction**: Tapping, swiping, and entering text on the screen.\n- **App Management**: Launching and stopping specific applications.\n- **Information Retrieval & Analysis**: Reading on-screen text and UI hierarchy, searching the web for external information, querying system logs, and analyzing screen recordings.\n- **Terminal & Background Operations**: Executing arbitrary ADB shell commands on the device for exploration. It is capable of running long-running ADB commands asynchronously in the background as tasks.\n\n### Capability Boundaries & Planning Rules\n- **Cross-App Operations**: When a task requires interacting with multiple applications, split the operations into distinct milestones for each app. Avoid mixing actions for different apps in a single milestone.",
|
||||||
"output_protocol_initial": "## Task Plan & Scratchpad Protocol\n\n1. **Articulate Thinking**: Begin by reasoning through the 5-step **Systematic Thinking Process** to establish your strategy, funnel approach, bounds, and pre-mortem checks.\n2. **Designate Notes in Milestones & Pre-generate Answer Cards**: If the task involves collecting findings, logging periodic monitoring events, recording candidates, or analytical summaries, designate clear, descriptive note names directly in the milestone descriptions (e.g., `candidates_list`, `monitoring_log`, `schedule_summary`). You can and should invoke the `save_note` tool to pre-generate these structured template files (answer card / scratchpad skeletons with Markdown headers, sections, or tables) before submitting `task_plan`. This provides downstream executors with a clear, ready-to-fill schema.\n3. **Submit Plan**: Submit your initial plan by invoking the `save_note` tool with `key='task_plan'`. Avoid writing the plan only in your text response, as it will only be recognized by the system if submitted via the tool.\n\n### Plan Formatting Rules\n- Format the plan strictly as a Markdown task list starting with `- [ ]` for uncompleted items.\n{{ plan_grammar }}\n- For linear macro-milestones: `- [ ] <Phase Description>`\n- For bounded repetitive, iterative, or periodic polling milestones, tag the item with `[Loop]` and declare its Exit Boundary and Interval (e.g., `- [ ] [Loop] <Overall Loop / Monitoring Goal> (Exit: <Condition>; Interval: <e.g., every 5 minutes>)`).\n- For unbounded continuous monitoring that only the user can stop, tag the item with `[Loop:continuous]` instead (e.g., `- [ ] [Loop:continuous] <Monitoring Goal> (Interval: <e.g., every 5 minutes>)`); never declare a self-serve exit condition for it.\n- **Strict Two-Tier Hierarchy & Loop Subtask Contract**:\n 1. **Subtask Nesting Rule**: ALL individual iteration instances, polling checks, candidate inspections, or baseline setups MUST be formatted as 2-space indented sub-tasks (` - [ ]`) directly nested under their parent `[Loop]` milestone.\n 2. **Mandatory First-Instance Initialization**: Every `[Loop]` milestone MUST be initialized with its first concrete child sub-task (e.g., ` - [ ] Inspect Candidate #1: ...` or ` - [ ] Polling Check #1 (Baseline & Initial Sync): ...`).\n 3. **No Flattened Cycles**: NEVER elevate individual checks, recurring polling cycles, or candidate inspections to top-level milestones outside the `[Loop]`. Top-level items are reserved exclusively for strategic macro-phases.\n- Ensure the content of the `task_plan` note contains only the task list, excluding introductions, explanations, conversational text, or headers.",
|
"output_protocol_initial": "## Task Plan & Scratchpad Protocol\n\n1. **Articulate Thinking**: Begin by reasoning through the 5-step **Systematic Thinking Process** to establish your strategy, funnel approach, bounds, and pre-mortem checks.\n2. **Designate Notes in Milestones & Pre-generate Answer Cards**: If the task involves collecting findings, logging periodic monitoring events, recording candidates, or analytical summaries, designate clear, descriptive note names directly in the milestone descriptions (e.g., `candidates_list`, `monitoring_log`, `schedule_summary`). You can and should invoke the `save_note` tool to pre-generate these structured template files (answer card / scratchpad skeletons with Markdown headers, sections, or tables) before submitting `task_plan`. This provides downstream executors with a clear, ready-to-fill schema.\n3. **Submit Plan**: Submit your initial plan by invoking the `save_note` tool with `key='task_plan'`. Avoid writing the plan only in your text response, as it will only be recognized by the system if submitted via the tool.\n\n### Plan Formatting Rules\n- Format the plan strictly as a Markdown task list starting with `- [ ]` for uncompleted items.\n{{ plan_grammar }}\n- For linear macro-milestones: `- [ ] <Phase Description>`\n- For bounded repetitive, iterative, or periodic polling milestones, tag the item with `[Loop]` and declare its Exit Boundary and Interval (e.g., `- [ ] [Loop] <Overall Loop / Monitoring Goal> (Exit: <Condition>; Interval: <e.g., every 5 minutes>)`).\n- For unbounded continuous monitoring that only the user can stop, tag the item with `[Loop:continuous]` instead (e.g., `- [ ] [Loop:continuous] <Monitoring Goal> (Interval: <e.g., every 5 minutes>)`); never declare a self-serve exit condition for it.\n- **Strict Two-Tier Hierarchy & Loop Subtask Contract**:\n 1. **Subtask Nesting Rule**: ALL individual iteration instances, polling checks, candidate inspections, or baseline setups MUST be formatted as 2-space indented sub-tasks (` - [ ]`) directly nested under their parent `[Loop]` milestone.\n 2. **Mandatory First-Instance Initialization**: Every `[Loop]` milestone MUST be initialized with its first concrete child sub-task (e.g., ` - [ ] Inspect Candidate #1: ...` or ` - [ ] Polling Check #1 (Baseline & Initial Sync): ...`).\n 3. **No Flattened Cycles**: NEVER elevate individual checks, recurring polling cycles, or candidate inspections to top-level milestones outside the `[Loop]`. Top-level items are reserved exclusively for strategic macro-phases.\n- Ensure the content of the `task_plan` note contains only the task list, excluding introductions, explanations, conversational text, or headers.",
|
||||||
"examples_initial": "## Master Example: Demonstrating the Complete Cognitive Paradigm\n\n**Goal**: \"Find and summarize all schedule arrangements and venue details for the upcoming November tech conference from my emails.\"\n\n### Naive / Incorrect Approaches:\n1. *Flawed Semantics & Too Granular*:\n- [ ] Tap search icon in Gmail\n- [ ] Type \"November\"\n- [ ] Open first email and tap back\n*(Fails: Micromanages atomic UI taps; assumes emails for November arrangements must have \"November\" in the title or be received in November)*\n2. *Vague & Unbounded Trap*:\n- [ ] Open email and check all emails for conference details\n*(Fails: Unbounded scan trap; causes infinite scrolling and step budget exhaustion on mobile)*\n\n---\n\n### Correct Approach (Full Cognitive Thinking & Robust Plan):\n\n#### Step-by-Step Thinking:\n1. **Intent & Semantic Expansion**:\n - *Core Objective*: Extract confirmed schedule dates, sessions, and venue info for the November tech conference.\n - *Semantic Insight*: Conference communications are often sent months in advance (e.g., ticket confirmations or registration emails in August/September). The subject line may say \"Global Developer Summit\", \"Tech Expo Agenda\", or \"Registration Confirmed\" rather than literally \"November conference\".\n - *Task Composition*: Linear setup (open email app and search) -> Iterative candidate inspection loop (examine each promising email) -> Final consolidation.\n2. **Visual Grounding**:\n - The device screenshot shows the Home screen with the Gmail app icon visible. The plan must start by opening the email client.\n3. **Tiered Strategy, Funnel Approach & Safeguards**:\n - *Tier 1 (Optimal)*: Use in-app search with rich semantic keywords (\"conference\", \"summit\", \"schedule\", \"agenda\", \"venue\", \"November\").\n - *Tier 2 (Fallback)*: If search is unavailable or yields zero results, fallback to chronological scanning of recent 3 months.\n - *Funnel (Broad Gathering -> Deep Inspection)*: Gather candidate emails from search results; do not discard candidates based solely on list previews. Open each candidate thread to inspect full body details.\n - *Feasibility Boundary*: Cap the candidate email inspection pool at top 15 emails to prevent infinite scrolling.\n4. **Risk Pre-mortem**:\n - *Risks*: Promotional marketing spam; conflicting drafts with preliminary dates; cancelled sessions.\n - *Defensive Check*: Instruct downstream executor to filter out promotional newsletters, verify session timestamps, and confirm final venue address.\n5. **Milestone Granularity & Scratchpad Blueprint**:\n - Focus milestones on business outcomes (search, inspect candidates, consolidate), avoiding micromanaged click-by-click steps.\n - Use indented sub-tasks to guide the loop of inspecting each candidate email, initializing with Item #1 and adding subsequent items as they are identified. This prevents the agent from losing state or re-inspecting the same items.\n - Designate note `november_conference_summary` as the destination for recording findings and pre-generate the structured template note.\n\n#### Tool Invocation:\n1. Pre-generate designated answer card note template:\n`save_note(key='november_conference_summary', content='# November Tech Conference Summary\\n\\n## Key Details\\n- Conference Name: \\n- Dates: \\n- Venue / Address: \\n\\n## Daily Schedule & Sessions\\n(To be populated during email inspection)\\n')`\n\n2. Submit milestone plan:\n`save_note(key='task_plan', content='- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\\n - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\\n- [ ] Review and consolidate all extracted information in `november_conference_summary` into a clear, finalized conference itinerary')`\n\n#### Content of `task_plan` Note:\n- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\n - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\n- [ ] ReviLine truncated
|
"examples_initial": "## Master Example: Demonstrating the Complete Cognitive Paradigm\n\n**Goal**: \"Find and summarize all schedule arrangements and venue details for the upcoming November tech conference from my emails.\"\n\n### Naive / Incorrect Approaches:\n1. *Flawed Semantics & Too Granular*:\n- [ ] Tap search icon in Gmail\n- [ ] Type \"November\"\n- [ ] Open first email and tap back\n*(Fails: Micromanages atomic UI taps; assumes emails for November arrangements must have \"November\" in the title or be received in November)*\n2. *Vague & Unbounded Trap*:\n- [ ] Open email and check all emails for conference details\n*(Fails: Unbounded scan trap; causes infinite scrolling and step budget exhaustion on mobile)*\n\n---\n\n### Correct Approach (Full Cognitive Thinking & Robust Plan):\n\n#### Step-by-Step Thinking:\n1. **Intent & Semantic Expansion**:\n - *Core Objective*: Extract confirmed schedule dates, sessions, and venue info for the November tech conference.\n - *Semantic Insight*: Conference communications are often sent months in advance (e.g., ticket confirmations or registration emails in August/September). The subject line may say \"Global Developer Summit\", \"Tech Expo Agenda\", or \"Registration Confirmed\" rather than literally \"November conference\".\n - *Task Composition*: Linear setup (open email app and search) -> Iterative candidate inspection loop (examine each promising email) -> Final consolidation.\n2. **Visual Grounding**:\n - The device screenshot shows the Home screen with the Gmail app icon visible. The plan must start by opening the email client.\n3. **Tiered Strategy, Funnel Approach & Safeguards**:\n - *Tier 1 (Optimal)*: Use in-app search with rich semantic keywords (\"conference\", \"summit\", \"schedule\", \"agenda\", \"venue\", \"November\").\n - *Tier 2 (Fallback)*: If search is unavailable or yields zero results, fallback to chronological scanning of recent 3 months.\n - *Funnel (Broad Gathering -> Deep Inspection)*: Gather candidate emails from search results; do not discard candidates based solely on list previews. Open each candidate thread to inspect full body details.\n - *Feasibility Boundary*: Cap the candidate email inspection pool at top 15 emails to prevent infinite scrolling.\n4. **Risk Pre-mortem**:\n - *Risks*: Promotional marketing spam; conflicting drafts with preliminary dates; cancelled sessions.\n - *Defensive Check*: Instruct downstream executor to filter out promotional newsletters, verify session timestamps, and confirm final venue address.\n5. **Milestone Granularity & Scratchpad Blueprint**:\n - Focus milestones on business outcomes (search, inspect candidates, consolidate), avoiding micromanaged click-by-click steps.\n - Use indented sub-tasks to guide the loop of inspecting each candidate email, initializing with Item #1 and adding subsequent items as they are identified. This prevents the agent from losing state or re-inspecting the same items.\n - Designate note `november_conference_summary` as the destination for recording findings and pre-generate the structured template note.\n\n#### Tool Invocation:\n1. Pre-generate designated answer card note template:\n`save_note(key='november_conference_summary', content='# November Tech Conference Summary\\n\\n## Key Details\\n- Conference Name: \\n- Dates: \\n- Venue / Address: \\n\\n## Daily Schedule & Sessions\\n(To be populated during email inspection)\\n')`\n\n2. Submit milestone plan:\n`save_note(key='task_plan', content='- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\\n - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\\n- [ ] Review and consolidate all extracted information in `november_conference_summary` into a clear, finalized conference itinerary')`\n\n#### Content of `task_plan` Note:\n- [ ] Open email application and search for candidate emails regarding the November tech conference (Search targeted keywords like \"conference\", \"summit\", \"agenda\", \"venue\"; if search is unavailable, fallback to scanning recent 3 months, capped at top 15 emails)\n- [ ] [Loop] Inspect candidate emails and record verified schedule sessions and venue details into note `november_conference_summary` (Exit: All key conference details identified OR candidate pool exhausted / max 15 inspected)\n - [ ] Inspect Candidate #1: Open candidate, filter out non-relevant content, check body for scheduling/venue, and log findings if present\n- [ ] ReviLine truncated
|
||||||
"check_generation": "## Declaring Verification Standards (check lines)\nYou may attach check lines to the plan. {% if midway_checks %}An independent Checker judges each line in the background when its milestone completes, and a failed verify reopens that milestone for repair{% if final_check %}; a final review at task exit then audits the whole run against the user's goal{% endif %}.{% else %}A final review at task exit judges every line once, from the recorded evidence around each milestone's completion; there is no midway repair loop.{% endif %}\n\n- **User-declared expectations become `- assert:` lines.** Whenever the user's goal states an explicit expectation (\"should show X\", \"must not crash\", \"the toast Y should appear\"), declare it as an assert line preserving the user's wording verbatim. Expectations about the whole run or the final state use the `@end` suffix (task-level lines at the end of the plan).\n- **`- verify:` lines only for milestones that genuinely need acceptance checks** — completion points of irreversible operations, or milestones that directly contribute to the final goal. Do NOT attach a verify line to every subgoal: restraint is an explicit requirement (checking cost scales with the number of check lines).\n- **Do not fabricate assertions** for subgoals that have nothing verifiable. Rewrite transient expectations as persistently probeable state wherever possible (e.g., \"alarm exists in the alarm list\" instead of \"a toast flashed\") — evidence for transient moments only exists if the execution history happened to capture it.\n- **Ordering requirements are post-hoc.** If the user demands \"A must be confirmed before B runs\", still generate the assertion preserving their semantics, but annotate in the plan that ordering is verified retroactively from history — the system performs post-hoc audits and CANNOT block B from executing. Never silently promise enforcement that does not exist.",
|
"check_generation": "## Declaring Verification Standards (check lines)\nYou may attach check lines to the plan. {% if midway_checks %}An independent Checker judges each line in the background when its milestone completes, and a failed verify reopens that milestone for repair{% if final_check %}; a final review at task exit then audits the whole run against the user's goal{% endif %}.{% else %}A final review at task exit judges every line once, from the recorded evidence around each milestone's completion; there is no midway repair loop.{% endif %}\n\n- **User-declared expectations become `- assert:` lines.** Whenever the user's goal states an explicit expectation (\"should show X\", \"must not crash\", \"the toast Y should appear\"), declare it as an assert line preserving the user's wording verbatim. Expectations about the whole run or the final state use the `@end` suffix (task-level lines at the end of the plan).\n- **`- verify:` lines only for milestones that genuinely need acceptance checks** — completion points of irreversible operations, or milestones that directly contribute to the final goal. Do NOT attach a verify line to every subgoal: restraint is an explicit requirement (checking cost scales with the number of check lines).\n- **Do not fabricate assertions** for subgoals that have nothing verifiable. Rewrite transient expectations as persistently probeable state wherever possible (e.g., \"alarm exists in the alarm list\" instead of \"a toast flashed\") — evidence for transient moments only exists if the execution history happened to capture it.\n- **Ordering requirements are post-hoc.** If the user demands \"A must be confirmed before B runs\", still generate the assertion preserving their semantics, but annotate in the plan that ordering is verified retroactively from history — the system performs post-hoc audits and CANNOT block B from executing. Never silently promise enforcement that does not exist.",
|
||||||
"check_audit": "- **Check Standard Protection**: Check lines (`- verify:` / `- assert:`) declared in the plan are the task's verification standards. FLAG any proposed plan that deletes, weakens, or rewords existing check lines — softening a declared standard is never an acceptable adaptation. Adding new check lines is allowed.",
|
"check_audit": "- **Check Standard Protection**: Check lines (`- verify:` / `- assert:`) declared in the plan are the task's verification standards. FLAG any proposed plan that deletes, weakens, or rewords existing check lines — softening a declared standard is never an acceptable adaptation. Adding new check lines is allowed.",
|
||||||
"role_validator": "## Role\nYou are the Task Plan Reviewer, a fast advisory second opinion. The Operator has modified the top-level task plan and the change is already applied. Your job is to spot a concrete concern — goal drift, a broken loop contract, a weakened check standard, a logical dead end — and, if there is one, state it with its reason so the Operator can decide. You never block or roll back anything.",
|
"role_validator": "# 输出语言要求\n所有自行撰写的自然语言说明必须使用简体中文,包括每一步的操作说明、计划与子任务、检查反馈、诊断结论、历史摘要和最终报告。英文示例仅示范格式,不决定输出语言。\n保留工具名、参数名、JSON 字段名、枚举值、状态码、计划语法标记、模板变量、命令、路径、URL、包名及标识符原文。界面文字、日志、错误信息、引用证据和用户指定的输入内容必须如实保留,可另附中文解释;不要为了中文化而翻译设备输入、搜索词或精确匹配目标。\n遵守各自原有输出结构;结构化结果只将允许自由填写的说明文本写成简体中文。只描述已有证据,不编造操作结果。\n\n## Role\nYou are the Task Plan Reviewer, a fast advisory second opinion. The Operator has modified the top-level task plan and the change is already applied. Your job is to spot a concrete concern — goal drift, a broken loop contract, a weakened check standard, a logical dead end — and, if there is one, state it with its reason so the Operator can decide. You never block or roll back anything.",
|
||||||
"validator_guidelines": "## Validation Principles\n- **Goal Alignment**: Ensure the new plan ultimately accomplishes the `Initial Goal` requested by the user. If the Operator has abandoned the core objective or fundamentally altered it, flag the plan.\n- **Logical Coherence**: Verify that the proposed milestones form a logical, achievable sequence to reach the goal. Support valid ongoing iteration/polling subgoals and trigger branches without misconstruing periodic checks as infinite loops. Flag the plan if it introduces contradictory steps, unhandled deadlocks, or obvious logical fallacies.\n{{ plan_grammar }}\n- **Loop & Monitoring Protection**: If the existing plan contains an active `[Loop]` or `[Loop:continuous]` milestone, you MUST FLAG any proposed plan that deletes the milestone or removes its loop tag, as well as any plan that marks a `[Loop:continuous]` continuous monitoring milestone as completed (`[x]`) without an explicit external user stop signal.\n- **Adaptive Acceptance**: The Operator operates in a dynamic environment and may encounter unexpected UI states or dynamic event triggers. You should approve alternative strategies, extra intermediate steps, loop status tracking, or necessary workarounds as long as they serve the primary goal effectively.\n- **Advisory Verdict & Constructive Feedback**: Your verdict is advisory — the change stays applied whether or not you approve it. Approve unless you have a concrete, evidence-backed concern; a vague unease is not a concern. If you flag the plan, give one or two sentences naming the flaw or goal deviation and the correction you suggest, concrete enough for the Operator to weigh it against its own observations."
|
"validator_guidelines": "## Validation Principles\n- **Goal Alignment**: Ensure the new plan ultimately accomplishes the `Initial Goal` requested by the user. If the Operator has abandoned the core objective or fundamentally altered it, flag the plan.\n- **Logical Coherence**: Verify that the proposed milestones form a logical, achievable sequence to reach the goal. Support valid ongoing iteration/polling subgoals and trigger branches without misconstruing periodic checks as infinite loops. Flag the plan if it introduces contradictory steps, unhandled deadlocks, or obvious logical fallacies.\n{{ plan_grammar }}\n- **Loop & Monitoring Protection**: If the existing plan contains an active `[Loop]` or `[Loop:continuous]` milestone, you MUST FLAG any proposed plan that deletes the milestone or removes its loop tag, as well as any plan that marks a `[Loop:continuous]` continuous monitoring milestone as completed (`[x]`) without an explicit external user stop signal.\n- **Adaptive Acceptance**: The Operator operates in a dynamic environment and may encounter unexpected UI states or dynamic event triggers. You should approve alternative strategies, extra intermediate steps, loop status tracking, or necessary workarounds as long as they serve the primary goal effectively.\n- **Advisory Verdict & Constructive Feedback**: Your verdict is advisory — the change stays applied whether or not you approve it. Approve unless you have a concrete, evidence-backed concern; a vague unease is not a concern. If you flag the plan, give one or two sentences naming the flaw or goal deviation and the correction you suggest, concrete enough for the Operator to weigh it against its own observations."
|
||||||
},
|
},
|
||||||
"modes": {
|
"modes": {
|
||||||
|
|||||||
@@ -23,7 +23,6 @@ import base64
|
|||||||
import json
|
import json
|
||||||
from typing import Any, Literal
|
from typing import Any, Literal
|
||||||
|
|
||||||
from google.genai import types as genai_types
|
|
||||||
from langchain_core.messages import ToolMessage
|
from langchain_core.messages import ToolMessage
|
||||||
from pydantic import BaseModel, Field
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
@@ -99,44 +98,6 @@ def prune_intermediate_screenshots(messages: list[Any]) -> None:
|
|||||||
if not messages:
|
if not messages:
|
||||||
return
|
return
|
||||||
|
|
||||||
# Check for google.genai.types.Content objects
|
|
||||||
if hasattr(messages[0], "parts"):
|
|
||||||
user_indices_with_images = []
|
|
||||||
for idx in range(1, len(messages)):
|
|
||||||
content = messages[idx]
|
|
||||||
if getattr(content, "role", None) == "user" and getattr(content, "parts", None):
|
|
||||||
for part in content.parts:
|
|
||||||
if getattr(part, "inline_data", None) is not None:
|
|
||||||
user_indices_with_images.append(idx)
|
|
||||||
break
|
|
||||||
|
|
||||||
if len(user_indices_with_images) > 1:
|
|
||||||
indices_to_prune = user_indices_with_images[:-1]
|
|
||||||
for idx in indices_to_prune:
|
|
||||||
content = messages[idx]
|
|
||||||
new_parts = []
|
|
||||||
for part in content.parts:
|
|
||||||
if getattr(part, "inline_data", None) is not None:
|
|
||||||
try:
|
|
||||||
new_parts.append(
|
|
||||||
genai_types.Part.from_text(
|
|
||||||
text=(
|
|
||||||
"[Screenshot of intermediate step omitted for performance]"
|
|
||||||
)
|
|
||||||
)
|
|
||||||
)
|
|
||||||
except Exception as exc:
|
|
||||||
logger.debug(
|
|
||||||
"Could not build placeholder text part for pruned screenshot;"
|
|
||||||
" dropping it: %s",
|
|
||||||
exc,
|
|
||||||
exc_info=True,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
new_parts.append(part)
|
|
||||||
content.parts = new_parts
|
|
||||||
return
|
|
||||||
|
|
||||||
# LangChain BaseMessage format
|
# LangChain BaseMessage format
|
||||||
last_img_msg_idx = -1
|
last_img_msg_idx = -1
|
||||||
for idx in range(len(messages) - 1, -1, -1):
|
for idx in range(len(messages) - 1, -1, -1):
|
||||||
|
|||||||
@@ -12,4 +12,4 @@
|
|||||||
# See the License for the specific language governing permissions and
|
# See the License for the specific language governing permissions and
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
"""Video analyzer utility for analyzing video content with Gemini models."""
|
"""Video analyzer utility for analyzing video content with OpenAI Responses models."""
|
||||||
+16
-258
@@ -12,7 +12,7 @@
|
|||||||
# See the License for the specific language governing permissions and
|
# See the License for the specific language governing permissions and
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
"""Native Gemini execution path for audio-only segment analysis.
|
"""Responses execution path for audio-only segment analysis.
|
||||||
|
|
||||||
Extracted from ``video_analyzer.py`` as a pure structural split. Patchable
|
Extracted from ``video_analyzer.py`` as a pure structural split. Patchable
|
||||||
collaborators are looked up late through the facade module so
|
collaborators are looked up late through the facade module so
|
||||||
@@ -23,24 +23,12 @@ keep working.
|
|||||||
import asyncio
|
import asyncio
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from google.genai import types
|
|
||||||
|
|
||||||
from artemis.agents.video_analyzer import video_analyzer as _va
|
from artemis.agents.video_analyzer import video_analyzer as _va
|
||||||
from artemis.agents.video_analyzer.chunk_conversation import (
|
|
||||||
_MISSING_SUMMARY_ERROR,
|
|
||||||
_NO_TOOL_CALL_ERROR,
|
|
||||||
_SUBMIT_NOW_WARNING,
|
|
||||||
_WRONG_TOOL_ERROR,
|
|
||||||
SUB_AGENT_MAX_TURNS,
|
|
||||||
_append_rejection,
|
|
||||||
_clean_text,
|
|
||||||
_tool_name,
|
|
||||||
)
|
|
||||||
from artemis.agents.video_analyzer.reliability import (
|
from artemis.agents.video_analyzer.reliability import (
|
||||||
SubAgentAnswerExhausted,
|
SubAgentAnswerExhausted,
|
||||||
classify_video_failure,
|
classify_video_failure,
|
||||||
)
|
)
|
||||||
from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
|
|
||||||
from artemis.data_engine.trace import TraceSpan
|
from artemis.data_engine.trace import TraceSpan
|
||||||
from artemis.llm.reliability import retry_policy_for
|
from artemis.llm.reliability import retry_policy_for
|
||||||
from artemis.utils.logger import get_logger
|
from artemis.utils.logger import get_logger
|
||||||
@@ -72,7 +60,7 @@ async def exec_analyze_audio_only(
|
|||||||
return "Error: start_time must be a numeric recording-relative timestamp."
|
return "Error: start_time must be a numeric recording-relative timestamp."
|
||||||
if not str(specific_query or "").strip():
|
if not str(specific_query or "").strip():
|
||||||
return "Error: specific_query must describe the audio evidence to find."
|
return "Error: specific_query must describe the audio evidence to find."
|
||||||
max_retries = analyzer.native_max_retries
|
max_retries = analyzer.max_retries
|
||||||
current_start = round(start_time, 1) if isinstance(start_time, (int, float)) else start_time
|
current_start = round(start_time, 1) if isinstance(start_time, (int, float)) else start_time
|
||||||
current_end = round(end_time, 1) if isinstance(end_time, (int, float)) else end_time
|
current_end = round(end_time, 1) if isinstance(end_time, (int, float)) else end_time
|
||||||
|
|
||||||
@@ -134,37 +122,25 @@ async def exec_analyze_audio_only(
|
|||||||
analyzer, result, media.actual_start, start_time, end_time, specific_query
|
analyzer, result, media.actual_start, start_time, end_time, specific_query
|
||||||
)
|
)
|
||||||
|
|
||||||
if not analyzer.use_native_gemini:
|
audio_result = await analyzer._exec_analyze_audio_universal(
|
||||||
audio_result = await analyzer._exec_analyze_audio_universal(
|
audio_path=media.audio_path,
|
||||||
audio_path=media.audio_path,
|
start_time=current_start,
|
||||||
start_time=current_start,
|
end_time=current_end,
|
||||||
end_time=current_end,
|
actual_start=media.actual_start,
|
||||||
actual_start=media.actual_start,
|
prompt_with_context=media.prompt_with_context,
|
||||||
prompt_with_context=media.prompt_with_context,
|
specific_query=specific_query,
|
||||||
specific_query=specific_query,
|
)
|
||||||
)
|
summary, analysis = analyzer._parse_chunk_result(audio_result)
|
||||||
summary, analysis = analyzer._parse_chunk_result(audio_result)
|
analyzer.blackboard.complete_segment(
|
||||||
analyzer.blackboard.complete_segment(
|
|
||||||
current_start,
|
|
||||||
current_end,
|
|
||||||
specific_query,
|
|
||||||
lease_owner,
|
|
||||||
summary,
|
|
||||||
analysis,
|
|
||||||
modality="audio",
|
|
||||||
)
|
|
||||||
return audio_result
|
|
||||||
|
|
||||||
return await _run_native_audio_conversation(
|
|
||||||
analyzer,
|
|
||||||
media,
|
|
||||||
current_start,
|
current_start,
|
||||||
current_end,
|
current_end,
|
||||||
start_time,
|
|
||||||
end_time,
|
|
||||||
specific_query,
|
specific_query,
|
||||||
lease_owner,
|
lease_owner,
|
||||||
|
summary,
|
||||||
|
analysis,
|
||||||
|
modality="audio",
|
||||||
)
|
)
|
||||||
|
return audio_result
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
# Same contract as the video sub-agent loop: classification
|
# Same contract as the video sub-agent loop: classification
|
||||||
@@ -232,49 +208,6 @@ async def _finish_failed_audio(
|
|||||||
):
|
):
|
||||||
"""Records terminal audio failure, attempting the universal fallback first."""
|
"""Records terminal audio failure, attempting the universal fallback first."""
|
||||||
failure = classify_video_failure(error)
|
failure = classify_video_failure(error)
|
||||||
if (
|
|
||||||
analyzer.use_native_gemini
|
|
||||||
and failure.should_fallback
|
|
||||||
and media.audio_path is not None
|
|
||||||
and media.actual_start is not None
|
|
||||||
and media.prompt_with_context is not None
|
|
||||||
):
|
|
||||||
try:
|
|
||||||
logger.warning(
|
|
||||||
f"Native audio analysis failed [{failure.category.value}]; using universal fallback"
|
|
||||||
)
|
|
||||||
_va._record_llm_event(
|
|
||||||
"llm_fallback",
|
|
||||||
{
|
|
||||||
"reason": "native_audio_analysis_failed",
|
|
||||||
"category": failure.category.value,
|
|
||||||
"error": str(error)[:500],
|
|
||||||
},
|
|
||||||
)
|
|
||||||
fallback_result = await analyzer._exec_analyze_audio_universal(
|
|
||||||
audio_path=media.audio_path,
|
|
||||||
start_time=current_start,
|
|
||||||
end_time=current_end,
|
|
||||||
actual_start=media.actual_start,
|
|
||||||
prompt_with_context=media.prompt_with_context,
|
|
||||||
specific_query=specific_query,
|
|
||||||
force_fallback=True,
|
|
||||||
)
|
|
||||||
summary, analysis = analyzer._parse_chunk_result(fallback_result)
|
|
||||||
analyzer.blackboard.complete_segment(
|
|
||||||
current_start,
|
|
||||||
current_end,
|
|
||||||
specific_query,
|
|
||||||
lease_owner,
|
|
||||||
summary,
|
|
||||||
analysis,
|
|
||||||
modality="audio",
|
|
||||||
)
|
|
||||||
return fallback_result
|
|
||||||
except Exception as fallback_error:
|
|
||||||
error = fallback_error
|
|
||||||
failure = classify_video_failure(error)
|
|
||||||
retryable = None
|
|
||||||
analyzer.blackboard.fail_segment(
|
analyzer.blackboard.fail_segment(
|
||||||
current_start,
|
current_start,
|
||||||
current_end,
|
current_end,
|
||||||
@@ -366,178 +299,3 @@ def _build_audio_prompt(
|
|||||||
)
|
)
|
||||||
prompt_with_context += f"{specific_query}"
|
prompt_with_context += f"{specific_query}"
|
||||||
return prompt_with_context
|
return prompt_with_context
|
||||||
|
|
||||||
|
|
||||||
async def _run_native_audio_conversation(
|
|
||||||
analyzer,
|
|
||||||
media: _AudioMedia,
|
|
||||||
current_start: float,
|
|
||||||
current_end: float,
|
|
||||||
start_time: float,
|
|
||||||
end_time: float | None,
|
|
||||||
specific_query: str,
|
|
||||||
lease_owner: str,
|
|
||||||
) -> str:
|
|
||||||
"""Uploads the audio and drives the native Gemini audio agent to a committed answer."""
|
|
||||||
file = None
|
|
||||||
try:
|
|
||||||
with TraceSpan(name="upload_audio_to_gemini") as span:
|
|
||||||
file = await analyzer.upload_and_poll_file(media.audio_path)
|
|
||||||
span.result = f"Uploaded {file.name}"
|
|
||||||
logger.info(f"Invoking Gemini for audio-only task with model {analyzer.model_name}...")
|
|
||||||
|
|
||||||
sub_agent_contents = [
|
|
||||||
types.Content(
|
|
||||||
role="user",
|
|
||||||
parts=[
|
|
||||||
types.Part.from_uri(file_uri=file.uri, mime_type=file.mime_type)
|
|
||||||
if hasattr(file, "uri")
|
|
||||||
else file,
|
|
||||||
types.Part.from_text(text=media.prompt_with_context),
|
|
||||||
],
|
|
||||||
)
|
|
||||||
]
|
|
||||||
|
|
||||||
(
|
|
||||||
final_summary,
|
|
||||||
final_analysis,
|
|
||||||
final_confidence_score,
|
|
||||||
) = await _drive_audio_agent_loop(analyzer, sub_agent_contents)
|
|
||||||
|
|
||||||
end_val = end_time if end_time is not None else "unknown"
|
|
||||||
analyzer._record_blackboard_entry(
|
|
||||||
{
|
|
||||||
"start": start_time,
|
|
||||||
"end": end_val,
|
|
||||||
"target": specific_query,
|
|
||||||
"summary": final_summary,
|
|
||||||
"confidence_score": final_confidence_score,
|
|
||||||
},
|
|
||||||
modality="audio",
|
|
||||||
)
|
|
||||||
|
|
||||||
end_str = f"{end_time:.1f}s" if end_time is not None else "unknown"
|
|
||||||
audio_result = (
|
|
||||||
f"[from {start_time:.1f}s to {end_str}] Summary:"
|
|
||||||
f" {final_summary} Analysis: {final_analysis}"
|
|
||||||
)
|
|
||||||
analyzer.blackboard.complete_segment(
|
|
||||||
current_start,
|
|
||||||
current_end,
|
|
||||||
specific_query,
|
|
||||||
lease_owner,
|
|
||||||
final_summary,
|
|
||||||
final_analysis,
|
|
||||||
modality="audio",
|
|
||||||
)
|
|
||||||
return audio_result
|
|
||||||
finally:
|
|
||||||
if file:
|
|
||||||
logger.info(f"Cleaning up cloud file {file.name}...")
|
|
||||||
try:
|
|
||||||
await asyncio.wait_for(
|
|
||||||
analyzer.client.aio.files.delete(name=file.name),
|
|
||||||
timeout=30,
|
|
||||||
)
|
|
||||||
analyzer.cloud_files_to_cleanup.discard(file.name)
|
|
||||||
except Exception as ce:
|
|
||||||
logger.error(f"Failed to delete cloud file {file.name}: {ce}")
|
|
||||||
|
|
||||||
|
|
||||||
async def _drive_audio_agent_loop(analyzer, sub_agent_contents: list) -> tuple[str, str, float]:
|
|
||||||
"""Return a validated (summary, analysis, confidence_score).
|
|
||||||
|
|
||||||
Raise ``SubAgentAnswerExhausted`` if no answer is accepted within the turn limit.
|
|
||||||
"""
|
|
||||||
rejection = _NO_TOOL_CALL_ERROR
|
|
||||||
|
|
||||||
for turn in range(1, SUB_AGENT_MAX_TURNS + 1):
|
|
||||||
if turn == SUB_AGENT_MAX_TURNS - 1:
|
|
||||||
sub_agent_contents.append(
|
|
||||||
types.Content(
|
|
||||||
role="user",
|
|
||||||
parts=[types.Part.from_text(text=_SUBMIT_NOW_WARNING)],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
with TraceSpan(name="gemini_generate_audio_content") as span:
|
|
||||||
response = await asyncio.wait_for(
|
|
||||||
analyzer.client.aio.models.generate_content(
|
|
||||||
model=analyzer.model_name,
|
|
||||||
contents=sub_agent_contents,
|
|
||||||
config=types.GenerateContentConfig(
|
|
||||||
system_instruction=analyzer.audio_system_prompt,
|
|
||||||
tools=[types.Tool(function_declarations=[analyzer.submit_answer_tool])],
|
|
||||||
safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
|
|
||||||
),
|
|
||||||
),
|
|
||||||
timeout=analyzer.model_call_timeout_seconds,
|
|
||||||
)
|
|
||||||
|
|
||||||
full_text = response.text or ""
|
|
||||||
function_calls = response.function_calls or []
|
|
||||||
|
|
||||||
if function_calls:
|
|
||||||
answer, rejection = _handle_audio_function_calls(
|
|
||||||
sub_agent_contents, response, function_calls, full_text
|
|
||||||
)
|
|
||||||
if answer is not None:
|
|
||||||
return answer
|
|
||||||
else:
|
|
||||||
rejection = _NO_TOOL_CALL_ERROR
|
|
||||||
sub_agent_contents.append(
|
|
||||||
types.Content(
|
|
||||||
role="model",
|
|
||||||
parts=[types.Part.from_text(text=full_text)],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
raise SubAgentAnswerExhausted(SUB_AGENT_MAX_TURNS, rejection)
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_audio_answer(summary: str, confidence_score) -> str | None:
|
|
||||||
"""Return a validation error for the audio answer, or None if valid."""
|
|
||||||
if not summary:
|
|
||||||
return _MISSING_SUMMARY_ERROR
|
|
||||||
if (
|
|
||||||
not isinstance(confidence_score, (int, float))
|
|
||||||
or isinstance(confidence_score, bool)
|
|
||||||
or not (0.0 <= float(confidence_score) <= 1.0)
|
|
||||||
):
|
|
||||||
return _BAD_CONFIDENCE_ERROR
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _handle_audio_function_calls(
|
|
||||||
sub_agent_contents: list,
|
|
||||||
response,
|
|
||||||
function_calls: list,
|
|
||||||
full_text: str,
|
|
||||||
) -> tuple[tuple[str, str, float] | None, str | None]:
|
|
||||||
"""Return (answer, rejection), appending any rejection to the conversation."""
|
|
||||||
original_parts = response.candidates[0].content.parts
|
|
||||||
for fc in function_calls:
|
|
||||||
if _tool_name(fc.name) != "submit_answer":
|
|
||||||
continue
|
|
||||||
args = dict(fc.args or {})
|
|
||||||
summary = _clean_text(args.get("summary"))
|
|
||||||
analysis = _clean_text(args.get("analysis"))
|
|
||||||
|
|
||||||
rejection = _validate_audio_answer(summary, args.get("confidence_score"))
|
|
||||||
if rejection is not None:
|
|
||||||
_append_rejection(
|
|
||||||
sub_agent_contents, original_parts, function_calls, full_text, fc.name, rejection
|
|
||||||
)
|
|
||||||
return None, rejection
|
|
||||||
|
|
||||||
return (summary, analysis or "No analysis provided.", float(args["confidence_score"])), None
|
|
||||||
|
|
||||||
_append_rejection(
|
|
||||||
sub_agent_contents,
|
|
||||||
original_parts,
|
|
||||||
function_calls,
|
|
||||||
full_text,
|
|
||||||
function_calls[0].name,
|
|
||||||
_WRONG_TOOL_ERROR,
|
|
||||||
)
|
|
||||||
return None, _WRONG_TOOL_ERROR
|
|
||||||
@@ -1,455 +0,0 @@
|
|||||||
# Copyright 2026 Google LLC
|
|
||||||
#
|
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
||||||
# you may not use this file except in compliance with the License.
|
|
||||||
# You may obtain a copy of the License at
|
|
||||||
#
|
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
#
|
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
# See the License for the specific language governing permissions and
|
|
||||||
# limitations under the License.
|
|
||||||
|
|
||||||
"""Analyze video chunks through the Gemini Interactions API.
|
|
||||||
|
|
||||||
``processing: "agentic"`` lets the model search and revisit an uploaded clip.
|
|
||||||
Validation errors are returned as function results, with video context retained
|
|
||||||
through ``previous_interaction_id``. Only accepted answers reach the blackboard.
|
|
||||||
|
|
||||||
The stream consumer reconstructs function arguments from step deltas. Completion
|
|
||||||
events may omit steps, so it fetches the interaction when reconstruction fails.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import copy
|
|
||||||
from dataclasses import dataclass, field
|
|
||||||
import json
|
|
||||||
from types import SimpleNamespace
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from artemis.agents.video_analyzer import gemini_files
|
|
||||||
from artemis.agents.video_analyzer import video_analyzer as _va
|
|
||||||
from artemis.agents.video_analyzer.chunk_conversation import (
|
|
||||||
_NO_TOOL_CALL_ERROR,
|
|
||||||
_SUBMIT_NOW_WARNING,
|
|
||||||
_WRONG_TOOL_ERROR,
|
|
||||||
SUB_AGENT_MAX_TURNS,
|
|
||||||
SubAgentAnswerExhausted,
|
|
||||||
_ChunkMedia,
|
|
||||||
_clean_text,
|
|
||||||
_persist_timeline_events,
|
|
||||||
_tool_name,
|
|
||||||
_validate_answer,
|
|
||||||
)
|
|
||||||
from artemis.agents.video_analyzer.universal_tools import UNIVERSAL_SUBMIT_ANSWER_TOOL
|
|
||||||
from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan
|
|
||||||
from artemis.llm.google import normalize_usage
|
|
||||||
from artemis.utils.logger import get_logger
|
|
||||||
|
|
||||||
logger = get_logger(__name__)
|
|
||||||
|
|
||||||
STREAM_START_TIMEOUT_SECONDS = 60.0
|
|
||||||
INTERACTION_READ_TIMEOUT_SECONDS = 30.0
|
|
||||||
|
|
||||||
# Interaction statuses after which no follow-up turn can be sent
|
|
||||||
# (``google.genai`` ``InteractionStatus``).
|
|
||||||
_TERMINAL_FAILURE_STATUSES = frozenset({"failed", "cancelled", "incomplete", "budget_exceeded"})
|
|
||||||
|
|
||||||
|
|
||||||
def build_interactions_submit_tool() -> dict[str, Any]:
|
|
||||||
"""The ``submit_answer`` declaration in Interactions API (flat JSON schema) form.
|
|
||||||
|
|
||||||
Derived from the universal (OpenAI-style) declaration so the schema is
|
|
||||||
written once; ``timeline_events`` is required to match the native path.
|
|
||||||
"""
|
|
||||||
function = UNIVERSAL_SUBMIT_ANSWER_TOOL["function"]
|
|
||||||
parameters = copy.deepcopy(function["parameters"])
|
|
||||||
required = list(parameters.get("required", []))
|
|
||||||
if "timeline_events" not in required:
|
|
||||||
required.append("timeline_events")
|
|
||||||
parameters["required"] = required
|
|
||||||
return {
|
|
||||||
"type": "function",
|
|
||||||
"name": function["name"],
|
|
||||||
"description": function["description"],
|
|
||||||
"parameters": parameters,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _steps(interaction: Any) -> list:
|
|
||||||
return list(getattr(interaction, "steps", None) or [])
|
|
||||||
|
|
||||||
|
|
||||||
def _find_function_call(interaction: Any):
|
|
||||||
"""The last ``function_call`` step of an interaction, if any."""
|
|
||||||
for step in reversed(_steps(interaction)):
|
|
||||||
if getattr(step, "type", None) == "function_call":
|
|
||||||
return step
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _raise_if_failed(interaction: Any) -> None:
|
|
||||||
"""Raises when an interaction ended in a terminal failure state.
|
|
||||||
|
|
||||||
A follow-up with ``previous_interaction_id`` onto a failed interaction is
|
|
||||||
rejected by the API, so the failure surfaces here as an exception and
|
|
||||||
flows through the shared retry / classification layer instead.
|
|
||||||
"""
|
|
||||||
status = str(getattr(interaction, "status", None) or "").lower()
|
|
||||||
errors = list(getattr(interaction, "errors", None) or [])
|
|
||||||
if status not in _TERMINAL_FAILURE_STATUSES and not errors:
|
|
||||||
return
|
|
||||||
details = "; ".join(
|
|
||||||
f"[{getattr(error, 'code', None)}] {getattr(error, 'message', error)}" for error in errors
|
|
||||||
)
|
|
||||||
raise RuntimeError(
|
|
||||||
f"Gemini Interactions interaction {getattr(interaction, 'id', None)} ended with"
|
|
||||||
f" status {status or 'unknown'}" + (f": {details}" if details else "")
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def run_agentic_chunk_conversation(
|
|
||||||
analyzer,
|
|
||||||
media: _ChunkMedia,
|
|
||||||
current_start: float,
|
|
||||||
current_end: float,
|
|
||||||
start_time: float,
|
|
||||||
end_time: float | None,
|
|
||||||
specific_query: str,
|
|
||||||
lease_owner: str,
|
|
||||||
) -> str:
|
|
||||||
"""Uploads the chunk and drives the agentic sub-agent to a committed answer."""
|
|
||||||
file = None
|
|
||||||
try:
|
|
||||||
with TraceSpan(name="upload_video_to_gemini") as span:
|
|
||||||
file = await analyzer.upload_and_poll_file(media.compressed_path)
|
|
||||||
span.result = f"Uploaded {file.name}"
|
|
||||||
logger.info(
|
|
||||||
f"Invoking Gemini agentic video understanding for sub-agent task with model"
|
|
||||||
f" {analyzer.model_name} (streaming)..."
|
|
||||||
)
|
|
||||||
trace_id = CURRENT_TRACE_ID.get()
|
|
||||||
|
|
||||||
if analyzer.ctx.data_engine and trace_id:
|
|
||||||
analyzer.ctx.data_engine.record_trace(
|
|
||||||
type="llm_call",
|
|
||||||
name="video_sub_agent",
|
|
||||||
payload={
|
|
||||||
"contents": [
|
|
||||||
f"file://{file.name}",
|
|
||||||
media.prompt_with_context,
|
|
||||||
],
|
|
||||||
"system_instruction": analyzer.sub_system_prompt,
|
|
||||||
"video_processing": "agentic",
|
|
||||||
},
|
|
||||||
parent_trace_id=trace_id,
|
|
||||||
)
|
|
||||||
|
|
||||||
final_summary, final_analysis = await _drive_agentic_loop(
|
|
||||||
analyzer,
|
|
||||||
file,
|
|
||||||
trace_id,
|
|
||||||
media,
|
|
||||||
start_time,
|
|
||||||
end_time,
|
|
||||||
specific_query,
|
|
||||||
)
|
|
||||||
|
|
||||||
end_str = f"{current_end:.1f}s" if current_end is not None else "unknown"
|
|
||||||
chunk_result = (
|
|
||||||
f"[from {current_start:.1f}s to {end_str}] Summary:"
|
|
||||||
f" {final_summary} Analysis: {final_analysis}"
|
|
||||||
)
|
|
||||||
analyzer.blackboard.complete_segment(
|
|
||||||
current_start,
|
|
||||||
current_end,
|
|
||||||
specific_query,
|
|
||||||
lease_owner,
|
|
||||||
final_summary,
|
|
||||||
final_analysis,
|
|
||||||
)
|
|
||||||
return chunk_result
|
|
||||||
finally:
|
|
||||||
if file:
|
|
||||||
await gemini_files.delete_cloud_file(
|
|
||||||
analyzer.client, file.name, analyzer.cloud_files_to_cleanup
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _initial_request(analyzer, file, prompt: str, tool: dict[str, Any]) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"model": analyzer.model_name,
|
|
||||||
"input": [
|
|
||||||
{
|
|
||||||
"type": "video",
|
|
||||||
"uri": file.uri,
|
|
||||||
"mime_type": file.mime_type,
|
|
||||||
"processing": "agentic",
|
|
||||||
},
|
|
||||||
{"type": "text", "text": prompt},
|
|
||||||
],
|
|
||||||
"system_instruction": analyzer.sub_system_prompt,
|
|
||||||
"tools": [tool],
|
|
||||||
"stream": True,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _follow_up_request(
|
|
||||||
analyzer, previous_id: str, steps: list[dict[str, Any]], tool: dict[str, Any]
|
|
||||||
) -> dict[str, Any]:
|
|
||||||
"""A stateful follow-up turn: the server keeps the video context.
|
|
||||||
|
|
||||||
Only the conversation state is inherited through
|
|
||||||
``previous_interaction_id``; ``system_instruction`` and ``tools`` are
|
|
||||||
interaction-scoped and must be re-sent on every turn.
|
|
||||||
"""
|
|
||||||
return {
|
|
||||||
"model": analyzer.model_name,
|
|
||||||
"previous_interaction_id": previous_id,
|
|
||||||
"input": steps,
|
|
||||||
"system_instruction": analyzer.sub_system_prompt,
|
|
||||||
"tools": [tool],
|
|
||||||
"stream": True,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def _function_result(call_id: str, error: str) -> dict[str, Any]:
|
|
||||||
return {
|
|
||||||
"type": "function_result",
|
|
||||||
"call_id": call_id,
|
|
||||||
"name": "submit_answer",
|
|
||||||
"result": {"error": error},
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
async def _drive_agentic_loop(
|
|
||||||
analyzer,
|
|
||||||
file,
|
|
||||||
trace_id,
|
|
||||||
media: _ChunkMedia,
|
|
||||||
start_time: float,
|
|
||||||
end_time: float | None,
|
|
||||||
specific_query: str,
|
|
||||||
) -> tuple[str, str]:
|
|
||||||
"""Return a validated answer and persist its timeline events.
|
|
||||||
|
|
||||||
Raise ``SubAgentAnswerExhausted`` if no answer is accepted within the turn limit.
|
|
||||||
"""
|
|
||||||
tool = build_interactions_submit_tool()
|
|
||||||
request = _initial_request(analyzer, file, media.prompt_with_context, tool)
|
|
||||||
rejection = _NO_TOOL_CALL_ERROR
|
|
||||||
|
|
||||||
for turn in range(1, SUB_AGENT_MAX_TURNS + 1):
|
|
||||||
interaction = await _stream_agentic_turn(analyzer, request, trace_id)
|
|
||||||
interaction_id = getattr(interaction, "id", None) or ""
|
|
||||||
call = _find_function_call(interaction)
|
|
||||||
|
|
||||||
if call is None:
|
|
||||||
rejection = _NO_TOOL_CALL_ERROR
|
|
||||||
steps = [{"type": "text", "text": _SUBMIT_NOW_WARNING}]
|
|
||||||
elif _tool_name(getattr(call, "name", None)) != "submit_answer":
|
|
||||||
rejection = _WRONG_TOOL_ERROR
|
|
||||||
steps = [_function_result(call.id, rejection)]
|
|
||||||
else:
|
|
||||||
args = dict(getattr(call, "arguments", None) or {})
|
|
||||||
timeline_events = args.get("timeline_events", [])
|
|
||||||
if not isinstance(timeline_events, list):
|
|
||||||
timeline_events = []
|
|
||||||
summary = _clean_text(args.get("summary"))
|
|
||||||
analysis = _clean_text(args.get("analysis"))
|
|
||||||
|
|
||||||
rejection = _validate_answer(summary, timeline_events, start_time, end_time)
|
|
||||||
if rejection is None:
|
|
||||||
await _persist_timeline_events(
|
|
||||||
analyzer, timeline_events, media, start_time, end_time, specific_query
|
|
||||||
)
|
|
||||||
return summary, analysis or "No analysis provided."
|
|
||||||
steps = [_function_result(call.id, rejection)]
|
|
||||||
|
|
||||||
if turn < SUB_AGENT_MAX_TURNS:
|
|
||||||
request = _follow_up_request(analyzer, interaction_id, steps, tool)
|
|
||||||
|
|
||||||
raise SubAgentAnswerExhausted(SUB_AGENT_MAX_TURNS, rejection)
|
|
||||||
|
|
||||||
|
|
||||||
async def _stream_agentic_turn(analyzer, request: dict[str, Any], trace_id):
|
|
||||||
"""Runs one traced streaming Interactions turn and returns the completed interaction."""
|
|
||||||
with TraceSpan(name="gemini_agentic_video_call") as span:
|
|
||||||
span.payload = {
|
|
||||||
"model": request.get("model"),
|
|
||||||
"video_processing": "agentic",
|
|
||||||
"previous_interaction_id": request.get("previous_interaction_id"),
|
|
||||||
}
|
|
||||||
stream = await asyncio.wait_for(
|
|
||||||
analyzer.client.aio.interactions.create(**request),
|
|
||||||
timeout=STREAM_START_TIMEOUT_SECONDS,
|
|
||||||
)
|
|
||||||
interaction = await asyncio.wait_for(
|
|
||||||
_consume_agentic_stream(analyzer, span, stream, trace_id),
|
|
||||||
timeout=analyzer.agentic_call_timeout_seconds,
|
|
||||||
)
|
|
||||||
call = _find_function_call(interaction)
|
|
||||||
span.result = (
|
|
||||||
f"Function call: {_tool_name(getattr(call, 'name', None))}"
|
|
||||||
if call is not None
|
|
||||||
else f"Status: {getattr(interaction, 'status', 'unknown')}"
|
|
||||||
)
|
|
||||||
return interaction
|
|
||||||
|
|
||||||
|
|
||||||
@dataclass
|
|
||||||
class _PendingStep:
|
|
||||||
"""A step under construction while its start/delta/stop events stream in."""
|
|
||||||
|
|
||||||
kind: str
|
|
||||||
call_id: str | None = None
|
|
||||||
name: str | None = None
|
|
||||||
seed_arguments: dict[str, Any] = field(default_factory=dict)
|
|
||||||
arguments_json: list[str] = field(default_factory=list)
|
|
||||||
text: list[str] = field(default_factory=list)
|
|
||||||
|
|
||||||
def finish(self):
|
|
||||||
"""Materializes the step in the shape the SDK's typed steps expose.
|
|
||||||
|
|
||||||
Returns ``None`` for a function call whose streamed arguments are not
|
|
||||||
valid JSON: a half-built call must not stand in for the real one, so
|
|
||||||
the step is dropped and the interaction is re-read by id instead.
|
|
||||||
"""
|
|
||||||
if self.kind == "function_call":
|
|
||||||
arguments: dict[str, Any] = dict(self.seed_arguments)
|
|
||||||
raw = "".join(self.arguments_json)
|
|
||||||
if raw.strip():
|
|
||||||
try:
|
|
||||||
parsed = json.loads(raw)
|
|
||||||
except ValueError:
|
|
||||||
logger.warning(
|
|
||||||
f"Could not parse streamed arguments for {self.name}: {raw[:200]!r};"
|
|
||||||
" dropping the step"
|
|
||||||
)
|
|
||||||
return None
|
|
||||||
if isinstance(parsed, dict):
|
|
||||||
arguments = parsed
|
|
||||||
return SimpleNamespace(
|
|
||||||
type="function_call", id=self.call_id, name=self.name, arguments=arguments
|
|
||||||
)
|
|
||||||
if self.kind == "model_output":
|
|
||||||
return SimpleNamespace(
|
|
||||||
type="model_output",
|
|
||||||
content=[SimpleNamespace(type="text", text="".join(self.text))],
|
|
||||||
)
|
|
||||||
return SimpleNamespace(type=self.kind, id=self.call_id)
|
|
||||||
|
|
||||||
|
|
||||||
async def _consume_agentic_stream(analyzer, span, stream, trace_id):
|
|
||||||
"""Consumes Interactions SSE events, rebuilding steps and mirroring progress.
|
|
||||||
|
|
||||||
Returns an interaction-shaped object (``id`` / ``status`` / ``steps`` /
|
|
||||||
``usage``). The interaction is re-read by id when the stream ended
|
|
||||||
without a completion event, or when it stopped at ``requires_action``
|
|
||||||
without a parsable function call.
|
|
||||||
"""
|
|
||||||
data_engine = analyzer.ctx.data_engine if trace_id else None
|
|
||||||
interaction_id: str | None = None
|
|
||||||
status: str | None = None
|
|
||||||
completed_steps: list | None = None
|
|
||||||
usage: dict[str, int] | None = None
|
|
||||||
pending: dict[int, _PendingStep] = {}
|
|
||||||
steps: list = []
|
|
||||||
processing_calls = 0
|
|
||||||
|
|
||||||
def _emit(text: str, *, thinking: bool) -> None:
|
|
||||||
if data_engine and text:
|
|
||||||
data_engine.stream_output(trace_id, text, is_thinking=thinking)
|
|
||||||
|
|
||||||
async for event in stream:
|
|
||||||
data = getattr(event, "data", event)
|
|
||||||
event_type = getattr(data, "event_type", None)
|
|
||||||
|
|
||||||
if event_type == "interaction.created":
|
|
||||||
created = getattr(data, "interaction", None)
|
|
||||||
interaction_id = getattr(created, "id", None) or interaction_id
|
|
||||||
elif event_type == "step.start":
|
|
||||||
step = getattr(data, "step", None)
|
|
||||||
index = getattr(data, "index", None)
|
|
||||||
kind = str(getattr(step, "type", None) or "")
|
|
||||||
pending[index] = _PendingStep(
|
|
||||||
kind=kind,
|
|
||||||
call_id=getattr(step, "id", None) or getattr(step, "call_id", None),
|
|
||||||
name=getattr(step, "name", None),
|
|
||||||
seed_arguments=dict(getattr(step, "arguments", None) or {}),
|
|
||||||
)
|
|
||||||
if kind == "processing_call":
|
|
||||||
processing_calls += 1
|
|
||||||
_emit(
|
|
||||||
f"Inspecting the recording (video lookup {processing_calls})\n",
|
|
||||||
thinking=True,
|
|
||||||
)
|
|
||||||
elif event_type == "step.delta":
|
|
||||||
delta = getattr(data, "delta", None)
|
|
||||||
delta_type = getattr(delta, "type", None)
|
|
||||||
current = pending.get(getattr(data, "index", None))
|
|
||||||
if delta_type == "arguments_delta":
|
|
||||||
fragment = getattr(delta, "arguments", None)
|
|
||||||
if current is not None and isinstance(fragment, str):
|
|
||||||
current.arguments_json.append(fragment)
|
|
||||||
elif delta_type == "text":
|
|
||||||
text = getattr(delta, "text", None)
|
|
||||||
if current is not None and isinstance(text, str):
|
|
||||||
current.text.append(text)
|
|
||||||
_emit(text or "", thinking=False)
|
|
||||||
elif delta_type == "thought_summary":
|
|
||||||
content = getattr(delta, "content", None)
|
|
||||||
_emit(getattr(content, "text", None) or "", thinking=True)
|
|
||||||
elif event_type == "step.stop":
|
|
||||||
current = pending.pop(getattr(data, "index", None), None)
|
|
||||||
finished = current.finish() if current is not None else None
|
|
||||||
if finished is not None:
|
|
||||||
steps.append(finished)
|
|
||||||
usage = normalize_usage(getattr(data, "usage", None)) or usage
|
|
||||||
elif event_type == "interaction.completed":
|
|
||||||
completed = getattr(data, "interaction", None)
|
|
||||||
interaction_id = getattr(completed, "id", None) or interaction_id
|
|
||||||
status = getattr(completed, "status", None) or status
|
|
||||||
completed_steps = getattr(completed, "steps", None) or None
|
|
||||||
usage = normalize_usage(getattr(completed, "usage", None)) or usage
|
|
||||||
_raise_if_failed(completed)
|
|
||||||
elif event_type == "interaction.status_update":
|
|
||||||
status = getattr(data, "status", None) or status
|
|
||||||
elif event_type == "error":
|
|
||||||
error = getattr(data, "error", None)
|
|
||||||
raise RuntimeError(
|
|
||||||
"Gemini Interactions stream error"
|
|
||||||
f" [{getattr(error, 'code', None)}]: {getattr(error, 'message', None)}"
|
|
||||||
)
|
|
||||||
|
|
||||||
for current in pending.values():
|
|
||||||
finished = current.finish()
|
|
||||||
if finished is not None:
|
|
||||||
steps.append(finished)
|
|
||||||
span.payload["processing_calls"] = processing_calls
|
|
||||||
if completed_steps:
|
|
||||||
steps = list(completed_steps)
|
|
||||||
|
|
||||||
needs_reread = status is None or (
|
|
||||||
status == "requires_action" and _find_function_call(SimpleNamespace(steps=steps)) is None
|
|
||||||
)
|
|
||||||
if needs_reread:
|
|
||||||
if not interaction_id:
|
|
||||||
raise RuntimeError("Gemini Interactions stream ended without an interaction")
|
|
||||||
logger.info(f"Re-reading interaction {interaction_id} (status={status})")
|
|
||||||
fetched = await asyncio.wait_for(
|
|
||||||
analyzer.client.aio.interactions.get(interaction_id),
|
|
||||||
timeout=INTERACTION_READ_TIMEOUT_SECONDS,
|
|
||||||
)
|
|
||||||
_raise_if_failed(fetched)
|
|
||||||
status = getattr(fetched, "status", None) or status
|
|
||||||
steps = _steps(fetched) or steps
|
|
||||||
usage = normalize_usage(getattr(fetched, "usage", None)) or usage
|
|
||||||
|
|
||||||
if usage:
|
|
||||||
span.payload["usage_metadata"] = usage
|
|
||||||
return SimpleNamespace(id=interaction_id, status=status, steps=steps, usage=usage)
|
|
||||||
@@ -1,491 +0,0 @@
|
|||||||
# Copyright 2026 Google LLC
|
|
||||||
#
|
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
||||||
# you may not use this file except in compliance with the License.
|
|
||||||
# You may obtain a copy of the License at
|
|
||||||
#
|
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
#
|
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
# See the License for the specific language governing permissions and
|
|
||||||
# limitations under the License.
|
|
||||||
|
|
||||||
"""Native Gemini sub-agent conversation and submit_answer handling for chunks.
|
|
||||||
|
|
||||||
Extracted from ``video_analyzer.py`` as a pure structural split. Patchable
|
|
||||||
collaborators are looked up late through the facade module so
|
|
||||||
``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")`` targets
|
|
||||||
keep working.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import glob
|
|
||||||
from pathlib import Path
|
|
||||||
import re
|
|
||||||
|
|
||||||
from google.genai import types
|
|
||||||
|
|
||||||
from artemis.agents.video_analyzer import gemini_files
|
|
||||||
from artemis.agents.video_analyzer import video_analyzer as _va
|
|
||||||
from artemis.agents.video_analyzer.reliability import SubAgentAnswerExhausted
|
|
||||||
from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
|
|
||||||
from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan
|
|
||||||
from artemis.llm.google import normalize_usage
|
|
||||||
from artemis.utils.logger import get_logger
|
|
||||||
|
|
||||||
logger = get_logger(__name__)
|
|
||||||
|
|
||||||
#: Turns the chunk sub-agent gets to produce an accepted ``submit_answer``
|
|
||||||
#: (shared by the static and agentic paths).
|
|
||||||
SUB_AGENT_MAX_TURNS = 2
|
|
||||||
|
|
||||||
_SUBMIT_NOW_WARNING = "[WARNING] You must call the submit_answer tool now, time is running out."
|
|
||||||
_WRONG_TOOL_ERROR = "Tool not recognized. Please use submit_answer."
|
|
||||||
_NO_TOOL_CALL_ERROR = "The model replied with text only and never called submit_answer."
|
|
||||||
_MISSING_SUMMARY_ERROR = (
|
|
||||||
"submit_answer was called without a 'summary'. Call it again with a non-empty"
|
|
||||||
" 'summary' (and 'analysis') describing what you observed."
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _tool_name(name: str | None) -> str:
|
|
||||||
name = name or ""
|
|
||||||
return name.split(":")[-1] if ":" in name else name
|
|
||||||
|
|
||||||
|
|
||||||
def _clean_text(value) -> str:
|
|
||||||
return str(value or "").strip().replace("\n", " ")
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_answer(
|
|
||||||
summary: str, timeline_events: list, start_time: float, end_time: float | None
|
|
||||||
) -> str | None:
|
|
||||||
"""Return a validation error for submit_answer, or None if valid."""
|
|
||||||
if not summary:
|
|
||||||
return _MISSING_SUMMARY_ERROR
|
|
||||||
return _validate_timeline_events(timeline_events, start_time, end_time)
|
|
||||||
|
|
||||||
|
|
||||||
class _ChunkMedia:
|
|
||||||
"""Mutable holder for per-attempt media artifacts shared with failure handling."""
|
|
||||||
|
|
||||||
__slots__ = ("path", "compressed_path", "actual_start", "prompt_with_context")
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
self.path: Path | None = None
|
|
||||||
self.compressed_path: Path | None = None
|
|
||||||
self.actual_start: float | None = None
|
|
||||||
self.prompt_with_context: str | None = None
|
|
||||||
|
|
||||||
|
|
||||||
async def _run_native_chunk_conversation(
|
|
||||||
analyzer,
|
|
||||||
media: _ChunkMedia,
|
|
||||||
current_start: float,
|
|
||||||
current_end: float,
|
|
||||||
start_time: float,
|
|
||||||
end_time: float | None,
|
|
||||||
specific_query: str,
|
|
||||||
lease_owner: str,
|
|
||||||
) -> str:
|
|
||||||
"""Uploads the chunk and drives the native Gemini sub-agent to a committed answer."""
|
|
||||||
file = None
|
|
||||||
try:
|
|
||||||
with TraceSpan(name="upload_video_to_gemini") as span:
|
|
||||||
file = await analyzer.upload_and_poll_file(media.compressed_path)
|
|
||||||
span.result = f"Uploaded {file.name}"
|
|
||||||
logger.info(
|
|
||||||
f"Invoking Gemini for sub-agent task with model {analyzer.model_name} (streaming)..."
|
|
||||||
)
|
|
||||||
trace_id = CURRENT_TRACE_ID.get()
|
|
||||||
|
|
||||||
if analyzer.ctx.data_engine and trace_id:
|
|
||||||
analyzer.ctx.data_engine.record_trace(
|
|
||||||
type="llm_call",
|
|
||||||
name="video_sub_agent",
|
|
||||||
payload={
|
|
||||||
"contents": [
|
|
||||||
f"file://{file.name}",
|
|
||||||
media.prompt_with_context,
|
|
||||||
],
|
|
||||||
"system_instruction": analyzer.sub_system_prompt,
|
|
||||||
},
|
|
||||||
parent_trace_id=trace_id,
|
|
||||||
)
|
|
||||||
|
|
||||||
sub_agent_contents = [
|
|
||||||
types.Content(
|
|
||||||
role="user",
|
|
||||||
parts=[
|
|
||||||
types.Part.from_uri(file_uri=file.uri, mime_type=file.mime_type)
|
|
||||||
if hasattr(file, "uri")
|
|
||||||
else file,
|
|
||||||
types.Part.from_text(text=media.prompt_with_context),
|
|
||||||
],
|
|
||||||
)
|
|
||||||
]
|
|
||||||
|
|
||||||
final_summary, final_analysis = await _drive_sub_agent_loop(
|
|
||||||
analyzer,
|
|
||||||
sub_agent_contents,
|
|
||||||
trace_id,
|
|
||||||
media,
|
|
||||||
start_time,
|
|
||||||
end_time,
|
|
||||||
specific_query,
|
|
||||||
)
|
|
||||||
|
|
||||||
end_str = f"{current_end:.1f}s" if current_end is not None else "unknown"
|
|
||||||
chunk_result = (
|
|
||||||
f"[from {current_start:.1f}s to {end_str}] Summary:"
|
|
||||||
f" {final_summary} Analysis: {final_analysis}"
|
|
||||||
)
|
|
||||||
analyzer.blackboard.complete_segment(
|
|
||||||
current_start,
|
|
||||||
current_end,
|
|
||||||
specific_query,
|
|
||||||
lease_owner,
|
|
||||||
final_summary,
|
|
||||||
final_analysis,
|
|
||||||
)
|
|
||||||
return chunk_result
|
|
||||||
finally:
|
|
||||||
if file:
|
|
||||||
await gemini_files.delete_cloud_file(
|
|
||||||
analyzer.client, file.name, analyzer.cloud_files_to_cleanup
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def _drive_sub_agent_loop(
|
|
||||||
analyzer,
|
|
||||||
sub_agent_contents: list,
|
|
||||||
trace_id,
|
|
||||||
media: _ChunkMedia,
|
|
||||||
start_time: float,
|
|
||||||
end_time: float | None,
|
|
||||||
specific_query: str,
|
|
||||||
) -> tuple[str, str]:
|
|
||||||
"""Return a validated answer and persist its timeline events.
|
|
||||||
|
|
||||||
Raise ``SubAgentAnswerExhausted`` if no answer is accepted within the turn limit.
|
|
||||||
"""
|
|
||||||
rejection = _NO_TOOL_CALL_ERROR
|
|
||||||
|
|
||||||
for turn in range(1, SUB_AGENT_MAX_TURNS + 1):
|
|
||||||
if turn == SUB_AGENT_MAX_TURNS - 1:
|
|
||||||
sub_agent_contents.append(
|
|
||||||
types.Content(
|
|
||||||
role="user",
|
|
||||||
parts=[types.Part.from_text(text=_SUBMIT_NOW_WARNING)],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
full_text, function_calls, original_parts = await _stream_sub_agent_turn(
|
|
||||||
analyzer, sub_agent_contents, trace_id
|
|
||||||
)
|
|
||||||
|
|
||||||
if function_calls:
|
|
||||||
answer, rejection = await _handle_sub_agent_calls(
|
|
||||||
analyzer,
|
|
||||||
sub_agent_contents,
|
|
||||||
function_calls,
|
|
||||||
original_parts,
|
|
||||||
full_text,
|
|
||||||
media,
|
|
||||||
start_time,
|
|
||||||
end_time,
|
|
||||||
specific_query,
|
|
||||||
)
|
|
||||||
if answer is not None:
|
|
||||||
return answer
|
|
||||||
else:
|
|
||||||
rejection = _NO_TOOL_CALL_ERROR
|
|
||||||
sub_agent_contents.append(
|
|
||||||
types.Content(
|
|
||||||
role="model",
|
|
||||||
parts=[types.Part.from_text(text=full_text)],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
raise SubAgentAnswerExhausted(SUB_AGENT_MAX_TURNS, rejection)
|
|
||||||
|
|
||||||
|
|
||||||
async def _stream_sub_agent_turn(
|
|
||||||
analyzer, sub_agent_contents: list, trace_id
|
|
||||||
) -> tuple[str, list, list]:
|
|
||||||
"""Starts one streamed Gemini turn and consumes it fully."""
|
|
||||||
with TraceSpan(name="gemini_stream_content_main") as span:
|
|
||||||
stream = await asyncio.wait_for(
|
|
||||||
analyzer.client.aio.models.generate_content_stream(
|
|
||||||
model=analyzer.model_name,
|
|
||||||
contents=sub_agent_contents,
|
|
||||||
config=types.GenerateContentConfig(
|
|
||||||
system_instruction=analyzer.sub_system_prompt,
|
|
||||||
tools=[types.Tool(function_declarations=[analyzer.submit_answer_tool])],
|
|
||||||
safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
|
|
||||||
),
|
|
||||||
),
|
|
||||||
timeout=60,
|
|
||||||
)
|
|
||||||
|
|
||||||
full_text, function_calls, original_parts = await asyncio.wait_for(
|
|
||||||
_read_sub_agent_stream(analyzer, span, stream, trace_id),
|
|
||||||
timeout=analyzer.model_call_timeout_seconds,
|
|
||||||
)
|
|
||||||
return full_text, function_calls, original_parts
|
|
||||||
|
|
||||||
|
|
||||||
async def _read_sub_agent_stream(analyzer, span, target_stream, current_trace_id):
|
|
||||||
"""Consumes a streamed response, mirroring text/thoughts into the trace."""
|
|
||||||
text = ""
|
|
||||||
function_calls = []
|
|
||||||
accumulated_parts = []
|
|
||||||
async for chunk in target_stream:
|
|
||||||
usage = normalize_usage(getattr(chunk, "usage_metadata", None))
|
|
||||||
if usage:
|
|
||||||
span.payload["usage_metadata"] = usage
|
|
||||||
|
|
||||||
chunk_text = chunk.text or ""
|
|
||||||
if chunk_text:
|
|
||||||
text += chunk_text
|
|
||||||
if analyzer.ctx.data_engine and current_trace_id:
|
|
||||||
analyzer.ctx.data_engine.stream_output(current_trace_id, chunk_text)
|
|
||||||
if chunk.function_calls:
|
|
||||||
function_calls.extend(chunk.function_calls)
|
|
||||||
|
|
||||||
candidates = getattr(chunk, "candidates", None)
|
|
||||||
if candidates and isinstance(candidates, list) and len(candidates) > 0:
|
|
||||||
content = getattr(candidates[0], "content", None)
|
|
||||||
parts = getattr(content, "parts", None) if content else None
|
|
||||||
if parts:
|
|
||||||
for part in parts:
|
|
||||||
accumulated_parts.append(part)
|
|
||||||
if getattr(part, "thought", False) and part.text:
|
|
||||||
if analyzer.ctx.data_engine and current_trace_id:
|
|
||||||
analyzer.ctx.data_engine.stream_output(
|
|
||||||
current_trace_id,
|
|
||||||
part.text,
|
|
||||||
is_thinking=True,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
if chunk.function_calls:
|
|
||||||
for fc in chunk.function_calls:
|
|
||||||
accumulated_parts.append(types.Part(function_call=fc))
|
|
||||||
|
|
||||||
return text, function_calls, accumulated_parts
|
|
||||||
|
|
||||||
|
|
||||||
async def _handle_sub_agent_calls(
|
|
||||||
analyzer,
|
|
||||||
sub_agent_contents: list,
|
|
||||||
function_calls: list,
|
|
||||||
original_parts: list,
|
|
||||||
full_text: str,
|
|
||||||
media: _ChunkMedia,
|
|
||||||
start_time: float,
|
|
||||||
end_time: float | None,
|
|
||||||
specific_query: str,
|
|
||||||
) -> tuple[tuple[str, str] | None, str | None]:
|
|
||||||
"""Return ``(answer, rejection)`` after processing one turn's tool calls.
|
|
||||||
|
|
||||||
Persist events for accepted answers; append rejected calls to the conversation
|
|
||||||
with an error for the model to correct.
|
|
||||||
"""
|
|
||||||
for fc in function_calls:
|
|
||||||
if _tool_name(fc.name) != "submit_answer":
|
|
||||||
continue
|
|
||||||
args = dict(fc.args or {})
|
|
||||||
timeline_events = args.get("timeline_events", [])
|
|
||||||
if not isinstance(timeline_events, list):
|
|
||||||
timeline_events = []
|
|
||||||
summary = _clean_text(args.get("summary"))
|
|
||||||
analysis = _clean_text(args.get("analysis"))
|
|
||||||
|
|
||||||
rejection = _validate_answer(summary, timeline_events, start_time, end_time)
|
|
||||||
if rejection is not None:
|
|
||||||
_append_rejection(
|
|
||||||
sub_agent_contents, original_parts, function_calls, full_text, fc.name, rejection
|
|
||||||
)
|
|
||||||
return None, rejection
|
|
||||||
|
|
||||||
await _persist_timeline_events(
|
|
||||||
analyzer, timeline_events, media, start_time, end_time, specific_query
|
|
||||||
)
|
|
||||||
return (summary, analysis or "No analysis provided."), None
|
|
||||||
|
|
||||||
_append_rejection(
|
|
||||||
sub_agent_contents,
|
|
||||||
original_parts,
|
|
||||||
function_calls,
|
|
||||||
full_text,
|
|
||||||
function_calls[0].name,
|
|
||||||
_WRONG_TOOL_ERROR,
|
|
||||||
)
|
|
||||||
return None, _WRONG_TOOL_ERROR
|
|
||||||
|
|
||||||
|
|
||||||
def _validate_timeline_events(
|
|
||||||
timeline_events: list, start_time: float, end_time: float | None
|
|
||||||
) -> str | None:
|
|
||||||
"""Returns the first validation error message for the submitted events, if any."""
|
|
||||||
for event in timeline_events:
|
|
||||||
if (
|
|
||||||
"confidence_score" not in event
|
|
||||||
or not isinstance(
|
|
||||||
event["confidence_score"],
|
|
||||||
(int, float),
|
|
||||||
)
|
|
||||||
or isinstance(event["confidence_score"], bool)
|
|
||||||
or not (0.0 <= float(event["confidence_score"]) <= 1.0)
|
|
||||||
):
|
|
||||||
return (
|
|
||||||
"Error: 'confidence_score' is"
|
|
||||||
" missing or out of bounds in"
|
|
||||||
" one of the timeline_events."
|
|
||||||
" It must be a float between"
|
|
||||||
" 0.0 and 1.0 inclusive. Fix"
|
|
||||||
" the payload and call"
|
|
||||||
" submit_answer again."
|
|
||||||
)
|
|
||||||
|
|
||||||
if (
|
|
||||||
"verification_timestamp_secs" not in event
|
|
||||||
or not isinstance(
|
|
||||||
event["verification_timestamp_secs"],
|
|
||||||
(int, float),
|
|
||||||
)
|
|
||||||
or isinstance(event["verification_timestamp_secs"], bool)
|
|
||||||
):
|
|
||||||
return (
|
|
||||||
"Error:"
|
|
||||||
" 'verification_timestamp_secs'"
|
|
||||||
" is missing or invalid in one"
|
|
||||||
" of the timeline_events. Fix"
|
|
||||||
" the payload and call"
|
|
||||||
" submit_answer again."
|
|
||||||
)
|
|
||||||
|
|
||||||
verification_ts = float(
|
|
||||||
event.get(
|
|
||||||
"verification_timestamp_secs",
|
|
||||||
start_time,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
confidence = float(event.get("confidence_score", 0.0)) # noqa: F841
|
|
||||||
|
|
||||||
if verification_ts < start_time or (end_time is not None and verification_ts > end_time):
|
|
||||||
return (
|
|
||||||
"Error:"
|
|
||||||
" 'verification_timestamp_secs'"
|
|
||||||
f" {verification_ts} must fall"
|
|
||||||
" within the start_time"
|
|
||||||
f" ({start_time}) and end_time"
|
|
||||||
f" ({end_time}) boundaries. Fix"
|
|
||||||
" the payload and call"
|
|
||||||
" submit_answer again."
|
|
||||||
)
|
|
||||||
return None
|
|
||||||
|
|
||||||
|
|
||||||
def _append_rejection(
|
|
||||||
sub_agent_contents: list,
|
|
||||||
original_parts: list,
|
|
||||||
function_calls: list,
|
|
||||||
full_text: str,
|
|
||||||
response_name: str,
|
|
||||||
error_msg: str,
|
|
||||||
) -> None:
|
|
||||||
"""Appends the model turn plus an error function response to the conversation."""
|
|
||||||
sub_agent_contents.append(
|
|
||||||
types.Content(
|
|
||||||
role="model",
|
|
||||||
parts=original_parts
|
|
||||||
if original_parts
|
|
||||||
else [types.Part(function_call=fc) for fc in function_calls]
|
|
||||||
+ ([types.Part.from_text(text=full_text)] if full_text else []),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
sub_agent_contents.append(
|
|
||||||
types.Content(
|
|
||||||
role="user",
|
|
||||||
parts=[
|
|
||||||
types.Part.from_function_response(
|
|
||||||
name=response_name,
|
|
||||||
response={"error": (error_msg)},
|
|
||||||
)
|
|
||||||
],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def _persist_timeline_events(
|
|
||||||
analyzer,
|
|
||||||
timeline_events: list,
|
|
||||||
media: _ChunkMedia,
|
|
||||||
start_time: float,
|
|
||||||
end_time: float | None,
|
|
||||||
specific_query: str,
|
|
||||||
) -> None:
|
|
||||||
"""Captures verification screenshots and records validated events on the blackboard."""
|
|
||||||
for event in timeline_events:
|
|
||||||
verification_ts = float(
|
|
||||||
event.get(
|
|
||||||
"verification_timestamp_secs",
|
|
||||||
start_time,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
confidence = float(event.get("confidence_score", 0.0))
|
|
||||||
relative_offset = max(0.0, verification_ts - media.actual_start)
|
|
||||||
|
|
||||||
out_dir = _va.get_temp_dir("video_analyzer")
|
|
||||||
out_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
existing_imgs = glob.glob(f"{out_dir}/img_*.jpg")
|
|
||||||
max_idx = -1
|
|
||||||
for img in existing_imgs:
|
|
||||||
match = re.search(r"img_(\d+)\.jpg$", img)
|
|
||||||
if match:
|
|
||||||
max_idx = max(max_idx, int(match.group(1)))
|
|
||||||
next_idx = max_idx + 1
|
|
||||||
screenshot_output_path = out_dir / f"img_{next_idx}.jpg"
|
|
||||||
while screenshot_output_path.exists():
|
|
||||||
next_idx += 1
|
|
||||||
screenshot_output_path = out_dir / f"img_{next_idx}.jpg"
|
|
||||||
screenshot_output_path.touch()
|
|
||||||
analyzer.local_files_to_cleanup.add(screenshot_output_path)
|
|
||||||
ffmpeg_cmd = [
|
|
||||||
"ffmpeg",
|
|
||||||
"-ss",
|
|
||||||
str(relative_offset),
|
|
||||||
"-i",
|
|
||||||
str(media.path),
|
|
||||||
"-vframes",
|
|
||||||
"1",
|
|
||||||
"-q:v",
|
|
||||||
"2",
|
|
||||||
"-y",
|
|
||||||
str(screenshot_output_path),
|
|
||||||
]
|
|
||||||
try:
|
|
||||||
proc = await asyncio.create_subprocess_exec(
|
|
||||||
*ffmpeg_cmd,
|
|
||||||
stdout=asyncio.subprocess.PIPE,
|
|
||||||
stderr=asyncio.subprocess.PIPE,
|
|
||||||
)
|
|
||||||
await proc.communicate()
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"Failed to extract frame: {e}")
|
|
||||||
|
|
||||||
screenshot_path_str = str(screenshot_output_path)
|
|
||||||
entry = {
|
|
||||||
"start": event.get("start_time", start_time),
|
|
||||||
"end": event.get("end_time", end_time),
|
|
||||||
"target": specific_query,
|
|
||||||
"summary": event.get("transcription", ""),
|
|
||||||
"confidence_score": confidence,
|
|
||||||
}
|
|
||||||
if Path(screenshot_output_path).exists():
|
|
||||||
entry["screenshot"] = str(screenshot_path_str)
|
|
||||||
|
|
||||||
analyzer._record_blackboard_entry(entry)
|
|
||||||
+29
-126
@@ -12,29 +12,23 @@
|
|||||||
# See the License for the specific language governing permissions and
|
# See the License for the specific language governing permissions and
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
"""Native Gemini execution path for a single video chunk sub-agent.
|
"""Responses execution path for a single video chunk sub-agent.
|
||||||
|
|
||||||
Extracted from ``video_analyzer.py`` as a pure structural split. Patchable
|
Extracted from ``video_analyzer.py`` as a pure structural split. Patchable
|
||||||
collaborators (``get_controller``, ``compress_video_for_api``, semaphores,
|
collaborators (``get_controller``, ``compress_video_for_api``, semaphores,
|
||||||
LLM event recorders, ...) are looked up late through the facade module so
|
LLM event recorders, ...) are looked up late through the facade module so
|
||||||
``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")`` targets
|
``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")`` targets
|
||||||
keep working. The sub-agent conversation itself lives in
|
keep working. The sub-agent conversation itself lives in
|
||||||
``chunk_conversation``.
|
``universal_engine``.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
from artemis.agents.video_analyzer import video_analyzer as _va
|
from artemis.agents.video_analyzer import video_analyzer as _va
|
||||||
from artemis.agents.video_analyzer.chunk_agentic import run_agentic_chunk_conversation
|
|
||||||
from artemis.agents.video_analyzer.chunk_conversation import (
|
|
||||||
_ChunkMedia,
|
|
||||||
_run_native_chunk_conversation,
|
|
||||||
)
|
|
||||||
from artemis.agents.video_analyzer.reliability import (
|
from artemis.agents.video_analyzer.reliability import (
|
||||||
AgenticVideoDegraded,
|
AgenticVideoDegraded,
|
||||||
classify_video_failure,
|
classify_video_failure,
|
||||||
is_agentic_rejection,
|
|
||||||
)
|
)
|
||||||
from artemis.llm.reliability import retry_policy_for
|
from artemis.llm.reliability import retry_policy_for
|
||||||
from artemis.utils.logger import get_logger
|
from artemis.utils.logger import get_logger
|
||||||
@@ -42,11 +36,23 @@ from artemis.utils.logger import get_logger
|
|||||||
logger = get_logger(__name__)
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
|
class _ChunkMedia:
|
||||||
|
"""Mutable holder for per-attempt media artifacts shared with failure handling."""
|
||||||
|
|
||||||
|
__slots__ = ("path", "compressed_path", "actual_start", "prompt_with_context")
|
||||||
|
|
||||||
|
def __init__(self):
|
||||||
|
self.path: Path | None = None
|
||||||
|
self.compressed_path: Path | None = None
|
||||||
|
self.actual_start: float | None = None
|
||||||
|
self.prompt_with_context: str | None = None
|
||||||
|
|
||||||
|
|
||||||
async def exec_single_chunk(
|
async def exec_single_chunk(
|
||||||
analyzer, start_time: float, end_time: float | None, specific_query: str
|
analyzer, start_time: float, end_time: float | None, specific_query: str
|
||||||
) -> str:
|
) -> str:
|
||||||
"""Analyzes one closed chunk interval, with claim/lease and retry handling."""
|
"""Analyzes one closed chunk interval, with claim/lease and retry handling."""
|
||||||
max_retries = analyzer.native_max_retries
|
max_retries = analyzer.max_retries
|
||||||
current_start = round(start_time, 1) if isinstance(start_time, (int, float)) else start_time
|
current_start = round(start_time, 1) if isinstance(start_time, (int, float)) else start_time
|
||||||
current_end = round(end_time, 1) if isinstance(end_time, (int, float)) else end_time
|
current_end = round(end_time, 1) if isinstance(end_time, (int, float)) else end_time
|
||||||
|
|
||||||
@@ -85,7 +91,6 @@ async def exec_single_chunk(
|
|||||||
while attempt <= max_retries:
|
while attempt <= max_retries:
|
||||||
# The mode this attempt was launched in: a parallel chunk may degrade
|
# The mode this attempt was launched in: a parallel chunk may degrade
|
||||||
# the run while this one is still in flight.
|
# the run while this one is still in flight.
|
||||||
attempt_mode = analyzer.video_processing
|
|
||||||
try:
|
try:
|
||||||
logger.info(
|
logger.info(
|
||||||
f"Attempt {attempt + 1}/{max_retries + 1} for sub-agent:"
|
f"Attempt {attempt + 1}/{max_retries + 1} for sub-agent:"
|
||||||
@@ -108,49 +113,25 @@ async def exec_single_chunk(
|
|||||||
)
|
)
|
||||||
_ensure_fits_static_ceiling(analyzer, current_start, current_end)
|
_ensure_fits_static_ceiling(analyzer, current_start, current_end)
|
||||||
|
|
||||||
if not analyzer.use_native_gemini:
|
chunk_result = await analyzer._exec_single_chunk_universal(
|
||||||
chunk_result = await analyzer._exec_single_chunk_universal(
|
compressed_path=media.compressed_path,
|
||||||
compressed_path=media.compressed_path,
|
raw_path=media.path,
|
||||||
raw_path=media.path,
|
start_time=current_start,
|
||||||
start_time=current_start,
|
end_time=current_end,
|
||||||
end_time=current_end,
|
actual_start=media.actual_start,
|
||||||
actual_start=media.actual_start,
|
prompt_with_context=media.prompt_with_context,
|
||||||
prompt_with_context=media.prompt_with_context,
|
specific_query=specific_query,
|
||||||
specific_query=specific_query,
|
)
|
||||||
)
|
summary, analysis = analyzer._parse_chunk_result(chunk_result)
|
||||||
summary, analysis = analyzer._parse_chunk_result(chunk_result)
|
analyzer.blackboard.complete_segment(
|
||||||
analyzer.blackboard.complete_segment(
|
|
||||||
current_start,
|
|
||||||
current_end,
|
|
||||||
specific_query,
|
|
||||||
lease_owner,
|
|
||||||
summary,
|
|
||||||
analysis,
|
|
||||||
)
|
|
||||||
return chunk_result
|
|
||||||
|
|
||||||
if analyzer.video_processing == "agentic":
|
|
||||||
return await run_agentic_chunk_conversation(
|
|
||||||
analyzer,
|
|
||||||
media,
|
|
||||||
current_start,
|
|
||||||
current_end,
|
|
||||||
start_time,
|
|
||||||
end_time,
|
|
||||||
specific_query,
|
|
||||||
lease_owner,
|
|
||||||
)
|
|
||||||
|
|
||||||
return await _run_native_chunk_conversation(
|
|
||||||
analyzer,
|
|
||||||
media,
|
|
||||||
current_start,
|
current_start,
|
||||||
current_end,
|
current_end,
|
||||||
start_time,
|
|
||||||
end_time,
|
|
||||||
specific_query,
|
specific_query,
|
||||||
lease_owner,
|
lease_owner,
|
||||||
|
summary,
|
||||||
|
analysis,
|
||||||
)
|
)
|
||||||
|
return chunk_result
|
||||||
|
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
# Retry decisions are owned by the shared classification layer.
|
# Retry decisions are owned by the shared classification layer.
|
||||||
@@ -159,22 +140,6 @@ async def exec_single_chunk(
|
|||||||
# stream output from the failed attempt is discarded, never
|
# stream output from the failed attempt is discarded, never
|
||||||
# surfaced as a complete answer.
|
# surfaced as a complete answer.
|
||||||
failure = classify_video_failure(e)
|
failure = classify_video_failure(e)
|
||||||
if attempt_mode == "agentic" and is_agentic_rejection(failure, e):
|
|
||||||
# Switch to static and let the coordinator resize the interval.
|
|
||||||
_degrade_agentic_to_static(analyzer, e, failure)
|
|
||||||
return await _finish_failed_chunk(
|
|
||||||
analyzer,
|
|
||||||
media,
|
|
||||||
current_start,
|
|
||||||
current_end,
|
|
||||||
specific_query,
|
|
||||||
lease_owner,
|
|
||||||
AgenticVideoDegraded(
|
|
||||||
f"Agentic video processing rejected ({e}); re-plan"
|
|
||||||
f" {current_start:.1f}s-{current_end:.1f}s at the static chunk size"
|
|
||||||
),
|
|
||||||
retryable=False,
|
|
||||||
)
|
|
||||||
attempts_so_far = attempt + 1
|
attempts_so_far = attempt + 1
|
||||||
if not failure.retryable:
|
if not failure.retryable:
|
||||||
logger.warning(
|
logger.warning(
|
||||||
@@ -229,26 +194,6 @@ async def exec_single_chunk(
|
|||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _degrade_agentic_to_static(analyzer, error: Exception, failure) -> None:
|
|
||||||
"""Switches the rest of this run to static video understanding (idempotent)."""
|
|
||||||
if analyzer.video_processing != "agentic":
|
|
||||||
return
|
|
||||||
logger.warning(
|
|
||||||
f"Agentic video processing rejected ({error}); the rest of this run"
|
|
||||||
" analyzes video in static mode"
|
|
||||||
)
|
|
||||||
_va._record_llm_event(
|
|
||||||
"llm_fallback",
|
|
||||||
{
|
|
||||||
"reason": "agentic_video_rejected",
|
|
||||||
"category": failure.category.value,
|
|
||||||
"error": str(error)[:500],
|
|
||||||
},
|
|
||||||
)
|
|
||||||
analyzer.video_processing = "static"
|
|
||||||
analyzer.chunk_size_seconds = analyzer.static_chunk_size_seconds
|
|
||||||
|
|
||||||
|
|
||||||
def _ensure_fits_static_ceiling(analyzer, current_start: float, current_end: float) -> None:
|
def _ensure_fits_static_ceiling(analyzer, current_start: float, current_end: float) -> None:
|
||||||
"""Refuses to analyze a clip sized for the agentic ceiling once the run is static.
|
"""Refuses to analyze a clip sized for the agentic ceiling once the run is static.
|
||||||
|
|
||||||
@@ -280,48 +225,6 @@ async def _finish_failed_chunk(
|
|||||||
):
|
):
|
||||||
"""Records terminal failure, attempting the universal fallback first when eligible."""
|
"""Records terminal failure, attempting the universal fallback first when eligible."""
|
||||||
failure = classify_video_failure(error)
|
failure = classify_video_failure(error)
|
||||||
if (
|
|
||||||
analyzer.use_native_gemini
|
|
||||||
and failure.should_fallback
|
|
||||||
and media.path is not None
|
|
||||||
and media.compressed_path is not None
|
|
||||||
and media.actual_start is not None
|
|
||||||
and media.prompt_with_context is not None
|
|
||||||
):
|
|
||||||
try:
|
|
||||||
logger.warning("Native video chunk exhausted retries; using universal fallback")
|
|
||||||
_va._record_llm_event(
|
|
||||||
"llm_fallback",
|
|
||||||
{
|
|
||||||
"reason": "native_video_chunk_failed",
|
|
||||||
"category": failure.category.value,
|
|
||||||
"error": str(error)[:500],
|
|
||||||
},
|
|
||||||
)
|
|
||||||
fallback_result = await analyzer._exec_single_chunk_universal(
|
|
||||||
compressed_path=media.compressed_path,
|
|
||||||
raw_path=media.path,
|
|
||||||
start_time=current_start,
|
|
||||||
end_time=current_end,
|
|
||||||
actual_start=media.actual_start,
|
|
||||||
prompt_with_context=media.prompt_with_context,
|
|
||||||
specific_query=specific_query,
|
|
||||||
force_fallback=True,
|
|
||||||
)
|
|
||||||
summary, analysis = analyzer._parse_chunk_result(fallback_result)
|
|
||||||
analyzer.blackboard.complete_segment(
|
|
||||||
current_start,
|
|
||||||
current_end,
|
|
||||||
specific_query,
|
|
||||||
lease_owner,
|
|
||||||
summary,
|
|
||||||
analysis,
|
|
||||||
)
|
|
||||||
return fallback_result
|
|
||||||
except Exception as fallback_error:
|
|
||||||
error = fallback_error
|
|
||||||
failure = classify_video_failure(error)
|
|
||||||
retryable = None
|
|
||||||
analyzer.blackboard.fail_segment(
|
analyzer.blackboard.fail_segment(
|
||||||
current_start,
|
current_start,
|
||||||
current_end,
|
current_end,
|
||||||
@@ -1,162 +0,0 @@
|
|||||||
# Copyright 2026 Google LLC
|
|
||||||
#
|
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
||||||
# you may not use this file except in compliance with the License.
|
|
||||||
# You may obtain a copy of the License at
|
|
||||||
#
|
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
#
|
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
# See the License for the specific language governing permissions and
|
|
||||||
# limitations under the License.
|
|
||||||
|
|
||||||
"""Gemini File API helpers: upload/poll lifecycle and abandoned-file cleanup."""
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
from datetime import datetime
|
|
||||||
from pathlib import Path
|
|
||||||
import time
|
|
||||||
|
|
||||||
from artemis.utils.logger import get_logger
|
|
||||||
|
|
||||||
try:
|
|
||||||
from datetime import UTC
|
|
||||||
except ImportError:
|
|
||||||
UTC = UTC
|
|
||||||
|
|
||||||
logger = get_logger(__name__)
|
|
||||||
|
|
||||||
_LAST_CLEANUP_TIME = 0.0
|
|
||||||
_CLEANUP_LOCK = asyncio.Lock()
|
|
||||||
|
|
||||||
|
|
||||||
async def cleanup_abandoned_gemini_files(client) -> None:
|
|
||||||
"""Scan and delete remaining cloud video files whose creation time has elapsed standard TTL."""
|
|
||||||
global _LAST_CLEANUP_TIME
|
|
||||||
if _CLEANUP_LOCK.locked():
|
|
||||||
return
|
|
||||||
|
|
||||||
async with _CLEANUP_LOCK:
|
|
||||||
now_ts = time.time()
|
|
||||||
if now_ts - _LAST_CLEANUP_TIME < 3600:
|
|
||||||
return
|
|
||||||
_LAST_CLEANUP_TIME = now_ts
|
|
||||||
|
|
||||||
try:
|
|
||||||
pager = await asyncio.wait_for(client.aio.files.list(), timeout=30)
|
|
||||||
files = []
|
|
||||||
async for f in pager:
|
|
||||||
files.append(f)
|
|
||||||
now = datetime.now(UTC)
|
|
||||||
files_to_delete = []
|
|
||||||
|
|
||||||
for f in files:
|
|
||||||
display_name = getattr(f, "display_name", "") or ""
|
|
||||||
|
|
||||||
if (
|
|
||||||
display_name.startswith("compressed_")
|
|
||||||
or display_name.startswith("audio_")
|
|
||||||
or "artemis" in display_name
|
|
||||||
):
|
|
||||||
create_time = getattr(f, "created_at", None) or getattr(f, "create_time", None)
|
|
||||||
if not create_time:
|
|
||||||
continue
|
|
||||||
|
|
||||||
if isinstance(create_time, str):
|
|
||||||
try:
|
|
||||||
parsed_time = datetime.fromisoformat(create_time.replace("Z", "+00:00"))
|
|
||||||
except Exception:
|
|
||||||
continue
|
|
||||||
elif isinstance(create_time, datetime):
|
|
||||||
parsed_time = create_time
|
|
||||||
else:
|
|
||||||
continue
|
|
||||||
|
|
||||||
if parsed_time.tzinfo is None:
|
|
||||||
parsed_time = parsed_time.replace(tzinfo=UTC)
|
|
||||||
|
|
||||||
age_seconds = (now - parsed_time).total_seconds()
|
|
||||||
|
|
||||||
if age_seconds > 7200:
|
|
||||||
files_to_delete.append(f)
|
|
||||||
logger.info(
|
|
||||||
f"Marked expired cloud asset for deletion: {f.name}"
|
|
||||||
f" ({display_name}), Age: {age_seconds / 3600:.1f}h"
|
|
||||||
)
|
|
||||||
|
|
||||||
if files_to_delete:
|
|
||||||
logger.info(f"Purging {len(files_to_delete)} expired cloud assets in parallel...")
|
|
||||||
tasks = [
|
|
||||||
asyncio.wait_for(client.aio.files.delete(name=f.name), timeout=30)
|
|
||||||
for f in files_to_delete
|
|
||||||
]
|
|
||||||
await asyncio.gather(*tasks, return_exceptions=True)
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"Routine cloud maintenance skipped: {e}")
|
|
||||||
|
|
||||||
|
|
||||||
async def delete_cloud_file(
|
|
||||||
client, file_name: str, cloud_files_to_cleanup: set, *, timeout: float = 30.0
|
|
||||||
) -> None:
|
|
||||||
"""Best-effort deletion of an uploaded file; failures are logged, never raised."""
|
|
||||||
logger.info(f"Cleaning up cloud file {file_name}...")
|
|
||||||
try:
|
|
||||||
await asyncio.wait_for(client.aio.files.delete(name=file_name), timeout=timeout)
|
|
||||||
cloud_files_to_cleanup.discard(file_name)
|
|
||||||
except Exception as ce:
|
|
||||||
logger.error(f"Failed to delete cloud file {file_name}: {ce}")
|
|
||||||
|
|
||||||
|
|
||||||
async def upload_and_poll_file(client, compressed_path: Path, cloud_files_to_cleanup: set) -> any:
|
|
||||||
"""Upload a media file to the Gemini File API and poll until it is ACTIVE."""
|
|
||||||
logger.info(f"Uploading {compressed_path} to Gemini File API...")
|
|
||||||
file_size_mb = compressed_path.stat().st_size / (1024 * 1024)
|
|
||||||
upload_timeout = max(30.0, min(120.0, file_size_mb * 2.0))
|
|
||||||
|
|
||||||
file = await asyncio.wait_for(
|
|
||||||
client.aio.files.upload(file=compressed_path),
|
|
||||||
timeout=upload_timeout,
|
|
||||||
)
|
|
||||||
cloud_files_to_cleanup.add(file.name)
|
|
||||||
|
|
||||||
max_wait = max(60, min(180, 60 + int(file_size_mb * 2)))
|
|
||||||
|
|
||||||
wait_interval = 0.5
|
|
||||||
start_wait = time.time()
|
|
||||||
retry_count = 0
|
|
||||||
max_retries = 3
|
|
||||||
|
|
||||||
while True:
|
|
||||||
try:
|
|
||||||
f_state = await asyncio.wait_for(client.aio.files.get(name=file.name), timeout=20)
|
|
||||||
retry_count = 0
|
|
||||||
except Exception as poll_error:
|
|
||||||
retry_count += 1
|
|
||||||
logger.warning(
|
|
||||||
f"Temporary issue polling file {file.name} (attempt"
|
|
||||||
f" {retry_count}/{max_retries}): {poll_error}"
|
|
||||||
)
|
|
||||||
if retry_count > max_retries:
|
|
||||||
raise RuntimeError(
|
|
||||||
f"Failed to poll Gemini File API after {max_retries} attempts: {poll_error}"
|
|
||||||
)
|
|
||||||
await asyncio.sleep(2.0)
|
|
||||||
continue
|
|
||||||
|
|
||||||
if f_state.state.name == "ACTIVE":
|
|
||||||
break
|
|
||||||
elif f_state.state.name == "FAILED":
|
|
||||||
raise RuntimeError(f"Gemini File API processing failed for {file.name}")
|
|
||||||
|
|
||||||
if time.time() - start_wait > max_wait:
|
|
||||||
raise TimeoutError(
|
|
||||||
f"Gemini File API processing timeout for {file.name}"
|
|
||||||
f" (Waited {max_wait}s for size {file_size_mb:.1f}MB)"
|
|
||||||
)
|
|
||||||
|
|
||||||
logger.info(f"File {file.name} is {f_state.state.name}, waiting {wait_interval}s...")
|
|
||||||
await asyncio.sleep(wait_interval)
|
|
||||||
wait_interval = min(3.0, wait_interval * 1.5)
|
|
||||||
return file
|
|
||||||
@@ -1,541 +0,0 @@
|
|||||||
# Copyright 2026 Google LLC
|
|
||||||
#
|
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
||||||
# you may not use this file except in compliance with the License.
|
|
||||||
# You may obtain a copy of the License at
|
|
||||||
#
|
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
#
|
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
# See the License for the specific language governing permissions and
|
|
||||||
# limitations under the License.
|
|
||||||
|
|
||||||
"""Native Gemini coordinator loop for the video analyzer main agent.
|
|
||||||
|
|
||||||
Extracted from ``video_analyzer.py`` as a pure structural split. Patchable
|
|
||||||
collaborators are looked up late through the facade module so
|
|
||||||
``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")`` targets
|
|
||||||
keep working.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import asyncio
|
|
||||||
import os
|
|
||||||
from pathlib import Path
|
|
||||||
import shutil
|
|
||||||
import tempfile
|
|
||||||
|
|
||||||
from google.genai import types
|
|
||||||
|
|
||||||
from artemis.agents.video_analyzer import video_analyzer as _va
|
|
||||||
from artemis.agents.video_analyzer.reliability import classify_video_failure
|
|
||||||
from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
|
|
||||||
from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan
|
|
||||||
from artemis.llm.google import normalize_usage
|
|
||||||
from artemis.utils.logger import get_logger
|
|
||||||
|
|
||||||
logger = get_logger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
class _DummyResponse:
|
|
||||||
"""Aggregated view over a consumed stream, shaped like a full response."""
|
|
||||||
|
|
||||||
def __init__(self, text, function_calls, parts):
|
|
||||||
self.text = text
|
|
||||||
self.function_calls = function_calls
|
|
||||||
self.candidates = [
|
|
||||||
types.Candidate(
|
|
||||||
content=types.Content(
|
|
||||||
role="model",
|
|
||||||
parts=parts if parts else ([types.Part.from_text(text=text)] if text else []),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
async def run_native(
|
|
||||||
analyzer,
|
|
||||||
ctx,
|
|
||||||
time_description: str,
|
|
||||||
purpose: str,
|
|
||||||
system_prompt: str,
|
|
||||||
temperature: float,
|
|
||||||
thinking_level,
|
|
||||||
) -> tuple[str, str]:
|
|
||||||
"""Drives the native Gemini coordinator loop to a final answer."""
|
|
||||||
tools_declaration = analyzer.tools_declaration
|
|
||||||
|
|
||||||
contents = [
|
|
||||||
types.Content(
|
|
||||||
role="user",
|
|
||||||
parts=[
|
|
||||||
types.Part.from_text(
|
|
||||||
text=(f"Time description: {time_description}\nPurpose: {purpose}")
|
|
||||||
)
|
|
||||||
],
|
|
||||||
)
|
|
||||||
]
|
|
||||||
|
|
||||||
agent_outcome = ""
|
|
||||||
status = "success"
|
|
||||||
max_iterations = 10
|
|
||||||
iterations = 0
|
|
||||||
last_ledger_index = 0
|
|
||||||
|
|
||||||
try:
|
|
||||||
while iterations < max_iterations:
|
|
||||||
iterations += 1
|
|
||||||
logger.info(f"Iteration {iterations}: Invoking Native Gemini SDK...")
|
|
||||||
|
|
||||||
analyzer._clean_blackboard()
|
|
||||||
|
|
||||||
last_ledger_index = _append_ledger_diff(analyzer, contents, last_ledger_index)
|
|
||||||
|
|
||||||
is_final_turn = iterations == max_iterations
|
|
||||||
_append_turn_guidance(contents, iterations, max_iterations, is_final_turn)
|
|
||||||
|
|
||||||
response = await _stream_main_turn(
|
|
||||||
analyzer,
|
|
||||||
ctx,
|
|
||||||
contents,
|
|
||||||
system_prompt,
|
|
||||||
temperature,
|
|
||||||
thinking_level,
|
|
||||||
tools_declaration,
|
|
||||||
is_final_turn,
|
|
||||||
)
|
|
||||||
|
|
||||||
function_calls = response.function_calls
|
|
||||||
if not function_calls:
|
|
||||||
agent_outcome = response.text
|
|
||||||
break
|
|
||||||
|
|
||||||
if response.candidates and response.candidates[0].content:
|
|
||||||
contents.append(response.candidates[0].content)
|
|
||||||
else:
|
|
||||||
contents.append(
|
|
||||||
types.Content(
|
|
||||||
role="model",
|
|
||||||
parts=[types.Part(function_call=fc) for fc in function_calls],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
tool_response_parts = await _execute_native_tool_calls(analyzer, function_calls)
|
|
||||||
|
|
||||||
if tool_response_parts:
|
|
||||||
contents.append(types.Content(role="user", parts=tool_response_parts))
|
|
||||||
|
|
||||||
if iterations >= max_iterations and not agent_outcome:
|
|
||||||
agent_outcome = (
|
|
||||||
"Error: Video analyzer agent reached maximum iterations"
|
|
||||||
" without settling on a final answer."
|
|
||||||
)
|
|
||||||
status = "error"
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Video analyzer failed: {e}")
|
|
||||||
failure = classify_video_failure(e)
|
|
||||||
if failure.should_fallback:
|
|
||||||
logger.warning(
|
|
||||||
f"Native coordinator failed with {failure.category.value}; "
|
|
||||||
"continuing through universal fallback"
|
|
||||||
)
|
|
||||||
agent_outcome, status = await analyzer._run_universal(
|
|
||||||
time_description,
|
|
||||||
purpose,
|
|
||||||
system_prompt,
|
|
||||||
force_fallback=True,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
agent_outcome = f"Video analysis failed: {e}"
|
|
||||||
status = "error"
|
|
||||||
finally:
|
|
||||||
analyzer._record_reliability_metrics()
|
|
||||||
await _cleanup_native_run(analyzer)
|
|
||||||
|
|
||||||
return agent_outcome, status
|
|
||||||
|
|
||||||
|
|
||||||
def _append_ledger_diff(analyzer, contents: list, last_ledger_index: int) -> int:
|
|
||||||
"""Appends new blackboard entries (and proofs) since the last coordinator turn."""
|
|
||||||
new_entries = analyzer.blackboard_entries[last_ledger_index:]
|
|
||||||
if analyzer.enable_ledger and new_entries:
|
|
||||||
lines = []
|
|
||||||
image_parts = []
|
|
||||||
for e in new_entries:
|
|
||||||
line = f"[{e['start']}s - {e['end']}s] SUMMARY: {e['summary']}"
|
|
||||||
screenshot_file = e.get("screenshot") or e.get("screenshot_path")
|
|
||||||
if screenshot_file and Path(screenshot_file).exists():
|
|
||||||
line += f" PROOF: {Path(screenshot_file).name}"
|
|
||||||
image_parts.append(
|
|
||||||
types.Part.from_text(text=f"PROOF: {Path(screenshot_file).name}")
|
|
||||||
)
|
|
||||||
image_parts.append(
|
|
||||||
types.Part.from_bytes(
|
|
||||||
data=Path(screenshot_file).read_bytes(),
|
|
||||||
mime_type="image/jpeg",
|
|
||||||
)
|
|
||||||
)
|
|
||||||
lines.append(line)
|
|
||||||
diff_text = "NEW BLACKBOARD ENTRIES ADDED: " + " | ".join(lines)
|
|
||||||
contents.append(
|
|
||||||
types.Content(
|
|
||||||
role="user",
|
|
||||||
parts=[types.Part.from_text(text=diff_text)],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
if image_parts:
|
|
||||||
contents.append(types.Content(role="user", parts=image_parts))
|
|
||||||
last_ledger_index = len(analyzer.blackboard_entries)
|
|
||||||
return last_ledger_index
|
|
||||||
|
|
||||||
|
|
||||||
def _append_turn_guidance(
|
|
||||||
contents: list, iterations: int, max_iterations: int, is_final_turn: bool
|
|
||||||
) -> None:
|
|
||||||
"""Appends per-turn steering messages (final-turn notice or progress nudge)."""
|
|
||||||
if is_final_turn:
|
|
||||||
contents.append(
|
|
||||||
types.Content(
|
|
||||||
role="user",
|
|
||||||
parts=[
|
|
||||||
types.Part.from_text(
|
|
||||||
text=(
|
|
||||||
"This is your final iteration; all"
|
|
||||||
" tools are stripped, and you must"
|
|
||||||
" provide your final answer directly."
|
|
||||||
)
|
|
||||||
)
|
|
||||||
],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
if iterations > 1:
|
|
||||||
contents.append(
|
|
||||||
types.Content(
|
|
||||||
role="user",
|
|
||||||
parts=[
|
|
||||||
types.Part.from_text(
|
|
||||||
text=(
|
|
||||||
"You have not completed the video"
|
|
||||||
" analysis yet (iteration"
|
|
||||||
f" {iterations} of"
|
|
||||||
f" {max_iterations})."
|
|
||||||
)
|
|
||||||
)
|
|
||||||
],
|
|
||||||
)
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def _record_main_llm_trace(
|
|
||||||
ctx, trace_id, contents: list, system_prompt: str, tools_declaration, is_final_turn: bool
|
|
||||||
) -> None:
|
|
||||||
"""Records the sanitized coordinator LLM call into the trace store."""
|
|
||||||
serialized_contents = []
|
|
||||||
for c in contents:
|
|
||||||
if hasattr(c, "model_dump"):
|
|
||||||
dumped = c.model_dump()
|
|
||||||
if isinstance(dumped, dict) and "parts" in dumped and dumped["parts"]:
|
|
||||||
for p in dumped["parts"]:
|
|
||||||
if (
|
|
||||||
isinstance(p, dict)
|
|
||||||
and "inline_data" in p
|
|
||||||
and p["inline_data"]
|
|
||||||
and "data" in p["inline_data"]
|
|
||||||
):
|
|
||||||
p["inline_data"]["data"] = "<image_bytes_sanitized>"
|
|
||||||
serialized_contents.append(dumped)
|
|
||||||
else:
|
|
||||||
serialized_contents.append(str(c))
|
|
||||||
|
|
||||||
ctx.data_engine.record_trace(
|
|
||||||
type="llm_call",
|
|
||||||
name="video_analyzer_main",
|
|
||||||
payload={
|
|
||||||
"contents": serialized_contents,
|
|
||||||
"system_instruction": system_prompt,
|
|
||||||
"tools": (
|
|
||||||
[t.name for t in tools_declaration]
|
|
||||||
if (tools_declaration and not is_final_turn)
|
|
||||||
else []
|
|
||||||
),
|
|
||||||
},
|
|
||||||
parent_trace_id=trace_id,
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def _stream_main_turn(
|
|
||||||
analyzer,
|
|
||||||
ctx,
|
|
||||||
contents: list,
|
|
||||||
system_prompt: str,
|
|
||||||
temperature: float,
|
|
||||||
thinking_level,
|
|
||||||
tools_declaration,
|
|
||||||
is_final_turn: bool,
|
|
||||||
):
|
|
||||||
"""Runs one traced, retry-wrapped streaming turn of the main coordinator."""
|
|
||||||
with TraceSpan(name="gemini_main_agent_call") as span:
|
|
||||||
trace_id = CURRENT_TRACE_ID.get()
|
|
||||||
|
|
||||||
if ctx.data_engine and trace_id:
|
|
||||||
_record_main_llm_trace(
|
|
||||||
ctx, trace_id, contents, system_prompt, tools_declaration, is_final_turn
|
|
||||||
)
|
|
||||||
|
|
||||||
async def run_stream():
|
|
||||||
return await _consume_main_stream(
|
|
||||||
analyzer,
|
|
||||||
ctx,
|
|
||||||
span,
|
|
||||||
trace_id,
|
|
||||||
contents,
|
|
||||||
system_prompt,
|
|
||||||
temperature,
|
|
||||||
thinking_level,
|
|
||||||
tools_declaration,
|
|
||||||
is_final_turn,
|
|
||||||
)
|
|
||||||
|
|
||||||
response = await _va._invoke_with_retry(
|
|
||||||
lambda: asyncio.wait_for(run_stream(), timeout=300),
|
|
||||||
"Native Gemini video coordinator",
|
|
||||||
)
|
|
||||||
span.result = (
|
|
||||||
f"Function calls: {len(response.function_calls)}"
|
|
||||||
if response.function_calls
|
|
||||||
else "Final answer"
|
|
||||||
)
|
|
||||||
return response
|
|
||||||
|
|
||||||
|
|
||||||
async def _consume_main_stream(
|
|
||||||
analyzer,
|
|
||||||
ctx,
|
|
||||||
span,
|
|
||||||
trace_id,
|
|
||||||
contents: list,
|
|
||||||
system_prompt: str,
|
|
||||||
temperature: float,
|
|
||||||
thinking_level,
|
|
||||||
tools_declaration,
|
|
||||||
is_final_turn: bool,
|
|
||||||
) -> _DummyResponse:
|
|
||||||
"""Starts the coordinator stream and aggregates it into a response object."""
|
|
||||||
stream = await asyncio.wait_for(
|
|
||||||
analyzer.client.aio.models.generate_content_stream(
|
|
||||||
model=analyzer.model_name,
|
|
||||||
contents=contents,
|
|
||||||
config=types.GenerateContentConfig(
|
|
||||||
system_instruction=system_prompt,
|
|
||||||
temperature=temperature,
|
|
||||||
tools=[]
|
|
||||||
if is_final_turn
|
|
||||||
else [types.Tool(function_declarations=tools_declaration)],
|
|
||||||
safety_settings=SAFETY_SETTINGS_BLOCK_NONE,
|
|
||||||
**(
|
|
||||||
{"thinking_config": types.ThinkingConfig(thinking_level=thinking_level)}
|
|
||||||
if thinking_level
|
|
||||||
else {}
|
|
||||||
),
|
|
||||||
),
|
|
||||||
),
|
|
||||||
timeout=60,
|
|
||||||
)
|
|
||||||
|
|
||||||
text = ""
|
|
||||||
accumulated_parts = []
|
|
||||||
function_calls = []
|
|
||||||
|
|
||||||
async for chunk in stream:
|
|
||||||
usage = normalize_usage(getattr(chunk, "usage_metadata", None))
|
|
||||||
if usage:
|
|
||||||
span.payload["usage_metadata"] = usage
|
|
||||||
|
|
||||||
chunk_text = chunk.text or ""
|
|
||||||
text += chunk_text
|
|
||||||
if ctx.data_engine and trace_id and chunk_text:
|
|
||||||
ctx.data_engine.stream_output(trace_id, chunk_text, is_thinking=False)
|
|
||||||
|
|
||||||
if chunk.function_calls:
|
|
||||||
function_calls.extend(chunk.function_calls)
|
|
||||||
|
|
||||||
candidates = getattr(chunk, "candidates", None)
|
|
||||||
if candidates and isinstance(candidates, list) and len(candidates) > 0:
|
|
||||||
content = getattr(candidates[0], "content", None)
|
|
||||||
parts = getattr(content, "parts", None) if content else None
|
|
||||||
if parts:
|
|
||||||
for part in parts:
|
|
||||||
accumulated_parts.append(part)
|
|
||||||
if getattr(part, "thought", False) and part.text:
|
|
||||||
if ctx.data_engine and trace_id:
|
|
||||||
ctx.data_engine.stream_output(
|
|
||||||
trace_id,
|
|
||||||
part.text,
|
|
||||||
is_thinking=True,
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
if chunk.function_calls:
|
|
||||||
for fc in chunk.function_calls:
|
|
||||||
accumulated_parts.append(types.Part(function_call=fc))
|
|
||||||
|
|
||||||
return _DummyResponse(text, function_calls, accumulated_parts)
|
|
||||||
|
|
||||||
|
|
||||||
async def _execute_native_tool_calls(analyzer, function_calls: list) -> list:
|
|
||||||
"""Executes coordinator tool calls (parallel where safe) and collects responses."""
|
|
||||||
sub_agent_calls = [
|
|
||||||
fc
|
|
||||||
for fc in function_calls
|
|
||||||
if (fc.name.split(":")[-1] if ":" in fc.name else fc.name) == "spawn_sub_agent"
|
|
||||||
]
|
|
||||||
audio_only_calls = [
|
|
||||||
fc
|
|
||||||
for fc in function_calls
|
|
||||||
if (fc.name.split(":")[-1] if ":" in fc.name else fc.name) == "analyze_audio_only"
|
|
||||||
]
|
|
||||||
other_calls = [
|
|
||||||
fc
|
|
||||||
for fc in function_calls
|
|
||||||
if (fc.name.split(":")[-1] if ":" in fc.name else fc.name)
|
|
||||||
not in ["spawn_sub_agent", "analyze_audio_only"]
|
|
||||||
]
|
|
||||||
|
|
||||||
tool_response_parts = []
|
|
||||||
|
|
||||||
if sub_agent_calls:
|
|
||||||
await _run_spawn_calls(analyzer, sub_agent_calls, tool_response_parts)
|
|
||||||
|
|
||||||
if audio_only_calls:
|
|
||||||
await _run_audio_calls(analyzer, audio_only_calls, tool_response_parts)
|
|
||||||
|
|
||||||
await _run_other_calls(analyzer, other_calls, tool_response_parts)
|
|
||||||
|
|
||||||
return tool_response_parts
|
|
||||||
|
|
||||||
|
|
||||||
async def _run_spawn_calls(analyzer, sub_agent_calls: list, tool_response_parts: list) -> None:
|
|
||||||
"""Runs spawn_sub_agent calls in parallel and appends their responses."""
|
|
||||||
logger.info(f"Executing {len(sub_agent_calls)} sub-agent calls in parallel...")
|
|
||||||
|
|
||||||
async def bounded_spawn(fc):
|
|
||||||
# exec_spawn_sub_agent limits each actual model leaf.
|
|
||||||
# Holding the same semaphore around the coordinator
|
|
||||||
# would deadlock when all slots wait for child slots.
|
|
||||||
return await analyzer.exec_spawn_sub_agent(
|
|
||||||
fc.args.get("start_time"),
|
|
||||||
fc.args.get("end_time"),
|
|
||||||
fc.args.get("specific_query"),
|
|
||||||
)
|
|
||||||
|
|
||||||
tasks = [bounded_spawn(fc) for fc in sub_agent_calls]
|
|
||||||
results = await asyncio.gather(*tasks, return_exceptions=True)
|
|
||||||
|
|
||||||
for fc, result in zip(sub_agent_calls, results):
|
|
||||||
if isinstance(result, Exception):
|
|
||||||
error_msg = f"Sub-agent failed. Error: {result}"
|
|
||||||
logger.error(error_msg)
|
|
||||||
tool_response_parts.append(
|
|
||||||
types.Part.from_function_response(name=fc.name, response={"error": error_msg})
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
tool_response_parts.append(
|
|
||||||
types.Part.from_function_response(name=fc.name, response={"result": result})
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def _run_audio_calls(analyzer, audio_only_calls: list, tool_response_parts: list) -> None:
|
|
||||||
"""Runs analyze_audio_only calls in parallel and appends their responses."""
|
|
||||||
logger.info(f"Executing {len(audio_only_calls)} analyze_audio_only calls in parallel...")
|
|
||||||
|
|
||||||
async def bounded_audio_spawn(fc):
|
|
||||||
async with _va.API_SEMAPHORE:
|
|
||||||
return await analyzer.exec_analyze_audio_only(
|
|
||||||
fc.args.get("start_time"),
|
|
||||||
fc.args.get("end_time"),
|
|
||||||
fc.args.get("specific_query"),
|
|
||||||
)
|
|
||||||
|
|
||||||
tasks = [bounded_audio_spawn(fc) for fc in audio_only_calls]
|
|
||||||
results = await asyncio.gather(*tasks, return_exceptions=True)
|
|
||||||
|
|
||||||
for fc, result in zip(audio_only_calls, results):
|
|
||||||
if isinstance(result, Exception):
|
|
||||||
error_msg = f"Audio analysis failed. Error: {result}"
|
|
||||||
logger.error(error_msg)
|
|
||||||
tool_response_parts.append(
|
|
||||||
types.Part.from_function_response(name=fc.name, response={"error": error_msg})
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
tool_response_parts.append(
|
|
||||||
types.Part.from_function_response(name=fc.name, response={"result": result})
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def _run_other_calls(analyzer, other_calls: list, tool_response_parts: list) -> None:
|
|
||||||
"""Runs remaining coordinator tool calls sequentially."""
|
|
||||||
for fc in other_calls:
|
|
||||||
name = fc.name.split(":")[-1] if ":" in fc.name else fc.name
|
|
||||||
args = fc.args
|
|
||||||
logger.info(f"Executing tool '{name}' sequentially...")
|
|
||||||
|
|
||||||
if name == "extract_segment_metadata":
|
|
||||||
try:
|
|
||||||
res = await analyzer.exec_extract_segment_metadata(
|
|
||||||
args.get("start_time"), args.get("end_time")
|
|
||||||
)
|
|
||||||
tool_response_parts.append(
|
|
||||||
types.Part.from_function_response(name=name, response={"result": res})
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
tool_response_parts.append(
|
|
||||||
types.Part.from_function_response(name=name, response={"error": str(e)})
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
async def _cleanup_native_run(analyzer) -> None:
|
|
||||||
"""Releases remaining cloud files and temporary local media after a run."""
|
|
||||||
if analyzer.cloud_files_to_cleanup:
|
|
||||||
logger.info(
|
|
||||||
f"Cleaning up {len(analyzer.cloud_files_to_cleanup)} remaining"
|
|
||||||
" cloud files in parallel..."
|
|
||||||
)
|
|
||||||
tasks = [
|
|
||||||
asyncio.wait_for(analyzer.client.aio.files.delete(name=name), timeout=30)
|
|
||||||
for name in list(analyzer.cloud_files_to_cleanup)
|
|
||||||
]
|
|
||||||
await asyncio.gather(*tasks, return_exceptions=True)
|
|
||||||
analyzer.cloud_files_to_cleanup.clear()
|
|
||||||
|
|
||||||
if os.environ.get("KEEP_VIDEOS") or os.environ.get("ARTEMIS_DEBUG"):
|
|
||||||
logger.info("Skipping cleanup of local video files and directories due to debug mode.")
|
|
||||||
else:
|
|
||||||
for path in list(analyzer.local_files_to_cleanup):
|
|
||||||
try:
|
|
||||||
if path.exists():
|
|
||||||
logger.info(f"Cleaning up local file {path}...")
|
|
||||||
path.unlink()
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Failed to delete local file {path}: {e}")
|
|
||||||
|
|
||||||
for dir_path in list(analyzer.local_dirs_to_cleanup):
|
|
||||||
try:
|
|
||||||
resolved = dir_path.resolve()
|
|
||||||
system_dirs = {
|
|
||||||
Path("/").resolve(),
|
|
||||||
Path("/tmp").resolve(),
|
|
||||||
Path("/var/tmp").resolve(),
|
|
||||||
Path.home().resolve(),
|
|
||||||
Path(tempfile.gettempdir()).resolve(),
|
|
||||||
}
|
|
||||||
if resolved in system_dirs:
|
|
||||||
logger.debug(f"Skipping deletion of root/system directory: {dir_path}")
|
|
||||||
continue
|
|
||||||
|
|
||||||
if dir_path.exists():
|
|
||||||
logger.info(f"Cleaning up local directory {dir_path}...")
|
|
||||||
shutil.rmtree(dir_path, ignore_errors=True)
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Failed to delete local directory {dir_path}: {e}")
|
|
||||||
@@ -35,6 +35,8 @@ from langchain_core.messages import (
|
|||||||
ToolMessage,
|
ToolMessage,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
from artemis.llm.structured import content_to_text
|
||||||
|
|
||||||
from artemis.agents.video_analyzer import video_analyzer as _va
|
from artemis.agents.video_analyzer import video_analyzer as _va
|
||||||
from artemis.agents.video_analyzer.reliability import (
|
from artemis.agents.video_analyzer.reliability import (
|
||||||
SubAgentAnswerExhausted,
|
SubAgentAnswerExhausted,
|
||||||
@@ -241,7 +243,7 @@ def _extract_submit_payload(response) -> tuple[str, str, list]:
|
|||||||
return final_summary, final_analysis, list(args.get("timeline_events") or [])
|
return final_summary, final_analysis, list(args.get("timeline_events") or [])
|
||||||
|
|
||||||
raw_content = getattr(response, "content", None)
|
raw_content = getattr(response, "content", None)
|
||||||
text_content = str(raw_content).strip() if raw_content else ""
|
text_content = content_to_text(raw_content).strip()
|
||||||
if text_content:
|
if text_content:
|
||||||
return text_content[:200].replace("\n", " "), text_content, []
|
return text_content[:200].replace("\n", " "), text_content, []
|
||||||
|
|
||||||
@@ -435,7 +437,7 @@ async def run_universal(
|
|||||||
messages.append(response)
|
messages.append(response)
|
||||||
|
|
||||||
if not getattr(response, "tool_calls", None):
|
if not getattr(response, "tool_calls", None):
|
||||||
agent_outcome = str(response.content)
|
agent_outcome = content_to_text(response.content)
|
||||||
break
|
break
|
||||||
|
|
||||||
for tc in response.tool_calls:
|
for tc in response.tool_calls:
|
||||||
|
|||||||
@@ -14,14 +14,11 @@
|
|||||||
|
|
||||||
"""Tool schema declarations for the video analyzer.
|
"""Tool schema declarations for the video analyzer.
|
||||||
|
|
||||||
Holds both the OpenAI-style function schemas used by the universal
|
OpenAI function schemas for the Responses video analysis engine.
|
||||||
(LangChain) engine and the builders for the native Gemini
|
|
||||||
``FunctionDeclaration`` equivalents. Pure data/declarations only.
|
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from google.genai import types
|
|
||||||
|
|
||||||
UNIVERSAL_SUBMIT_ANSWER_TOOL: dict[str, Any] = {
|
UNIVERSAL_SUBMIT_ANSWER_TOOL: dict[str, Any] = {
|
||||||
"type": "function",
|
"type": "function",
|
||||||
@@ -172,152 +169,3 @@ UNIVERSAL_MAIN_TOOLS: list[dict[str, Any]] = [
|
|||||||
},
|
},
|
||||||
},
|
},
|
||||||
]
|
]
|
||||||
|
|
||||||
|
|
||||||
def build_native_tools_declaration() -> list[types.FunctionDeclaration]:
|
|
||||||
"""Builds the native Gemini declarations for the coordinator tools."""
|
|
||||||
return [
|
|
||||||
types.FunctionDeclaration(
|
|
||||||
name="extract_segment_metadata",
|
|
||||||
description=(
|
|
||||||
"Retrieves metadata (such as duration_seconds and"
|
|
||||||
" file_size_mb) for a specific video segment. Does not"
|
|
||||||
" return the video content or perform content analysis."
|
|
||||||
),
|
|
||||||
parameters=types.Schema(
|
|
||||||
type=types.Type.OBJECT,
|
|
||||||
properties={
|
|
||||||
"start_time": types.Schema(
|
|
||||||
type=types.Type.NUMBER,
|
|
||||||
description="Start time in seconds",
|
|
||||||
),
|
|
||||||
"end_time": types.Schema(
|
|
||||||
type=types.Type.NUMBER,
|
|
||||||
description=(
|
|
||||||
"End time in seconds (optional, defaults to latest available time)"
|
|
||||||
),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
required=["start_time"],
|
|
||||||
),
|
|
||||||
),
|
|
||||||
types.FunctionDeclaration(
|
|
||||||
name="spawn_sub_agent",
|
|
||||||
description=(
|
|
||||||
"Spawns a sub-agent to analyze both visual and audio"
|
|
||||||
" content of a video segment. Parallelizes chunk analysis"
|
|
||||||
" across workers. Returns combined summary and analysis."
|
|
||||||
),
|
|
||||||
parameters=types.Schema(
|
|
||||||
type=types.Type.OBJECT,
|
|
||||||
properties={
|
|
||||||
"start_time": types.Schema(
|
|
||||||
type=types.Type.NUMBER,
|
|
||||||
description="Start time in seconds",
|
|
||||||
),
|
|
||||||
"end_time": types.Schema(
|
|
||||||
type=types.Type.NUMBER,
|
|
||||||
description=(
|
|
||||||
"End time in seconds. Use the maximum available time if not specified."
|
|
||||||
),
|
|
||||||
),
|
|
||||||
"specific_query": types.Schema(
|
|
||||||
type=types.Type.STRING,
|
|
||||||
description=("Specific query or intent for the sub-agent"),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
required=["start_time", "specific_query"],
|
|
||||||
),
|
|
||||||
),
|
|
||||||
types.FunctionDeclaration(
|
|
||||||
name="analyze_audio_only",
|
|
||||||
description=(
|
|
||||||
"Extracts and analyzes only the audio track from a video"
|
|
||||||
" segment. More efficient for non-visual tasks."
|
|
||||||
),
|
|
||||||
parameters=types.Schema(
|
|
||||||
type=types.Type.OBJECT,
|
|
||||||
properties={
|
|
||||||
"start_time": types.Schema(
|
|
||||||
type=types.Type.NUMBER,
|
|
||||||
description="Start time in seconds",
|
|
||||||
),
|
|
||||||
"end_time": types.Schema(
|
|
||||||
type=types.Type.NUMBER,
|
|
||||||
description="End time in seconds (optional)",
|
|
||||||
),
|
|
||||||
"specific_query": types.Schema(
|
|
||||||
type=types.Type.STRING,
|
|
||||||
description=("Specific query or intent for the audio content"),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
required=["start_time", "specific_query"],
|
|
||||||
),
|
|
||||||
),
|
|
||||||
]
|
|
||||||
|
|
||||||
|
|
||||||
def build_submit_answer_declaration() -> types.FunctionDeclaration:
|
|
||||||
"""Builds the native Gemini declaration for the sub-agent submit_answer tool."""
|
|
||||||
return types.FunctionDeclaration(
|
|
||||||
name="submit_answer",
|
|
||||||
description=("Submit the final findings for the requested segment analysis."),
|
|
||||||
parameters=types.Schema(
|
|
||||||
type=types.Type.OBJECT,
|
|
||||||
properties={
|
|
||||||
"summary": types.Schema(
|
|
||||||
type=types.Type.STRING,
|
|
||||||
description=("A short paragraph summarizing of your findings."),
|
|
||||||
),
|
|
||||||
"analysis": types.Schema(
|
|
||||||
type=types.Type.STRING,
|
|
||||||
description="A short paragraph of detailed analysis.",
|
|
||||||
),
|
|
||||||
"timeline_events": types.Schema(
|
|
||||||
type=types.Type.ARRAY,
|
|
||||||
description=(
|
|
||||||
"List of detected events matching the query. If no"
|
|
||||||
" relevant events occurred in the segment, return"
|
|
||||||
" an empty array."
|
|
||||||
),
|
|
||||||
items=types.Schema(
|
|
||||||
type=types.Type.OBJECT,
|
|
||||||
properties={
|
|
||||||
"start_time": types.Schema(
|
|
||||||
type=types.Type.NUMBER,
|
|
||||||
description="Event start time (seconds)",
|
|
||||||
),
|
|
||||||
"end_time": types.Schema(
|
|
||||||
type=types.Type.NUMBER,
|
|
||||||
description="Event end time (seconds)",
|
|
||||||
),
|
|
||||||
"transcription": types.Schema(
|
|
||||||
type=types.Type.STRING,
|
|
||||||
description=(
|
|
||||||
"A concise transcription of what happened in the event. "
|
|
||||||
),
|
|
||||||
),
|
|
||||||
"confidence_score": types.Schema(
|
|
||||||
type=types.Type.NUMBER,
|
|
||||||
description=("Confidence score between 0 and 1"),
|
|
||||||
),
|
|
||||||
"verification_timestamp_secs": types.Schema(
|
|
||||||
type=types.Type.NUMBER,
|
|
||||||
description=(
|
|
||||||
"Exact timestamp (seconds) for verification screenshot."
|
|
||||||
),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
required=[
|
|
||||||
"start_time",
|
|
||||||
"end_time",
|
|
||||||
"transcription",
|
|
||||||
"confidence_score",
|
|
||||||
"verification_timestamp_secs",
|
|
||||||
],
|
|
||||||
),
|
|
||||||
),
|
|
||||||
},
|
|
||||||
required=["summary", "analysis", "timeline_events"],
|
|
||||||
),
|
|
||||||
)
|
|
||||||
@@ -14,9 +14,8 @@
|
|||||||
|
|
||||||
"""VideoAnalyzer facade: entry point and shared runtime state.
|
"""VideoAnalyzer facade: entry point and shared runtime state.
|
||||||
|
|
||||||
The execution engines live in sibling modules (``chunk_native``,
|
The execution engines live in sibling modules (``chunk_runner``,
|
||||||
``audio_native``, ``universal_engine``, ``native_coordinator``,
|
``audio_runner``, ``universal_engine``, ``universal_tools``) and resolve their patchable
|
||||||
``gemini_files``, ``universal_tools``) and resolve their patchable
|
|
||||||
collaborators through THIS module's namespace, so existing
|
collaborators through THIS module's namespace, so existing
|
||||||
``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")``
|
``mock.patch("artemis.agents.video_analyzer.video_analyzer.<name>")``
|
||||||
targets keep working. Keep the imports below in place even when the facade
|
targets keep working. Keep the imports below in place even when the facade
|
||||||
@@ -36,8 +35,6 @@ import tempfile
|
|||||||
import time
|
import time
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from google import genai
|
|
||||||
from google.genai import types
|
|
||||||
from langchain_core.messages import (
|
from langchain_core.messages import (
|
||||||
BaseMessage,
|
BaseMessage,
|
||||||
HumanMessage,
|
HumanMessage,
|
||||||
@@ -49,9 +46,6 @@ from artemis.agents.video_analyzer.conflict_resolution import (
|
|||||||
ConflictResolutionService,
|
ConflictResolutionService,
|
||||||
)
|
)
|
||||||
from artemis.agents.video_analyzer.blackboard import get_video_blackboard
|
from artemis.agents.video_analyzer.blackboard import get_video_blackboard
|
||||||
from artemis.agents.video_analyzer.gemini_files import (
|
|
||||||
cleanup_abandoned_gemini_files,
|
|
||||||
)
|
|
||||||
from artemis.agents.video_analyzer.reliability import (
|
from artemis.agents.video_analyzer.reliability import (
|
||||||
AgenticVideoDegraded,
|
AgenticVideoDegraded,
|
||||||
VideoCircuitBreaker,
|
VideoCircuitBreaker,
|
||||||
@@ -60,19 +54,11 @@ from artemis.agents.video_analyzer.reliability import (
|
|||||||
from artemis.agents.video_analyzer.universal_tools import (
|
from artemis.agents.video_analyzer.universal_tools import (
|
||||||
UNIVERSAL_MAIN_TOOLS,
|
UNIVERSAL_MAIN_TOOLS,
|
||||||
UNIVERSAL_SUBMIT_ANSWER_TOOL,
|
UNIVERSAL_SUBMIT_ANSWER_TOOL,
|
||||||
build_native_tools_declaration,
|
|
||||||
build_submit_answer_declaration,
|
|
||||||
)
|
)
|
||||||
from artemis.config import get_temp_dir, settings
|
from artemis.config import get_temp_dir, settings
|
||||||
from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
|
|
||||||
from artemis.context import ArtemisContext
|
from artemis.context import ArtemisContext
|
||||||
from artemis.controllers.controller_factory import get_controller
|
from artemis.controllers.controller_factory import get_controller
|
||||||
from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan, trace
|
from artemis.data_engine.trace import CURRENT_TRACE_ID, TraceSpan, trace
|
||||||
from artemis.llm.google import (
|
|
||||||
is_gemini_model,
|
|
||||||
resolve_video_processing,
|
|
||||||
strip_provider_prefix,
|
|
||||||
)
|
|
||||||
from artemis.llm.reliability import retry_policy_for
|
from artemis.llm.reliability import retry_policy_for
|
||||||
from artemis.services.llm import _record_llm_event, _record_llm_retry, get_llm
|
from artemis.services.llm import _record_llm_event, _record_llm_retry, get_llm
|
||||||
from artemis.utils.logger import get_logger
|
from artemis.utils.logger import get_logger
|
||||||
@@ -159,16 +145,8 @@ class VideoAnalyzer:
|
|||||||
getattr(agent_config, "enable_video_ledger", True),
|
getattr(agent_config, "enable_video_ledger", True),
|
||||||
)
|
)
|
||||||
self.static_chunk_size_seconds = float(
|
self.static_chunk_size_seconds = float(
|
||||||
getattr(self.video_config, "chunk_size_seconds", 60.0)
|
getattr(self.video_config, "chunk_size_seconds", settings.ARTEMIS_VIDEO_CHUNK_SECONDS)
|
||||||
)
|
)
|
||||||
self.agentic_chunk_size_seconds = float(
|
|
||||||
getattr(self.video_config, "agentic_chunk_size_seconds", 600.0)
|
|
||||||
)
|
|
||||||
self.agentic_call_timeout_seconds = float(
|
|
||||||
getattr(self.video_config, "agentic_call_timeout_seconds", 300.0)
|
|
||||||
)
|
|
||||||
# Effective chunk ceiling; re-derived once the engine picks a
|
|
||||||
# video-processing mode (agentic sub-agents take long clips).
|
|
||||||
self.chunk_size_seconds = self.static_chunk_size_seconds
|
self.chunk_size_seconds = self.static_chunk_size_seconds
|
||||||
self.video_processing = "static"
|
self.video_processing = "static"
|
||||||
self.min_chunk_seconds = float(getattr(self.video_config, "min_chunk_seconds", 4.0))
|
self.min_chunk_seconds = float(getattr(self.video_config, "min_chunk_seconds", 4.0))
|
||||||
@@ -178,9 +156,15 @@ class VideoAnalyzer:
|
|||||||
self.max_dense_action_frames = int(
|
self.max_dense_action_frames = int(
|
||||||
getattr(self.video_config, "max_dense_action_frames", 24)
|
getattr(self.video_config, "max_dense_action_frames", 24)
|
||||||
)
|
)
|
||||||
self.native_max_retries = int(getattr(self.video_config, "native_max_retries", 1))
|
self.max_retries = int(
|
||||||
|
getattr(self.video_config, "max_retries", settings.ARTEMIS_VIDEO_MAX_RETRIES)
|
||||||
|
)
|
||||||
self.model_call_timeout_seconds = float(
|
self.model_call_timeout_seconds = float(
|
||||||
getattr(self.video_config, "model_call_timeout_seconds", 120.0)
|
getattr(
|
||||||
|
self.video_config,
|
||||||
|
"model_call_timeout_seconds",
|
||||||
|
settings.ARTEMIS_VIDEO_TIMEOUT_SECONDS,
|
||||||
|
)
|
||||||
)
|
)
|
||||||
breaker = getattr(ctx, "_video_circuit_breaker", None)
|
breaker = getattr(ctx, "_video_circuit_breaker", None)
|
||||||
if breaker is None:
|
if breaker is None:
|
||||||
@@ -210,7 +194,6 @@ class VideoAnalyzer:
|
|||||||
audio_prompt_path.read_text(encoding="utf-8") if audio_prompt_path.exists() else ""
|
audio_prompt_path.read_text(encoding="utf-8") if audio_prompt_path.exists() else ""
|
||||||
)
|
)
|
||||||
|
|
||||||
self._init_tools()
|
|
||||||
self._init_engine()
|
self._init_engine()
|
||||||
|
|
||||||
def _get_universal_llm(self, *, use_fallback: bool = False):
|
def _get_universal_llm(self, *, use_fallback: bool = False):
|
||||||
@@ -406,65 +389,13 @@ class VideoAnalyzer:
|
|||||||
analysis = match.group(2).strip() if match.group(2) else None
|
analysis = match.group(2).strip() if match.group(2) else None
|
||||||
return summary, analysis
|
return summary, analysis
|
||||||
|
|
||||||
def _init_tools(self) -> None:
|
|
||||||
"""Initializes native and sub-agent tool declarations."""
|
|
||||||
self.tools_declaration = build_native_tools_declaration()
|
|
||||||
self.submit_answer_tool = build_submit_answer_declaration()
|
|
||||||
|
|
||||||
def _init_engine(self) -> None:
|
def _init_engine(self) -> None:
|
||||||
"""Initializes model engine and decides whether to use native Gemini or Universal path."""
|
llm_config = getattr(self.ctx, "llm_config", None)
|
||||||
ctx = self.ctx
|
|
||||||
llm_config = getattr(ctx, "llm_config", None)
|
|
||||||
utils_cfg = getattr(llm_config, "utils", None) if llm_config else None
|
utils_cfg = getattr(llm_config, "utils", None) if llm_config else None
|
||||||
llm_cfg = getattr(utils_cfg, "video_analyzer", None) if utils_cfg else None
|
llm_cfg = getattr(utils_cfg, "video_analyzer", None) if utils_cfg else None
|
||||||
self.model_name = strip_provider_prefix(
|
self.model_name = getattr(llm_cfg, "model", None) or settings.OPENAI_MODEL
|
||||||
llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.8-flash"
|
self.video_processing = "static"
|
||||||
)
|
self.chunk_size_seconds = self.static_chunk_size_seconds
|
||||||
|
|
||||||
has_google_key = bool(
|
|
||||||
settings.GOOGLE_API_KEY and settings.GOOGLE_API_KEY.get_secret_value()
|
|
||||||
)
|
|
||||||
|
|
||||||
# If client is already set on ctx or explicit Gemini configuration
|
|
||||||
self.client = getattr(ctx, "_genai_client", None)
|
|
||||||
if self.client is not None:
|
|
||||||
self.use_native_gemini = True
|
|
||||||
elif has_google_key and is_gemini_model(self.model_name):
|
|
||||||
try:
|
|
||||||
self.client = genai.Client(api_key=settings.GOOGLE_API_KEY.get_secret_value())
|
|
||||||
ctx._genai_client = self.client
|
|
||||||
self.use_native_gemini = True
|
|
||||||
asyncio.create_task(cleanup_abandoned_gemini_files(self.client))
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(
|
|
||||||
f"Failed to initialize native Gemini client: {e}. Using universal engine."
|
|
||||||
)
|
|
||||||
self.use_native_gemini = False
|
|
||||||
else:
|
|
||||||
self.use_native_gemini = False
|
|
||||||
self._resolve_video_processing()
|
|
||||||
|
|
||||||
def _resolve_video_processing(self) -> None:
|
|
||||||
"""Picks static vs agentic video understanding for the sub-agents.
|
|
||||||
|
|
||||||
Agentic processing (Gemini Interactions API) needs the native client
|
|
||||||
and a supporting model; the configured knob (``auto`` / ``agentic`` /
|
|
||||||
``static``) decides the rest. The chunk ceiling follows the mode
|
|
||||||
because agentic sub-agents are built to take long clips.
|
|
||||||
"""
|
|
||||||
configured = getattr(self.video_config, "processing", "auto")
|
|
||||||
mode = "static"
|
|
||||||
if self.use_native_gemini:
|
|
||||||
mode = resolve_video_processing(configured, self.model_name)
|
|
||||||
self.video_processing = mode
|
|
||||||
self.chunk_size_seconds = (
|
|
||||||
self.agentic_chunk_size_seconds if mode == "agentic" else self.static_chunk_size_seconds
|
|
||||||
)
|
|
||||||
if mode == "agentic":
|
|
||||||
logger.info(
|
|
||||||
f"Video sub-agents use agentic video understanding on {self.model_name}"
|
|
||||||
f" (chunk ceiling {self.chunk_size_seconds:.0f}s)"
|
|
||||||
)
|
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def blackboard_ledger(self) -> str:
|
def blackboard_ledger(self) -> str:
|
||||||
@@ -476,13 +407,6 @@ class VideoAnalyzer:
|
|||||||
lines.append(f"{e['start']}s - {e['end']}s: {e['summary']}")
|
lines.append(f"{e['start']}s - {e['end']}s: {e['summary']}")
|
||||||
return " | ".join(lines)
|
return " | ".join(lines)
|
||||||
|
|
||||||
async def upload_and_poll_file(self, compressed_path: Path) -> any:
|
|
||||||
from artemis.agents.video_analyzer import gemini_files
|
|
||||||
|
|
||||||
return await gemini_files.upload_and_poll_file(
|
|
||||||
self.client, compressed_path, self.cloud_files_to_cleanup
|
|
||||||
)
|
|
||||||
|
|
||||||
def get_overlapping_warnings(self, start_time: float, end_time: float | None) -> list[dict]:
|
def get_overlapping_warnings(self, start_time: float, end_time: float | None) -> list[dict]:
|
||||||
warnings = []
|
warnings = []
|
||||||
if not isinstance(start_time, (int, float)) or (
|
if not isinstance(start_time, (int, float)) or (
|
||||||
@@ -739,17 +663,17 @@ class VideoAnalyzer:
|
|||||||
async def _exec_single_chunk(
|
async def _exec_single_chunk(
|
||||||
self, start_time: float, end_time: float | None, specific_query: str
|
self, start_time: float, end_time: float | None, specific_query: str
|
||||||
) -> str:
|
) -> str:
|
||||||
from artemis.agents.video_analyzer import chunk_native
|
from artemis.agents.video_analyzer import chunk_runner
|
||||||
|
|
||||||
return await chunk_native.exec_single_chunk(self, start_time, end_time, specific_query)
|
return await chunk_runner.exec_single_chunk(self, start_time, end_time, specific_query)
|
||||||
|
|
||||||
@trace(type="tool", name="analyze_audio_only")
|
@trace(type="tool", name="analyze_audio_only")
|
||||||
async def exec_analyze_audio_only(
|
async def exec_analyze_audio_only(
|
||||||
self, start_time: float, end_time: float | None, specific_query: str
|
self, start_time: float, end_time: float | None, specific_query: str
|
||||||
) -> str:
|
) -> str:
|
||||||
from artemis.agents.video_analyzer import audio_native
|
from artemis.agents.video_analyzer import audio_runner
|
||||||
|
|
||||||
return await audio_native.exec_analyze_audio_only(
|
return await audio_runner.exec_analyze_audio_only(
|
||||||
self, start_time, end_time, specific_query
|
self, start_time, end_time, specific_query
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -837,15 +761,12 @@ class VideoAnalyzer:
|
|||||||
utils_cfg = getattr(llm_config, "utils", None) if llm_config else None
|
utils_cfg = getattr(llm_config, "utils", None) if llm_config else None
|
||||||
llm_cfg = getattr(utils_cfg, "video_analyzer", None) if utils_cfg else None
|
llm_cfg = getattr(utils_cfg, "video_analyzer", None) if utils_cfg else None
|
||||||
temperature = 0.2
|
temperature = 0.2
|
||||||
thinking_level = None
|
|
||||||
if llm_cfg:
|
if llm_cfg:
|
||||||
self.model_name = strip_provider_prefix(llm_cfg.model)
|
self.model_name = llm_cfg.model
|
||||||
if getattr(llm_cfg, "temperature", None) is not None:
|
if getattr(llm_cfg, "temperature", None) is not None:
|
||||||
temperature = llm_cfg.temperature
|
temperature = llm_cfg.temperature
|
||||||
if getattr(llm_cfg, "thinking_level", None) is not None:
|
|
||||||
thinking_level = llm_cfg.thinking_level
|
|
||||||
else:
|
else:
|
||||||
self.model_name = "gemini-3.8-flash"
|
self.model_name = settings.OPENAI_MODEL
|
||||||
|
|
||||||
# Track files for cleanup
|
# Track files for cleanup
|
||||||
self.local_files_to_cleanup = set()
|
self.local_files_to_cleanup = set()
|
||||||
@@ -868,24 +789,11 @@ class VideoAnalyzer:
|
|||||||
" proceeding.\n"
|
" proceeding.\n"
|
||||||
)
|
)
|
||||||
|
|
||||||
if not self.use_native_gemini:
|
try:
|
||||||
try:
|
return await self._run_universal(
|
||||||
return await self._run_universal(
|
time_description=time_description,
|
||||||
time_description=time_description,
|
purpose=purpose,
|
||||||
purpose=purpose,
|
system_prompt=system_prompt,
|
||||||
system_prompt=system_prompt,
|
)
|
||||||
)
|
finally:
|
||||||
finally:
|
await self._cleanup_universal_resources()
|
||||||
await self._cleanup_universal_resources()
|
|
||||||
|
|
||||||
from artemis.agents.video_analyzer import native_coordinator
|
|
||||||
|
|
||||||
return await native_coordinator.run_native(
|
|
||||||
self,
|
|
||||||
ctx,
|
|
||||||
time_description,
|
|
||||||
purpose,
|
|
||||||
system_prompt,
|
|
||||||
temperature,
|
|
||||||
thinking_level,
|
|
||||||
)
|
|
||||||
@@ -57,14 +57,10 @@ from artemis.config.constants import (
|
|||||||
ENV_ADB_HOST,
|
ENV_ADB_HOST,
|
||||||
ENV_ADB_PORT,
|
ENV_ADB_PORT,
|
||||||
ENV_ADB_SERVER_SOCKET,
|
ENV_ADB_SERVER_SOCKET,
|
||||||
ENV_ANTHROPIC_API_KEY,
|
|
||||||
ENV_ANTIGRAVITY_APP_DIR,
|
ENV_ANTIGRAVITY_APP_DIR,
|
||||||
ENV_ANTIGRAVITY_LS_ADDRESS,
|
ENV_ANTIGRAVITY_LS_ADDRESS,
|
||||||
ENV_DATA_ENGINE_DB_PATH,
|
ENV_DATA_ENGINE_DB_PATH,
|
||||||
ENV_EVENTS_OUTPUT_PATH,
|
ENV_EVENTS_OUTPUT_PATH,
|
||||||
ENV_GCP_API_KEY,
|
|
||||||
ENV_GEMINI_API_KEY,
|
|
||||||
ENV_GOOGLE_API_KEY,
|
|
||||||
ENV_KEEP_VIDEOS,
|
ENV_KEEP_VIDEOS,
|
||||||
ENV_ARTEMIS_APP_DIR,
|
ENV_ARTEMIS_APP_DIR,
|
||||||
ENV_ARTEMIS_CLOUD_GATEWAY_URL,
|
ENV_ARTEMIS_CLOUD_GATEWAY_URL,
|
||||||
@@ -83,11 +79,9 @@ from artemis.config.constants import (
|
|||||||
ENV_ARTEMIS_TRACES_DIR,
|
ENV_ARTEMIS_TRACES_DIR,
|
||||||
ENV_ARTEMIS_USE_FILE_API,
|
ENV_ARTEMIS_USE_FILE_API,
|
||||||
ENV_ARTEMIS_USE_USER_DIR,
|
ENV_ARTEMIS_USE_USER_DIR,
|
||||||
ENV_OPEN_ROUTER_API_KEY,
|
|
||||||
ENV_OPENAI_API_KEY,
|
ENV_OPENAI_API_KEY,
|
||||||
ENV_OPENAI_BASE_URL,
|
ENV_OPENAI_BASE_URL,
|
||||||
ENV_RESULTS_OUTPUT_PATH,
|
ENV_RESULTS_OUTPUT_PATH,
|
||||||
ENV_XAI_API_KEY,
|
|
||||||
IPC_PORT_FILENAME,
|
IPC_PORT_FILENAME,
|
||||||
LLM_CONFIG_FILENAME,
|
LLM_CONFIG_FILENAME,
|
||||||
LLM_CONFIG_OVERRIDE_FILENAME,
|
LLM_CONFIG_OVERRIDE_FILENAME,
|
||||||
@@ -110,7 +104,6 @@ from artemis.config.llm import (
|
|||||||
initialize_llm_config,
|
initialize_llm_config,
|
||||||
load_llm_config_override,
|
load_llm_config_override,
|
||||||
parse_llm_config,
|
parse_llm_config,
|
||||||
validate_vertex_ai_credentials,
|
|
||||||
)
|
)
|
||||||
from artemis.config.output import (
|
from artemis.config.output import (
|
||||||
OutputConfig,
|
OutputConfig,
|
||||||
@@ -206,7 +199,6 @@ __all__ = [
|
|||||||
"get_default_llm_config",
|
"get_default_llm_config",
|
||||||
"deep_merge_llm_config",
|
"deep_merge_llm_config",
|
||||||
"load_llm_config_override",
|
"load_llm_config_override",
|
||||||
"validate_vertex_ai_credentials",
|
|
||||||
# Agent & Tool Config
|
# Agent & Tool Config
|
||||||
"AgentGlobalConfig",
|
"AgentGlobalConfig",
|
||||||
"CheckerConfig",
|
"CheckerConfig",
|
||||||
@@ -290,14 +282,8 @@ __all__ = [
|
|||||||
"ENV_ADB_HOST",
|
"ENV_ADB_HOST",
|
||||||
"ENV_ADB_PORT",
|
"ENV_ADB_PORT",
|
||||||
"ENV_ADB_SERVER_SOCKET",
|
"ENV_ADB_SERVER_SOCKET",
|
||||||
"ENV_GOOGLE_API_KEY",
|
|
||||||
"ENV_GEMINI_API_KEY",
|
|
||||||
"ENV_GCP_API_KEY",
|
|
||||||
"ENV_OPENAI_API_KEY",
|
"ENV_OPENAI_API_KEY",
|
||||||
"ENV_OPENAI_BASE_URL",
|
"ENV_OPENAI_BASE_URL",
|
||||||
"ENV_ANTHROPIC_API_KEY",
|
|
||||||
"ENV_OPEN_ROUTER_API_KEY",
|
|
||||||
"ENV_XAI_API_KEY",
|
|
||||||
"ENV_ARTEMIS_EXPLORER_VERSION",
|
"ENV_ARTEMIS_EXPLORER_VERSION",
|
||||||
"ENV_ARTEMIS_DEFAULT_PROFILE",
|
"ENV_ARTEMIS_DEFAULT_PROFILE",
|
||||||
"ENV_ARTEMIS_DEFAULT_MODEL",
|
"ENV_ARTEMIS_DEFAULT_MODEL",
|
||||||
|
|||||||
+14
-38
@@ -26,7 +26,8 @@ from artemis.config.constants import (
|
|||||||
ExplorerVersion,
|
ExplorerVersion,
|
||||||
)
|
)
|
||||||
from artemis.config.paths import ROOT_DIR, get_config_path
|
from artemis.config.paths import ROOT_DIR, get_config_path
|
||||||
from artemis.llm.google import VideoProcessing
|
from artemis.config.settings import settings
|
||||||
|
from artemis.config.core.interpolator import interpolate_config_value
|
||||||
from artemis.utils.file import load_jsonc
|
from artemis.utils.file import load_jsonc
|
||||||
from artemis.utils.logger import get_logger
|
from artemis.utils.logger import get_logger
|
||||||
|
|
||||||
@@ -48,7 +49,7 @@ class VideoAnalyzerConfig(BaseModel):
|
|||||||
description="Whether to enable video action ledger state tracking.",
|
description="Whether to enable video action ledger state tracking.",
|
||||||
)
|
)
|
||||||
chunk_size_seconds: float = Field(
|
chunk_size_seconds: float = Field(
|
||||||
default=60.0,
|
default_factory=lambda: settings.ARTEMIS_VIDEO_CHUNK_SECONDS,
|
||||||
ge=5.0,
|
ge=5.0,
|
||||||
le=300.0,
|
le=300.0,
|
||||||
description="Maximum duration of an initial video-analysis chunk.",
|
description="Maximum duration of an initial video-analysis chunk.",
|
||||||
@@ -95,44 +96,20 @@ class VideoAnalyzerConfig(BaseModel):
|
|||||||
le=120,
|
le=120,
|
||||||
description="Maximum additional action-proximal keyframes per chunk.",
|
description="Maximum additional action-proximal keyframes per chunk.",
|
||||||
)
|
)
|
||||||
native_max_retries: int = Field(
|
max_retries: int = Field(
|
||||||
default=1,
|
default_factory=lambda: settings.ARTEMIS_VIDEO_MAX_RETRIES,
|
||||||
ge=0,
|
ge=0,
|
||||||
le=5,
|
le=5,
|
||||||
description=(
|
description=(
|
||||||
"Retries against the native video provider before switching to the "
|
"Retries against the video model before switching to the "
|
||||||
"configured universal fallback. A value of 1 means two total attempts."
|
"configured universal fallback. A value of 1 means two total attempts."
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
model_call_timeout_seconds: float = Field(
|
model_call_timeout_seconds: float = Field(
|
||||||
default=120.0,
|
default_factory=lambda: settings.ARTEMIS_VIDEO_TIMEOUT_SECONDS,
|
||||||
ge=15.0,
|
ge=15.0,
|
||||||
le=600.0,
|
le=600.0,
|
||||||
description="Hard timeout for one native or universal video-model response.",
|
description="Hard timeout for one Responses video-model response.",
|
||||||
)
|
|
||||||
processing: VideoProcessing = Field(
|
|
||||||
default="auto",
|
|
||||||
description=(
|
|
||||||
"How Gemini sub-agents watch a clip. 'agentic' lets the model search,"
|
|
||||||
" scan and re-watch the clip on demand (Gemini Interactions API);"
|
|
||||||
" 'static' samples it at a fixed frame rate; 'auto' picks agentic on"
|
|
||||||
" Gemini 3.6+ Flash models and static everywhere else."
|
|
||||||
),
|
|
||||||
)
|
|
||||||
agentic_chunk_size_seconds: float = Field(
|
|
||||||
default=600.0,
|
|
||||||
ge=30.0,
|
|
||||||
le=1800.0,
|
|
||||||
description=(
|
|
||||||
"Maximum clip length handed to one agentic sub-agent. Static mode keeps"
|
|
||||||
" using chunk_size_seconds."
|
|
||||||
),
|
|
||||||
)
|
|
||||||
agentic_call_timeout_seconds: float = Field(
|
|
||||||
default=300.0,
|
|
||||||
ge=30.0,
|
|
||||||
le=1200.0,
|
|
||||||
description="Hard timeout for one agentic sub-agent response.",
|
|
||||||
)
|
)
|
||||||
model_config = {"extra": "allow"}
|
model_config = {"extra": "allow"}
|
||||||
|
|
||||||
@@ -214,12 +191,12 @@ class CheckerConfig(BaseModel):
|
|||||||
description="Maximum concurrently running checkpoint attempts.",
|
description="Maximum concurrently running checkpoint attempts.",
|
||||||
)
|
)
|
||||||
checkpoint_timeout: float = Field(
|
checkpoint_timeout: float = Field(
|
||||||
default=180.0,
|
default_factory=lambda: settings.ARTEMIS_CHECKPOINT_TIMEOUT_SECONDS,
|
||||||
gt=0,
|
gt=0,
|
||||||
description="Per-attempt timeout (seconds) for one checkpoint check.",
|
description="Per-attempt timeout (seconds) for one checkpoint check.",
|
||||||
)
|
)
|
||||||
settlement_timeout: float = Field(
|
settlement_timeout: float = Field(
|
||||||
default=120.0,
|
default_factory=lambda: settings.ARTEMIS_SETTLEMENT_TIMEOUT_SECONDS,
|
||||||
gt=0,
|
gt=0,
|
||||||
description="Aggregate timeout (seconds) for the exit settlement barrier.",
|
description="Aggregate timeout (seconds) for the exit settlement barrier.",
|
||||||
)
|
)
|
||||||
@@ -343,8 +320,7 @@ class ExplorerConfig(BaseModel):
|
|||||||
caching: bool | None = Field(
|
caching: bool | None = Field(
|
||||||
default=None,
|
default=None,
|
||||||
description=(
|
description=(
|
||||||
"Gemini explicit context caching for multi-turn Explorer tiers. ``None``"
|
"Legacy compatibility field; Responses prompt caching is managed by the endpoint."
|
||||||
" (default) uses the tier's own default: off for pro, on for ultra."
|
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
model_config = {"extra": "allow"}
|
model_config = {"extra": "allow"}
|
||||||
@@ -391,7 +367,7 @@ class StepSummarizerConfig(BaseModel):
|
|||||||
description="Whether to asynchronously summarize historical steps to replace pruned images.",
|
description="Whether to asynchronously summarize historical steps to replace pruned images.",
|
||||||
)
|
)
|
||||||
model: str = Field(
|
model: str = Field(
|
||||||
default="gemini-2.5-flash-lite",
|
default_factory=lambda: settings.OPENAI_MODEL,
|
||||||
description="Lightweight model used for background step state summarization.",
|
description="Lightweight model used for background step state summarization.",
|
||||||
)
|
)
|
||||||
prune_history_xml: bool = Field(
|
prune_history_xml: bool = Field(
|
||||||
@@ -688,7 +664,7 @@ class MemoryChunkingConfig(BaseModel):
|
|||||||
),
|
),
|
||||||
)
|
)
|
||||||
model: str = Field(
|
model: str = Field(
|
||||||
default="gemini-3.8-flash",
|
default_factory=lambda: settings.OPENAI_MODEL,
|
||||||
description="Model used for the chunk-level StepCapsuleLens (bands ①+②).",
|
description="Model used for the chunk-level StepCapsuleLens (bands ①+②).",
|
||||||
)
|
)
|
||||||
max_chunks: int = Field(
|
max_chunks: int = Field(
|
||||||
@@ -1089,7 +1065,7 @@ def load_agent_config(
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
with open(resolved_path, encoding="utf-8") as f:
|
with open(resolved_path, encoding="utf-8") as f:
|
||||||
data = load_jsonc(f)
|
data = interpolate_config_value(load_jsonc(f))
|
||||||
if isinstance(data, dict) and "agent" in data:
|
if isinstance(data, dict) and "agent" in data:
|
||||||
return AgentGlobalConfig.model_validate(data["agent"])
|
return AgentGlobalConfig.model_validate(data["agent"])
|
||||||
return AgentGlobalConfig.model_validate(data)
|
return AgentGlobalConfig.model_validate(data)
|
||||||
|
|||||||
@@ -53,17 +53,11 @@ ENV_ARTEMIS_HIERARCHY_BACKEND = "ARTEMIS_HIERARCHY_BACKEND"
|
|||||||
ENV_ARTEMIS_HELPER_AUTO_INSTALL = "ARTEMIS_HELPER_AUTO_INSTALL"
|
ENV_ARTEMIS_HELPER_AUTO_INSTALL = "ARTEMIS_HELPER_AUTO_INSTALL"
|
||||||
|
|
||||||
# LLM & Vision OCR API Keys
|
# LLM & Vision OCR API Keys
|
||||||
ENV_GOOGLE_API_KEY = "GOOGLE_API_KEY"
|
|
||||||
ENV_GEMINI_API_KEY = "GEMINI_API_KEY"
|
|
||||||
ENV_GCP_API_KEY = "GCP_API_KEY"
|
|
||||||
ENV_OCR_API_KEY = "OCR_API_KEY"
|
ENV_OCR_API_KEY = "OCR_API_KEY"
|
||||||
ENV_VISION_API_KEY = "VISION_API_KEY"
|
ENV_VISION_API_KEY = "VISION_API_KEY"
|
||||||
ENV_API_KEY = "API_KEY"
|
ENV_API_KEY = "API_KEY"
|
||||||
ENV_OPENAI_API_KEY = "OPENAI_API_KEY"
|
ENV_OPENAI_API_KEY = "OPENAI_API_KEY"
|
||||||
ENV_OPENAI_BASE_URL = "OPENAI_BASE_URL"
|
ENV_OPENAI_BASE_URL = "OPENAI_BASE_URL"
|
||||||
ENV_ANTHROPIC_API_KEY = "ANTHROPIC_API_KEY"
|
|
||||||
ENV_OPEN_ROUTER_API_KEY = "OPEN_ROUTER_API_KEY"
|
|
||||||
ENV_XAI_API_KEY = "XAI_API_KEY"
|
|
||||||
|
|
||||||
# Explorer & Agent Defaults
|
# Explorer & Agent Defaults
|
||||||
ENV_ARTEMIS_EXPLORER_VERSION = "ARTEMIS_EXPLORER_VERSION"
|
ENV_ARTEMIS_EXPLORER_VERSION = "ARTEMIS_EXPLORER_VERSION"
|
||||||
@@ -110,7 +104,7 @@ DEFAULT_ACTION_TIMEOUT = 30.0
|
|||||||
DEFAULT_STREAM_PING_INTERVAL = 15.0
|
DEFAULT_STREAM_PING_INTERVAL = 15.0
|
||||||
|
|
||||||
DEFAULT_PROFILE = "pro"
|
DEFAULT_PROFILE = "pro"
|
||||||
DEFAULT_MODEL = "gemini-2.5-flash"
|
DEFAULT_MODEL = "gpt-5-mini"
|
||||||
DEFAULT_EXPLORER_VERSION: Literal["flash", "pro", "ultra"] = "flash"
|
DEFAULT_EXPLORER_VERSION: Literal["flash", "pro", "ultra"] = "flash"
|
||||||
|
|
||||||
|
|
||||||
@@ -118,9 +112,7 @@ DEFAULT_EXPLORER_VERSION: Literal["flash", "pro", "ultra"] = "flash"
|
|||||||
# Literals and Type Aliases
|
# Literals and Type Aliases
|
||||||
# ==============================================================================
|
# ==============================================================================
|
||||||
|
|
||||||
LLMProvider = Literal[
|
LLMProvider = Literal["openai"]
|
||||||
"openai", "google", "openrouter", "xai", "vertexai", "anthropic", "ollama", "vllm", "custom"
|
|
||||||
]
|
|
||||||
ExplorerVersion = Literal["flash", "pro", "ultra"]
|
ExplorerVersion = Literal["flash", "pro", "ultra"]
|
||||||
|
|
||||||
LLMUtilsNode = Literal[
|
LLMUtilsNode = Literal[
|
||||||
|
|||||||
@@ -74,7 +74,7 @@ class CascadingConfigEngine:
|
|||||||
# 3. Workspace .env
|
# 3. Workspace .env
|
||||||
ws_env = self.workspace_root / ".env"
|
ws_env = self.workspace_root / ".env"
|
||||||
if ws_env.exists():
|
if ws_env.exists():
|
||||||
load_dotenv(dotenv_path=ws_env, override=True, encoding="utf-8")
|
load_dotenv(dotenv_path=ws_env, override=False, encoding="utf-8")
|
||||||
|
|
||||||
def load_raw_config(
|
def load_raw_config(
|
||||||
self,
|
self,
|
||||||
|
|||||||
+19
-46
@@ -15,12 +15,11 @@
|
|||||||
"""LLM provider, model hierarchy, fallback chaining, and configuration loaders."""
|
"""LLM provider, model hierarchy, fallback chaining, and configuration loaders."""
|
||||||
|
|
||||||
import os
|
import os
|
||||||
|
from artemis.config.core.interpolator import interpolate_config_value
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
from typing import Any, Literal
|
from typing import Any, Literal
|
||||||
|
|
||||||
import google.auth
|
from pydantic import Field, BaseModel, ValidationError
|
||||||
from google.auth.exceptions import DefaultCredentialsError
|
|
||||||
from pydantic import BaseModel, ValidationError
|
|
||||||
|
|
||||||
from artemis.config.constants import (
|
from artemis.config.constants import (
|
||||||
LLM_CONFIG_FILENAME,
|
LLM_CONFIG_FILENAME,
|
||||||
@@ -36,18 +35,6 @@ from artemis.utils.logger import get_logger
|
|||||||
logger = get_logger(__name__)
|
logger = get_logger(__name__)
|
||||||
|
|
||||||
|
|
||||||
def validate_vertex_ai_credentials() -> None:
|
|
||||||
"""Validate Google Application Default Credentials for VertexAI provider."""
|
|
||||||
try:
|
|
||||||
_, project = google.auth.default()
|
|
||||||
if not project:
|
|
||||||
raise Exception("VertexAI requires a Google Cloud project to be set.")
|
|
||||||
except DefaultCredentialsError as e:
|
|
||||||
raise Exception(
|
|
||||||
f"VertexAI requires valid Google Application Default Credentials (ADC): {e}"
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
class _CyFunctionDetectorMeta(type):
|
class _CyFunctionDetectorMeta(type):
|
||||||
def __instancecheck__(self, instance):
|
def __instancecheck__(self, instance):
|
||||||
name = type(instance).__name__
|
name = type(instance).__name__
|
||||||
@@ -72,13 +59,14 @@ class LLM(BaseModel):
|
|||||||
"""Base model representing an LLM model provider and runtime parameters."""
|
"""Base model representing an LLM model provider and runtime parameters."""
|
||||||
|
|
||||||
model_config = {"ignored_types": (CyFunctionDetector,)}
|
model_config = {"ignored_types": (CyFunctionDetector,)}
|
||||||
provider: LLMProvider
|
provider: LLMProvider = "openai"
|
||||||
model: str
|
model: str = Field(default_factory=lambda: settings.OPENAI_MODEL)
|
||||||
|
api_base: str | None = None
|
||||||
|
timeout: float = Field(default_factory=lambda: settings.OPENAI_TIMEOUT_SECONDS, gt=0)
|
||||||
|
max_retries: int = Field(default_factory=lambda: settings.OPENAI_MAX_RETRIES, ge=0)
|
||||||
|
max_tokens: int | None = Field(default=None, gt=0)
|
||||||
temperature: float | None = None
|
temperature: float | None = None
|
||||||
thinking_budget: int | None = None
|
|
||||||
thinking_level: Literal["minimal", "low", "medium", "high"] | None = None
|
|
||||||
reasoning_effort: Literal["none", "low", "medium", "high"] | None = None
|
reasoning_effort: Literal["none", "low", "medium", "high"] | None = None
|
||||||
include_thoughts: bool | None = None
|
|
||||||
enable_grounding: bool | None = None
|
enable_grounding: bool | None = None
|
||||||
|
|
||||||
def validate_provider(self, name: str) -> None:
|
def validate_provider(self, name: str) -> None:
|
||||||
@@ -86,20 +74,6 @@ class LLM(BaseModel):
|
|||||||
if self.provider == "openai":
|
if self.provider == "openai":
|
||||||
if not settings.OPENAI_API_KEY:
|
if not settings.OPENAI_API_KEY:
|
||||||
raise Exception(f"{name} requires OPENAI_API_KEY in .env")
|
raise Exception(f"{name} requires OPENAI_API_KEY in .env")
|
||||||
elif self.provider == "google":
|
|
||||||
if not settings.GOOGLE_API_KEY:
|
|
||||||
raise Exception(f"{name} requires GOOGLE_API_KEY in .env")
|
|
||||||
elif self.provider == "vertexai":
|
|
||||||
validate_vertex_ai_credentials()
|
|
||||||
elif self.provider == "anthropic":
|
|
||||||
if not (settings.ANTHROPIC_API_KEY or os.environ.get("ANTHROPIC_API_KEY")):
|
|
||||||
raise Exception(f"{name} requires ANTHROPIC_API_KEY in .env")
|
|
||||||
elif self.provider == "openrouter":
|
|
||||||
if not settings.OPEN_ROUTER_API_KEY:
|
|
||||||
raise Exception(f"{name} requires OPEN_ROUTER_API_KEY in .env")
|
|
||||||
elif self.provider == "xai":
|
|
||||||
if not settings.XAI_API_KEY:
|
|
||||||
raise Exception(f"{name} requires XAI_API_KEY in .env")
|
|
||||||
|
|
||||||
def __str__(self) -> str:
|
def __str__(self) -> str:
|
||||||
return f"{self.provider}/{self.model}"
|
return f"{self.provider}/{self.model}"
|
||||||
@@ -110,7 +84,6 @@ class LLMWithFallback(LLM):
|
|||||||
|
|
||||||
fallback: LLM
|
fallback: LLM
|
||||||
fix_model: str | None = None
|
fix_model: str | None = None
|
||||||
timeout: float | None = None
|
|
||||||
|
|
||||||
def __str__(self) -> str:
|
def __str__(self) -> str:
|
||||||
return f"{self.provider}/{self.model} (fallback: {self.fallback})"
|
return f"{self.provider}/{self.model} (fallback: {self.fallback})"
|
||||||
@@ -118,14 +91,14 @@ class LLMWithFallback(LLM):
|
|||||||
|
|
||||||
def lightweight_judge_default() -> "LLMWithFallback":
|
def lightweight_judge_default() -> "LLMWithFallback":
|
||||||
"""Factory default for the lightweight judge nodes (pixel safety net and
|
"""Factory default for the lightweight judge nodes (pixel safety net and
|
||||||
planner validation): a flash-lite model at temperature 0."""
|
planner validation): the configured Responses model at temperature 0."""
|
||||||
return LLMWithFallback(
|
return LLMWithFallback(
|
||||||
provider="google",
|
provider="openai",
|
||||||
model="gemini-3.5-flash-lite",
|
model=settings.OPENAI_MODEL,
|
||||||
temperature=0.0,
|
temperature=0.0,
|
||||||
fallback=LLM(
|
fallback=LLM(
|
||||||
provider="google",
|
provider="openai",
|
||||||
model="gemini-3.1-flash-lite",
|
model=settings.OPENAI_MODEL,
|
||||||
temperature=0.0,
|
temperature=0.0,
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
@@ -236,11 +209,11 @@ def _expand_default_into_nodes(config_dict: dict) -> dict:
|
|||||||
default_model_cfg = config_dict.get(
|
default_model_cfg = config_dict.get(
|
||||||
"default",
|
"default",
|
||||||
{
|
{
|
||||||
"provider": "google",
|
"provider": "openai",
|
||||||
"model": "gemini-3.8-flash",
|
"model": settings.OPENAI_MODEL,
|
||||||
"fallback": {
|
"fallback": {
|
||||||
"provider": "google",
|
"provider": "openai",
|
||||||
"model": "gemini-3.7-flash",
|
"model": settings.OPENAI_MODEL,
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
@@ -310,7 +283,7 @@ def parse_llm_config() -> LLMConfig:
|
|||||||
|
|
||||||
try:
|
try:
|
||||||
with open(config_path, encoding="utf-8") as f:
|
with open(config_path, encoding="utf-8") as f:
|
||||||
config_dict = load_jsonc(f)
|
config_dict = interpolate_config_value(load_jsonc(f))
|
||||||
expanded_dict = _expand_default_into_nodes(config_dict)
|
expanded_dict = _expand_default_into_nodes(config_dict)
|
||||||
return LLMConfig.model_validate(expanded_dict)
|
return LLMConfig.model_validate(expanded_dict)
|
||||||
except Exception as e:
|
except Exception as e:
|
||||||
@@ -361,7 +334,7 @@ def load_llm_config_override(path: Path | str) -> LLMConfig:
|
|||||||
if resolved_path.exists():
|
if resolved_path.exists():
|
||||||
logger.info(f"Loading custom LLM config from {resolved_path.resolve()}...")
|
logger.info(f"Loading custom LLM config from {resolved_path.resolve()}...")
|
||||||
with open(resolved_path, encoding="utf-8") as f:
|
with open(resolved_path, encoding="utf-8") as f:
|
||||||
override_config_dict = load_jsonc(f)
|
override_config_dict = interpolate_config_value(load_jsonc(f))
|
||||||
else:
|
else:
|
||||||
logger.warning(f"Custom LLM config not found at {path} - using default config")
|
logger.warning(f"Custom LLM config not found at {path} - using default config")
|
||||||
|
|
||||||
|
|||||||
+21
-71
@@ -26,16 +26,10 @@ from artemis.config.constants import (
|
|||||||
DEFAULT_ADB_PORT,
|
DEFAULT_ADB_PORT,
|
||||||
DEFAULT_MODEL,
|
DEFAULT_MODEL,
|
||||||
DEFAULT_PROFILE,
|
DEFAULT_PROFILE,
|
||||||
ENV_ANTHROPIC_API_KEY,
|
|
||||||
ENV_DATA_ENGINE_DB_PATH,
|
ENV_DATA_ENGINE_DB_PATH,
|
||||||
ENV_GCP_API_KEY,
|
|
||||||
ENV_GEMINI_API_KEY,
|
|
||||||
ENV_GOOGLE_API_KEY,
|
|
||||||
ENV_OCR_API_KEY,
|
ENV_OCR_API_KEY,
|
||||||
ENV_OPEN_ROUTER_API_KEY,
|
|
||||||
ENV_OPENAI_API_KEY,
|
ENV_OPENAI_API_KEY,
|
||||||
ENV_VISION_API_KEY,
|
ENV_VISION_API_KEY,
|
||||||
ENV_XAI_API_KEY,
|
|
||||||
)
|
)
|
||||||
from artemis.config.paths import (
|
from artemis.config.paths import (
|
||||||
GLOBAL_APP_DIR,
|
GLOBAL_APP_DIR,
|
||||||
@@ -94,12 +88,6 @@ class Settings(BaseSettings):
|
|||||||
|
|
||||||
# LLM Provider Authentication
|
# LLM Provider Authentication
|
||||||
OPENAI_API_KEY: SecretStr | None = None
|
OPENAI_API_KEY: SecretStr | None = None
|
||||||
GOOGLE_API_KEY: SecretStr | None = None
|
|
||||||
GEMINI_API_KEY: SecretStr | None = None
|
|
||||||
GCP_API_KEY: SecretStr | None = None
|
|
||||||
ANTHROPIC_API_KEY: SecretStr | None = None
|
|
||||||
XAI_API_KEY: SecretStr | None = None
|
|
||||||
OPEN_ROUTER_API_KEY: SecretStr | None = None
|
|
||||||
|
|
||||||
# Google Cloud Vision OCR Authentication
|
# Google Cloud Vision OCR Authentication
|
||||||
OCR_API_KEY: SecretStr | None = None
|
OCR_API_KEY: SecretStr | None = None
|
||||||
@@ -108,6 +96,19 @@ class Settings(BaseSettings):
|
|||||||
|
|
||||||
# Custom Provider Endpoints
|
# Custom Provider Endpoints
|
||||||
OPENAI_BASE_URL: str | None = None
|
OPENAI_BASE_URL: str | None = None
|
||||||
|
OPENAI_MODEL: str = Field(default="gpt-5-mini", min_length=1)
|
||||||
|
OPENAI_TIMEOUT_SECONDS: float = Field(default=120.0, gt=0)
|
||||||
|
OPENAI_MAX_RETRIES: int = Field(default=0, ge=0)
|
||||||
|
ARTEMIS_CREDENTIAL_TIMEOUT_SECONDS: float = Field(default=12.0, gt=0)
|
||||||
|
ARTEMIS_MODEL_CACHE_TTL_SECONDS: float = Field(default=10.0, ge=0)
|
||||||
|
ARTEMIS_VIDEO_TIMEOUT_SECONDS: float = Field(default=120.0, gt=0)
|
||||||
|
ARTEMIS_VIDEO_MAX_RETRIES: int = Field(default=1, ge=0)
|
||||||
|
ARTEMIS_VIDEO_CHUNK_SECONDS: float = Field(default=60.0, gt=0)
|
||||||
|
ARTEMIS_CHECKPOINT_TIMEOUT_SECONDS: float = Field(default=180.0, gt=0)
|
||||||
|
ARTEMIS_SETTLEMENT_TIMEOUT_SECONDS: float = Field(default=120.0, gt=0)
|
||||||
|
ARTEMIS_VIDEO_ENABLED: bool = True
|
||||||
|
ARTEMIS_OUTPUTTER_ENABLED: bool = True
|
||||||
|
ARTEMIS_STEP_SUMMARIZER_ENABLED: bool = True
|
||||||
|
|
||||||
# Android ADB Connectivity
|
# Android ADB Connectivity
|
||||||
ADB_HOST: str | None = Field(default=DEFAULT_ADB_HOST)
|
ADB_HOST: str | None = Field(default=DEFAULT_ADB_HOST)
|
||||||
@@ -125,16 +126,6 @@ class Settings(BaseSettings):
|
|||||||
ARTEMIS_DEFAULT_PROFILE: str = Field(default=DEFAULT_PROFILE)
|
ARTEMIS_DEFAULT_PROFILE: str = Field(default=DEFAULT_PROFILE)
|
||||||
ARTEMIS_DEFAULT_MODEL: str = Field(default=DEFAULT_MODEL)
|
ARTEMIS_DEFAULT_MODEL: str = Field(default=DEFAULT_MODEL)
|
||||||
|
|
||||||
# Explorer Tool Settings (the tier itself is configured in artemis.jsonc or
|
|
||||||
# via ARTEMIS_EXPLORER_VERSION; see artemis.config.agent.ExplorerConfig)
|
|
||||||
EXPLORER_CACHING: bool | None = Field(
|
|
||||||
default=None,
|
|
||||||
description=(
|
|
||||||
"Environment-level override for Explorer context caching; unset"
|
|
||||||
" defers to the agent configuration and the tier default."
|
|
||||||
),
|
|
||||||
)
|
|
||||||
|
|
||||||
# LLM Reliability
|
# LLM Reliability
|
||||||
LLM_PAUSE_TIMEOUT_SECONDS: float = Field(
|
LLM_PAUSE_TIMEOUT_SECONDS: float = Field(
|
||||||
default=900.0,
|
default=900.0,
|
||||||
@@ -154,16 +145,10 @@ class Settings(BaseSettings):
|
|||||||
|
|
||||||
@model_validator(mode="after")
|
@model_validator(mode="after")
|
||||||
def fallback_api_keys(self) -> "Settings":
|
def fallback_api_keys(self) -> "Settings":
|
||||||
"""Normalize Google / Gemini / GCP and OCR API keys and filter placeholders."""
|
"""Normalize OpenAI and OCR API keys and filter placeholders."""
|
||||||
# Sanitize any placeholder values loaded from environment or .env
|
# Sanitize any placeholder values loaded from environment or .env
|
||||||
for attr in (
|
for attr in (
|
||||||
"OPENAI_API_KEY",
|
"OPENAI_API_KEY",
|
||||||
"GOOGLE_API_KEY",
|
|
||||||
"GEMINI_API_KEY",
|
|
||||||
"GCP_API_KEY",
|
|
||||||
"ANTHROPIC_API_KEY",
|
|
||||||
"XAI_API_KEY",
|
|
||||||
"OPEN_ROUTER_API_KEY",
|
|
||||||
"OCR_API_KEY",
|
"OCR_API_KEY",
|
||||||
"VISION_API_KEY",
|
"VISION_API_KEY",
|
||||||
"API_KEY",
|
"API_KEY",
|
||||||
@@ -172,12 +157,6 @@ class Settings(BaseSettings):
|
|||||||
if val and is_placeholder_key(val):
|
if val and is_placeholder_key(val):
|
||||||
setattr(self, attr, None)
|
setattr(self, attr, None)
|
||||||
|
|
||||||
if not self.GOOGLE_API_KEY:
|
|
||||||
if self.GEMINI_API_KEY:
|
|
||||||
self.GOOGLE_API_KEY = self.GEMINI_API_KEY
|
|
||||||
elif self.GCP_API_KEY:
|
|
||||||
self.GOOGLE_API_KEY = self.GCP_API_KEY
|
|
||||||
|
|
||||||
# Fallback for OCR & Vision API keys
|
# Fallback for OCR & Vision API keys
|
||||||
if not self.OCR_API_KEY and self.VISION_API_KEY:
|
if not self.OCR_API_KEY and self.VISION_API_KEY:
|
||||||
self.OCR_API_KEY = self.VISION_API_KEY
|
self.OCR_API_KEY = self.VISION_API_KEY
|
||||||
@@ -187,27 +166,18 @@ class Settings(BaseSettings):
|
|||||||
"""Get API key for a specified provider.
|
"""Get API key for a specified provider.
|
||||||
|
|
||||||
Args:
|
Args:
|
||||||
provider: Provider name (e.g., 'google', 'gemini', 'openai', 'ocr', 'vision').
|
provider: Provider name ('openai', 'ocr', or 'vision').
|
||||||
|
|
||||||
Returns:
|
Returns:
|
||||||
SecretStr containing the API key or None if not configured.
|
SecretStr containing the API key or None if not configured.
|
||||||
"""
|
"""
|
||||||
from artemis.llm.google.provider import is_google_family_provider
|
|
||||||
|
|
||||||
provider_lower = provider.lower()
|
provider_lower = provider.lower()
|
||||||
key: SecretStr | None = None
|
key: SecretStr | None = None
|
||||||
if is_google_family_provider(provider_lower):
|
if provider_lower in ("ocr", "vision", "google_vision"):
|
||||||
key = self.GOOGLE_API_KEY or self.GEMINI_API_KEY or self.GCP_API_KEY
|
|
||||||
elif provider_lower in ("ocr", "vision", "google_vision"):
|
|
||||||
key = self.OCR_API_KEY or self.VISION_API_KEY
|
key = self.OCR_API_KEY or self.VISION_API_KEY
|
||||||
elif provider_lower == "openai":
|
elif provider_lower == "openai":
|
||||||
key = self.OPENAI_API_KEY
|
key = self.OPENAI_API_KEY
|
||||||
elif provider_lower in ("anthropic", "claude"):
|
|
||||||
key = self.ANTHROPIC_API_KEY
|
|
||||||
elif provider_lower == "openrouter":
|
|
||||||
key = self.OPEN_ROUTER_API_KEY
|
|
||||||
elif provider_lower in ("xai", "grok"):
|
|
||||||
key = self.XAI_API_KEY
|
|
||||||
|
|
||||||
if key and not is_placeholder_key(key):
|
if key and not is_placeholder_key(key):
|
||||||
return key
|
return key
|
||||||
@@ -222,35 +192,14 @@ class Settings(BaseSettings):
|
|||||||
persist_to_env: Whether to save the key to the app directory's .env file.
|
persist_to_env: Whether to save the key to the app directory's .env file.
|
||||||
"""
|
"""
|
||||||
secret = SecretStr(key)
|
secret = SecretStr(key)
|
||||||
from artemis.llm.google.provider import is_google_family_provider
|
|
||||||
|
|
||||||
provider_lower = provider.lower()
|
provider_lower = provider.lower()
|
||||||
env_key_name = None
|
env_key_name = None
|
||||||
|
|
||||||
if is_google_family_provider(provider_lower):
|
if provider_lower == "openai":
|
||||||
self.GOOGLE_API_KEY = secret
|
|
||||||
self.GEMINI_API_KEY = secret
|
|
||||||
self.GCP_API_KEY = secret
|
|
||||||
env_key_name = ENV_GEMINI_API_KEY
|
|
||||||
os.environ[ENV_GOOGLE_API_KEY] = key
|
|
||||||
os.environ[ENV_GEMINI_API_KEY] = key
|
|
||||||
os.environ[ENV_GCP_API_KEY] = key
|
|
||||||
elif provider_lower == "openai":
|
|
||||||
self.OPENAI_API_KEY = secret
|
self.OPENAI_API_KEY = secret
|
||||||
env_key_name = ENV_OPENAI_API_KEY
|
env_key_name = ENV_OPENAI_API_KEY
|
||||||
os.environ[ENV_OPENAI_API_KEY] = key
|
os.environ[ENV_OPENAI_API_KEY] = key
|
||||||
elif provider_lower in ("anthropic", "claude"):
|
|
||||||
self.ANTHROPIC_API_KEY = secret
|
|
||||||
env_key_name = ENV_ANTHROPIC_API_KEY
|
|
||||||
os.environ[ENV_ANTHROPIC_API_KEY] = key
|
|
||||||
elif provider_lower == "openrouter":
|
|
||||||
self.OPEN_ROUTER_API_KEY = secret
|
|
||||||
env_key_name = ENV_OPEN_ROUTER_API_KEY
|
|
||||||
os.environ[ENV_OPEN_ROUTER_API_KEY] = key
|
|
||||||
elif provider_lower == "xai":
|
|
||||||
self.XAI_API_KEY = secret
|
|
||||||
env_key_name = ENV_XAI_API_KEY
|
|
||||||
os.environ[ENV_XAI_API_KEY] = key
|
|
||||||
elif provider_lower in ("ocr", "vision", "google_vision"):
|
elif provider_lower in ("ocr", "vision", "google_vision"):
|
||||||
self.OCR_API_KEY = secret
|
self.OCR_API_KEY = secret
|
||||||
self.VISION_API_KEY = secret
|
self.VISION_API_KEY = secret
|
||||||
@@ -258,6 +207,9 @@ class Settings(BaseSettings):
|
|||||||
os.environ[ENV_OCR_API_KEY] = key
|
os.environ[ENV_OCR_API_KEY] = key
|
||||||
os.environ[ENV_VISION_API_KEY] = key
|
os.environ[ENV_VISION_API_KEY] = key
|
||||||
|
|
||||||
|
else:
|
||||||
|
raise ValueError(f"Unsupported credential provider: {provider}")
|
||||||
|
|
||||||
if persist_to_env and env_key_name:
|
if persist_to_env and env_key_name:
|
||||||
target_env_files = [get_env_file()]
|
target_env_files = [get_env_file()]
|
||||||
seen_paths = set()
|
seen_paths = set()
|
||||||
@@ -275,9 +227,7 @@ class Settings(BaseSettings):
|
|||||||
|
|
||||||
# Determine keys to update
|
# Determine keys to update
|
||||||
keys_to_update = [env_key_name]
|
keys_to_update = [env_key_name]
|
||||||
if is_google_family_provider(provider_lower):
|
if provider_lower in ("ocr", "vision", "google_vision"):
|
||||||
keys_to_update = [ENV_GEMINI_API_KEY, ENV_GOOGLE_API_KEY, ENV_GCP_API_KEY]
|
|
||||||
elif provider_lower in ("ocr", "vision", "google_vision"):
|
|
||||||
keys_to_update = [ENV_OCR_API_KEY, ENV_VISION_API_KEY]
|
keys_to_update = [ENV_OCR_API_KEY, ENV_VISION_API_KEY]
|
||||||
|
|
||||||
new_lines = []
|
new_lines = []
|
||||||
|
|||||||
@@ -20,28 +20,3 @@ VALIDATOR_POLL_TIMEOUT = 2.0
|
|||||||
VALIDATOR_POLL_INTERVAL = 0.2
|
VALIDATOR_POLL_INTERVAL = 0.2
|
||||||
VALIDATOR_OCR_INTERVAL = 1.0
|
VALIDATOR_OCR_INTERVAL = 1.0
|
||||||
VALIDATOR_UI_HIERARCHY_TIMEOUT = 1.0
|
VALIDATOR_UI_HIERARCHY_TIMEOUT = 1.0
|
||||||
|
|
||||||
from google.genai import types as _genai_types
|
|
||||||
|
|
||||||
SAFETY_SETTINGS_BLOCK_NONE = [
|
|
||||||
_genai_types.SafetySetting(
|
|
||||||
category=_genai_types.HarmCategory.HARM_CATEGORY_HATE_SPEECH,
|
|
||||||
threshold=_genai_types.HarmBlockThreshold.BLOCK_NONE,
|
|
||||||
),
|
|
||||||
_genai_types.SafetySetting(
|
|
||||||
category=_genai_types.HarmCategory.HARM_CATEGORY_HARASSMENT,
|
|
||||||
threshold=_genai_types.HarmBlockThreshold.BLOCK_NONE,
|
|
||||||
),
|
|
||||||
_genai_types.SafetySetting(
|
|
||||||
category=_genai_types.HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT,
|
|
||||||
threshold=_genai_types.HarmBlockThreshold.BLOCK_NONE,
|
|
||||||
),
|
|
||||||
_genai_types.SafetySetting(
|
|
||||||
category=_genai_types.HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT,
|
|
||||||
threshold=_genai_types.HarmBlockThreshold.BLOCK_NONE,
|
|
||||||
),
|
|
||||||
_genai_types.SafetySetting(
|
|
||||||
category=_genai_types.HarmCategory.HARM_CATEGORY_CIVIC_INTEGRITY,
|
|
||||||
threshold=_genai_types.HarmBlockThreshold.BLOCK_NONE,
|
|
||||||
),
|
|
||||||
]
|
|
||||||
@@ -26,11 +26,11 @@ from artemis.core.diagnostics.schema import (
|
|||||||
|
|
||||||
|
|
||||||
class LLMCredentialsProbe(BaseProbe):
|
class LLMCredentialsProbe(BaseProbe):
|
||||||
"""Probe verifying Gemini and multi-provider multimodal LLM credentials."""
|
"""Readiness depends on an OpenAI credential, never merely a base URL."""
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def probe_id(self) -> str:
|
def probe_id(self) -> str:
|
||||||
return "gemini_api_key"
|
return "openai_api_key"
|
||||||
|
|
||||||
@property
|
@property
|
||||||
def category(self) -> ProbeCategory:
|
def category(self) -> ProbeCategory:
|
||||||
@@ -40,192 +40,38 @@ class LLMCredentialsProbe(BaseProbe):
|
|||||||
def is_blocker(self) -> bool:
|
def is_blocker(self) -> bool:
|
||||||
return True
|
return True
|
||||||
|
|
||||||
def _mask_key(self, key_str: str) -> str:
|
|
||||||
"""Helper to safely mask an API credential for display."""
|
|
||||||
if len(key_str) > 10:
|
|
||||||
return f"{key_str[:6]}...{key_str[-4:]}"
|
|
||||||
return "***"
|
|
||||||
|
|
||||||
async def probe(self) -> ProbeResult:
|
async def probe(self) -> ProbeResult:
|
||||||
import os
|
key = settings.get_api_key("openai")
|
||||||
|
configured = bool(key)
|
||||||
from artemis.config.settings import is_placeholder_key
|
providers = (
|
||||||
|
[
|
||||||
gemini_key = settings.get_api_key("google")
|
|
||||||
openai_key = settings.get_api_key("openai")
|
|
||||||
claude_key = settings.get_api_key("anthropic")
|
|
||||||
openrouter_key = settings.get_api_key("openrouter")
|
|
||||||
xai_key = settings.get_api_key("xai")
|
|
||||||
ocr_key = settings.get_api_key("ocr")
|
|
||||||
|
|
||||||
configured_providers: list[dict[str, Any]] = []
|
|
||||||
api_keys_map: dict[str, str] = {}
|
|
||||||
if gemini_key and not is_placeholder_key(gemini_key):
|
|
||||||
g_val = gemini_key.get_secret_value()
|
|
||||||
configured_providers.append(
|
|
||||||
{
|
|
||||||
"provider": "google",
|
|
||||||
"label": "Gemini",
|
|
||||||
"masked": self._mask_key(g_val),
|
|
||||||
"raw_key": g_val,
|
|
||||||
"key": g_val,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
api_keys_map["google"] = g_val
|
|
||||||
api_keys_map["gemini"] = g_val
|
|
||||||
if openai_key and not is_placeholder_key(openai_key):
|
|
||||||
o_val = openai_key.get_secret_value()
|
|
||||||
configured_providers.append(
|
|
||||||
{
|
{
|
||||||
"provider": "openai",
|
"provider": "openai",
|
||||||
"label": "ChatGPT",
|
"label": "OpenAI Responses",
|
||||||
"masked": self._mask_key(o_val),
|
"masked": "****" + key.get_secret_value()[-4:],
|
||||||
"raw_key": o_val,
|
|
||||||
"key": o_val,
|
|
||||||
}
|
}
|
||||||
)
|
]
|
||||||
api_keys_map["openai"] = o_val
|
if key
|
||||||
if claude_key and not is_placeholder_key(claude_key):
|
else []
|
||||||
c_val = claude_key.get_secret_value()
|
|
||||||
configured_providers.append(
|
|
||||||
{
|
|
||||||
"provider": "anthropic",
|
|
||||||
"label": "Claude",
|
|
||||||
"masked": self._mask_key(c_val),
|
|
||||||
"raw_key": c_val,
|
|
||||||
"key": c_val,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
api_keys_map["anthropic"] = c_val
|
|
||||||
if openrouter_key and not is_placeholder_key(openrouter_key):
|
|
||||||
or_val = openrouter_key.get_secret_value()
|
|
||||||
configured_providers.append(
|
|
||||||
{
|
|
||||||
"provider": "openrouter",
|
|
||||||
"label": "OpenRouter",
|
|
||||||
"masked": self._mask_key(or_val),
|
|
||||||
"raw_key": or_val,
|
|
||||||
"key": or_val,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
api_keys_map["openrouter"] = or_val
|
|
||||||
if xai_key and not is_placeholder_key(xai_key):
|
|
||||||
x_val = xai_key.get_secret_value()
|
|
||||||
configured_providers.append(
|
|
||||||
{
|
|
||||||
"provider": "xai",
|
|
||||||
"label": "xAI (Grok)",
|
|
||||||
"masked": self._mask_key(x_val),
|
|
||||||
"raw_key": x_val,
|
|
||||||
"key": x_val,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
api_keys_map["xai"] = x_val
|
|
||||||
|
|
||||||
# Detect any custom model endpoints or environment variables defined in files
|
|
||||||
for env_var, label, prov_id in [
|
|
||||||
("DEEPSEEK_API_KEY", "DeepSeek", "deepseek"),
|
|
||||||
("GROQ_API_KEY", "Groq", "groq"),
|
|
||||||
("OPENAI_BASE_URL", "Custom OpenAI Endpoint", "custom"),
|
|
||||||
("OLLAMA_BASE_URL", "Local Ollama", "ollama"),
|
|
||||||
("VLLM_BASE_URL", "vLLM Endpoint", "vllm"),
|
|
||||||
("VERTEX_AI_PROJECT", "Google Cloud Vertex AI", "vertexai"),
|
|
||||||
]:
|
|
||||||
val = os.environ.get(env_var)
|
|
||||||
if val and val.strip() and not is_placeholder_key(val.strip()):
|
|
||||||
configured_providers.append(
|
|
||||||
{
|
|
||||||
"provider": prov_id,
|
|
||||||
"label": label,
|
|
||||||
"masked": self._mask_key(val.strip()),
|
|
||||||
"raw_key": val.strip(),
|
|
||||||
"key": val.strip(),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
api_keys_map[prov_id] = val.strip()
|
|
||||||
|
|
||||||
if ocr_key and not is_placeholder_key(ocr_key):
|
|
||||||
api_keys_map["ocr"] = ocr_key.get_secret_value()
|
|
||||||
|
|
||||||
current_active_key = (
|
|
||||||
gemini_key.get_secret_value()
|
|
||||||
if gemini_key
|
|
||||||
else (configured_providers[0]["key"] if configured_providers else "")
|
|
||||||
)
|
)
|
||||||
|
|
||||||
metadata = {
|
|
||||||
"configured_count": len(configured_providers),
|
|
||||||
"providers": configured_providers,
|
|
||||||
"has_ocr_key": ocr_key is not None,
|
|
||||||
"current_key": current_active_key,
|
|
||||||
"current_gemini_key": gemini_key.get_secret_value() if gemini_key else "",
|
|
||||||
"api_keys": api_keys_map,
|
|
||||||
}
|
|
||||||
|
|
||||||
# Case 1: Gemini API Key configured (Standard / Recommended)
|
|
||||||
if gemini_key:
|
|
||||||
masked = self._mask_key(gemini_key.get_secret_value())
|
|
||||||
return ProbeResult(
|
|
||||||
id=self.probe_id,
|
|
||||||
category=self.category,
|
|
||||||
title="Multimodal LLM API Key",
|
|
||||||
status=ProbeStatus.PASS,
|
|
||||||
is_blocker=self.is_blocker,
|
|
||||||
summary="Active (Gemini)",
|
|
||||||
description=f"Gemini multimodal API credential is active ({masked}) and ready for vision and reasoning.",
|
|
||||||
metadata=metadata,
|
|
||||||
actions=[
|
|
||||||
ProbeAction(
|
|
||||||
action_type="hint",
|
|
||||||
label="Provider Active",
|
|
||||||
payload="Gemini 2.5 Flash / Pro Multimodal Engine is enabled.",
|
|
||||||
)
|
|
||||||
],
|
|
||||||
)
|
|
||||||
|
|
||||||
# Case 2: Other LLM provider configured
|
|
||||||
if configured_providers:
|
|
||||||
first_p = configured_providers[0]
|
|
||||||
return ProbeResult(
|
|
||||||
id=self.probe_id,
|
|
||||||
category=self.category,
|
|
||||||
title="Multimodal LLM API Key",
|
|
||||||
status=ProbeStatus.PASS,
|
|
||||||
is_blocker=self.is_blocker,
|
|
||||||
summary=f"Active ({first_p['label']})",
|
|
||||||
description=f"{first_p['label']} multimodal API credential is active ({first_p['masked']}).",
|
|
||||||
metadata=metadata,
|
|
||||||
actions=[
|
|
||||||
ProbeAction(
|
|
||||||
action_type="hint",
|
|
||||||
label="Provider Active",
|
|
||||||
payload=f"{first_p['label']} multimodal vision endpoint is active.",
|
|
||||||
)
|
|
||||||
],
|
|
||||||
)
|
|
||||||
|
|
||||||
# Case 3: No LLM key configured
|
|
||||||
return ProbeResult(
|
return ProbeResult(
|
||||||
id=self.probe_id,
|
id=self.probe_id,
|
||||||
category=self.category,
|
category=self.category,
|
||||||
title="Multimodal LLM API Key",
|
title="OpenAI API 密钥",
|
||||||
status=ProbeStatus.FAIL,
|
status=ProbeStatus.PASS if configured else ProbeStatus.FAIL,
|
||||||
is_blocker=self.is_blocker,
|
is_blocker=True,
|
||||||
summary="Key Missing",
|
summary="已配置 OpenAI Responses" if configured else "缺少 API 密钥",
|
||||||
description="No Multimodal LLM credential (e.g. GEMINI_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY) found in environment or .env file.",
|
description="模型统一通过 OpenAI Responses 接口调用。"
|
||||||
metadata=metadata,
|
if configured
|
||||||
actions=[
|
else "请在环境变量或 .env 中设置 OPENAI_API_KEY。",
|
||||||
ProbeAction(
|
metadata={
|
||||||
action_type="command",
|
"configured_count": len(providers),
|
||||||
label="Run Artemis Init",
|
"providers": providers,
|
||||||
payload="artemis init",
|
"has_ocr_key": bool(settings.get_api_key("ocr")),
|
||||||
),
|
},
|
||||||
ProbeAction(
|
actions=[]
|
||||||
action_type="link",
|
if configured
|
||||||
label="Get Google AI Studio Key",
|
else [ProbeAction(action_type="command", label="初始化配置", payload="artemis init")],
|
||||||
payload="https://aistudio.google.com/app/apikey",
|
|
||||||
),
|
|
||||||
],
|
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
@@ -267,7 +113,7 @@ class VisionOCRProbe(BaseProbe):
|
|||||||
is_blocker=False,
|
is_blocker=False,
|
||||||
summary="Active & Configured",
|
summary="Active & Configured",
|
||||||
description=f"Google Cloud Vision OCR ({masked}) is active for image text recognition.",
|
description=f"Google Cloud Vision OCR ({masked}) is active for image text recognition.",
|
||||||
metadata={"configured": True, "masked_key": masked, "key": val, "raw_key": val},
|
metadata={"configured": True, "masked_key": masked},
|
||||||
actions=[
|
actions=[
|
||||||
ProbeAction(
|
ProbeAction(
|
||||||
action_type="hint",
|
action_type="hint",
|
||||||
|
|||||||
@@ -17,7 +17,7 @@
|
|||||||
import platform
|
import platform
|
||||||
import sys
|
import sys
|
||||||
|
|
||||||
from artemis.config import parse_llm_config
|
from artemis.config import parse_llm_config, settings
|
||||||
from artemis.core.diagnostics.probes.base import BaseProbe
|
from artemis.core.diagnostics.probes.base import BaseProbe
|
||||||
from artemis.core.diagnostics.schema import (
|
from artemis.core.diagnostics.schema import (
|
||||||
ProbeAction,
|
ProbeAction,
|
||||||
@@ -67,7 +67,7 @@ class PythonRuntimeProbe(BaseProbe):
|
|||||||
title="Python Runtime Environment",
|
title="Python Runtime Environment",
|
||||||
status=ProbeStatus.PASS,
|
status=ProbeStatus.PASS,
|
||||||
is_blocker=self.is_blocker,
|
is_blocker=self.is_blocker,
|
||||||
summary=f"Python {py_ver_str} Ready",
|
summary=f"Python {py_ver_str} 已就绪",
|
||||||
description=f"Python {py_ver_str} ({arch}, 64-bit) detected at {executable}. Satisfies requirement (>= 3.12).",
|
description=f"Python {py_ver_str} ({arch}, 64-bit) detected at {executable}. Satisfies requirement (>= 3.12).",
|
||||||
metadata=metadata,
|
metadata=metadata,
|
||||||
actions=[
|
actions=[
|
||||||
@@ -121,7 +121,7 @@ class SystemConfigProbe(BaseProbe):
|
|||||||
async def probe(self) -> ProbeResult:
|
async def probe(self) -> ProbeResult:
|
||||||
try:
|
try:
|
||||||
cfg = parse_llm_config()
|
cfg = parse_llm_config()
|
||||||
planner_model = getattr(getattr(cfg, "planner", None), "model", "Gemini 2.5 Flash")
|
planner_model = getattr(getattr(cfg, "planner", None), "model", settings.OPENAI_MODEL)
|
||||||
metadata = {
|
metadata = {
|
||||||
"valid": True,
|
"valid": True,
|
||||||
"config_file": "config/artemis.jsonc",
|
"config_file": "config/artemis.jsonc",
|
||||||
@@ -133,7 +133,7 @@ class SystemConfigProbe(BaseProbe):
|
|||||||
title="Artemis System Configuration",
|
title="Artemis System Configuration",
|
||||||
status=ProbeStatus.PASS,
|
status=ProbeStatus.PASS,
|
||||||
is_blocker=self.is_blocker,
|
is_blocker=self.is_blocker,
|
||||||
summary="Config Valid",
|
summary="配置有效",
|
||||||
description=f"System configuration loaded and validated (Planner model: {planner_model}).",
|
description=f"System configuration loaded and validated (Planner model: {planner_model}).",
|
||||||
metadata=metadata,
|
metadata=metadata,
|
||||||
actions=[
|
actions=[
|
||||||
|
|||||||
@@ -72,7 +72,7 @@ class ToolchainProbe(BaseProbe):
|
|||||||
title="Video Recording Toolchain",
|
title="Video Recording Toolchain",
|
||||||
status=ProbeStatus.PASS,
|
status=ProbeStatus.PASS,
|
||||||
is_blocker=self.is_blocker,
|
is_blocker=self.is_blocker,
|
||||||
summary="Ready (FFmpeg + scrcpy)",
|
summary="录屏工具已就绪(FFmpeg + scrcpy)",
|
||||||
description=f"FFmpeg ({ffmpeg_path}) and scrcpy ({scrcpy_path}) are available. Live stream and high-speed video replays are active.",
|
description=f"FFmpeg ({ffmpeg_path}) and scrcpy ({scrcpy_path}) are available. Live stream and high-speed video replays are active.",
|
||||||
metadata=metadata,
|
metadata=metadata,
|
||||||
actions=[
|
actions=[
|
||||||
@@ -132,7 +132,7 @@ class ToolchainProbe(BaseProbe):
|
|||||||
title="Video Recording Toolchain",
|
title="Video Recording Toolchain",
|
||||||
status=ProbeStatus.FAIL,
|
status=ProbeStatus.FAIL,
|
||||||
is_blocker=self.is_blocker,
|
is_blocker=self.is_blocker,
|
||||||
summary=f"Missing {missing_str}",
|
summary=f"缺少 {missing_str}",
|
||||||
description=f"Video toolchain is required for screen streaming and test replay recording. Missing: {missing_str}.",
|
description=f"Video toolchain is required for screen streaming and test replay recording. Missing: {missing_str}.",
|
||||||
metadata=metadata,
|
metadata=metadata,
|
||||||
actions=actions,
|
actions=actions,
|
||||||
|
|||||||
@@ -39,7 +39,7 @@ CHECK_ORDER: dict[str, int] = {
|
|||||||
"python_runtime": 0,
|
"python_runtime": 0,
|
||||||
"system_config": 1,
|
"system_config": 1,
|
||||||
"integration_host": 2,
|
"integration_host": 2,
|
||||||
"gemini_api_key": 3,
|
"openai_api_key": 3,
|
||||||
"android_adb": 4,
|
"android_adb": 4,
|
||||||
"toolchain": 5,
|
"toolchain": 5,
|
||||||
"vision_ocr_key": 6,
|
"vision_ocr_key": 6,
|
||||||
|
|||||||
@@ -78,7 +78,7 @@ class ProbeResult(BaseModel):
|
|||||||
"""Outcome of a single readiness probe check."""
|
"""Outcome of a single readiness probe check."""
|
||||||
|
|
||||||
id: str = Field(
|
id: str = Field(
|
||||||
description="Unique identifier for the probe (e.g. 'android_adb', 'gemini_api_key')"
|
description="Unique identifier for the probe (e.g. 'android_adb', 'openai_api_key')"
|
||||||
)
|
)
|
||||||
category: ProbeCategory = Field(description="Category of the probe")
|
category: ProbeCategory = Field(description="Category of the probe")
|
||||||
title: str = Field(description="Human-readable title (e.g. 'Device / Emulator Connected')")
|
title: str = Field(description="Human-readable title (e.g. 'Device / Emulator Connected')")
|
||||||
|
|||||||
@@ -36,21 +36,7 @@ def init_command() -> None:
|
|||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
# 1. Select Provider
|
env_key, provider_name = "OPENAI_API_KEY", "OpenAI Responses"
|
||||||
console.print("\n[bold]Step 1: Choose Primary LLM Provider[/bold]")
|
|
||||||
console.print(" [1] Google Gemini (Recommended - Default)")
|
|
||||||
console.print(" [2] OpenAI (GPT-4o)")
|
|
||||||
console.print(" [3] Anthropic (Claude 3.5 Sonnet)")
|
|
||||||
console.print(" [4] OpenRouter")
|
|
||||||
|
|
||||||
choice = Prompt.ask("Select provider", choices=["1", "2", "3", "4"], default="1")
|
|
||||||
provider_map = {
|
|
||||||
"1": ("GEMINI_API_KEY", "Google Gemini"),
|
|
||||||
"2": ("OPENAI_API_KEY", "OpenAI"),
|
|
||||||
"3": ("ANTHROPIC_API_KEY", "Anthropic"),
|
|
||||||
"4": ("OPEN_ROUTER_API_KEY", "OpenRouter"),
|
|
||||||
}
|
|
||||||
env_key, provider_name = provider_map[choice]
|
|
||||||
|
|
||||||
# 2. Enter API Key
|
# 2. Enter API Key
|
||||||
console.print(f"\n[bold]Step 2: Enter your {provider_name} API Key[/bold]")
|
console.print(f"\n[bold]Step 2: Enter your {provider_name} API Key[/bold]")
|
||||||
@@ -126,7 +112,7 @@ def init_command() -> None:
|
|||||||
"",
|
"",
|
||||||
"# Execution Defaults",
|
"# Execution Defaults",
|
||||||
"ARTEMIS_DEFAULT_PROFILE=pro",
|
"ARTEMIS_DEFAULT_PROFILE=pro",
|
||||||
"ARTEMIS_DEFAULT_MODEL=gemini-2.5-flash",
|
"OPENAI_MODEL=gpt-5-mini",
|
||||||
"ARTEMIS_TRACES_DIR=./traces",
|
"ARTEMIS_TRACES_DIR=./traces",
|
||||||
"",
|
"",
|
||||||
]
|
]
|
||||||
|
|||||||
@@ -503,7 +503,7 @@ def run_command(
|
|||||||
Panel(
|
Panel(
|
||||||
f"[bold red]✖ Authentication Error:[/bold red] {err_msg}\n\n"
|
f"[bold red]✖ Authentication Error:[/bold red] {err_msg}\n\n"
|
||||||
"💡 [bold cyan]Quick Fix:[/bold cyan] Run [bold green]artemis init[/bold green] to configure your API key in 10 seconds,\n"
|
"💡 [bold cyan]Quick Fix:[/bold cyan] Run [bold green]artemis init[/bold green] to configure your API key in 10 seconds,\n"
|
||||||
"or add [bold]GEMINI_API_KEY=your_key[/bold] to [dim].env[/dim].",
|
"or add [bold]OPENAI_API_KEY=your_key[/bold] to [dim].env[/dim].",
|
||||||
title="Missing API Key",
|
title="Missing API Key",
|
||||||
expand=False,
|
expand=False,
|
||||||
)
|
)
|
||||||
|
|||||||
@@ -1,57 +0,0 @@
|
|||||||
# Copyright 2026 Google LLC
|
|
||||||
#
|
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
||||||
# you may not use this file except in compliance with the License.
|
|
||||||
# You may obtain a copy of the License at
|
|
||||||
#
|
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
#
|
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
# See the License for the specific language governing permissions and
|
|
||||||
# limitations under the License.
|
|
||||||
|
|
||||||
"""Google / Gemini specific knowledge shared by every call site.
|
|
||||||
|
|
||||||
Two concerns live here so they are defined exactly once:
|
|
||||||
|
|
||||||
* :mod:`artemis.llm.google.provider` - "is this endpoint Google?" and the
|
|
||||||
Gemini model capability table (thinking levels, agentic video).
|
|
||||||
* :mod:`artemis.llm.google.usage` - token-usage normalization across the
|
|
||||||
LangChain, ``generate_content`` and Interactions API response shapes.
|
|
||||||
|
|
||||||
Both modules are dependency-free (no settings, no SDK client) so they can be
|
|
||||||
imported from configuration code without creating import cycles.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from artemis.llm.google.provider import (
|
|
||||||
VideoProcessing,
|
|
||||||
gemini_version,
|
|
||||||
is_agentic_video_auto_eligible,
|
|
||||||
is_gemini_model,
|
|
||||||
is_google_chat_model,
|
|
||||||
is_google_family_provider,
|
|
||||||
is_google_provider,
|
|
||||||
resolve_video_processing,
|
|
||||||
strip_provider_prefix,
|
|
||||||
supports_agentic_video,
|
|
||||||
supports_thinking_level,
|
|
||||||
)
|
|
||||||
from artemis.llm.google.usage import normalize_usage, usage_from_message
|
|
||||||
|
|
||||||
__all__ = [
|
|
||||||
"VideoProcessing",
|
|
||||||
"gemini_version",
|
|
||||||
"is_agentic_video_auto_eligible",
|
|
||||||
"is_gemini_model",
|
|
||||||
"is_google_chat_model",
|
|
||||||
"is_google_family_provider",
|
|
||||||
"is_google_provider",
|
|
||||||
"normalize_usage",
|
|
||||||
"resolve_video_processing",
|
|
||||||
"strip_provider_prefix",
|
|
||||||
"supports_agentic_video",
|
|
||||||
"supports_thinking_level",
|
|
||||||
"usage_from_message",
|
|
||||||
]
|
|
||||||
@@ -1,185 +0,0 @@
|
|||||||
# Copyright 2026 Google LLC
|
|
||||||
#
|
|
||||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
|
||||||
# you may not use this file except in compliance with the License.
|
|
||||||
# You may obtain a copy of the License at
|
|
||||||
#
|
|
||||||
# http://www.apache.org/licenses/LICENSE-2.0
|
|
||||||
#
|
|
||||||
# Unless required by applicable law or agreed to in writing, software
|
|
||||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
|
||||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
|
||||||
# See the License for the specific language governing permissions and
|
|
||||||
# limitations under the License.
|
|
||||||
|
|
||||||
"""Google provider identity and the Gemini model capability table.
|
|
||||||
|
|
||||||
Provider identity has two tiers:
|
|
||||||
|
|
||||||
* :func:`is_google_provider` - the Gemini Developer API (``google`` /
|
|
||||||
``gemini``). Native ``google-genai`` features (Files API, explicit caching,
|
|
||||||
the Interactions API) are only reachable here.
|
|
||||||
* :func:`is_google_family_provider` - additionally Vertex AI, which shares
|
|
||||||
Google credentials but not the developer-API-only features.
|
|
||||||
|
|
||||||
Model IDs use ``gemini-<major>.<minor>-<tier>[<suffix>]``. Release suffixes
|
|
||||||
(``preview``, ``latest``, ``exp``, and numeric tokens) retain base-family
|
|
||||||
capabilities. Variants such as ``image`` and ``tts`` are excluded from agentic
|
|
||||||
video support, but their version still determines thinking-level support.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import re
|
|
||||||
from typing import Any, Literal, NamedTuple
|
|
||||||
|
|
||||||
VideoProcessing = Literal["auto", "agentic", "static"]
|
|
||||||
ResolvedVideoProcessing = Literal["agentic", "static"]
|
|
||||||
|
|
||||||
_GOOGLE_API_PROVIDERS = frozenset({"google", "gemini"})
|
|
||||||
_GOOGLE_FAMILY_PROVIDERS = _GOOGLE_API_PROVIDERS | frozenset({"vertexai", "vertex"})
|
|
||||||
|
|
||||||
_GEMINI_MODEL_RE = re.compile(r"gemini-(\d+)\.(\d+)-(flash-lite|flash|pro)", re.IGNORECASE)
|
|
||||||
_RELEASE_SUFFIX_TOKENS = frozenset({"preview", "latest", "exp"})
|
|
||||||
|
|
||||||
|
|
||||||
class _GeminiModel(NamedTuple):
|
|
||||||
version: tuple[int, int]
|
|
||||||
tier: str
|
|
||||||
#: False for variant ids (``-image``, ``-tts``, ...) that carry their own
|
|
||||||
#: capabilities and must not inherit the family's answers.
|
|
||||||
is_base: bool
|
|
||||||
|
|
||||||
|
|
||||||
# Agentic video understanding (Interactions API, ``processing: "agentic"``)
|
|
||||||
# launched on Gemini 3.6/3.7/3.8 Flash and 3.5 Flash-Lite. Later releases in
|
|
||||||
# the same families are assumed to keep it.
|
|
||||||
_AGENTIC_VIDEO_MIN_FLASH = (3, 6)
|
|
||||||
_AGENTIC_VIDEO_MIN_FLASH_LITE = (3, 5)
|
|
||||||
|
|
||||||
|
|
||||||
def _normalize_provider(value: Any) -> str:
|
|
||||||
if value is None:
|
|
||||||
return ""
|
|
||||||
value = getattr(value, "value", value)
|
|
||||||
return str(value).lower().replace("_", "").replace("-", "").strip()
|
|
||||||
|
|
||||||
|
|
||||||
def is_google_provider(value: Any) -> bool:
|
|
||||||
"""True for the Gemini Developer API (``google`` / ``gemini``).
|
|
||||||
|
|
||||||
Accepts a provider name, a provider enum, or anything exposing ``.value``.
|
|
||||||
"""
|
|
||||||
return _normalize_provider(value) in _GOOGLE_API_PROVIDERS
|
|
||||||
|
|
||||||
|
|
||||||
def is_google_family_provider(value: Any) -> bool:
|
|
||||||
"""True for every provider that authenticates with Google credentials."""
|
|
||||||
return _normalize_provider(value) in _GOOGLE_FAMILY_PROVIDERS
|
|
||||||
|
|
||||||
|
|
||||||
def is_google_chat_model(model: Any) -> bool:
|
|
||||||
"""True when a LangChain chat model is backed by Google (by class name).
|
|
||||||
|
|
||||||
Only used when no endpoint metadata is available; the class-name sniff is
|
|
||||||
the historical fallback and is kept in one place here.
|
|
||||||
"""
|
|
||||||
cls = getattr(model, "__class__", None)
|
|
||||||
name = cls.__name__ if cls else ""
|
|
||||||
return "Google" in name or "Gemini" in name or "VertexAI" in name
|
|
||||||
|
|
||||||
|
|
||||||
def strip_provider_prefix(model_name: Any) -> str:
|
|
||||||
"""Drops a ``provider/`` prefix (``google/gemini-3.8-flash`` -> ``gemini-3.8-flash``)."""
|
|
||||||
name = str(model_name or "")
|
|
||||||
return name.rsplit("/", 1)[-1] if "/" in name else name
|
|
||||||
|
|
||||||
|
|
||||||
def is_gemini_model(model_name: Any) -> bool:
|
|
||||||
"""True when the model name refers to any Gemini model."""
|
|
||||||
return "gemini" in strip_provider_prefix(model_name).lower()
|
|
||||||
|
|
||||||
|
|
||||||
def _is_release_suffix(rest: str) -> bool:
|
|
||||||
"""Whether what follows the tier is a release suffix (see the module docstring)."""
|
|
||||||
if not rest:
|
|
||||||
return True
|
|
||||||
if not rest.startswith("-"):
|
|
||||||
return False
|
|
||||||
return all(token in _RELEASE_SUFFIX_TOKENS or token.isdigit() for token in rest[1:].split("-"))
|
|
||||||
|
|
||||||
|
|
||||||
def _parse_gemini_model(model_name: Any) -> _GeminiModel | None:
|
|
||||||
name = strip_provider_prefix(model_name).lower()
|
|
||||||
match = _GEMINI_MODEL_RE.search(name)
|
|
||||||
if not match:
|
|
||||||
return None
|
|
||||||
return _GeminiModel(
|
|
||||||
version=(int(match.group(1)), int(match.group(2))),
|
|
||||||
tier=match.group(3),
|
|
||||||
is_base=_is_release_suffix(name[match.end() :]),
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def gemini_version(model_name: Any) -> tuple[int, int] | None:
|
|
||||||
"""``(major, minor)`` parsed from a ``gemini-X.Y-*`` name, else ``None``."""
|
|
||||||
parsed = _parse_gemini_model(model_name)
|
|
||||||
return parsed.version if parsed else None
|
|
||||||
|
|
||||||
|
|
||||||
def _gemini_tier(model_name: Any) -> str | None:
|
|
||||||
"""The tier of a *base* model id; ``None`` for variants and non-Gemini names."""
|
|
||||||
parsed = _parse_gemini_model(model_name)
|
|
||||||
return parsed.tier if parsed and parsed.is_base else None
|
|
||||||
|
|
||||||
|
|
||||||
def supports_thinking_level(model_name: Any) -> bool:
|
|
||||||
"""Whether the model accepts ``thinking_level`` (Gemini 3 and later).
|
|
||||||
|
|
||||||
Gemini 1.x/2.x only understand ``thinking_budget``. Names that do not
|
|
||||||
carry a parsable version (e.g. the robotics ER models) are assumed to be
|
|
||||||
current and therefore accept it.
|
|
||||||
"""
|
|
||||||
version = gemini_version(model_name)
|
|
||||||
return version is None or version[0] >= 3
|
|
||||||
|
|
||||||
|
|
||||||
def supports_agentic_video(model_name: Any) -> bool:
|
|
||||||
"""Whether the Gemini API accepts ``processing: "agentic"`` for this model.
|
|
||||||
|
|
||||||
Only base Flash / Flash-Lite ids qualify; variants (``-image``, ``-tts``,
|
|
||||||
``-live``, ...) do not inherit the family's support.
|
|
||||||
"""
|
|
||||||
parsed = _parse_gemini_model(model_name)
|
|
||||||
if parsed is None or not parsed.is_base:
|
|
||||||
return False
|
|
||||||
if parsed.tier == "flash":
|
|
||||||
return parsed.version >= _AGENTIC_VIDEO_MIN_FLASH
|
|
||||||
if parsed.tier == "flash-lite":
|
|
||||||
return parsed.version >= _AGENTIC_VIDEO_MIN_FLASH_LITE
|
|
||||||
return False
|
|
||||||
|
|
||||||
|
|
||||||
def is_agentic_video_auto_eligible(model_name: Any) -> bool:
|
|
||||||
"""Models that get agentic video processing under ``processing: "auto"``.
|
|
||||||
|
|
||||||
Flash-Lite is deliberately excluded: it accepts the mode but mis-reads
|
|
||||||
timestamps in practice, so it must be opted into explicitly.
|
|
||||||
"""
|
|
||||||
return supports_agentic_video(model_name) and _gemini_tier(model_name) == "flash"
|
|
||||||
|
|
||||||
|
|
||||||
def resolve_video_processing(configured: Any, model_name: Any) -> ResolvedVideoProcessing:
|
|
||||||
"""Turns the configured video-processing knob into a concrete mode.
|
|
||||||
|
|
||||||
``static`` is always honoured. ``agentic`` is honoured only for models
|
|
||||||
that support it and silently degrades otherwise. ``auto`` (and any
|
|
||||||
unrecognised value) enables agentic processing for the auto-eligible
|
|
||||||
models and keeps static processing for everything else.
|
|
||||||
"""
|
|
||||||
mode = str(configured or "auto").strip().lower() if isinstance(configured, str) else "auto"
|
|
||||||
if mode == "static":
|
|
||||||
return "static"
|
|
||||||
if mode == "agentic":
|
|
||||||
return "agentic" if supports_agentic_video(model_name) else "static"
|
|
||||||
return "agentic" if is_agentic_video_auto_eligible(model_name) else "static"
|
|
||||||
+43
-323
@@ -12,12 +12,7 @@
|
|||||||
# See the License for the specific language governing permissions and
|
# See the License for the specific language governing permissions and
|
||||||
# limitations under the License.
|
# limitations under the License.
|
||||||
|
|
||||||
"""Dynamic Multi-Provider Model Router for ARTEMIS.
|
"""Single transport for all Artemis models: OpenAI Responses API."""
|
||||||
|
|
||||||
Provides role-based LLM/VLM dispatching, dynamic fallback chains,
|
|
||||||
and unified provider configuration across Gemini, Vertex AI, OpenAI,
|
|
||||||
Anthropic, OpenRouter, and local Ollama/vLLM endpoints.
|
|
||||||
"""
|
|
||||||
|
|
||||||
from enum import StrEnum
|
from enum import StrEnum
|
||||||
import hashlib
|
import hashlib
|
||||||
@@ -28,94 +23,34 @@ from langchain_core.language_models.chat_models import BaseChatModel
|
|||||||
from pydantic import BaseModel, Field
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
from artemis.config.settings import settings
|
from artemis.config.settings import settings
|
||||||
from artemis.llm.google.provider import supports_thinking_level
|
|
||||||
from artemis.utils.logger import get_logger
|
|
||||||
|
|
||||||
logger = get_logger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
class ModelProvider(StrEnum):
|
class ModelProvider(StrEnum):
|
||||||
GOOGLE = "google"
|
|
||||||
GEMINI = "google"
|
|
||||||
VERTEX_AI = "vertexai"
|
|
||||||
OPENAI = "openai"
|
OPENAI = "openai"
|
||||||
ANTHROPIC = "anthropic"
|
|
||||||
OPENROUTER = "openrouter"
|
|
||||||
XAI = "xai"
|
|
||||||
OLLAMA = "ollama"
|
|
||||||
VLLM = "vllm"
|
|
||||||
CUSTOM = "custom"
|
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def from_string(cls, val: Any) -> "ModelProvider":
|
def from_string(cls, val: Any) -> "ModelProvider":
|
||||||
"""Converts a string or enum into a canonical ModelProvider.
|
if val is None or str(val).strip().lower() in ("", "openai"):
|
||||||
|
return cls.OPENAI
|
||||||
``None``/empty means "use the default" (Google); an unrecognized
|
raise ValueError(f"Unsupported LLM provider {val!r}; only OpenAI Responses is supported")
|
||||||
non-empty value raises instead of silently routing to Gemini with
|
|
||||||
the wrong credentials.
|
|
||||||
"""
|
|
||||||
if isinstance(val, cls):
|
|
||||||
return val
|
|
||||||
if val is None:
|
|
||||||
return cls.GOOGLE
|
|
||||||
s = str(val).lower().replace("_", "").replace("-", "").strip()
|
|
||||||
if not s:
|
|
||||||
return cls.GOOGLE
|
|
||||||
mapping = {
|
|
||||||
"google": cls.GOOGLE,
|
|
||||||
"gemini": cls.GOOGLE,
|
|
||||||
"vertexai": cls.VERTEX_AI,
|
|
||||||
"vertex": cls.VERTEX_AI,
|
|
||||||
"openai": cls.OPENAI,
|
|
||||||
"anthropic": cls.ANTHROPIC,
|
|
||||||
"claude": cls.ANTHROPIC,
|
|
||||||
"openrouter": cls.OPENROUTER,
|
|
||||||
"xai": cls.XAI,
|
|
||||||
"grok": cls.XAI,
|
|
||||||
"ollama": cls.OLLAMA,
|
|
||||||
"vllm": cls.VLLM,
|
|
||||||
"custom": cls.CUSTOM,
|
|
||||||
}
|
|
||||||
provider = mapping.get(s)
|
|
||||||
if provider is None:
|
|
||||||
raise ValueError(
|
|
||||||
f"Unknown LLM provider {val!r}. Valid providers: {sorted(set(mapping))}"
|
|
||||||
)
|
|
||||||
return provider
|
|
||||||
|
|
||||||
|
|
||||||
class ModelEndpoint(BaseModel):
|
class ModelEndpoint(BaseModel):
|
||||||
"""Configuration definition for an LLM/VLM model endpoint."""
|
provider: ModelProvider = ModelProvider.OPENAI
|
||||||
|
model_name: str = Field(default_factory=lambda: settings.OPENAI_MODEL)
|
||||||
provider: ModelProvider = Field(default=ModelProvider.GOOGLE, description="Model provider")
|
api_key: str | None = Field(default=None, repr=False)
|
||||||
model_name: str = Field(default="gemini-2.5-flash", description="Model name identifier")
|
api_base: str | None = None
|
||||||
api_key: str | None = Field(default=None, description="API Key or secret")
|
temperature: float | None = None
|
||||||
api_base: str | None = Field(default=None, description="Custom API endpoint base URL")
|
max_tokens: int | None = Field(default=None, gt=0)
|
||||||
temperature: float = Field(default=0.0, description="Sampling temperature")
|
timeout_seconds: float = Field(default_factory=lambda: settings.OPENAI_TIMEOUT_SECONDS, gt=0)
|
||||||
max_tokens: int | None = Field(default=None, description="Maximum completion tokens")
|
max_retries: int = Field(default_factory=lambda: settings.OPENAI_MAX_RETRIES, ge=0)
|
||||||
timeout_seconds: float = Field(default=60.0, description="Request timeout in seconds")
|
is_multimodal: bool = True
|
||||||
is_multimodal: bool = Field(
|
reasoning_effort: str | None = None
|
||||||
default=True, description="Whether endpoint accepts image/video inputs"
|
enable_grounding: bool = False
|
||||||
)
|
|
||||||
reasoning_effort: str | None = Field(
|
|
||||||
default=None, description="Reasoning effort: 'low', 'medium', 'high'"
|
|
||||||
)
|
|
||||||
thinking_budget: int | None = Field(default=None, description="Thinking budget token count")
|
|
||||||
thinking_level: str | None = Field(
|
|
||||||
default=None, description="Thinking level: 'minimal', 'low', 'medium', 'high'"
|
|
||||||
)
|
|
||||||
include_thoughts: bool | None = Field(
|
|
||||||
default=None, description="Whether to include thought traces in response"
|
|
||||||
)
|
|
||||||
enable_grounding: bool = Field(
|
|
||||||
default=False,
|
|
||||||
description="Whether to enable Google Search grounding for Gemini endpoints",
|
|
||||||
)
|
|
||||||
|
|
||||||
def cache_key(self) -> tuple:
|
def cache_key(self) -> tuple:
|
||||||
"""Key the client cache by endpoint settings, hashing the API key."""
|
key = self.api_key or (
|
||||||
api_key_digest = (
|
settings.OPENAI_API_KEY.get_secret_value() if settings.OPENAI_API_KEY else ""
|
||||||
hashlib.sha256(self.api_key.encode("utf-8")).hexdigest()[:16] if self.api_key else None
|
|
||||||
)
|
)
|
||||||
return (
|
return (
|
||||||
self.provider.value,
|
self.provider.value,
|
||||||
@@ -123,81 +58,15 @@ class ModelEndpoint(BaseModel):
|
|||||||
self.temperature,
|
self.temperature,
|
||||||
self.max_tokens,
|
self.max_tokens,
|
||||||
self.timeout_seconds,
|
self.timeout_seconds,
|
||||||
self.thinking_budget,
|
self.max_retries,
|
||||||
self.thinking_level,
|
|
||||||
self.include_thoughts,
|
|
||||||
self.reasoning_effort,
|
self.reasoning_effort,
|
||||||
self.enable_grounding,
|
self.enable_grounding,
|
||||||
self.api_base,
|
self.api_base or settings.OPENAI_BASE_URL,
|
||||||
api_key_digest,
|
hashlib.sha256(key.encode()).hexdigest(),
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
||||||
def _patch_langchain_google_genai():
|
|
||||||
"""Patches ChatGoogleGenerativeAI._process_tool_config to preserve and ensure include_server_side_tool_invocations."""
|
|
||||||
try:
|
|
||||||
from langchain_google_genai.chat_models import ChatGoogleGenerativeAI
|
|
||||||
from google.genai.types import ToolConfig
|
|
||||||
|
|
||||||
if getattr(ChatGoogleGenerativeAI, "_is_artemis_patched", False):
|
|
||||||
return
|
|
||||||
|
|
||||||
original_process = ChatGoogleGenerativeAI._process_tool_config
|
|
||||||
|
|
||||||
def patched_process_tool_config(self, tool_choice, tool_config, formatted_tools):
|
|
||||||
config = original_process(self, tool_choice, tool_config, formatted_tools)
|
|
||||||
|
|
||||||
has_builtin = False
|
|
||||||
has_functions = False
|
|
||||||
if formatted_tools:
|
|
||||||
for tool in formatted_tools:
|
|
||||||
tool_dict = tool if isinstance(tool, dict) else getattr(tool, "__dict__", {})
|
|
||||||
if any(
|
|
||||||
tool_dict.get(k) is not None
|
|
||||||
for k in (
|
|
||||||
"google_search",
|
|
||||||
"code_execution",
|
|
||||||
"google_maps",
|
|
||||||
"google_search_retrieval",
|
|
||||||
)
|
|
||||||
):
|
|
||||||
has_builtin = True
|
|
||||||
if tool_dict.get("function_declarations"):
|
|
||||||
has_functions = True
|
|
||||||
|
|
||||||
if tool_config:
|
|
||||||
normalized = (
|
|
||||||
ToolConfig.model_validate(tool_config)
|
|
||||||
if isinstance(tool_config, dict)
|
|
||||||
else tool_config
|
|
||||||
)
|
|
||||||
if getattr(normalized, "include_server_side_tool_invocations", None) is not None:
|
|
||||||
if config is None:
|
|
||||||
config = ToolConfig(
|
|
||||||
include_server_side_tool_invocations=normalized.include_server_side_tool_invocations
|
|
||||||
)
|
|
||||||
else:
|
|
||||||
config.include_server_side_tool_invocations = (
|
|
||||||
normalized.include_server_side_tool_invocations
|
|
||||||
)
|
|
||||||
|
|
||||||
if has_builtin and has_functions:
|
|
||||||
if config is None:
|
|
||||||
config = ToolConfig(include_server_side_tool_invocations=True)
|
|
||||||
else:
|
|
||||||
config.include_server_side_tool_invocations = True
|
|
||||||
|
|
||||||
return config
|
|
||||||
|
|
||||||
ChatGoogleGenerativeAI._process_tool_config = patched_process_tool_config
|
|
||||||
ChatGoogleGenerativeAI._is_artemis_patched = True
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"Could not patch ChatGoogleGenerativeAI._process_tool_config: {e}")
|
|
||||||
|
|
||||||
|
|
||||||
class ModelFactory:
|
class ModelFactory:
|
||||||
"""Unified factory for instantiating and caching LangChain chat models across providers."""
|
|
||||||
|
|
||||||
_cache: dict[tuple, BaseChatModel] = {}
|
_cache: dict[tuple, BaseChatModel] = {}
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
@@ -209,181 +78,32 @@ class ModelFactory:
|
|||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def create_model(cls, endpoint: ModelEndpoint) -> BaseChatModel:
|
def create_model(cls, endpoint: ModelEndpoint) -> BaseChatModel:
|
||||||
|
ModelProvider.from_string(endpoint.provider)
|
||||||
if os.environ.get("ARTEMIS_FAKE_LLM") == "1":
|
if os.environ.get("ARTEMIS_FAKE_LLM") == "1":
|
||||||
from artemis.llm.fake_model import FakeChatModel
|
from artemis.llm.fake_model import FakeChatModel
|
||||||
|
|
||||||
delay = float(os.environ.get("ARTEMIS_FAKE_LLM_DELAY_S", "0") or 0)
|
return FakeChatModel(
|
||||||
logger.warning(
|
delay_s=float(os.environ.get("ARTEMIS_FAKE_LLM_DELAY_S", "0") or 0)
|
||||||
f"ARTEMIS_FAKE_LLM=1 — returning FakeChatModel (delay={delay}s) "
|
|
||||||
f"instead of {endpoint.provider}/{endpoint.model_name}"
|
|
||||||
)
|
|
||||||
return FakeChatModel(delay_s=delay)
|
|
||||||
|
|
||||||
provider = ModelProvider.from_string(endpoint.provider)
|
|
||||||
|
|
||||||
if provider == ModelProvider.GOOGLE:
|
|
||||||
_patch_langchain_google_genai()
|
|
||||||
from langchain_google_genai import (
|
|
||||||
ChatGoogleGenerativeAI,
|
|
||||||
HarmBlockThreshold,
|
|
||||||
HarmCategory,
|
|
||||||
)
|
)
|
||||||
|
|
||||||
api_key = (
|
from langchain_openai import ChatOpenAI
|
||||||
endpoint.api_key
|
|
||||||
or (settings.GOOGLE_API_KEY.get_secret_value() if settings.GOOGLE_API_KEY else None)
|
|
||||||
or os.environ.get("GOOGLE_API_KEY")
|
|
||||||
or os.environ.get("GEMINI_API_KEY")
|
|
||||||
)
|
|
||||||
# Gemini 1.x/2.x only understand thinking_budget.
|
|
||||||
thinking_level = (
|
|
||||||
endpoint.thinking_level if supports_thinking_level(endpoint.model_name) else None
|
|
||||||
)
|
|
||||||
|
|
||||||
kwargs: dict[str, Any] = {
|
api_key = endpoint.api_key or (
|
||||||
"model": endpoint.model_name,
|
settings.OPENAI_API_KEY.get_secret_value() if settings.OPENAI_API_KEY else None
|
||||||
"temperature": endpoint.temperature,
|
)
|
||||||
"max_output_tokens": endpoint.max_tokens,
|
if not api_key:
|
||||||
"api_key": api_key,
|
raise ValueError("请在环境变量或 .env 中配置 OPENAI_API_KEY")
|
||||||
"timeout": endpoint.timeout_seconds,
|
kwargs = {
|
||||||
"thinking_budget": endpoint.thinking_budget,
|
"model": endpoint.model_name,
|
||||||
"thinking_level": thinking_level,
|
"api_key": api_key,
|
||||||
"include_thoughts": endpoint.include_thoughts,
|
"base_url": endpoint.api_base or settings.OPENAI_BASE_URL or None,
|
||||||
"safety_settings": {
|
"timeout": endpoint.timeout_seconds,
|
||||||
HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_NONE,
|
"max_retries": endpoint.max_retries,
|
||||||
HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_NONE,
|
"temperature": endpoint.temperature,
|
||||||
HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
|
"max_tokens": endpoint.max_tokens,
|
||||||
HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,
|
"reasoning_effort": endpoint.reasoning_effort,
|
||||||
},
|
"use_responses_api": True,
|
||||||
}
|
"output_version": "responses/v1",
|
||||||
return ChatGoogleGenerativeAI(**{k: v for k, v in kwargs.items() if v is not None})
|
"store": False,
|
||||||
|
}
|
||||||
elif provider == ModelProvider.VERTEX_AI:
|
return ChatOpenAI(**{k: v for k, v in kwargs.items() if v is not None})
|
||||||
from langchain_google_vertexai import (
|
|
||||||
ChatVertexAI,
|
|
||||||
HarmBlockThreshold,
|
|
||||||
HarmCategory,
|
|
||||||
)
|
|
||||||
|
|
||||||
kwargs = {
|
|
||||||
"model_name": endpoint.model_name,
|
|
||||||
"temperature": endpoint.temperature,
|
|
||||||
"max_output_tokens": endpoint.max_tokens,
|
|
||||||
"timeout": endpoint.timeout_seconds,
|
|
||||||
"thinking_budget": endpoint.thinking_budget,
|
|
||||||
"safety_settings": {
|
|
||||||
HarmCategory.HARM_CATEGORY_HATE_SPEECH: HarmBlockThreshold.BLOCK_NONE,
|
|
||||||
HarmCategory.HARM_CATEGORY_HARASSMENT: HarmBlockThreshold.BLOCK_NONE,
|
|
||||||
HarmCategory.HARM_CATEGORY_SEXUALLY_EXPLICIT: HarmBlockThreshold.BLOCK_NONE,
|
|
||||||
HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: HarmBlockThreshold.BLOCK_NONE,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
return ChatVertexAI(**{k: v for k, v in kwargs.items() if v is not None})
|
|
||||||
|
|
||||||
elif provider == ModelProvider.OPENAI:
|
|
||||||
from langchain_openai import ChatOpenAI
|
|
||||||
|
|
||||||
api_key = (
|
|
||||||
endpoint.api_key
|
|
||||||
or (settings.OPENAI_API_KEY.get_secret_value() if settings.OPENAI_API_KEY else None)
|
|
||||||
or os.environ.get("OPENAI_API_KEY", "EMPTY")
|
|
||||||
)
|
|
||||||
base_url = endpoint.api_base or (
|
|
||||||
str(settings.OPENAI_BASE_URL) if settings.OPENAI_BASE_URL else None
|
|
||||||
)
|
|
||||||
kwargs = {
|
|
||||||
"model": endpoint.model_name,
|
|
||||||
"temperature": endpoint.temperature,
|
|
||||||
"max_tokens": endpoint.max_tokens,
|
|
||||||
"api_key": api_key,
|
|
||||||
"base_url": base_url,
|
|
||||||
"timeout": endpoint.timeout_seconds,
|
|
||||||
}
|
|
||||||
if endpoint.reasoning_effort:
|
|
||||||
kwargs["reasoning_effort"] = endpoint.reasoning_effort
|
|
||||||
return ChatOpenAI(**{k: v for k, v in kwargs.items() if v is not None})
|
|
||||||
|
|
||||||
elif provider == ModelProvider.ANTHROPIC:
|
|
||||||
from langchain_anthropic import ChatAnthropic
|
|
||||||
|
|
||||||
api_key = (
|
|
||||||
endpoint.api_key
|
|
||||||
or (
|
|
||||||
settings.ANTHROPIC_API_KEY.get_secret_value()
|
|
||||||
if settings.ANTHROPIC_API_KEY
|
|
||||||
else None
|
|
||||||
)
|
|
||||||
or os.environ.get("ANTHROPIC_API_KEY")
|
|
||||||
)
|
|
||||||
kwargs = {
|
|
||||||
"model": endpoint.model_name,
|
|
||||||
"temperature": endpoint.temperature,
|
|
||||||
"api_key": api_key,
|
|
||||||
"timeout": endpoint.timeout_seconds,
|
|
||||||
}
|
|
||||||
budget = endpoint.thinking_budget
|
|
||||||
if not budget and endpoint.reasoning_effort:
|
|
||||||
effort_map = {"low": 2048, "medium": 8192, "high": 32768}
|
|
||||||
budget = effort_map.get(endpoint.reasoning_effort.lower())
|
|
||||||
if budget:
|
|
||||||
kwargs["thinking"] = {"type": "enabled", "budget_tokens": budget}
|
|
||||||
kwargs["temperature"] = 1.0
|
|
||||||
return ChatAnthropic(**{k: v for k, v in kwargs.items() if v is not None})
|
|
||||||
|
|
||||||
elif provider == ModelProvider.OPENROUTER:
|
|
||||||
from langchain_openai import ChatOpenAI
|
|
||||||
|
|
||||||
api_key = (
|
|
||||||
endpoint.api_key
|
|
||||||
or (
|
|
||||||
settings.OPEN_ROUTER_API_KEY.get_secret_value()
|
|
||||||
if settings.OPEN_ROUTER_API_KEY
|
|
||||||
else None
|
|
||||||
)
|
|
||||||
or os.environ.get("OPEN_ROUTER_API_KEY")
|
|
||||||
)
|
|
||||||
return ChatOpenAI(
|
|
||||||
model=endpoint.model_name,
|
|
||||||
temperature=endpoint.temperature,
|
|
||||||
api_key=api_key,
|
|
||||||
base_url=endpoint.api_base or "https://openrouter.ai/api/v1",
|
|
||||||
timeout=endpoint.timeout_seconds,
|
|
||||||
)
|
|
||||||
|
|
||||||
elif provider == ModelProvider.XAI:
|
|
||||||
from langchain_openai import ChatOpenAI
|
|
||||||
|
|
||||||
api_key = (
|
|
||||||
endpoint.api_key
|
|
||||||
or (settings.XAI_API_KEY.get_secret_value() if settings.XAI_API_KEY else None)
|
|
||||||
or os.environ.get("XAI_API_KEY")
|
|
||||||
)
|
|
||||||
return ChatOpenAI(
|
|
||||||
model=endpoint.model_name,
|
|
||||||
temperature=endpoint.temperature,
|
|
||||||
api_key=api_key,
|
|
||||||
base_url=endpoint.api_base or "https://api.x.ai/v1",
|
|
||||||
timeout=endpoint.timeout_seconds,
|
|
||||||
)
|
|
||||||
|
|
||||||
elif provider in (ModelProvider.OLLAMA, ModelProvider.VLLM, ModelProvider.CUSTOM):
|
|
||||||
from langchain_openai import ChatOpenAI
|
|
||||||
|
|
||||||
api_key = endpoint.api_key or os.environ.get("OPENAI_API_KEY", "EMPTY")
|
|
||||||
base_url = endpoint.api_base or os.environ.get(
|
|
||||||
"OPENAI_BASE_URL", "http://localhost:8000/v1"
|
|
||||||
)
|
|
||||||
kwargs = {
|
|
||||||
"model": endpoint.model_name,
|
|
||||||
"temperature": endpoint.temperature,
|
|
||||||
"max_tokens": endpoint.max_tokens,
|
|
||||||
"api_key": api_key,
|
|
||||||
"base_url": base_url,
|
|
||||||
"timeout": endpoint.timeout_seconds,
|
|
||||||
}
|
|
||||||
if endpoint.reasoning_effort:
|
|
||||||
kwargs["reasoning_effort"] = endpoint.reasoning_effort
|
|
||||||
return ChatOpenAI(**{k: v for k, v in kwargs.items() if v is not None})
|
|
||||||
|
|
||||||
else:
|
|
||||||
raise ValueError(f"Unsupported model provider: {provider}")
|
|
||||||
File renamed without changes.
@@ -35,6 +35,7 @@ context is rebuilt only when a compression event replaces or folds turns.
|
|||||||
"""
|
"""
|
||||||
|
|
||||||
import asyncio
|
import asyncio
|
||||||
|
from artemis.config.settings import settings
|
||||||
from dataclasses import dataclass, field
|
from dataclasses import dataclass, field
|
||||||
import json
|
import json
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
@@ -45,7 +46,6 @@ from uuid import uuid4
|
|||||||
|
|
||||||
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
|
from langchain_core.messages import BaseMessage, HumanMessage, SystemMessage
|
||||||
|
|
||||||
from artemis.llm.google import is_google_provider
|
|
||||||
from artemis.memory.step_memory import JobKey, StepLens, StepMemoryService
|
from artemis.memory.step_memory import JobKey, StepLens, StepMemoryService
|
||||||
from artemis.memory.transcript import format_session_offset
|
from artemis.memory.transcript import format_session_offset
|
||||||
from artemis.utils.logger import get_logger
|
from artemis.utils.logger import get_logger
|
||||||
@@ -298,7 +298,7 @@ class StepCapsuleLens(StepLens):
|
|||||||
fallback_model_name: str | None = None,
|
fallback_model_name: str | None = None,
|
||||||
fallback_llm: Any | None = None,
|
fallback_llm: Any | None = None,
|
||||||
):
|
):
|
||||||
self._model_name = model_name or "gemini-3.8-flash"
|
self._model_name = model_name or settings.OPENAI_MODEL
|
||||||
self._llm = llm
|
self._llm = llm
|
||||||
self._ctx = ctx
|
self._ctx = ctx
|
||||||
# Availability hardening: `chunking.model` is a dedicated model with no
|
# Availability hardening: `chunking.model` is a dedicated model with no
|
||||||
@@ -323,16 +323,16 @@ class StepCapsuleLens(StepLens):
|
|||||||
|
|
||||||
def _get_llm(self):
|
def _get_llm(self):
|
||||||
if self._llm is None:
|
if self._llm is None:
|
||||||
from artemis.services.llm import get_google_llm
|
from artemis.services.llm import get_openai_llm
|
||||||
|
|
||||||
self._llm = get_google_llm(model_name=self._model_name, temperature=0.0)
|
self._llm = get_openai_llm(model_name=self._model_name, temperature=0.0)
|
||||||
return self._llm
|
return self._llm
|
||||||
|
|
||||||
def _get_fallback_llm(self):
|
def _get_fallback_llm(self):
|
||||||
if self._fallback_llm is None and self._fallback_model_name:
|
if self._fallback_llm is None and self._fallback_model_name:
|
||||||
from artemis.services.llm import get_google_llm
|
from artemis.services.llm import get_openai_llm
|
||||||
|
|
||||||
self._fallback_llm = get_google_llm(
|
self._fallback_llm = get_openai_llm(
|
||||||
model_name=self._fallback_model_name, temperature=0.0
|
model_name=self._fallback_model_name, temperature=0.0
|
||||||
)
|
)
|
||||||
return self._fallback_llm
|
return self._fallback_llm
|
||||||
@@ -891,7 +891,7 @@ class HistoryChunkManager:
|
|||||||
self._max_steps = int(getattr(cc, "max_steps", 12) or 12)
|
self._max_steps = int(getattr(cc, "max_steps", 12) or 12)
|
||||||
self._min_steps = max(1, min(self._max_steps, int(getattr(cc, "min_steps", 3) or 3)))
|
self._min_steps = max(1, min(self._max_steps, int(getattr(cc, "min_steps", 3) or 3)))
|
||||||
self._target_source_tokens = int(getattr(cc, "target_source_tokens", 2000) or 2000)
|
self._target_source_tokens = int(getattr(cc, "target_source_tokens", 2000) or 2000)
|
||||||
self._model_name = getattr(cc, "model", None) or "gemini-3.8-flash"
|
self._model_name = getattr(cc, "model", None) or settings.OPENAI_MODEL
|
||||||
self._max_chunks = int(getattr(cc, "max_chunks", 8) or 8)
|
self._max_chunks = int(getattr(cc, "max_chunks", 8) or 8)
|
||||||
# None uses max_chunks as the era cap.
|
# None uses max_chunks as the era cap.
|
||||||
self._max_eras = int(getattr(cc, "max_eras", None) or self._max_chunks)
|
self._max_eras = int(getattr(cc, "max_eras", None) or self._max_chunks)
|
||||||
@@ -965,7 +965,7 @@ class HistoryChunkManager:
|
|||||||
fallback = getattr(getattr(llm_cfg, "summarizer", None), "fallback", None)
|
fallback = getattr(getattr(llm_cfg, "summarizer", None), "fallback", None)
|
||||||
provider = str(getattr(fallback, "provider", "") or "")
|
provider = str(getattr(fallback, "provider", "") or "")
|
||||||
model = getattr(fallback, "model", None)
|
model = getattr(fallback, "model", None)
|
||||||
if model and is_google_provider(provider) and model != self._model_name:
|
if model and str(provider).lower() == "openai" and model != self._model_name:
|
||||||
return str(model)
|
return str(model)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
logger.debug(f"Capsule fallback model resolution skipped: {exc}", exc_info=True)
|
logger.debug(f"Capsule fallback model resolution skipped: {exc}", exc_info=True)
|
||||||
|
|||||||
@@ -1,299 +1,122 @@
|
|||||||
|
// 模型仅支持 OpenAI Responses;端点和密钥从 .env 读取。
|
||||||
{
|
{
|
||||||
// ==============================================================================
|
|
||||||
// ☕ ARTEMIS Autonomous Agent & Framework Configuration
|
|
||||||
// ==============================================================================
|
|
||||||
|
|
||||||
// ------------------------------------------------------------------------------
|
|
||||||
// 1. Global Default Model (Applied to all sub-agents unless overridden below)
|
|
||||||
// ------------------------------------------------------------------------------
|
|
||||||
"default": {
|
"default": {
|
||||||
"provider": "google",
|
"provider": "openai",
|
||||||
"model": "gemini-3.8-flash",
|
"model": "${env:OPENAI_MODEL:-gpt-5-mini}",
|
||||||
"thinking_level": "medium",
|
"timeout": "${env:OPENAI_TIMEOUT_SECONDS:-120}",
|
||||||
|
"max_retries": "${env:OPENAI_MAX_RETRIES:-0}",
|
||||||
"fallback": {
|
"fallback": {
|
||||||
"provider": "google",
|
"provider": "openai",
|
||||||
"model": "gemini-3.7-flash",
|
"model": "${env:OPENAI_MODEL:-gpt-5-mini}"
|
||||||
"thinking_level": "medium"
|
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"presets": {},
|
||||||
// ------------------------------------------------------------------------------
|
|
||||||
// 2. Ready-to-Use Presets (Switch easily in code or CLI)
|
|
||||||
// ------------------------------------------------------------------------------
|
|
||||||
"presets": {
|
|
||||||
"gemini-flagship": {
|
|
||||||
"provider": "google",
|
|
||||||
"model": "gemini-3.8-flash",
|
|
||||||
"fallback": { "provider": "google", "model": "gemini-3.7-flash" }
|
|
||||||
},
|
|
||||||
"gemini-flash": {
|
|
||||||
"provider": "google",
|
|
||||||
"model": "gemini-2.5-flash",
|
|
||||||
"fallback": { "provider": "google", "model": "gemini-2.0-flash" }
|
|
||||||
},
|
|
||||||
"openai-gpt4o": {
|
|
||||||
"provider": "openai",
|
|
||||||
"model": "gpt-4o",
|
|
||||||
"fallback": { "provider": "openai", "model": "gpt-4o-mini" }
|
|
||||||
},
|
|
||||||
"cost-saving": {
|
|
||||||
"provider": "google",
|
|
||||||
"model": "gemini-3.5-flash-lite",
|
|
||||||
"fallback": { "provider": "google", "model": "gemini-3.1-flash-lite" }
|
|
||||||
},
|
|
||||||
"local-ollama": {
|
|
||||||
"provider": "openai",
|
|
||||||
"model": "llama3.2-vision",
|
|
||||||
"fallback": { "provider": "openai", "model": "llama3.2-vision" }
|
|
||||||
}
|
|
||||||
},
|
|
||||||
|
|
||||||
// ------------------------------------------------------------------------------
|
|
||||||
// 3. Agent & Sub-Agent Node Customizations (Optional - Inherits from 'default')
|
|
||||||
// ------------------------------------------------------------------------------
|
|
||||||
"nodes": {
|
"nodes": {
|
||||||
// 🧠 Planner: High-level goal decomposition and strategic step planning
|
"planner": {},
|
||||||
"planner": {
|
"operator": {},
|
||||||
"thinking_level": "high"
|
|
||||||
},
|
|
||||||
|
|
||||||
// 🎯 Operator: Screen perception, element interaction, and direct execution
|
|
||||||
"operator": {
|
|
||||||
"thinking_level": "medium",
|
|
||||||
"include_thoughts": true
|
|
||||||
},
|
|
||||||
|
|
||||||
// 🎬 Video Analyzer: Multimodal screen recording verification and temporal analysis
|
|
||||||
// NOTE: Dual-path engine automatically routes:
|
|
||||||
// - Native Gemini Files API for Gemini models with Google API keys (long video file stream);
|
|
||||||
// Gemini 3.6+ Flash sub-agents use agentic video understanding by default (see video_analyzer.processing below)
|
|
||||||
// - Universal Multi-Model Engine for Claude 3.5/3.7, GPT-4o, OpenRouter, Vertex AI (dynamic keyframe extraction + multimodal tool calling)
|
|
||||||
"video_analyzer": {},
|
"video_analyzer": {},
|
||||||
|
"object_detector": {},
|
||||||
// 👁️ Object Detector & Explorer (Flash Mode): Visual coordinates grounding and bounding boxes
|
"explorer": {},
|
||||||
// CRITICAL REQUIREMENT: Standard LLMs (including standard Gemini Flash, GPT-4o, Claude 3.7) lack sub-pixel spatial coordinate fine-tuning.
|
|
||||||
// To achieve accurate [x, y] point and bounding box localization, this node MUST use specialized Gemini ER (Embodied Reasoning / Robotics) models:
|
|
||||||
// e.g., "gemini-robotics-er-2-preview". Non-ER models will fail spatial coordinate detection.
|
|
||||||
"object_detector": {
|
|
||||||
"model": "gemini-robotics-er-2-preview",
|
|
||||||
"fallback": {
|
|
||||||
"provider": "google",
|
|
||||||
"model": "gemini-robotics-er-2-preview"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
|
|
||||||
// 🔍 Explorer: Deep element grounding and multi-candidate visual search
|
|
||||||
// NOTE:
|
|
||||||
// - "flash" version delegates directly to object_detector (requires Gemini ER model above).
|
|
||||||
// - "pro" / "ultra" versions run multi-turn ReAct loops (supports both Native Gemini and Universal Multi-Model LangChain engines).
|
|
||||||
"explorer": {
|
|
||||||
"thinking_level": "medium"
|
|
||||||
},
|
|
||||||
|
|
||||||
// 📋 Summarizer: Aggregates step history and generates structured outcome
|
|
||||||
"summarizer": {},
|
"summarizer": {},
|
||||||
|
"hopper": {},
|
||||||
// 🔽 Hopper: Fast lightweight package name locator
|
|
||||||
"hopper": {
|
|
||||||
"model": "gemini-3.5-flash-lite",
|
|
||||||
"fallback": {
|
|
||||||
"provider": "google",
|
|
||||||
"model": "gemini-3.1-flash-lite"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
|
|
||||||
// 📝 Outputter: Synthesizes execution trace into final output report
|
|
||||||
"outputter": {},
|
"outputter": {},
|
||||||
|
"log_analyzer": {},
|
||||||
// 🪵 Log Analyzer: Error log diagnosis with search grounding
|
"diagnoser_expert": {},
|
||||||
"log_analyzer": {
|
|
||||||
"enable_grounding": true
|
|
||||||
},
|
|
||||||
|
|
||||||
// 🩺 Diagnoser Expert: Failure root cause diagnosis with search grounding
|
|
||||||
"diagnoser_expert": {
|
|
||||||
"enable_grounding": true
|
|
||||||
},
|
|
||||||
// 🏛️ Committee Council Members
|
|
||||||
"planner_avatar": {},
|
"planner_avatar": {},
|
||||||
"history_analyzer_expert": {},
|
"history_analyzer_expert": {},
|
||||||
// 🔎 Checker: Plan-declared checkpoint verification and exit final review.
|
"checker": {}
|
||||||
// Thoughts are streamed so the verifier's reasoning shows live in the timeline.
|
|
||||||
"checker": {
|
|
||||||
"thinking_level": "medium",
|
|
||||||
"include_thoughts": true
|
|
||||||
}
|
|
||||||
// 🎯 Planner Validation: Advisory review of milestone changes.
|
|
||||||
// Defaults to the same lightweight model as the pixel safety net
|
|
||||||
// (gemini-3.5-flash-lite, temperature 0); add "planner_validation": {...} here to override.
|
|
||||||
},
|
},
|
||||||
|
|
||||||
// ------------------------------------------------------------------------------
|
|
||||||
// 4. Runtime Mode & Agent Capabilities Configuration (Dual-Engine: Flash vs. Pro)
|
|
||||||
// ------------------------------------------------------------------------------
|
|
||||||
"agent": {
|
"agent": {
|
||||||
// ==========================================================================
|
|
||||||
// ⚡ 4.1 FLASH Profile Options (Reactive Single-Agent Lightweight Loop)
|
|
||||||
// ==========================================================================
|
|
||||||
// Best for: Simple, fast, deterministic tasks; runs unbounded by default.
|
|
||||||
// Execution: Observe-Think-Act single LLM reactive loop without graph overhead.
|
|
||||||
"flash": {
|
"flash": {
|
||||||
"max_turns": 0, // Maximum reactive turns before stopping (0 = unlimited; history is bounded by transcript compression, not by a turn cap)
|
"max_turns": 0,
|
||||||
"explorer_mode": "flash", // Explorer tier for the Flash runner: 'flash' | 'pro' | 'ultra' (see top-level "explorer")
|
"explorer_mode": "flash",
|
||||||
// 🧠 Step Summarizer: Asynchronous visual context compressor
|
|
||||||
// Summarizes intermediate steps in background without blocking the runner,
|
|
||||||
// replacing images only after strictly objective, high-density summaries are ready.
|
|
||||||
"step_summarizer": {
|
"step_summarizer": {
|
||||||
"enabled": true, // Whether background step summarization is active
|
"enabled": "${env:ARTEMIS_STEP_SUMMARIZER_ENABLED:-true}",
|
||||||
"model": "gemini-3.5-flash-lite", // Lightweight model for fast background inference
|
"model": "${env:OPENAI_MODEL:-gpt-5-mini}",
|
||||||
"prune_history_xml": true // Prune outdated heavy XML trees from historical steps
|
"prune_history_xml": true
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
// ==========================================================================
|
|
||||||
// 🚀 4.2 PRO Profile Options (Deep Multi-Agent Graph Closed-Loop)
|
|
||||||
// ==========================================================================
|
|
||||||
// Best for: Complex, exploratory, dynamic multi-branch tasks.
|
|
||||||
// Execution: LangGraph multi-node graph with a living task plan, a pre-execution safety net (single actions) / fast-action bursts (multi-action turns),
|
|
||||||
// execution-incident recovery handled by the Operator itself (no separate repair agent), and a read-only Checker for checkpoint verification + exit final review.
|
|
||||||
"pro": {
|
"pro": {
|
||||||
// 🧭 Explorer: Perception sub-agent under Pro profile ('flash', 'pro', or 'ultra')
|
|
||||||
"explorer": {
|
"explorer": {
|
||||||
"mode": "flash" // Explorer tier for Pro nodes (Operator/Validator): 'flash' | 'pro' | 'ultra'
|
"mode": "flash"
|
||||||
},
|
},
|
||||||
// 🎯 Planner Validation: Async verification when task plan milestones change
|
|
||||||
"planner_validation": {
|
"planner_validation": {
|
||||||
"enabled": true // On by default: every milestone text change is reviewed (advisory: hint + reason only, never rolled back)
|
"enabled": true
|
||||||
},
|
},
|
||||||
// 🏛️ Committee: Multi-Agent Council debate tool for Operator
|
|
||||||
"committee": {
|
"committee": {
|
||||||
"enabled": false,
|
"enabled": false,
|
||||||
"debate_rounds": 2
|
"debate_rounds": 2
|
||||||
},
|
},
|
||||||
// 🔎 Checker: Plan-declared checkpoint verification (optimistic async) + exit final review
|
|
||||||
"checker": {
|
"checker": {
|
||||||
"enabled": true, // Master switch (compat alias): false disables both gates below
|
"enabled": true,
|
||||||
"midway_checks": false, // Plan-declared midway checkpoints (off in the factory layering)
|
"midway_checks": false,
|
||||||
"final_check": true, // Exit final review against the user's original goal (on by default)
|
"final_check": true,
|
||||||
"max_iterations": 20,
|
"max_iterations": 20,
|
||||||
"final_check_max_attempts": 3,
|
"final_check_max_attempts": 3,
|
||||||
"checkpoint_max_repairs": 2,
|
"checkpoint_max_repairs": 2,
|
||||||
"max_concurrent_checkpoints": 3,
|
"max_concurrent_checkpoints": 3,
|
||||||
"checkpoint_timeout": 180.0,
|
"checkpoint_timeout": "${env:ARTEMIS_CHECKPOINT_TIMEOUT_SECONDS:-180}",
|
||||||
"settlement_timeout": 120.0,
|
"settlement_timeout": "${env:ARTEMIS_SETTLEMENT_TIMEOUT_SECONDS:-120}",
|
||||||
"assert_failure_policy": "continue", // 'continue' or 'halt' on assert failures
|
"assert_failure_policy": "continue",
|
||||||
"device_probes": true // Checker's enumerated read-only device probes
|
"device_probes": true
|
||||||
},
|
},
|
||||||
// 🎬 Video Analyzer & Screen Recording: Continuous screen recording via scrcpy and temporal analysis
|
|
||||||
"video_analyzer": {
|
"video_analyzer": {
|
||||||
"enabled": true, // Whether continuous video recording and video tools are active (default: auto-detected)
|
"enabled": "${env:ARTEMIS_VIDEO_ENABLED:-true}",
|
||||||
"enable_ledger": true
|
"enable_ledger": true
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
// ==========================================================================
|
|
||||||
// 🌐 4.3 Shared Capabilities & Global Overrides
|
|
||||||
// ==========================================================================
|
|
||||||
// Shared step memory and history compression
|
|
||||||
// Transcript ledger + layered compression for the Pro operator.
|
|
||||||
"memory": {
|
"memory": {
|
||||||
// Background summarization runtime (visual/capsule lenses)
|
|
||||||
"runtime": {
|
"runtime": {
|
||||||
"max_concurrency": 2, // 1→2 per 2026-09-01 baseline: serial queue pushed summary-ready P90 past the scrub window
|
"max_concurrency": 2,
|
||||||
"retry_limit": 3,
|
"retry_limit": 3,
|
||||||
"flush_timeout_s": 30
|
"flush_timeout_s": 30
|
||||||
},
|
},
|
||||||
"transcript": {
|
"transcript": {
|
||||||
// ROLLBACK SWITCH: set false to restore the legacy per-turn 2-message
|
|
||||||
// prompt path byte-for-byte (L2/L3 chunk compression rides this same
|
|
||||||
// flag). See services/token_meter.py for measured session token usage
|
|
||||||
// and cache-hit ratios.
|
|
||||||
"enabled": true,
|
"enabled": true,
|
||||||
"context_budget_tokens": 80000,
|
"context_budget_tokens": 80000,
|
||||||
// Below start_ratio of the operator's own measured prompt size, closed
|
|
||||||
// segments only prepare their summary; the original turns are kept in
|
|
||||||
// context and replaced once occupancy reaches this ratio. Measurements
|
|
||||||
// come from the operator; other agents' calls do not affect this gate.
|
|
||||||
"start_ratio": 0.35,
|
"start_ratio": 0.35,
|
||||||
"soft_ratio": 0.7,
|
"soft_ratio": 0.7,
|
||||||
"hard_ratio": 0.9,
|
"hard_ratio": 0.9,
|
||||||
"min_active_steps": 5,
|
"min_active_steps": 5,
|
||||||
// Screenshot edge: 6 while the operator's measured context is below
|
|
||||||
// start_ratio (nothing is being replaced yet), 3 from there up. A
|
|
||||||
// screenshot reaching the edge is replaced in place by its visual
|
|
||||||
// summary (the 'Current Screenshot' label leaves with it) and never
|
|
||||||
// restored; the depth only decides how long new ones stay.
|
|
||||||
"image_scrub_depth": 3,
|
"image_scrub_depth": 3,
|
||||||
"image_scrub_depth_relaxed": 6,
|
"image_scrub_depth_relaxed": 6,
|
||||||
"pending_grace_steps": 3,
|
"pending_grace_steps": 3,
|
||||||
// Text edge: once a newer observation exists (depth 1) the UI element
|
|
||||||
// list, the plan recitation and the per-turn ephemeral blocks
|
|
||||||
// (reminders, hints, guidance wrappers) leave the older observation.
|
|
||||||
// User instruction bodies remain in history. The text edge stays
|
|
||||||
// shallow on purpose: only the live observation carries an indexed
|
|
||||||
// element list, so a stale [n] index can never be picked up as a
|
|
||||||
// target.
|
|
||||||
"xml_scrub_depth": 1,
|
"xml_scrub_depth": 1,
|
||||||
// Local historical-state hint (no model call); distance calibrated
|
|
||||||
// 2026-09-01 from 460 dHash-stamped on-device steps (same screen <=4,
|
|
||||||
// different screens >=7).
|
|
||||||
"similarity_hint": true,
|
"similarity_hint": true,
|
||||||
"similarity_max_distance": 5
|
"similarity_max_distance": 5
|
||||||
},
|
},
|
||||||
// Segment (L2) chunk compression: three-band chunks, era folding
|
|
||||||
"chunking": {
|
"chunking": {
|
||||||
"max_steps": 12,
|
"max_steps": 12,
|
||||||
// Size/pressure closes wait until at least this many turns are
|
|
||||||
// eligible, so heavy turns do not become one-step chunks (one capsule
|
|
||||||
// call each); milestone closes are exempt, the hard line waives it.
|
|
||||||
"min_steps": 3,
|
"min_steps": 3,
|
||||||
"target_source_tokens": 2000,
|
"target_source_tokens": 2000,
|
||||||
"model": "gemini-3.8-flash",
|
"model": "${env:OPENAI_MODEL:-gpt-5-mini}",
|
||||||
"max_chunks": 8
|
"max_chunks": 8
|
||||||
// "max_eras": 8 // independent era cap; omitted = follows max_chunks
|
|
||||||
},
|
},
|
||||||
// search_history: deterministic keyword / step-range lookup over stored history
|
|
||||||
"recall": {
|
"recall": {
|
||||||
"enabled": true,
|
"enabled": true,
|
||||||
"max_results": 5,
|
"max_results": 5,
|
||||||
"max_text_tokens": 2000,
|
"max_text_tokens": 2000,
|
||||||
"screen_scan_steps": 150
|
"screen_scan_steps": 150
|
||||||
},
|
},
|
||||||
// replay_steps: full step replays (whole trailing steps are dropped over budget)
|
|
||||||
"replay": {
|
"replay": {
|
||||||
"max_steps": 5,
|
"max_steps": 5,
|
||||||
"max_tokens": 12000
|
"max_tokens": 12000
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
// 🎬 Screen Recording & Video Analyzer Configuration
|
|
||||||
"video_analyzer": {
|
"video_analyzer": {
|
||||||
"enabled": true, // Whether continuous screen recording via scrcpy is active (default: auto-detected)
|
"enabled": "${env:ARTEMIS_VIDEO_ENABLED:-true}",
|
||||||
"enable_ledger": true
|
"enable_ledger": true
|
||||||
// "processing": "auto", // How Gemini sub-agents watch a clip: auto (agentic on Gemini 3.6+ Flash, static elsewhere) | agentic | static
|
|
||||||
// "agentic_chunk_size_seconds": 600, // Longest clip one agentic sub-agent takes; static mode keeps chunk_size_seconds (60)
|
|
||||||
// "agentic_call_timeout_seconds": 300 // Hard timeout for one agentic sub-agent response
|
|
||||||
},
|
},
|
||||||
// 📝 Outputter: Post-execution report synthesis and structured schema extraction
|
|
||||||
// Reviews action history, notes, and verifies goal achievement to synthesize final output.
|
|
||||||
"outputter": {
|
"outputter": {
|
||||||
"enabled": true, // Whether Outputter is mounted to synthesize final outputs
|
"enabled": "${env:ARTEMIS_OUTPUTTER_ENABLED:-true}",
|
||||||
"force_synthesis": false // Force Outputter synthesis even without explicit output schema
|
"force_synthesis": false
|
||||||
},
|
},
|
||||||
// 🧭 Explorer tier: the perception sub-agent behind `ask_explorer`.
|
|
||||||
// Tiers are a user setting; calling agents never see which tier is active.
|
|
||||||
// flash - one-shot visual detection on the screenshot (fastest)
|
|
||||||
// pro - 3-turn reasoning loop: UI-tree search + coordinate audit + detection
|
|
||||||
// ultra - 8-turn loop adding zoom, OCR and pixel-level image processing (slowest)
|
|
||||||
// `flash.explorer_mode` and `pro.explorer.mode` above are the per-profile knobs;
|
|
||||||
// `caching` (Gemini explicit context caching) defaults per tier: off for pro, on for ultra.
|
|
||||||
"explorer": {
|
"explorer": {
|
||||||
"default_version": "flash" // Fallback tier when no profile knob applies
|
"default_version": "flash"
|
||||||
// "caching": true, // Uncomment to force context caching for pro/ultra
|
|
||||||
},
|
},
|
||||||
// Advanced: per-agent tier override, e.g. {"validator": "ultra"}. Takes
|
|
||||||
// precedence over the profile knobs; leave empty to let them decide.
|
|
||||||
"explorer_versions": {},
|
"explorer_versions": {},
|
||||||
// 🚫 Per-agent Tool Denylists
|
|
||||||
"denylisted_tools": {
|
"denylisted_tools": {
|
||||||
"explorer": [] // Perception tools to withhold from the Explorer, e.g. "ask_image_processor"
|
"explorer": []
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -15,7 +15,7 @@
|
|||||||
limitations under the License.
|
limitations under the License.
|
||||||
-->
|
-->
|
||||||
|
|
||||||
<html lang="en" data-beasties-container>
|
<html lang="zh-CN" data-beasties-container>
|
||||||
<head>
|
<head>
|
||||||
<meta charset="utf-8">
|
<meta charset="utf-8">
|
||||||
<title>artemis</title>
|
<title>artemis</title>
|
||||||
@@ -23,15 +23,15 @@
|
|||||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||||
<link rel="preconnect" href="https://fonts.googleapis.com">
|
<link rel="preconnect" href="https://fonts.googleapis.com">
|
||||||
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
|
||||||
<style>@font-face{font-family:'Material Symbols Outlined';font-style:normal;font-weight:100 700;src:url(https://fonts.gstatic.com/s/materialsymbolsoutlined/v368/kJEhBvYX7BgnkSrUwT8OhrdQw4oELdPIeeII9v6oFsI.woff2) format('woff2');}.material-symbols-outlined{font-family:'Material Symbols Outlined';font-weight:normal;font-style:normal;font-size:24px;line-height:1;letter-spacing:normal;text-transform:none;display:inline-block;white-space:nowrap;word-wrap:normal;direction:ltr;-webkit-font-feature-settings:'liga';-webkit-font-smoothing:antialiased;}</style>
|
<style>@font-face{font-family:'Material Symbols Outlined';font-style:normal;font-weight:100 700;src:url(https://fonts.gstatic.com/s/materialsymbolsoutlined/v374/kJEhBvYX7BgnkSrUwT8OhrdQw4oELdPIeeII9v6oFsI.woff2) format('woff2');}.material-symbols-outlined{font-family:'Material Symbols Outlined';font-weight:normal;font-style:normal;font-size:24px;line-height:1;letter-spacing:normal;text-transform:none;display:inline-block;white-space:nowrap;word-wrap:normal;direction:ltr;-webkit-font-feature-settings:'liga';-webkit-font-smoothing:antialiased;}</style>
|
||||||
<style>@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format('woff2');unicode-range:U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format('woff2');unicode-range:U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format('woff2');unicode-range:U+1F00-1FFF;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format('woff2');unicode-range:U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format('woff2');unicode-range:U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format('woff2');unicode-range:U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format('woff2');unicode-range:U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format('woff2');unicode-range:U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format('woff2');unicode-range:U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format('woff2');unicode-range:U+1F00-1FFF;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format('woff2');unicode-range:U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format('woff2');unicode-range:U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format('woff2');unicode-range:U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format('woff2');unicode-range:U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;}@font-face{font-family:'Inter';font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format('woff2');unicode-range:U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;}@font-face{font-family:'Inter';font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format('woff2');unicode-range:U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;}@font-face{font-family:'Inter';font-style:normal;font-weight:500;font-display:swap;src:url(https://fLine truncated
|
<style>@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format('woff2');unicode-range:U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format('woff2');unicode-range:U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format('woff2');unicode-range:U+1F00-1FFF;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format('woff2');unicode-range:U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format('woff2');unicode-range:U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format('woff2');unicode-range:U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;}@font-face{font-family:'Inter';font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format('woff2');unicode-range:U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format('woff2');unicode-range:U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format('woff2');unicode-range:U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format('woff2');unicode-range:U+1F00-1FFF;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format('woff2');unicode-range:U+0370-0377, U+037A-037F, U+0384-038A, U+038C, U+038E-03A1, U+03A3-03FF;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format('woff2');unicode-range:U+0102-0103, U+0110-0111, U+0128-0129, U+0168-0169, U+01A0-01A1, U+01AF-01B0, U+0300-0301, U+0303-0304, U+0308-0309, U+0323, U+0329, U+1EA0-1EF9, U+20AB;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format('woff2');unicode-range:U+0100-02BA, U+02BD-02C5, U+02C7-02CC, U+02CE-02D7, U+02DD-02FF, U+0304, U+0308, U+0329, U+1D00-1DBF, U+1E00-1E9F, U+1EF2-1EFF, U+2020, U+20A0-20AB, U+20AD-20C0, U+2113, U+2C60-2C7F, U+A720-A7FF;}@font-face{font-family:'Inter';font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format('woff2');unicode-range:U+0000-00FF, U+0131, U+0152-0153, U+02BB-02BC, U+02C6, U+02DA, U+02DC, U+0304, U+0308, U+0329, U+2000-206F, U+20AC, U+2122, U+2191, U+2193, U+2212, U+2215, U+FEFF, U+FFFD;}@font-face{font-family:'Inter';font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format('woff2');unicode-range:U+0460-052F, U+1C80-1C8A, U+20B4, U+2DE0-2DFF, U+A640-A69F, U+FE2E-FE2F;}@font-face{font-family:'Inter';font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format('woff2');unicode-range:U+0301, U+0400-045F, U+0490-0491, U+04B0-04B1, U+2116;}@font-face{font-family:'Inter';font-style:normal;font-weight:500;font-display:swap;src:url(https://fLine truncated
|
||||||
<link rel="icon" type="image/png" sizes="192x192" href="/logo.png?v=2">
|
<link rel="icon" type="image/png" sizes="192x192" href="/logo.png?v=2">
|
||||||
<link rel="icon" type="image/png" sizes="32x32" href="/logo.png?v=2">
|
<link rel="icon" type="image/png" sizes="32x32" href="/logo.png?v=2">
|
||||||
<link rel="icon" type="image/svg+xml" href="/logo.svg?v=2">
|
<link rel="icon" type="image/svg+xml" href="/logo.svg?v=2">
|
||||||
<link rel="shortcut icon" href="/favicon.ico?v=2">
|
<link rel="shortcut icon" href="/favicon.ico?v=2">
|
||||||
<link rel="apple-touch-icon" href="/logo.png?v=2">
|
<link rel="apple-touch-icon" href="/logo.png?v=2">
|
||||||
<style>@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:500;Line truncated
|
<style>@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:500;Line truncated
|
||||||
<body>
|
<body>
|
||||||
<app-root></app-root>
|
<app-root></app-root>
|
||||||
<script src="polyfills-5CFQRCPP.js" type="module"></script><script src="main-BGAFACUZ.js" type="module"></script></body>
|
<script src="polyfills-5CFQRCPP.js" type="module"></script><script src="main-WGGFA77O.js" type="module"></script></body>
|
||||||
</html>
|
</html>
|
||||||
@@ -1,18 +0,0 @@
|
|||||||
var aO=Object.defineProperty,sO=Object.defineProperties;var cO=Object.getOwnPropertyDescriptors;var vm=Object.getOwnPropertySymbols;var lO=Object.prototype.hasOwnProperty,dO=Object.prototype.propertyIsEnumerable;var ym=(n,t,e)=>t in n?aO(n,t,{enumerable:!0,configurable:!0,writable:!0,value:e}):n[t]=e,S=(n,t)=>{for(var e in t||={})lO.call(t,e)&&ym(n,e,t[e]);if(vm)for(var e of vm(t))dO.call(t,e)&&ym(n,e,t[e]);return n},F=(n,t)=>sO(n,cO(t));var be=(n,t,e)=>new Promise((o,i)=>{var r=c=>{try{s(e.next(c))}catch(p){i(p)}},a=c=>{try{s(e.throw(c))}catch(p){i(p)}},s=c=>c.done?o(c.value):Promise.resolve(c.value).then(r,a);s((e=e.apply(n,t)).next())});var mt=null,Ds=!1,$o=1,uO=null,nt=Symbol("SIGNAL");function Y(n){let t=mt;return mt=n,t}function Is(){return mt}var po={version:0,lastCleanEpoch:0,dirty:!1,producers:void 0,producersTail:void 0,consumers:void 0,consumersTail:void 0,recomputing:!1,consumerAllowSignalWrites:!1,consumerIsAlwaysLive:!1,kind:"unknown",producerMustRecompute:()=>!1,producerRecomputeValue:()=>{},consumerMarkedDirty:()=>{},consumerOnSignalRead:()=>{}};function Ho(n){if(Ds)throw new Error("");if(mt===null)return;mt.consumerOnSignalRead(n);let t=mt.producersTail;if(t!==void 0&&t.producer===n)return;let e,o=mt.recomputing;if(o&&(e=t!==void 0?t.nextProducer:mt.producers,e!==void 0&&e.producer===n)){mt.producersTail=e,e.lastReadVersion=n.version,e.knownValidAtEpoch=$o;return}let i=n.consumersTail;if(i!==void 0&&i.consumer===mt&&(!o||i.knownValidAtEpoch===$o))return;let r=$i(mt),a={producer:n,consumer:mt,nextProducer:e,prevConsumer:void 0,knownValidAtEpoch:$o,lastReadVersion:n.version,nextConsumer:void 0};mt.producersTail=a,t!==void 0?t.nextProducer=a:mt.producers=a,r&&km(n,a)}function xm(){$o++}function Uo(n){if(!($i(n)&&!n.dirty)&&!(!n.dirty&&n.lastCleanEpoch===$o)){if(!n.producerMustRecompute(n)&&!Wr(n)){ji(n);return}n.producerRecomputeValue(n),ji(n)}}function eu(n){if(n.consumers===void 0)return;let t=Ds;Ds=!0;try{for(let e=n.consumers;e!==void 0;e=e.nextConsumer){let o=e.consumer;o.dirty||gO(o)}}finally{Ds=t}}function tu(){return mt?.consumerAllowSignalWrites!==!1}function gO(n){n.dirty=!0,eu(n),n.consumerMarkedDirty?.(n)}function ji(n){n.dirty=!1,n.lastCleanEpoch=$o}function fo(n){return n&&wm(n),Y(n)}function wm(n){if(n.producersTail?.knownValidAtEpoch===$o){let t=n.producers;for(;t!==void 0;)t.knownValidAtEpoch=null,t=t.nextProducer}n.producersTail=void 0,n.recomputing=!0}function Go(n,t){Y(t),n&&Sm(n)}function Sm(n){n.recomputing=!1;let t=n.producersTail,e=t!==void 0?t.nextProducer:n.producers;if(e!==void 0){if($i(n))do e=nu(e);while(e!==void 0);t!==void 0?t.nextProducer=void 0:n.producers=void 0}}function Wr(n){for(let t=n.producers;t!==void 0;t=t.nextProducer){let e=t.producer,o=t.lastReadVersion;if(o!==e.version||(Uo(e),o!==e.version))return!0}return!1}function qo(n){if($i(n)){let t=n.producers;for(;t!==void 0;)t=nu(t)}n.producers=void 0,n.producersTail=void 0,n.consumers=void 0,n.consumersTail=void 0}function km(n,t){let e=n.consumersTail,o=$i(n);if(e!==void 0?(t.nextConsumer=e.nextConsumer,e.nextConsumer=t):(t.nextConsumer=void 0,n.consumers=t),t.prevConsumer=e,n.consumersTail=t,!o)for(let i=n.producers;i!==void 0;i=i.nextProducer)km(i.producer,i)}function nu(n){let t=n.producer,e=n.nextProducer,o=n.nextConsumer,i=n.prevConsumer;if(n.nextConsumer=void 0,n.prevConsumer=void 0,o!==void 0?o.prevConsumer=i:t.consumersTail=i,i!==void 0)i.nextConsumer=o;else if(t.consumers=o,!$i(t)){let r=t.producers;for(;r!==void 0;)r=nu(r)}return e}function $i(n){return n.consumerIsAlwaysLive||n.consumers!==void 0}function Kr(n){uO?.(n)}function Yr(n,t){return Object.is(n,t)}function Ts(n,t){let e=Object.create(pO);e.computation=n,t!==void 0&&(e.equal=t);let o=()=>{if(Uo(e),Ho(e),e.value===Cn)throw e.error;return e.value};return o[nt]=e,Kr(e),o}var zo=Symbol("UNSET"),Bo=Symbol("COMPUTING"),Cn=Symbol("ERRORED"),pO=F(S({},po),{value:zo,dirty:!0,error:null,equal:Yr,kind:"computed",producerMustRecompute(n){return n.value===zo||n.value===Bo},producerRecomputeValue(n){if(n.value===Bo)throw new Error("");let t=n.value;n.value=Bo;let e=fo(n),o,i=!1;try{o=n.computation(),Y(null),i=t!==zo&&t!==Cn&&o!==Cn&&n.equal(t,o)}catch(r){o=Cn,n.error=r}finally{Go(n,e)}if(i){n.value=t;return}n.value=o,n.version++}});function fO(){throw new Error}var Em=fO;function Dm(n){Em(n)}function ou(n){Em=n}var mO=null;function iu(n,t){let e=Object.create(Ns);e.value=n,t!==void 0&&(e.equal=t);let o=()=>Im(e);return o[nt]=e,Kr(e),[o,a=>Wo(e,a),a=>As(e,a)]}function Im(n){return Ho(n),n.value}function Wo(n,t){tu()||Dm(n),n.equal(n.value,t)||(n.value=t,_O(n))}function As(n,t){tu()||Dm(n),Wo(n,t(n.value))}var Ns=F(S({},po),{equal:Yr,value:void 0,kind:"signal"});function _O(n){n.version++,xm(),eu(n),mO?.(n)}var ru=F(S({},po),{consumerIsAlwaysLive:!0,consumerAllowSignalWrites:!0,dirty:!0,kind:"effect"});function au(n){if(n.dirty=!1,n.version>0&&!Wr(n))return;n.version++;let t=fo(n);try{n.cleanup(),n.fn()}finally{Go(n,t)}}var su;function Rs(){return
Line truncated
|
|
||||||
${e.map((o,i)=>`${i+1}) ${o.toString()}`).join(`
|
|
||||||
`)}`:"",this.name="UnsubscriptionError",this.errors=e});function Zr(n,t){if(n){let e=n.indexOf(t);0<=e&&n.splice(e,1)}}var Be=class n{constructor(t){this.initialTeardown=t,this.closed=!1,this._parentage=null,this._finalizers=null}unsubscribe(){let t;if(!this.closed){this.closed=!0;let{_parentage:e}=this;if(e)if(this._parentage=null,Array.isArray(e))for(let r of e)r.remove(this);else e.remove(this);let{initialTeardown:o}=this;if(ae(o))try{o()}catch(r){t=r instanceof Fs?r.errors:[r]}let{_finalizers:i}=this;if(i){this._finalizers=null;for(let r of i)try{Rm(r)}catch(a){t=t??[],a instanceof Fs?t=[...t,...a.errors]:t.push(a)}}if(t)throw new Fs(t)}}add(t){var e;if(t&&t!==this)if(this.closed)Rm(t);else{if(t instanceof n){if(t.closed||t._hasParent(this))return;t._addParent(this)}(this._finalizers=(e=this._finalizers)!==null&&e!==void 0?e:[]).push(t)}}_hasParent(t){let{_parentage:e}=this;return e===t||Array.isArray(e)&&e.includes(t)}_addParent(t){let{_parentage:e}=this;this._parentage=Array.isArray(e)?(e.push(t),e):e?[e,t]:t}_removeParent(t){let{_parentage:e}=this;e===t?this._parentage=null:Array.isArray(e)&&Zr(e,t)}remove(t){let{_finalizers:e}=this;e&&Zr(e,t),t instanceof n&&t._removeParent(this)}};Be.EMPTY=(()=>{let n=new Be;return n.closed=!0,n})();var du=Be.EMPTY;function Ls(n){return n instanceof Be||n&&"closed"in n&&ae(n.remove)&&ae(n.add)&&ae(n.unsubscribe)}function Rm(n){ae(n)?n():n.unsubscribe()}var en={onUnhandledError:null,onStoppedNotification:null,Promise:void 0,useDeprecatedSynchronousErrorHandling:!1,useDeprecatedNextContext:!1};var Hi={setTimeout(n,t,...e){let{delegate:o}=Hi;return o?.setTimeout?o.setTimeout(n,t,...e):setTimeout(n,t,...e)},clearTimeout(n){let{delegate:t}=Hi;return(t?.clearTimeout||clearTimeout)(n)},delegate:void 0};function Vs(n){Hi.setTimeout(()=>{let{onUnhandledError:t}=en;if(t)t(n);else throw n})}function Qr(){}var Fm=uu("C",void 0,void 0);function Lm(n){return uu("E",void 0,n)}function Vm(n){return uu("N",n,void 0)}function uu(n,t,e){return{kind:n,value:t,error:e}}var Ko=null;function Ui(n){if(en.useDeprecatedSynchronousErrorHandling){let t=!Ko;if(t&&(Ko={errorThrown:!1,error:null}),n(),t){let{errorThrown:e,error:o}=Ko;if(Ko=null,e)throw o}}else n()}function jm(n){en.useDeprecatedSynchronousErrorHandling&&Ko&&(Ko.errorThrown=!0,Ko.error=n)}var Yo=class extends Be{constructor(t){super(),this.isStopped=!1,t?(this.destination=t,Ls(t)&&t.add(this)):this.destination=bO}static create(t,e,o){return new zn(t,e,o)}next(t){this.isStopped?pu(Vm(t),this):this._next(t)}error(t){this.isStopped?pu(Lm(t),this):(this.isStopped=!0,this._error(t))}complete(){this.isStopped?pu(Fm,this):(this.isStopped=!0,this._complete())}unsubscribe(){this.closed||(this.isStopped=!0,super.unsubscribe(),this.destination=null)}_next(t){this.destination.next(t)}_error(t){try{this.destination.error(t)}finally{this.unsubscribe()}}_complete(){try{this.destination.complete()}finally{this.unsubscribe()}}},CO=Function.prototype.bind;function gu(n,t){return CO.call(n,t)}var fu=class{constructor(t){this.partialObserver=t}next(t){let{partialObserver:e}=this;if(e.next)try{e.next(t)}catch(o){js(o)}}error(t){let{partialObserver:e}=this;if(e.error)try{e.error(t)}catch(o){js(o)}else js(t)}complete(){let{partialObserver:t}=this;if(t.complete)try{t.complete()}catch(e){js(e)}}},zn=class extends Yo{constructor(t,e,o){super();let i;if(ae(t)||!t)i={next:t??void 0,error:e??void 0,complete:o??void 0};else{let r;this&&en.useDeprecatedNextContext?(r=Object.create(t),r.unsubscribe=()=>this.unsubscribe(),i={next:t.next&&gu(t.next,r),error:t.error&&gu(t.error,r),complete:t.complete&&gu(t.complete,r)}):i=t}this.destination=new fu(i)}};function js(n){en.useDeprecatedSynchronousErrorHandling?jm(n):Vs(n)}function MO(n){throw n}function pu(n,t){let{onStoppedNotification:e}=en;e&&Hi.setTimeout(()=>e(n,t))}var bO={closed:!0,next:Qr,error:MO,complete:Qr};var Gi=typeof Symbol=="function"&&Symbol.observable||"@@observable";function tn(n){return n}function $s(...n){return mu(n)}function mu(n){return n.length===0?tn:n.length===1?n[0]:function(e){return n.reduce((o,i)=>i(o),e)}}var ie=class n{constructor(t){t&&(this._subscribe=t)}lift(t){let e=new n;return e.source=this,e.operator=t,e}subscribe(t,e,o){let i=OO(t)?t:new zn(t,e,o);return Ui(()=>{let{operator:r,source:a}=this;i.add(r?r.call(i,a):a?this._subscribe(i):this._trySubscribe(i))}),i}_trySubscribe(t){try{return this._subscribe(t)}catch(e){t.error(e)}}forEach(t,e){return e=$m(e),new e((o,i)=>{let r=new zn({next:a=>{try{t(a)}catch(s){i(s),r.unsubscribe()}},error:i,complete:o});this.subscribe(r)})}_subscribe(t){var e;return(e=this.source)===null||e===void 0?void 0:e.subscribe(t)}[Gi](){return this}pipe(...t){return mu(t)(this)}toPromise(t){return t=$m(t),new t((e,o)=>{let i;this.subscribe(r=>i=r,r=>o(r),()=>e(i))})}};ie.create=n=>new ie(n);function $m(n){var t;return(t=n??en.Promise)!==null&&t!==void 0?t:Promise}function PO(n){return n&&ae(n.next)&&ae(n.error)&&ae(n.complete)}function OO(n){return n&&nLine truncated
|
|
||||||
`);return o>=0?e.slice(0,o):e}function fc(n,t){return n?t?`${n} ${t}`:n:t||""}var zO=he({__forward_ref__:he});function on(n){return n.__forward_ref__=on,n}function lt(n){return Lu(n)?n():n}function Lu(n){return typeof n=="function"&&Object.hasOwn(n,zO)&&n.__forward_ref__===on}function te(n){return{token:n.token,providedIn:n.providedIn||null,factory:n.factory,value:void 0}}function Pn(n){return{providers:n.providers||[],imports:n.imports||[]}}function sa(n){return BO(n,mc)}function Vu(n){return sa(n)!==null}function BO(n,t){return Object.hasOwn(n,t)&&n[t]||null}function HO(n){let t=n?.[mc]??null;return t||null}function xu(n){return n&&Object.hasOwn(n,cc)?n[cc]:null}var mc=he({\u0275prov:he}),cc=he({\u0275inj:he}),B=class{_desc;ngMetadataName="InjectionToken";\u0275prov;constructor(t,e){this._desc=t,this.\u0275prov=void 0,typeof e=="number"?this.__NG_ELEMENT_ID__=e:e!==void 0&&(this.\u0275prov=te({token:this,providedIn:e.providedIn||"root",factory:e.factory}))}get multi(){return this}toString(){return`InjectionToken ${this._desc}`}};function ju(n){return n&&!!n.\u0275providers}var ca=he({\u0275cmp:he}),la=he({\u0275dir:he}),$u=he({\u0275pipe:he}),zu=he({\u0275mod:he}),na=he({\u0275fac:he}),oi=he({__NG_ELEMENT_ID__:he}),s_=he({__NG_ENV_ID__:he});function p_(n){return _c(n,"@NgModule"),n[zu]||null}function ii(n){return _c(n,"@Component"),n[ca]||null}function Bu(n){return _c(n,"@Directive"),n[la]||null}function f_(n){return _c(n,"@Pipe"),n[$u]||null}function _c(n,t){if(n==null)throw new z(-919,!1)}function rn(n){return typeof n=="string"?n:n==null?"":String(n)}var m_=he({ngErrorCode:he}),UO=he({ngErrorMessage:he}),GO=he({ngTokenPath:he});function Hu(n,t){return __("",-200,t)}function hc(n,t){throw new z(-201,!1)}function __(n,t,e){let o=new z(t,n);return o[m_]=t,o[UO]=n,e&&(o[GO]=e),o}function qO(n){return n[m_]}var wu;function h_(){return wu}function _t(n){let t=wu;return wu=n,t}function Uu(n,t,e){let o=sa(n);if(o&&o.providedIn=="root")return o.value===void 0?o.value=o.factory():o.value;if(e&8)return null;if(t!==void 0)return t;hc(n,"")}var an=globalThis;var WO={},Xo=WO,KO="__NG_DI_FLAG__",Su=class{injector;constructor(t){this.injector=t}retrieve(t,e){let o=ei(e)||0;try{return this.injector.get(t,o&8?null:Xo,o)}catch(i){if(zi(i))return i;throw i}}};function YO(n,t=0){let e=Rs();if(e===void 0)throw new z(-203,!1);if(e===null)return Uu(n,void 0,t);{let o=ZO(t),i=e.retrieve(n,o);if(zi(i)){if(o.optional)return null;throw i}return i}}function Q(n,t=0){return(h_()||YO)(lt(n),t)}function D(n,t){return Q(n,ei(t))}function ei(n){return typeof n>"u"||typeof n=="number"?n:0|(n.optional&&8)|(n.host&&1)|(n.self&&2)|(n.skipSelf&&4)}function ZO(n){return{optional:!!(n&8),host:!!(n&1),self:!!(n&2),skipSelf:!!(n&4)}}function ku(n){let t=[];for(let e=0;e<n.length;e++){let o=lt(n[e]);if(Array.isArray(o)){if(o.length===0)throw new z(900,!1);let i,r=0;for(let a=0;a<o.length;a++){let s=o[a],c=QO(s);typeof c=="number"?c===-1?i=s.token:r|=c:i=s}t.push(Q(i,r))}else t.push(Q(o))}return t}function QO(n){return n[KO]}function Co(n,t){let e=Object.hasOwn(n,na);return e?n[na]:null}function C_(n,t,e){if(n.length!==t.length)return!1;for(let o=0;o<n.length;o++){let i=n[o],r=t[o];if(e&&(i=e(i),r=e(r)),r!==i)return!1}return!0}function M_(n){return n.flat(Number.POSITIVE_INFINITY)}function Cc(n,t){n.forEach(e=>Array.isArray(e)?Cc(e,t):t(e))}function Gu(n,t,e){t>=n.length?n.push(e):n.splice(t,0,e)}function da(n,t){return t>=n.length-1?n.pop():n.splice(t,1)[0]}function b_(n,t,e,o){let i=n.length;if(i==t)n.push(e,o);else if(i===1)n.push(o,n[0]),n[0]=e;else{for(i--,n.push(n[i-1],n[i]);i>t;){let r=i-2;n[i]=n[r],i--}n[t]=e,n[t+1]=o}}function Mc(n,t,e){let o=Ji(n,t);return o>=0?n[o|1]=e:(o=~o,b_(n,o,t,e)),o}function bc(n,t){let e=Ji(n,t);if(e>=0)return n[e|1]}function Ji(n,t){return JO(n,t,1)}function JO(n,t,e){let o=0,i=n.length>>e;for(;i!==o;){let r=o+(i-o>>1),a=n[r<<e];if(t===a)return r<<e;a>t?i=r:o=r+1}return~(i<<e)}var Mo={},ht=[],ri=new B(""),ua=new B("",-1),qu=new B(""),Zi=class{get(t,e=Xo){if(e===Xo){let i=__("",-201);throw i.name="\u0275NotFound",i}return e}};function bo(n){return{\u0275providers:n}}function P_(...n){return{\u0275providers:Wu(!0,n),\u0275fromNgModule:!0}}function Wu(n,...t){let e=[],o=new Set,i,r=a=>{e.push(a)};return Cc(t,a=>{let s=a;lc(s,r,[],o)&&(i||=[],i.push(s))}),i!==void 0&&O_(i,r),e}function O_(n,t){for(let e=0;e<n.length;e++){let{ngModule:o,providers:i}=n[e];Ku(i,r=>{t(r,o)})}}function lc(n,t,e,o){if(n=lt(n),!n)return!1;let i=null,r=xu(n),a=!r&&ii(n);if(!r&&!a){let c=n.ngModule;if(r=xu(c),r)i=c;else return!1}else{if(a&&!a.standalone)return!1;i=n}let s=o.has(i);if(a){if(s)return!1;if(o.add(i),a.dependencies){let c=typeof a.dependencies=="function"?a.dependencies():a.dependencies;for(let p of c)lc(p,t,e,o)}}else if(r){if(r.imports!=null&&!s){o.add(i);let p;Cc(r.imports,f=>{lc(f,t,e,o)&&(p||=[],p.push(f))}),p!==void 0&&O_(p,t)}if(!s){let p=Co(i)||(()=>new i);t({provide:i,useFactory:p,deps:ht},i),t({provide:qu,useValue:i,
Line truncated
|
|
||||||
`).forEach(e=>{let o=e.indexOf(":");if(o>0){let i=e.slice(0,o),r=e.slice(o+1).trim();this.addHeaderEntry(i,r)}})}:typeof Headers<"u"&&t instanceof Headers?(this.headers=new Map,t.forEach((e,o)=>{this.addHeaderEntry(o,e)})):this.lazyInit=()=>{this.headers=new Map,Object.entries(t).forEach(([e,o])=>{this.setHeaderEntries(e,o)})}:this.headers=new Map}has(t){return this.init(),this.headers.has(t.toLowerCase())}get(t){this.init();let e=this.headers.get(t.toLowerCase());return e&&e.length>0?e[0]:null}keys(){return this.init(),Array.from(this.normalizedNames.values())}getAll(t){return this.init(),this.headers.get(t.toLowerCase())||null}append(t,e){return this.clone({name:t,value:e,op:"a"})}set(t,e){return this.clone({name:t,value:e,op:"s"})}delete(t,e){return this.clone({name:t,value:e,op:"d"})}maybeSetNormalizedName(t,e){this.normalizedNames.has(e)||this.normalizedNames.set(e,t)}init(){this.lazyInit&&(this.lazyInit instanceof n?this.copyFrom(this.lazyInit):this.lazyInit(),this.lazyInit=null,this.lazyUpdate&&(this.lazyUpdate.forEach(t=>this.applyUpdate(t)),this.lazyUpdate=null))}copyFrom(t){t.init();for(let[e,o]of t.headers.entries())this.headers.set(e,o),this.normalizedNames.set(e,t.normalizedNames.get(e))}clone(t){let e=new n;return e.lazyInit=this.lazyInit&&this.lazyInit instanceof n?this.lazyInit:this,e.lazyUpdate=(this.lazyUpdate||[]).concat([t]),e}applyUpdate(t){let e=t.name.toLowerCase();switch(t.op){case"a":case"s":let o=t.value;if(typeof o=="string"&&(o=[o]),o.length===0)return;this.maybeSetNormalizedName(t.name,e);let i=t.op==="a"?(this.headers.get(e)||[]).slice():[];i.push(...o),this.headers.set(e,i);break;case"d":let r=t.value;if(r===void 0)this.headers.delete(e),this.normalizedNames.delete(e);else{let a=Array.isArray(r)?r:[r],s=this.headers.get(e);if(!s)return;s=s.filter(c=>a.indexOf(c)===-1),s.length===0?(this.headers.delete(e),this.normalizedNames.delete(e)):this.headers.set(e,s)}break}}addHeaderEntry(t,e){let o=t.toLowerCase();this.maybeSetNormalizedName(t,o),this.headers.has(o)?this.headers.get(o).push(e):this.headers.set(o,[e])}setHeaderEntries(t,e){let o=(Array.isArray(e)?e:[e]).map(r=>r.toString()),i=t.toLowerCase();this.headers.set(i,o),this.maybeSetNormalizedName(t,i)}forEach(t){this.init(),Array.from(this.normalizedNames.keys()).forEach(e=>t(this.normalizedNames.get(e),this.headers.get(e)))}};var Wl=class{map=new Map;set(t,e){return this.map.set(t,e),this}get(t){return this.map.has(t)||this.map.set(t,t.defaultValue()),this.map.get(t)}delete(t){return this.map.delete(t),this}has(t){return this.map.has(t)}keys(){return this.map.keys()}},Kl=class{encodeKey(t){return y0(t)}encodeValue(t){return y0(t)}decodeKey(t){return decodeURIComponent(t)}decodeValue(t){return decodeURIComponent(t)}};function Ck(n,t){let e=new Map;return n.length>0&&n.replace(/^\?/,"").split("&").forEach(i=>{let r=i.indexOf("="),[a,s]=r==-1?[t.decodeKey(i),""]:[t.decodeKey(i.slice(0,r)),t.decodeValue(i.slice(r+1))],c=e.get(a)||[];c.push(s),e.set(a,c)}),e}var Mk=/%(\d[a-f0-9])/gi,bk={40:"@","3A":":",24:"$","2C":",","3B":";","3D":"=","3F":"?","2F":"/"};function y0(n){return encodeURIComponent(n).replace(Mk,(t,e)=>bk[e]??t)}function ql(n){return`${n}`}var ao=class n{map;encoder;updates=null;cloneFrom=null;constructor(t={}){if(this.encoder=t.encoder||new Kl,t.fromString){if(t.fromObject)throw new z(2805,!1);this.map=Ck(t.fromString,this.encoder)}else t.fromObject?(this.map=new Map,Object.keys(t.fromObject).forEach(e=>{let o=t.fromObject[e],i=Array.isArray(o)?o.map(ql):[ql(o)];this.map.set(e,i)})):this.map=null}has(t){return this.init(),this.map.has(t)}get(t){this.init();let e=this.map.get(t);return e?e[0]:null}getAll(t){return this.init(),this.map.get(t)||null}keys(){return this.init(),Array.from(this.map.keys())}append(t,e){return this.clone({param:t,value:e,op:"a"})}appendAll(t){let e=[];return Object.keys(t).forEach(o=>{let i=t[o];Array.isArray(i)?i.forEach(r=>{e.push({param:o,value:r,op:"a"})}):e.push({param:o,value:i,op:"a"})}),this.clone(e)}set(t,e){return this.clone({param:t,value:e,op:"s"})}delete(t,e){return this.clone({param:t,value:e,op:"d"})}toString(){return this.init(),this.keys().map(t=>{let e=this.encoder.encodeKey(t);return this.map.get(t).map(o=>e+"="+this.encoder.encodeValue(o)).join("&")}).filter(t=>t!=="").join("&")}clone(t){let e=new n({encoder:this.encoder});return e.cloneFrom=this.cloneFrom||this,e.updates=(this.updates||[]).concat(t),e}init(){if(this.map===null&&(this.map=new Map),this.cloneFrom!==null){this.cloneFrom.init();for(let[t,e]of this.cloneFrom.map.entries())this.map.set(t,e);this.updates.forEach(t=>{switch(t.op){case"a":case"s":let e=t.op==="a"?(this.map.get(t.param)||[]).slice():[];e.push(ql(t.value)),this.map.set(t.param,e);break;case"d":if(t.value!==void 0){let o=(this.map.get(t.param)||[]).slice(),i=o.indexOf(ql(t.value));i!==-1&&o.splice(i,1),o.length>0?this.map.set(t.param,o):this.map.delete(t.param)}else{this.map.delete(t.param);break}}}),this.cloneFrom=this.updates=null}}};function Pk(
Line truncated
|
|
||||||
`)||"","boot_logs"))}),l(8,"span",29),u(9),d(),l(10,"span"),u(11),d()()(),l(12,"pre",202),M(13,yI,2,0)(14,xI,2,0,"div",203),d()()}if(n&2){let e=m(3);g(9),O(e.copiedId()==="boot_logs"?"check":"content_copy"),g(2),O(e.copiedId()==="boot_logs"?"Copied":"Copy Logs"),g(2),b(e.emulatorLaunchState()?.logs?.length?13:14)}}function SI(n,t){if(n&1){let e=L();l(0,"div",65)(1,"div",168)(2,"div",169)(3,"div",170)(4,"div",171)(5,"span",172),u(6,"sync"),d()(),l(7,"div",173)(8,"div",174)(9,"h4",175),u(10," Launching Android Emulator: "),l(11,"span",176),u(12),d()(),l(13,"span",177),u(14),d()(),l(15,"p",178),u(16),d()()(),l(17,"div",179)(18,"button",180),k("click",function(){v(e);let i=m(2);return y(i.toggleLaunchLogs())}),l(19,"span",29),u(20,"terminal"),d(),l(21,"span"),u(22),d()(),l(23,"button",181),k("click",function(){v(e);let i=m(2);return y(i.stopEmulator())}),l(24,"span",29),u(25,"cancel"),d(),l(26,"span"),u(27,"Stop"),d()()()(),l(28,"div",182)(29,"div",183)(30,"div",184),M(31,mI,2,0,"span",185)(32,_I,1,0,"span",186),d(),l(33,"span",187),u(34,"1. Process Start"),d()(),$(35,"div",188),l(36,"div",183)(37,"div",184),M(38,hI,2,0,"span",185)(39,CI,1,0,"span",186)(40,MI,2,0,"span"),d(),l(41,"span",187),u(42,"2. ADB Handshake"),d()(),$(43,"div",188),l(44,"div",183)(45,"div",184),M(46,bI,2,0,"span",185)(47,PI,1,0,"span",186)(48,OI,2,0,"span"),d(),l(49,"span",187),u(50,"3. Android OS Boot"),d()(),$(51,"div",188),l(52,"div",183)(53,"div",184)(54,"span",185),u(55,"check_circle"),d()(),l(56,"span",187),u(57,"4. Ready"),d()()(),l(58,"div",189)(59,"div",190),$(60,"div",191),d(),l(61,"div",192)(62,"span",193),u(63),d(),l(64,"span",194)(65,"span",195),u(66,"timer"),d(),u(67),d()()(),l(68,"div",196)(69,"span",197),u(70,"info"),d(),l(71,"span"),u(72,"Android virtualization cold boot usually takes 20-45 seconds. Live progress is actively tracked above\u2014please do not click launch repeatedly."),d()(),M(73,wI,15,3,"div",198),d()()}if(n&2){let e=m(2);g(12),O(e.emulatorLaunchState()?.avd_name||e.activeDevice()?.serial||"Virtual Device"),g(2),O(e.emulatorLaunchState()?.status==="booting"?"OS Booting":"Starting"),g(2),O(e.emulatorLaunchState()?.stage_message||"Initializing Android Virtual Device..."),g(2),w("active",e.showLaunchLogs()),g(4),O(e.showLaunchLogs()?"Hide Logs":"View Boot Logs"),g(7),w("done",e.emulatorLaunchState()?.progress_percent>=35)("active",e.emulatorLaunchState()?.status==="starting"),g(2),b(e.emulatorLaunchState()?.progress_percent>=35?31:32),g(4),w("done",e.emulatorLaunchState()?.progress_percent>=35),g(),w("done",e.emulatorLaunchState()?.progress_percent>=65)("active",e.emulatorLaunchState()?.status==="waiting_for_adb"),g(2),b(e.emulatorLaunchState()?.progress_percent>=65?38:e.emulatorLaunchState()?.status==="waiting_for_adb"?39:40),g(5),w("done",e.emulatorLaunchState()?.progress_percent>=65),g(),w("done",e.emulatorLaunchState()?.status==="ready")("active",e.emulatorLaunchState()?.status==="booting"||e.adbProbe()?.summary==="Device Booting"),g(2),b(e.emulatorLaunchState()?.status==="ready"?46:e.emulatorLaunchState()?.status==="booting"||e.adbProbe()?.summary==="Device Booting"?47:48),g(5),w("done",e.emulatorLaunchState()?.status==="ready"),g(),w("done",e.emulatorLaunchState()?.status==="ready"),g(8),Se("width",e.emulatorLaunchState()?.progress_percent||25,"%"),g(3),V("",e.emulatorLaunchState()?.progress_percent||25,"% Complete"),g(4),V(" Elapsed: ",e.emulatorLaunchState()?.elapsed_seconds||0,"s "),g(6),b(e.showLaunchLogs()?73:-1)}}function kI(n,t){if(n&1&&(l(0,"div",204),u(1),d()),n&2){let e=t.$implicit;g(),O(e)}}function EI(n,t){if(n&1){let e=L();l(0,"div",214)(1,"div",199)(2,"span"),u(3,"Process Output / Failure Logs:"),d(),l(4,"button",201),k("click",function(){v(e);let i=m(3);return y(i.copyToClipboard(i.emulatorLaunchState()?.logs?.join(`
|
|
||||||
`)||"","fail_logs"))}),l(5,"span",29),u(6),d(),l(7,"span"),u(8),d()()(),l(9,"pre",219),G(10,kI,2,1,"div",204,Ie),d()()}if(n&2){let e=m(3);g(6),O(e.copiedId()==="fail_logs"?"check":"content_copy"),g(2),O(e.copiedId()==="fail_logs"?"Copied":"Copy Logs"),g(2),q(e.emulatorLaunchState()?.logs)}}function DI(n,t){if(n&1){let e=L();l(0,"button",220),k("click",function(){v(e);let i=m(3);return y(i.launchAvdEmulator(i.emulatorLaunchState().avd_name))}),l(1,"span",29),u(2,"refresh"),d(),l(3,"span"),u(4),d()()}if(n&2){let e=m(3);R("disabled",e.isRemoteAdbServer()),g(4),V("Retry Launch (",e.emulatorLaunchState()?.avd_name,")")}}function II(n,t){if(n&1){let e=L();l(0,"div",66)(1,"div",205)(2,"div",206)(3,"div",207)(4,"div",208)(5,"span",209),u(6,"error"),d()(),l(7,"div",210)(8,"h4",211),u(9,"Emulator Launch Failed"),d(),l(10,"p",212),u(11),d()()(),l(12,"button",213),k("click",function(){v(e);let i=m(2);return y(i.dismissEmulatorStatus())}),l(13,"span",29),u(14,"close"),d()()(),M(15,EI,12,2,"div",214),l(16,"div",215),M(17,DI,5,2,"button",216),l(18,"button",217),k("click",function(){v(e);let i=m(2);return y(i.restartAdbServer())}),l(19,"span",29),u(20,"sync"),d(),l(21,"span"),u(22,"Restart ADB Server"),d()(),l(23,"button",218),k("click",function(){v(e);let i=m(2);return y(i.dismissEmulatorStatus())}),l(24,"span"),u(25,"Dismiss"),d()()()()()}if(n&2){let e=m(2);g(11),O(e.emulatorLaunchState()?.error||"The Android emulator process failed to start or crashed during initialization."),g(4),b(e.emulatorLaunchState()?.logs?.length?15:-1),g(2),b(e.emulatorLaunchState()?.avd_name?17:-1)}}function TI(n,t){if(n&1&&u(0),n&2){let e=m(3);We(" ",e.installedAvds().length," local virtual device",e.installedAvds().length>1?"s":""," available ")}}function AI(n,t){n&1&&u(0," Launch a local Android Virtual Device ")}function NI(n,t){if(n&1){let e=L();l(0,"div",231)(1,"div",234)(2,"span",29),u(3,"hub"),d(),l(4,"span"),u(5," Artemis is currently using "),l(6,"strong"),u(7),d(),u(8,". Switch to local ADB before using a local emulator, USB cable, or Wireless ADB. "),d()(),l(9,"button",235),k("click",function(){v(e);let i=m(3);return y(i.switchToLocalAdbServer())}),l(10,"span",29),u(11,"sync"),d(),l(12,"span"),u(13),d()()()}if(n&2){let e=m(3);g(7),We("",e.adbServerStatus()?.endpoint?.host,":",e.adbServerStatus()?.endpoint?.port),g(2),R("disabled",e.isSwitchingToLocalAdb()),g(),w("spinning",e.isSwitchingToLocalAdb()),g(3),O(e.isSwitchingToLocalAdb()?"Switching...":"Use Local ADB")}}function RI(n,t){if(n&1&&($(0,"span",252),l(1,"span"),u(2),d()),n&2){let e=m(6);g(2),We("",e.emulatorLaunchState()?.status==="booting"?"Booting OS":"Starting"," (",e.emulatorLaunchState()?.elapsed_seconds||0,"s)")}}function FI(n,t){n&1&&(l(0,"span",253),u(1,"play_arrow"),d(),l(2,"span"),u(3,"Launch"),d())}function LI(n,t){if(n&1){let e=L();l(0,"div",243)(1,"div",244)(2,"div",245)(3,"span",29),u(4,"smartphone"),d()(),l(5,"div",246)(6,"div",247)(7,"span",248),u(8),d(),l(9,"span",249),u(10,"Virtual Device"),d()()()(),l(11,"div",250)(12,"button",251),k("click",function(){let i=v(e).$implicit,r=m(5);return y(r.launchAvdEmulator(i))}),M(13,RI,3,2)(14,FI,4,0),d()()()}if(n&2){let e=t.$implicit,o=m(5);g(8),O(e),g(4),w("launching",o.launchingAvd()===e||o.isEmulatorLaunching()&&o.emulatorLaunchState()?.avd_name===e),R("disabled",o.isEmulatorLaunching()||o.isRemoteAdbServer())("title",o.isEmulatorLaunching()?"An emulator is currently launching...":"Launch "+e),g(),b(o.launchingAvd()===e||o.isEmulatorLaunching()&&o.emulatorLaunchState()?.avd_name===e?13:14)}}function VI(n,t){if(n&1&&(l(0,"div",236)(1,"div",238)(2,"div",239)(3,"span",120),u(4,"devices"),d(),l(5,"span",240),u(6,"Installed Virtual Devices (AVD)"),d(),l(7,"span",241),u(8),d()()(),l(9,"div",242),G(10,LI,15,6,"div",243,Ie),d()()),n&2){let e=m(4);g(8),V("",e.installedAvds().length," Available"),g(2),q(e.installedAvds())}}function jI(n,t){if(n&1&&(l(0,"div",264)(1,"div",266)(2,"div",267)(3,"span",268),u(4,"1"),d(),l(5,"span",269),u(6,"download"),d()(),l(7,"div",270),u(8,"Install Android Studio"),d(),l(9,"div",271),u(10),l(11,"a",272),u(12,"developer.android.com/studio"),d(),u(13," and install. "),d()(),l(14,"div",266)(15,"div",267)(16,"span",268),u(17,"2"),d(),l(18,"span",269),u(19,"developer_mode"),d()(),l(20,"div",270),u(21,"Open Device Manager"),d(),l(22,"div",271),u(23," On Studio welcome screen, open "),l(24,"strong"),u(25,"More Actions > Virtual Device Manager"),d(),u(26," (or "),l(27,"strong"),u(28,"Tools > Device Manager"),d(),u(29,"). "),d()(),l(30,"div",266)(31,"div",267)(32,"span",268),u(33,"3"),d(),l(34,"span",269),u(35,"add_circle"),d()(),l(36,"div",270),u(37,"Create Virtual Device"),d(),l(38,"div",271),u(39," Click "),l(40,"strong"),u(41,"Create Device"),d(),u(42,", choose "),l(43,"strong"),u(44,"Pixel 8"),d(),u(45,", select "),l(46,"strong"),u(47,"Android 14 (API 34)"),d(),u(48,", and click "),l(49,"strong"),u(50,"Finish"),d(),u(51,". "),d()(),l(52,"div",266)(53,"div",267)(54,"span",268),u(55,"4"),d(),l(56,"spa
Line truncated
|
|
||||||
`).map(t=>{let e=t.trim(),o=t.length-t.trimStart().length;if(e.startsWith("- [x] ")||e.startsWith("- [X] ")){let a=e.substring(6);return{type:"checked",segments:$n(a),indent:o}}if(e.startsWith("- [/] ")){let a=e.substring(6);return{type:"progress",segments:$n(a),indent:o}}if(e.startsWith("- [ ] ")){let a=e.substring(6);return{type:"unchecked",segments:$n(a),indent:o}}let i=e.match(/^[-*]\s*(verify|assert)(@end)?\s*:\s*(.*)$/i);if(i){let a=i[1].toLowerCase(),s=!!i[2],c=i[3].trim();return{type:"verify",checkKind:a,atEnd:s,segments:$n(c),indent:o}}let r=e.match(/^[-*]\s*finding\s*:\s*(.*)$/i);if(r){let a=r[1].trim();return{type:"finding",checkKind:"finding",segments:$n(a),indent:o}}if(e.startsWith("- ")||e.startsWith("* ")){let a=e.substring(2);return{type:"list-item",segments:$n(a),indent:o}}if(e.startsWith("# ")){let a=e.substring(2);return{type:"h1",segments:$n(a),indent:o}}if(e.startsWith("## ")){let a=e.substring(3);return{type:"h2",segments:$n(a),indent:o}}if(e.startsWith("### ")){let a=e.substring(4);return{type:"h3",segments:$n(a),indent:o}}return e?{type:"text",segments:$n(t),indent:o}:{type:"empty",segments:[],indent:0}}):[]}function ks(n){let t=Ss(n),e=[],o=[],i=null,r=null,a=0;for(let s of t){if(s.type==="empty")continue;if((s.type==="h1"||s.type==="h2")&&!i){i=s.segments.map(p=>p.text).join("");continue}["checked","progress","unchecked"].includes(s.type)&&(!r||s.indent<=(r._indent??0))?(a++,r={index:a,type:s.type,segments:s.segments,subSteps:[],checks:[],_indent:s.indent},e.push(r)):r&&s.indent>(r._indent??0)?s.type==="verify"||s.type==="assert"||s.type==="finding"?r.checks.push(s):r.subSteps.push(s):o.push(s)}return{title:i,milestones:e,otherLines:o}}function WP(n){if(!n)return null;let t=n.trim();if(t.includes("```json")){let e=t.split("```json");e.length>1&&(t=e[1].split("```")[0].trim())}else if(t.includes("```")){let e=t.split("```");e.length>1&&(t=e[1].split("```")[0].trim())}try{let e=JSON.parse(t);if(e&&typeof e=="object"&&"success"in e)return{success:!!e.success,reason:e.reason||"No reason provided."}}catch(e){}return null}function Vi(n){return n?n.replace(/`([^`]+)`/g,'<code class="inline-code">$1</code>').replace(/\*\*\*([^*]+)\*\*\*/g,"<strong><em>$1</em></strong>").replace(/___([^_]+)___/g,"<strong><em>$1</em></strong>").replace(/\*\*([^*]+)\*\*/g,"<strong>$1</strong>").replace(/__([^_]+)__/g,"<strong>$1</strong>").replace(/\*([^*]+)\*/g,"<em>$1</em>").replace(/\b_([^_]+)_\b/g,"<em>$1</em>").replace(/~~([^~]+)~~/g,"<del>$1</del>"):""}function Om(n){if(!n)return"";let o=n.replace(/<\/?(thought|short_term_memory|strong_term_memory|reasoning|plan|task_plan|call_tool)[^\n>]*>?/gi,"").replace(/\r\n/g,`
|
|
||||||
`).replace(/\r/g,`
|
|
||||||
`).replace(/\n{3,}/g,`
|
|
||||||
|
|
||||||
`).trim().replace(/&/g,"&").replace(/</g,"<").replace(/>/g,">").split(`
|
|
||||||
`),i=[],r=null,a=!1,s="",c=[],p=()=>{r&&(i.push(`</${r}>`),r=null)};for(let f=0;f<o.length;f++){let _=o[f],C=_.trim();if(C.startsWith("```")){if(a){let A=c.join(`
|
|
||||||
`);i.push(`<pre class="code-block"><code class="lang-${s}">${A}</code></pre>`),a=!1,c=[],s=""}else p(),a=!0,s=C.substring(3).trim(),c=[];continue}if(a){c.push(_);continue}let h=C.match(/^(#{1,6})\s+(.*)$/);if(h){p();let A=h[1].length,N=Vi(h[2]);i.push(`<h${A}>${N}</h${A}>`);continue}if(/^(\-{3,}|\*{3,}|_{3,})$/.test(C)){p(),i.push('<hr class="md-hr" />');continue}if(C.startsWith("> ")||C.startsWith(">")){p();let A=Vi(C.replace(/^>\s?/,""));i.push(`<blockquote>${A}</blockquote>`);continue}let P=_.match(/^(\s*)([\*\-])\s+(.*)$/);if(P){r!=="ul"&&(p(),r="ul",i.push("<ul>"));let A=P[1].length,E=Math.max(0,Math.floor((A-2)/2))*12,U=E?` style="margin-left: ${E}px"`:"",ne=P[3],J=ne.match(/^(verify|assert)(@end)?\s*:\s*(.*)$/i),X=ne.match(/^finding\s*:\s*(.*)$/i);if(J){let W=J[1].toLowerCase(),Me=J[2]?"@end":"",Ee=W==="assert"?" assert":"",ft=Vi(J[3].trim());i.push(`<li${U} class="md-verify-item"><span class="verify-badge${Ee}">${W}${Me}</span>${ft}</li>`)}else if(X){let W=Vi(X[1].trim());i.push(`<li${U} class="md-finding-item"><span class="finding-badge">finding</span>${W}</li>`)}else{let W=Vi(ne);i.push(`<li${U}>${W}</li>`)}continue}let T=_.match(/^(\s*)(\d+)\.\s+(.*)$/);if(T){r!=="ol"&&(p(),r="ol",i.push("<ol>"));let A=T[1].length,E=Math.max(0,Math.floor((A-2)/2))*12,U=E?` style="margin-left: ${E}px"`:"",ne=Vi(T[3]);i.push(`<li${U}>${ne}</li>`);continue}if(p(),C==="")i.push('<div class="md-para-gap"></div>');else{let A=Vi(_);i.push(`<div>${A}</div>`)}}if(a){let f=c.join(`
|
|
||||||
`);i.push(`<pre class="code-block"><code class="lang-${s}">${f}</code></pre>`)}return p(),i.join(`
|
|
||||||
`)}function Gr(n,t=0){if(n!=null){if(typeof n=="number")return n<1e11?n*1e3:n;let e=new Date(n).getTime();if(!isNaN(e)&&e>0)return e}return t}function KP(n,t){if(!t)return-1;let e=String(t);return n.findIndex(o=>o.type==="checker"&&o.data?.trace_id&&String(o.data.trace_id)===e)}function zT(n){for(let t=n.length-1;t>=0;t--)if(n[t].type==="checker"&&n[t].data?.isCompleted===!1)return t;return-1}function BT(n,t){let e=t.data||{};if(!e.event)return;if(e.event==="run_outcome"){let s="checker-outcome",c=n.findIndex(_=>_.id===s),p=F(S(S({},c>-1?n[c].data:{}),e),{phase:"outcome",status:"done",isCompleted:!0,generic_tools:[]}),f={id:s,type:"checker",timestamp:t.timestamp,data:p};c>-1?n[c]=f:n.push(f);return}if(!e.attempt_id)return;let o=`checker-${e.attempt_id}`,i=n.findIndex(s=>s.id===o),r=i>-1?n[i]:null,a=typeof e.ts=="number"?e.ts:t.timestamp?new Date(t.timestamp).getTime()/1e3:void 0;if(e.event==="attempt_started"){let s=F(S(S({},r?.data||{}),e),{status:r?.data?.isCompleted?r.data.status:"running",isCompleted:r?.data?.isCompleted??!1,started_at:r?.data?.started_at??a,verdicts:r?.data?.verdicts||[],findings:r?.data?.findings||[],generic_tools:r?.data?.generic_tools||[]}),c={id:o,type:"checker",timestamp:r?.timestamp||t.timestamp,data:s};r?n[i]=c:n.push(c);return}if(e.event==="attempt_finished"){let s=r?.data?.started_at??a,c=Array.isArray(e.verdicts)?e.verdicts:r?.data?.verdicts||[],p=r?.data?.items?.length?r.data.items:c.map(C=>({kind:C.kind,text:C.item_text,when:C.when})),f=F(S(S({},r?.data||{}),e),{status:e.status||"done",verdicts:c,items:p,findings:Array.isArray(e.findings)?e.findings:r?.data?.findings||[],generic_tools:r?.data?.generic_tools||[],started_at:s,finished_at:a,duration:s!==void 0&&a!==void 0?Math.max(0,a-s):void 0,isCompleted:!0}),_={id:o,type:"checker",timestamp:r?.timestamp||t.timestamp,data:f};r?n[i]=_:n.push(_)}}function HT(n){let t=new Map;n.forEach((e,o)=>{e.type==="checker"&&e.data?.trace_id&&t.set(String(e.data.trace_id),o)}),t.size!==0&&n.forEach((e,o)=>{if(e.type==="checker")return;let i=e.data?.generic_tools||[];if(i.length===0)return;let r=[],a=!1;for(let s of i){let c=s?.trace_id?String(s.trace_id):"",p=s?.parent_trace_id?String(s.parent_trace_id):"";if(c&&t.has(c)&&s?.type==="agent"){a=!0;continue}let f=p?t.get(p):void 0;if(f===void 0){r.push(s);continue}let _=n[f],C=_.data.generic_tools||[];C.some(h=>h.trace_id&&h.trace_id===s.trace_id)||(n[f]=F(S({},_),{data:F(S({},_.data),{generic_tools:[...C,s]})})),a=!0}a&&(n[o]=F(S({},e),{data:F(S({},e.data),{generic_tools:r})}))})}function ZP(n){if(!n||n.length===0)return[];let t=[];return n.forEach(e=>{if(e){if(e.type==="checker_event"){BT(t,e);return}if(e.type==="llm_stream"){let o=e.data?.execution_id||e.id,i=e.data?.step_id,r=e.data?.stream_type||"text",a=e.data?.text||"",s=e.data?.isCompleted??!1,c=KP(t,e.data?.parent_trace_id);if(c>-1){let f=t[c],_=S({},f.data),C=Array.isArray(_.stream_segments)?[..._.stream_segments]:[],h=C.findIndex(A=>A.execution_id===o&&A.stream_type===r),P={execution_id:o,stream_type:r==="thinking"?"thinking":"text",text:a,timestamp:h>-1?C[h].timestamp:e.timestamp,isCompleted:s};h>-1?C[h]=P:C.push(P),_.stream_segments=C;let T=A=>C.filter(N=>N.stream_type===A&&N.text.trim()).map(N=>N.text).join(`
|
|
||||||
|
|
||||||
`);_.operator_native_thinking=T("thinking"),_.operator_raw_thinking=T("text"),r==="thinking"?_.operator_native_thinking_timestamp=_.operator_native_thinking_timestamp||e.timestamp:_.operator_raw_thinking_timestamp=_.operator_raw_thinking_timestamp||e.timestamp,o&&!_.execution_id&&(_.execution_id=o),t[c]=F(S({},f),{data:_});return}let p=-1;if(i&&(p=t.findIndex(f=>f.id===`step-${i}`||f.data?.step_id===i)),p===-1&&o&&(p=t.findIndex(f=>f.data?.execution_id===o||f.id===`stream-${o}`)),p>-1){let f=t[p],_=S({},f.data);r==="thinking"?(_.operator_native_thinking=a,_.operator_native_thinking_timestamp=_.operator_native_thinking_timestamp||e.timestamp):(_.operator_raw_thinking=a,_.operator_raw_thinking_timestamp=_.operator_raw_thinking_timestamp||e.timestamp),i&&!_.step_id&&(_.step_id=i),o&&!_.execution_id&&(_.execution_id=o),_.isCompleted=s,t[p]=F(S({},f),{data:_})}else{let f=i?`step-${i}`:`stream-${o}`,_={execution_id:o,step_id:i,isCompleted:s,generic_tools:[]};r==="thinking"?(_.operator_native_thinking=a,_.operator_native_thinking_timestamp=e.timestamp):(_.operator_raw_thinking=a,_.operator_raw_thinking_timestamp=e.timestamp),t.push({id:f,type:i?"step":"llm_stream",timestamp:e.timestamp,data:_})}}else if(e.type==="step_recorded"||e.type==="step_updated"){let o=e.data.step_id||e.data.step_number||"unknown",i=t.findIndex(r=>r.id===`step-${o}`||r.data?.step_id===o);if(i>-1){let r=t[i],a=r.data.generic_tools||[],s=e.data.generic_tools||[],c=[...a];s.forEach(f=>{let _=c.findIndex(C=>C.trace_id&&f.trace_id&&C.trace_id===f.trace_id);_>-1?c[_]=S(S({},c[_]),f):c.push(f)});let p=F(S(S({},r.data),e.data),{operator_native_thinking:e.data.operator_native_thinking||r.data.operator_native_thinking||void 0,operator_raw_thinking:e.data.operator_raw_thinking||r.data.operator_raw_thinking||void 0,operator_native_thinking_timestamp:r.data.operator_native_thinking_timestamp||e.data.operator_native_thinking_timestamp||(e.data.operator_native_thinking?e.timestamp:void 0),operator_raw_thinking_timestamp:r.data.operator_raw_thinking_timestamp||e.data.operator_raw_thinking_timestamp||(e.data.operator_raw_thinking?e.timestamp:void 0),action_taken:e.data.action_taken||r.data.action_taken||void 0,last_execution_result:e.data.last_execution_result||r.data.last_execution_result||void 0,generic_tools:c,isCompleted:!0});t[i]=F(S({},r),{id:`step-${o}`,type:"step",timestamp:r.timestamp||e.timestamp,data:p})}else t.push({id:`step-${o}`,type:"step",timestamp:e.timestamp,data:F(S({},e.data),{isCompleted:!0,operator_native_thinking_timestamp:e.data.operator_native_thinking_timestamp||(e.data.operator_native_thinking?e.timestamp:void 0),operator_raw_thinking_timestamp:e.data.operator_raw_thinking_timestamp||(e.data.operator_raw_thinking?e.timestamp:void 0),generic_tools:e.data.generic_tools||[]})})}else if(e.type==="trace_recorded"){let o=e.data.type==="action",i=e.data.type==="tool"||o,r=e.data.type==="llm_call"&&(e.data.status==="failed"||e.data.status==="retrying");if(!i&&!r)return;let a=e.data.step_id,s=-1,c=KP(t,e.data.parent_trace_id);if(c===-1&&String(e.data.agent_name||"").toLowerCase()==="checker"&&(c=zT(t)),c>-1&&(s=c),s===-1&&e.data.trace_id&&(s=t.findIndex(p=>p.data?.generic_tools?.some(f=>f.trace_id===e.data.trace_id))),s===-1&&a&&(s=t.findIndex(p=>p.id===`step-${a}`||p.data?.step_id===a)),s===-1&&!a&&(s=t.findIndex(p=>p.id==="step-pre-planning"||p.data?.step_id==="pre-planning")),s===-1&&!a){let p=e.timestamp?new Date(e.timestamp).getTime():0,f=-1;for(let _=0;_<t.length;_++){let C=t[_],h=C.timestamp?new Date(C.timestamp).getTime():0;h<=p&&h>=f&&(f=h,s=_)}}if(s>-1){let p=t[s].data,f=p.generic_tools?[...p.generic_tools]:[],_=e.data.trace_id?f.findIndex(C=>C.trace_id===e.data.trace_id):-1;_>-1?f[_]=S(S({},f[_]),e.data):f.push(e.data),t[s]=F(S({},t[s]),{data:F(S({},p),{generic_tools:f})})}else{let p=a||"pre-planning";t.push({id:`step-${p}`,type:"step",timestamp:e.timestamp,data:{step_id:p,isCompleted:!0,generic_tools:[e.data]}})}}}}),HT(t),t.sort((e,o)=>{let i=e.timestamp?new Date(e.timestamp).getTime():0,r=o.timestamp?new Date(o.timestamp).getTime():0;if(Math.abs(i-r)>50)return i-r;let a=p=>p.data?.step_number!==void 0&&p.data.step_number!==null?Number(p.data.step_number):p.data?.step_id==="pre-planning"||p.data?.step_type==="planning"?0:99999,s=a(e),c=a(o);return s!==c&&s!==99999&&c!==99999?s-c:i-r}),t}var YP=new WeakMap;function UT(n){if(typeof n=="string")return{isImage:n.includes("data:image/")||n.includes("base64,"),length:n.length};if(n===null||typeof n!="object")return{isImage:!1,length:String(n).length};let t=YP.get(n);if(t)return t;let e="";try{e=JSON.stringify(n)||""}catch(i){e=String(n)}let o={isImage:e.includes("data:image/")||e.includes("base64,"),length:e.length};return YP.set(n,o),o}function GT(n){if(!n||!n.data)return{total:0,prompt:0,completion:0};let t=n.data;if(t.total_tokens!==void 0&&t.total_tokens!==null&&Number(t.total_tokens)>0){let s=t.token_usage?.prompt_tokens||t.token_usage?.input_tokens||0,c=t.token_usage?.complet
Line truncated
|
|
||||||
@@ -0,0 +1,157 @@
|
|||||||
|
var Ay=Object.defineProperty,Ry=Object.defineProperties;var Fy=Object.getOwnPropertyDescriptors;var eh=Object.getOwnPropertySymbols;var Ny=Object.prototype.hasOwnProperty,Ly=Object.prototype.propertyIsEnumerable;var th=(n,t,e)=>t in n?Ay(n,t,{enumerable:!0,configurable:!0,writable:!0,value:e}):n[t]=e,x=(n,t)=>{for(var e in t||={})Ny.call(t,e)&&th(n,e,t[e]);if(eh)for(var e of eh(t))Ly.call(t,e)&&th(n,e,t[e]);return n},F=(n,t)=>Ry(n,Fy(t));var ke=(n,t,e)=>new Promise((o,i)=>{var r=c=>{try{s(e.next(c))}catch(u){i(u)}},a=c=>{try{s(e.throw(c))}catch(u){i(u)}},s=c=>c.done?o(c.value):Promise.resolve(c.value).then(r,a);s((e=e.apply(n,t)).next())});var It=null,nc=!1,ci=1,Vy=null,lt=Symbol("SIGNAL");function Y(n){let t=It;return It=n,t}function oc(){return It}var To={version:0,lastCleanEpoch:0,dirty:!1,producers:void 0,producersTail:void 0,consumers:void 0,consumersTail:void 0,recomputing:!1,consumerAllowSignalWrites:!1,consumerIsAlwaysLive:!1,kind:"unknown",producerMustRecompute:()=>!1,producerRecomputeValue:()=>{},consumerMarkedDirty:()=>{},consumerOnSignalRead:()=>{}};function ui(n){if(nc)throw new Error("");if(It===null)return;It.consumerOnSignalRead(n);let t=It.producersTail;if(t!==void 0&&t.producer===n)return;let e,o=It.recomputing;if(o&&(e=t!==void 0?t.nextProducer:It.producers,e!==void 0&&e.producer===n)){It.producersTail=e,e.lastReadVersion=n.version,e.knownValidAtEpoch=ci;return}let i=n.consumersTail;if(i!==void 0&&i.consumer===It&&(!o||i.knownValidAtEpoch===ci))return;let r=lr(It),a={producer:n,consumer:It,nextProducer:e,prevConsumer:void 0,knownValidAtEpoch:ci,lastReadVersion:n.version,nextConsumer:void 0};It.producersTail=a,t!==void 0?t.nextProducer=a:It.producers=a,r&&rh(n,a)}function nh(){ci++}function gi(n){if(!(lr(n)&&!n.dirty)&&!(!n.dirty&&n.lastCleanEpoch===ci)){if(!n.producerMustRecompute(n)&&!fa(n)){cr(n);return}n.producerRecomputeValue(n),cr(n)}}function Yu(n){if(n.consumers===void 0)return;let t=nc;nc=!0;try{for(let e=n.consumers;e!==void 0;e=e.nextConsumer){let o=e.consumer;o.dirty||By(o)}}finally{nc=t}}function Ku(){return It?.consumerAllowSignalWrites!==!1}function By(n){n.dirty=!0,Yu(n),n.consumerMarkedDirty?.(n)}function cr(n){n.dirty=!1,n.lastCleanEpoch=ci}function Ao(n){return n&&oh(n),Y(n)}function oh(n){if(n.producersTail?.knownValidAtEpoch===ci){let t=n.producers;for(;t!==void 0;)t.knownValidAtEpoch=null,t=t.nextProducer}n.producersTail=void 0,n.recomputing=!0}function pi(n,t){Y(t),n&&ih(n)}function ih(n){n.recomputing=!1;let t=n.producersTail,e=t!==void 0?t.nextProducer:n.producers;if(e!==void 0){if(lr(n))do e=Zu(e);while(e!==void 0);t!==void 0?t.nextProducer=void 0:n.producers=void 0}}function fa(n){for(let t=n.producers;t!==void 0;t=t.nextProducer){let e=t.producer,o=t.lastReadVersion;if(o!==e.version||(gi(e),o!==e.version))return!0}return!1}function fi(n){if(lr(n)){let t=n.producers;for(;t!==void 0;)t=Zu(t)}n.producers=void 0,n.producersTail=void 0,n.consumers=void 0,n.consumersTail=void 0}function rh(n,t){let e=n.consumersTail,o=lr(n);if(e!==void 0?(t.nextConsumer=e.nextConsumer,e.nextConsumer=t):(t.nextConsumer=void 0,n.consumers=t),t.prevConsumer=e,n.consumersTail=t,!o)for(let i=n.producers;i!==void 0;i=i.nextProducer)rh(i.producer,i)}function Zu(n){let t=n.producer,e=n.nextProducer,o=n.nextConsumer,i=n.prevConsumer;if(n.nextConsumer=void 0,n.prevConsumer=void 0,o!==void 0?o.prevConsumer=i:t.consumersTail=i,i!==void 0)i.nextConsumer=o;else if(t.consumers=o,!lr(t)){let r=t.producers;for(;r!==void 0;)r=Zu(r)}return e}function lr(n){return n.consumerIsAlwaysLive||n.consumers!==void 0}function ma(n){Vy?.(n)}function _a(n,t){return Object.is(n,t)}function ic(n,t){let e=Object.create(jy);e.computation=n,t!==void 0&&(e.equal=t);let o=()=>{if(gi(e),ui(e),e.value===zn)throw e.error;return e.value};return o[lt]=e,ma(e),o}var li=Symbol("UNSET"),di=Symbol("COMPUTING"),zn=Symbol("ERRORED"),jy=F(x({},To),{value:li,dirty:!0,error:null,equal:_a,kind:"computed",producerMustRecompute(n){return n.value===li||n.value===di},producerRecomputeValue(n){if(n.value===di)throw new Error("");let t=n.value;n.value=di;let e=Ao(n),o,i=!1;try{o=n.computation(),Y(null),i=t!==li&&t!==zn&&o!==zn&&n.equal(t,o)}catch(r){o=zn,n.error=r}finally{pi(n,e)}if(i){n.value=t;return}n.value=o,n.version++}});function zy(){throw new Error}var ah=zy;function sh(n){ah(n)}function Xu(n){ah=n}var $y=null;function Qu(n,t){let e=Object.create(ac);e.value=n,t!==void 0&&(e.equal=t);let o=()=>ch(e);return o[lt]=e,ma(e),[o,a=>mi(e,a),a=>rc(e,a)]}function ch(n){return ui(n),n.value}function mi(n,t){Ku()||sh(n),n.equal(n.value,t)||(n.value=t,Hy(n))}function rc(n,t){Ku()||sh(n),mi(n,t(n.value))}var ac=F(x({},To),{equal:_a,value:void 0,kind:"signal"});function Hy(n){n.version++,nh(),Yu(n),$y?.(n)}var Ju=F(x({},To),{consumerIsAlwaysLive:!0,consumerAllowSignalWrites:!0,dirty:!0,kind:"effect"});function eg(n){if(n.dirty=!1,n.version>0&&!fa(n))return;n.version++;let t=Ao(n);try{n.cleanup(),n.fn()}finally{pi(n,t)}}var tg;function sc(){return
Line truncated
|
||||||
|
${e.map((o,i)=>`${i+1}) ${o.toString()}`).join(`
|
||||||
|
`)}`:"",this.name="UnsubscriptionError",this.errors=e});function _i(n,t){if(n){let e=n.indexOf(t);0<=e&&n.splice(e,1)}}var Oe=class n{constructor(t){this.initialTeardown=t,this.closed=!1,this._parentage=null,this._finalizers=null}unsubscribe(){let t;if(!this.closed){this.closed=!0;let{_parentage:e}=this;if(e)if(this._parentage=null,Array.isArray(e))for(let r of e)r.remove(this);else e.remove(this);let{initialTeardown:o}=this;if(de(o))try{o()}catch(r){t=r instanceof cc?r.errors:[r]}let{_finalizers:i}=this;if(i){this._finalizers=null;for(let r of i)try{gh(r)}catch(a){t=t??[],a instanceof cc?t=[...t,...a.errors]:t.push(a)}}if(t)throw new cc(t)}}add(t){var e;if(t&&t!==this)if(this.closed)gh(t);else{if(t instanceof n){if(t.closed||t._hasParent(this))return;t._addParent(this)}(this._finalizers=(e=this._finalizers)!==null&&e!==void 0?e:[]).push(t)}}_hasParent(t){let{_parentage:e}=this;return e===t||Array.isArray(e)&&e.includes(t)}_addParent(t){let{_parentage:e}=this;this._parentage=Array.isArray(e)?(e.push(t),e):e?[e,t]:t}_removeParent(t){let{_parentage:e}=this;e===t?this._parentage=null:Array.isArray(e)&&_i(e,t)}remove(t){let{_finalizers:e}=this;e&&_i(e,t),t instanceof n&&t._removeParent(this)}};Oe.EMPTY=(()=>{let n=new Oe;return n.closed=!0,n})();var ig=Oe.EMPTY;function lc(n){return n instanceof Oe||n&&"closed"in n&&de(n.remove)&&de(n.add)&&de(n.unsubscribe)}function gh(n){de(n)?n():n.unsubscribe()}var vn={onUnhandledError:null,onStoppedNotification:null,Promise:void 0,useDeprecatedSynchronousErrorHandling:!1,useDeprecatedNextContext:!1};var gr={setTimeout(n,t,...e){let{delegate:o}=gr;return o?.setTimeout?o.setTimeout(n,t,...e):setTimeout(n,t,...e)},clearTimeout(n){let{delegate:t}=gr;return(t?.clearTimeout||clearTimeout)(n)},delegate:void 0};function dc(n){gr.setTimeout(()=>{let{onUnhandledError:t}=vn;if(t)t(n);else throw n})}function ha(){}var ph=rg("C",void 0,void 0);function fh(n){return rg("E",void 0,n)}function mh(n){return rg("N",n,void 0)}function rg(n,t,e){return{kind:n,value:t,error:e}}var hi=null;function pr(n){if(vn.useDeprecatedSynchronousErrorHandling){let t=!hi;if(t&&(hi={errorThrown:!1,error:null}),n(),t){let{errorThrown:e,error:o}=hi;if(hi=null,e)throw o}}else n()}function _h(n){vn.useDeprecatedSynchronousErrorHandling&&hi&&(hi.errorThrown=!0,hi.error=n)}var Ci=class extends Oe{constructor(t){super(),this.isStopped=!1,t?(this.destination=t,lc(t)&&t.add(this)):this.destination=qy}static create(t,e,o){return new lo(t,e,o)}next(t){this.isStopped?sg(mh(t),this):this._next(t)}error(t){this.isStopped?sg(fh(t),this):(this.isStopped=!0,this._error(t))}complete(){this.isStopped?sg(ph,this):(this.isStopped=!0,this._complete())}unsubscribe(){this.closed||(this.isStopped=!0,super.unsubscribe(),this.destination=null)}_next(t){this.destination.next(t)}_error(t){try{this.destination.error(t)}finally{this.unsubscribe()}}_complete(){try{this.destination.complete()}finally{this.unsubscribe()}}},Wy=Function.prototype.bind;function ag(n,t){return Wy.call(n,t)}var cg=class{constructor(t){this.partialObserver=t}next(t){let{partialObserver:e}=this;if(e.next)try{e.next(t)}catch(o){uc(o)}}error(t){let{partialObserver:e}=this;if(e.error)try{e.error(t)}catch(o){uc(o)}else uc(t)}complete(){let{partialObserver:t}=this;if(t.complete)try{t.complete()}catch(e){uc(e)}}},lo=class extends Ci{constructor(t,e,o){super();let i;if(de(t)||!t)i={next:t??void 0,error:e??void 0,complete:o??void 0};else{let r;this&&vn.useDeprecatedNextContext?(r=Object.create(t),r.unsubscribe=()=>this.unsubscribe(),i={next:t.next&&ag(t.next,r),error:t.error&&ag(t.error,r),complete:t.complete&&ag(t.complete,r)}):i=t}this.destination=new cg(i)}};function uc(n){vn.useDeprecatedSynchronousErrorHandling?_h(n):dc(n)}function Gy(n){throw n}function sg(n,t){let{onStoppedNotification:e}=vn;e&&gr.setTimeout(()=>e(n,t))}var qy={closed:!0,next:ha,error:Gy,complete:ha};var fr=typeof Symbol=="function"&&Symbol.observable||"@@observable";function yn(n){return n}function gc(...n){return lg(n)}function lg(n){return n.length===0?yn:n.length===1?n[0]:function(e){return n.reduce((o,i)=>i(o),e)}}var te=class n{constructor(t){t&&(this._subscribe=t)}lift(t){let e=new n;return e.source=this,e.operator=t,e}subscribe(t,e,o){let i=Ky(t)?t:new lo(t,e,o);return pr(()=>{let{operator:r,source:a}=this;i.add(r?r.call(i,a):a?this._subscribe(i):this._trySubscribe(i))}),i}_trySubscribe(t){try{return this._subscribe(t)}catch(e){t.error(e)}}forEach(t,e){return e=hh(e),new e((o,i)=>{let r=new lo({next:a=>{try{t(a)}catch(s){i(s),r.unsubscribe()}},error:i,complete:o});this.subscribe(r)})}_subscribe(t){var e;return(e=this.source)===null||e===void 0?void 0:e.subscribe(t)}[fr](){return this}pipe(...t){return lg(t)(this)}toPromise(t){return t=hh(t),new t((e,o)=>{let i;this.subscribe(r=>i=r,r=>o(r),()=>e(i))})}};te.create=n=>new te(n);function hh(n){var t;return(t=n??vn.Promise)!==null&&t!==void 0?t:Promise}function Yy(n){return n&&de(n.next)&&de(n.error)&&de(n.complete)}function Ky(n){return n&&nLine truncated
|
||||||
|
`);return o>=0?e.slice(0,o):e}function Gc(n,t){return n?t?`${n} ${t}`:n:t||""}var fx=we({__forward_ref__:we});function yt(n){return n.__forward_ref__=yt,n}function Mt(n){return Ig(n)?n():n}function Ig(n){return typeof n=="function"&&Object.hasOwn(n,fx)&&n.__forward_ref__===yt}function ee(n){return{token:n.token,providedIn:n.providedIn||null,factory:n.factory,value:void 0}}function at(n){return{providers:n.providers||[],imports:n.imports||[]}}function Ea(n){return mx(n,qc)}function Tg(n){return Ea(n)!==null}function mx(n,t){return Object.hasOwn(n,t)&&n[t]||null}function _x(n){let t=n?.[qc]??null;return t||null}function Mg(n){return n&&Object.hasOwn(n,jc)?n[jc]:null}var qc=we({\u0275prov:we}),jc=we({\u0275inj:we}),$=class{_desc;ngMetadataName="InjectionToken";\u0275prov;constructor(t,e){this._desc=t,this.\u0275prov=void 0,typeof e=="number"?this.__NG_ELEMENT_ID__=e:e!==void 0&&(this.\u0275prov=ee({token:this,providedIn:e.providedIn||"root",factory:e.factory}))}get multi(){return this}toString(){return`InjectionToken ${this._desc}`}};function Ag(n){return n&&!!n.\u0275providers}var Da=we({\u0275cmp:we}),Ia=we({\u0275dir:we}),Rg=we({\u0275pipe:we}),Fg=we({\u0275mod:we}),ya=we({\u0275fac:we}),wi=we({__NG_ELEMENT_ID__:we}),zh=we({__NG_ENV_ID__:we});function qh(n){return Yc(n,"@NgModule"),n[Fg]||null}function Lo(n){return Yc(n,"@Component"),n[Da]||null}function Ng(n){return Yc(n,"@Directive"),n[Ia]||null}function Yh(n){return Yc(n,"@Pipe"),n[Rg]||null}function Yc(n,t){if(n==null)throw new j(-919,!1)}function wn(n){return typeof n=="string"?n:n==null?"":String(n)}var Kh=we({ngErrorCode:we}),hx=we({ngErrorMessage:we}),Cx=we({ngTokenPath:we});function Lg(n,t){return Zh("",-200,t)}function Kc(n,t){throw new j(-201,!1)}function Zh(n,t,e){let o=new j(t,n);return o[Kh]=t,o[hx]=n,e&&(o[Cx]=e),o}function Mx(n){return n[Kh]}var bg;function Xh(){return bg}function Rt(n){let t=bg;return bg=n,t}function Vg(n,t,e){let o=Ea(n);if(o&&o.providedIn=="root")return o.value===void 0?o.value=o.factory():o.value;if(e&8)return null;if(t!==void 0)return t;Kc(n,"")}var Sn=globalThis;var bx={},Oi=bx,Px="__NG_DI_FLAG__",Pg=class{injector;constructor(t){this.injector=t}retrieve(t,e){let o=vi(e)||0;try{return this.injector.get(t,o&8?null:Oi,o)}catch(i){if(dr(i))return i;throw i}}};function Ox(n,t=0){let e=sc();if(e===void 0)throw new j(-203,!1);if(e===null)return Vg(n,void 0,t);{let o=vx(t),i=e.retrieve(n,o);if(dr(i)){if(o.optional)return null;throw i}return i}}function Q(n,t=0){return(Xh()||Ox)(Mt(n),t)}function v(n,t){return Q(n,vi(t))}function vi(n){return typeof n>"u"||typeof n=="number"?n:0|(n.optional&&8)|(n.host&&1)|(n.self&&2)|(n.skipSelf&&4)}function vx(n){return{optional:!!(n&8),host:!!(n&1),self:!!(n&2),skipSelf:!!(n&4)}}function Og(n){let t=[];for(let e=0;e<n.length;e++){let o=Mt(n[e]);if(Array.isArray(o)){if(o.length===0)throw new j(900,!1);let i,r=0;for(let a=0;a<o.length;a++){let s=o[a],c=yx(s);typeof c=="number"?c===-1?i=s.token:r|=c:i=s}t.push(Q(i,r))}else t.push(Q(o))}return t}function yx(n){return n[Px]}function No(n,t){let e=Object.hasOwn(n,ya);return e?n[ya]:null}function Qh(n,t,e){if(n.length!==t.length)return!1;for(let o=0;o<n.length;o++){let i=n[o],r=t[o];if(e&&(i=e(i),r=e(r)),r!==i)return!1}return!0}function Jh(n){return n.flat(Number.POSITIVE_INFINITY)}function Zc(n,t){n.forEach(e=>Array.isArray(e)?Zc(e,t):t(e))}function Bg(n,t,e){t>=n.length?n.push(e):n.splice(t,0,e)}function Ta(n,t){return t>=n.length-1?n.pop():n.splice(t,1)[0]}function eC(n,t,e,o){let i=n.length;if(i==t)n.push(e,o);else if(i===1)n.push(o,n[0]),n[0]=e;else{for(i--,n.push(n[i-1],n[i]);i>t;){let r=i-2;n[i]=n[r],i--}n[t]=e,n[t+1]=o}}function Xc(n,t,e){let o=Or(n,t);return o>=0?n[o|1]=e:(o=~o,eC(n,o,t,e)),o}function Qc(n,t){let e=Or(n,t);if(e>=0)return n[e|1]}function Or(n,t){return xx(n,t,1)}function xx(n,t,e){let o=0,i=n.length>>e;for(;i!==o;){let r=o+(i-o>>1),a=n[r<<e];if(t===a)return r<<e;a>t?i=r:o=r+1}return~(i<<e)}var Vo={},Ft=[],Si=new $(""),Aa=new $("",-1),jg=new $(""),br=class{get(t,e=Oi){if(e===Oi){let i=Zh("",-201);throw i.name="\u0275NotFound",i}return e}};function Bo(n){return{\u0275providers:n}}function tC(...n){return{\u0275providers:zg(!0,n),\u0275fromNgModule:!0}}function zg(n,...t){let e=[],o=new Set,i,r=a=>{e.push(a)};return Zc(t,a=>{let s=a;zc(s,r,[],o)&&(i||=[],i.push(s))}),i!==void 0&&nC(i,r),e}function nC(n,t){for(let e=0;e<n.length;e++){let{ngModule:o,providers:i}=n[e];$g(i,r=>{t(r,o)})}}function zc(n,t,e,o){if(n=Mt(n),!n)return!1;let i=null,r=Mg(n),a=!r&&Lo(n);if(!r&&!a){let c=n.ngModule;if(r=Mg(c),r)i=c;else return!1}else{if(a&&!a.standalone)return!1;i=n}let s=o.has(i);if(a){if(s)return!1;if(o.add(i),a.dependencies){let c=typeof a.dependencies=="function"?a.dependencies():a.dependencies;for(let u of c)zc(u,t,e,o)}}else if(r){if(r.imports!=null&&!s){o.add(i);let u;Zc(r.imports,f=>{zc(f,t,e,o)&&(u||=[],u.push(f))}),u!==void 0&&nC(u,t)}if(!s){let u=No(i)||(()=>new i);t({provide:i,useFactory:u,deps:Ft},i),t({provide:jg,useValue:i,
Line truncated
|
||||||
|
`).forEach(e=>{let o=e.indexOf(":");if(o>0){let i=e.slice(0,o),r=e.slice(o+1).trim();this.addHeaderEntry(i,r)}})}:typeof Headers<"u"&&t instanceof Headers?(this.headers=new Map,t.forEach((e,o)=>{this.addHeaderEntry(o,e)})):this.lazyInit=()=>{this.headers=new Map,Object.entries(t).forEach(([e,o])=>{this.setHeaderEntries(e,o)})}:this.headers=new Map}has(t){return this.init(),this.headers.has(t.toLowerCase())}get(t){this.init();let e=this.headers.get(t.toLowerCase());return e&&e.length>0?e[0]:null}keys(){return this.init(),Array.from(this.normalizedNames.values())}getAll(t){return this.init(),this.headers.get(t.toLowerCase())||null}append(t,e){return this.clone({name:t,value:e,op:"a"})}set(t,e){return this.clone({name:t,value:e,op:"s"})}delete(t,e){return this.clone({name:t,value:e,op:"d"})}maybeSetNormalizedName(t,e){this.normalizedNames.has(e)||this.normalizedNames.set(e,t)}init(){this.lazyInit&&(this.lazyInit instanceof n?this.copyFrom(this.lazyInit):this.lazyInit(),this.lazyInit=null,this.lazyUpdate&&(this.lazyUpdate.forEach(t=>this.applyUpdate(t)),this.lazyUpdate=null))}copyFrom(t){t.init();for(let[e,o]of t.headers.entries())this.headers.set(e,o),this.normalizedNames.set(e,t.normalizedNames.get(e))}clone(t){let e=new n;return e.lazyInit=this.lazyInit&&this.lazyInit instanceof n?this.lazyInit:this,e.lazyUpdate=(this.lazyUpdate||[]).concat([t]),e}applyUpdate(t){let e=t.name.toLowerCase();switch(t.op){case"a":case"s":let o=t.value;if(typeof o=="string"&&(o=[o]),o.length===0)return;this.maybeSetNormalizedName(t.name,e);let i=t.op==="a"?(this.headers.get(e)||[]).slice():[];i.push(...o),this.headers.set(e,i);break;case"d":let r=t.value;if(r===void 0)this.headers.delete(e),this.normalizedNames.delete(e);else{let a=Array.isArray(r)?r:[r],s=this.headers.get(e);if(!s)return;s=s.filter(c=>a.indexOf(c)===-1),s.length===0?(this.headers.delete(e),this.normalizedNames.delete(e)):this.headers.set(e,s)}break}}addHeaderEntry(t,e){let o=t.toLowerCase();this.maybeSetNormalizedName(t,o),this.headers.has(o)?this.headers.get(o).push(e):this.headers.set(o,[e])}setHeaderEntries(t,e){let o=(Array.isArray(e)?e:[e]).map(r=>r.toString()),i=t.toLowerCase();this.headers.set(i,o),this.maybeSetNormalizedName(t,i)}forEach(t){this.init(),Array.from(this.normalizedNames.keys()).forEach(e=>t(this.normalizedNames.get(e),this.headers.get(e)))}};var Sd=class{map=new Map;set(t,e){return this.map.set(t,e),this}get(t){return this.map.has(t)||this.map.set(t,t.defaultValue()),this.map.get(t)}delete(t){return this.map.delete(t),this}has(t){return this.map.has(t)}keys(){return this.map.keys()}},kd=class{encodeKey(t){return cP(t)}encodeValue(t){return cP(t)}decodeKey(t){return decodeURIComponent(t)}decodeValue(t){return decodeURIComponent(t)}};function WE(n,t){let e=new Map;return n.length>0&&n.replace(/^\?/,"").split("&").forEach(i=>{let r=i.indexOf("="),[a,s]=r==-1?[t.decodeKey(i),""]:[t.decodeKey(i.slice(0,r)),t.decodeValue(i.slice(r+1))],c=e.get(a)||[];c.push(s),e.set(a,c)}),e}var GE=/%(\d[a-f0-9])/gi,qE={40:"@","3A":":",24:"$","2C":",","3B":";","3D":"=","3F":"?","2F":"/"};function cP(n){return encodeURIComponent(n).replace(GE,(t,e)=>qE[e]??t)}function wd(n){return`${n}`}var yo=class n{map;encoder;updates=null;cloneFrom=null;constructor(t={}){if(this.encoder=t.encoder||new kd,t.fromString){if(t.fromObject)throw new j(2805,!1);this.map=WE(t.fromString,this.encoder)}else t.fromObject?(this.map=new Map,Object.keys(t.fromObject).forEach(e=>{let o=t.fromObject[e],i=Array.isArray(o)?o.map(wd):[wd(o)];this.map.set(e,i)})):this.map=null}has(t){return this.init(),this.map.has(t)}get(t){this.init();let e=this.map.get(t);return e?e[0]:null}getAll(t){return this.init(),this.map.get(t)||null}keys(){return this.init(),Array.from(this.map.keys())}append(t,e){return this.clone({param:t,value:e,op:"a"})}appendAll(t){let e=[];return Object.keys(t).forEach(o=>{let i=t[o];Array.isArray(i)?i.forEach(r=>{e.push({param:o,value:r,op:"a"})}):e.push({param:o,value:i,op:"a"})}),this.clone(e)}set(t,e){return this.clone({param:t,value:e,op:"s"})}delete(t,e){return this.clone({param:t,value:e,op:"d"})}toString(){return this.init(),this.keys().map(t=>{let e=this.encoder.encodeKey(t);return this.map.get(t).map(o=>e+"="+this.encoder.encodeValue(o)).join("&")}).filter(t=>t!=="").join("&")}clone(t){let e=new n({encoder:this.encoder});return e.cloneFrom=this.cloneFrom||this,e.updates=(this.updates||[]).concat(t),e}init(){if(this.map===null&&(this.map=new Map),this.cloneFrom!==null){this.cloneFrom.init();for(let[t,e]of this.cloneFrom.map.entries())this.map.set(t,e);this.updates.forEach(t=>{switch(t.op){case"a":case"s":let e=t.op==="a"?(this.map.get(t.param)||[]).slice():[];e.push(wd(t.value)),this.map.set(t.param,e);break;case"d":if(t.value!==void 0){let o=(this.map.get(t.param)||[]).slice(),i=o.indexOf(wd(t.value));i!==-1&&o.splice(i,1),o.length>0?this.map.set(t.param,o):this.map.delete(t.param)}else{this.map.delete(t.param);break}}}),this.cloneFrom=this.updates=null}}};function YE(
Line truncated
|
||||||
|
|
||||||
|
`);n.operator_native_thinking=o("thinking"),n.operator_raw_thinking=o("text");let i=e("thinking")[0],r=e("text")[0];i&&!n.operator_native_thinking_timestamp&&(n.operator_native_thinking_timestamp=i.timestamp),r&&!n.operator_raw_thinking_timestamp&&(n.operator_raw_thinking_timestamp=r.timestamp)}function oA(n,t){let e=t.data||{};if(!e.event)return;if(e.event==="run_outcome"){let s="checker-outcome",c=n.findIndex(_=>_.id===s),u=F(x(x({},c>-1?n[c].data:{}),e),{phase:"outcome",status:"done",isCompleted:!0,generic_tools:[]}),f={id:s,type:"checker",timestamp:t.timestamp,data:u};c>-1?n[c]=f:n.push(f);return}if(!e.attempt_id)return;let o=`checker-${e.attempt_id}`,i=n.findIndex(s=>s.id===o),r=i>-1?n[i]:null,a=typeof e.ts=="number"?e.ts:t.timestamp?new Date(t.timestamp).getTime()/1e3:void 0;if(e.event==="attempt_started"){let s=F(x(x({},r?.data||{}),e),{status:r?.data?.isCompleted?r.data.status:"running",isCompleted:r?.data?.isCompleted??!1,started_at:r?.data?.started_at??a,verdicts:r?.data?.verdicts||[],findings:r?.data?.findings||[],generic_tools:r?.data?.generic_tools||[]}),c={id:o,type:"checker",timestamp:r?.timestamp||t.timestamp,data:s};r?n[i]=c:n.push(c);return}if(e.event==="attempt_finished"){let s=r?.data?.started_at??a,c=Array.isArray(e.verdicts)?e.verdicts:r?.data?.verdicts||[],u=r?.data?.items?.length?r.data.items:c.map(C=>({kind:C.kind,text:C.item_text,when:C.when})),f=nA(r?.data?.stream_segments,e.stream_segments),_=F(x(x({},r?.data||{}),e),{status:e.status||"done",verdicts:c,items:u,findings:Array.isArray(e.findings)?e.findings:r?.data?.findings||[],generic_tools:r?.data?.generic_tools||[],stream_segments:f,started_at:s,finished_at:a,duration:s!==void 0&&a!==void 0?Math.max(0,a-s):void 0,isCompleted:!0});f&&jv(_,f);let h={id:o,type:"checker",timestamp:r?.timestamp||t.timestamp,data:_};r?n[i]=h:n.push(h)}}function iA(n){let t=new Map;n.forEach((e,o)=>{e.type==="checker"&&e.data?.trace_id&&t.set(String(e.data.trace_id),o)}),t.size!==0&&n.forEach((e,o)=>{if(e.type==="checker")return;let i=e.data?.generic_tools||[];if(i.length===0)return;let r=[],a=!1;for(let s of i){let c=s?.trace_id?String(s.trace_id):"",u=s?.parent_trace_id?String(s.parent_trace_id):"";if(c&&t.has(c)&&s?.type==="agent"){a=!0;continue}let f=u?t.get(u):void 0;if(f===void 0){r.push(s);continue}let _=n[f],h=_.data.generic_tools||[];h.some(C=>C.trace_id&&C.trace_id===s.trace_id)||(n[f]=F(x({},_),{data:F(x({},_.data),{generic_tools:[...h,s]})})),a=!0}a&&(n[o]=F(x({},e),{data:F(x({},e.data),{generic_tools:r})}))})}function k_(n){return n?.type==="log"&&n?.name==="hierarchy_backend"&&!!n?.payload?.previous_backend}function zv(n){if(!n||n.length===0)return[];let t=[];return n.forEach(e=>{if(e){if(e.type==="checker_event"){oA(t,e);return}if(e.type==="llm_stream"){let o=e.data?.execution_id||e.id,i=e.data?.step_id,r=e.data?.stream_type||"text",a=e.data?.text||"",s=e.data?.isCompleted??!1,c=e.data?.isReset??!1,u=e.data?.resetMessage,f=Nv(t,e.data?.parent_trace_id);if(f>-1){let h=t[f],C=x({},h.data),O=Array.isArray(C.stream_segments)?[...C.stream_segments]:[],P=O.findIndex(R=>R.execution_id===o&&R.stream_type===r),T={execution_id:o,stream_type:r==="thinking"?"thinking":"text",text:a,timestamp:P>-1?O[P].timestamp:e.timestamp,isCompleted:s,isReset:c,resetMessage:u};P>-1?O[P]=T:O.push(T),C.stream_segments=O,r==="thinking"?C.operator_native_thinking_timestamp=C.operator_native_thinking_timestamp||e.timestamp:C.operator_raw_thinking_timestamp=C.operator_raw_thinking_timestamp||e.timestamp,jv(C,O),o&&!C.execution_id&&(C.execution_id=o),t[f]=F(x({},h),{data:C});return}let _=-1;if(i&&(_=t.findIndex(h=>h.id===`step-${i}`||h.data?.step_id===i)),_===-1&&o&&(_=t.findIndex(h=>h.data?.execution_id===o||h.id===`stream-${o}`)),_===-1&&!i&&(_=t.findIndex(h=>h.type==="llm_stream"&&h.data?.isReset)),_>-1){let h=t[_],C=x({},h.data),O=Array.isArray(C.stream_resets)?[...C.stream_resets]:[];if(c){C.isReset=!0,C.resetMessage=u;let P=u||Do,T=o||`reset-${e.timestamp||C.step_id||"stream"}`,R=O.findIndex(D=>D.id===T);R>-1?O[R]={id:T,message:P,isWaiting:!0,streamType:r}:O.push({id:T,message:P,isWaiting:!0,streamType:r}),C.stream_resets=O}else o&&C.execution_id!==o?(C.stream_resets=O.map(P=>F(x({},P),{isWaiting:!1})),C.isReset=!1,C.resetMessage=void 0,C.execution_id=o):!c&&a&&a.length>0&&O.some(P=>P.isWaiting)&&(C.stream_resets=O.map(P=>F(x({},P),{isWaiting:!1})),C.isReset=!1,C.resetMessage=void 0);r==="thinking"?(C.operator_native_thinking=a,C.operator_native_thinking_timestamp=C.operator_native_thinking_timestamp||e.timestamp):(C.operator_raw_thinking=a,C.operator_raw_thinking_timestamp=C.operator_raw_thinking_timestamp||e.timestamp),i&&!C.step_id&&(C.step_id=i),o&&!C.execution_id&&(C.execution_id=o),C.isCompleted=s,s&&C.stream_resets&&(C.stream_resets=C.stream_resets.map(P=>F(x({},P),{isWaiting:!1}))),t[_]=F(x({},h),{data:C})}else{let h=i?`step-${i}`:`stream-${o}`,C={execution_id:o,step_id:i,isCompleted:s,isReset:c,resetMessage:u,stream_resets:c?[{id:o||`reset-${e.timestamp|Line truncated
|
||||||
|
`)||"","boot_logs"))}),l(8,"span",29),g(9),d(),l(10,"span"),g(11),d()()(),l(12,"pre",123),M(13,UA,2,0)(14,WA,2,0,"div",124),d()()}if(n&2){let e=m(3);p(9),y(e.copiedId()==="boot_logs"?"check":"content_copy"),p(2),y(e.copiedId()==="boot_logs"?"Copied":"Copy Logs"),p(2),b(e.emulatorLaunchState()?.logs?.length?13:14)}}function qA(n,t){if(n&1){let e=B();l(0,"div",47)(1,"div",89)(2,"div",90)(3,"div",91)(4,"div",92)(5,"span",93),g(6,"sync"),d()(),l(7,"div",94)(8,"div",95)(9,"h4",96),g(10," \u6B63\u5728\u542F\u52A8 Android \u6A21\u62DF\u5668\uFF1A "),l(11,"span",97),g(12),d()(),l(13,"span",98),g(14),d()(),l(15,"p",99),g(16),d()()(),l(17,"div",100)(18,"button",101),I("click",function(){S(e);let i=m(2);return k(i.toggleLaunchLogs())}),l(19,"span",29),g(20,"terminal"),d(),l(21,"span"),g(22),d()(),l(23,"button",102),I("click",function(){S(e);let i=m(2);return k(i.stopEmulator())}),l(24,"span",29),g(25,"cancel"),d(),l(26,"span"),g(27,"\u505C\u6B62"),d()()()(),l(28,"div",103)(29,"div",104)(30,"div",105),M(31,FA,2,0,"span",106)(32,NA,1,0,"span",107),d(),l(33,"span",108),g(34,"1. \u542F\u52A8\u8FDB\u7A0B"),d()(),z(35,"div",109),l(36,"div",104)(37,"div",105),M(38,LA,2,0,"span",106)(39,VA,1,0,"span",107)(40,BA,2,0,"span"),d(),l(41,"span",108),g(42,"2. ADB \u8FDE\u63A5"),d()(),z(43,"div",109),l(44,"div",104)(45,"div",105),M(46,jA,2,0,"span",106)(47,zA,1,0,"span",107)(48,$A,2,0,"span"),d(),l(49,"span",108),g(50,"3. \u7CFB\u7EDF\u542F\u52A8"),d()(),z(51,"div",109),l(52,"div",104)(53,"div",105)(54,"span",106),g(55,"check_circle"),d()(),l(56,"span",108),g(57,"4. \u5C31\u7EEA"),d()()(),l(58,"div",110)(59,"div",111),z(60,"div",112),d(),l(61,"div",113)(62,"span",114),g(63),d(),l(64,"span",115)(65,"span",116),g(66,"timer"),d(),g(67),d()()(),l(68,"div",117)(69,"span",118),g(70,"info"),d(),l(71,"span"),g(72,"Android \u6A21\u62DF\u5668\u51B7\u542F\u52A8\u901A\u5E38\u9700\u8981 20\u201345 \u79D2\u3002\u4E0A\u65B9\u4F1A\u663E\u793A\u5B9E\u65F6\u8FDB\u5EA6\uFF0C\u8BF7\u52FF\u91CD\u590D\u70B9\u51FB\u542F\u52A8\u3002"),d()(),M(73,GA,15,3,"div",119),d()()}if(n&2){let e=m(2);p(12),y(e.emulatorLaunchState()?.avd_name||e.activeDevice()?.serial||"\u865A\u62DF\u8BBE\u5907"),p(2),y(e.emulatorLaunchState()?.status==="booting"?"OS Booting":"Starting"),p(2),y(e.emulatorLaunchState()?.stage_message||"Initializing Android Virtual Device..."),p(2),E("active",e.showLaunchLogs()),p(4),y(e.showLaunchLogs()?"Hide Logs":"View Boot Logs"),p(7),E("done",e.emulatorLaunchState()?.progress_percent>=35)("active",e.emulatorLaunchState()?.status==="starting"),p(2),b(e.emulatorLaunchState()?.progress_percent>=35?31:32),p(4),E("done",e.emulatorLaunchState()?.progress_percent>=35),p(),E("done",e.emulatorLaunchState()?.progress_percent>=65)("active",e.emulatorLaunchState()?.status==="waiting_for_adb"),p(2),b(e.emulatorLaunchState()?.progress_percent>=65?38:e.emulatorLaunchState()?.status==="waiting_for_adb"?39:40),p(5),E("done",e.emulatorLaunchState()?.progress_percent>=65),p(),E("done",e.emulatorLaunchState()?.status==="ready")("active",e.emulatorLaunchState()?.status==="booting"||e.adbProbe()?.summary==="Device Booting"),p(2),b(e.emulatorLaunchState()?.status==="ready"?46:e.emulatorLaunchState()?.status==="booting"||e.adbProbe()?.summary==="Device Booting"?47:48),p(5),E("done",e.emulatorLaunchState()?.status==="ready"),p(),E("done",e.emulatorLaunchState()?.status==="ready"),p(8),Ie("width",e.emulatorLaunchState()?.progress_percent||25,"%"),p(3),L("",e.emulatorLaunchState()?.progress_percent||25,"% Complete"),p(4),L(" Elapsed: ",e.emulatorLaunchState()?.elapsed_seconds||0,"s "),p(6),b(e.showLaunchLogs()?73:-1)}}function YA(n,t){if(n&1&&(l(0,"div",125),g(1),d()),n&2){let e=t.$implicit;p(),y(e)}}function KA(n,t){if(n&1){let e=B();l(0,"div",135)(1,"div",120)(2,"span"),g(3,"\u8FDB\u7A0B\u8F93\u51FA\u4E0E\u9519\u8BEF\u65E5\u5FD7\uFF1A"),d(),l(4,"button",122),I("click",function(){S(e);let i=m(3);return k(i.copyToClipboard(i.emulatorLaunchState()?.logs?.join(`
|
||||||
|
`)||"","fail_logs"))}),l(5,"span",29),g(6),d(),l(7,"span"),g(8),d()()(),l(9,"pre",140),W(10,YA,2,1,"div",125,ze),d()()}if(n&2){let e=m(3);p(6),y(e.copiedId()==="fail_logs"?"check":"content_copy"),p(2),y(e.copiedId()==="fail_logs"?"Copied":"Copy Logs"),p(2),G(e.emulatorLaunchState()?.logs)}}function ZA(n,t){if(n&1){let e=B();l(0,"button",141),I("click",function(){S(e);let i=m(3);return k(i.launchAvdEmulator(i.emulatorLaunchState().avd_name))}),l(1,"span",29),g(2,"refresh"),d(),l(3,"span"),g(4),d()()}if(n&2){let e=m(3);N("disabled",e.isRemoteAdbServer()),p(4),L("Retry Launch (",e.emulatorLaunchState()?.avd_name,")")}}function XA(n,t){if(n&1){let e=B();l(0,"div",48)(1,"div",126)(2,"div",127)(3,"div",128)(4,"div",129)(5,"span",130),g(6,"error"),d()(),l(7,"div",131)(8,"h4",132),g(9,"\u6A21\u62DF\u5668\u542F\u52A8\u5931\u8D25"),d(),l(10,"p",133),g(11),d()()(),l(12,"button",134),I("click",function(){S(e);let i=m(2);return k(i.dismissEmulatorStatus())}),l(13,"span",29),g(14,"close"),d()()(),M(15,KA,12,2,"div",135),l(16,"div",136),M(17,ZA,5,2,"button",137),l(18,"button",138),I("click",function(){S(e);let i=m(2);return k(i.restartAdbServer())}),l(19,"span",29),g(20,"sync"),d(),l(21,"span"),g(22,"\u91CD\u542F ADB \u670D\u52A1\u5668"),d()(),l(23,"button",139),I("click",function(){S(e);let i=m(2);return k(i.dismissEmulatorStatus())}),l(24,"span"),g(25,"\u5173\u95ED\u63D0\u793A"),d()()()()()}if(n&2){let e=m(2);p(11),y(e.emulatorLaunchState()?.error||"The Android emulator process failed to start or crashed during initialization."),p(4),b(e.emulatorLaunchState()?.logs?.length?15:-1),p(2),b(e.emulatorLaunchState()?.avd_name?17:-1)}}function QA(n,t){if(n&1&&g(0),n&2){let e=m(3);L(" ",e.installedAvds().length," \u4E2A\u672C\u5730\u865A\u62DF\u8BBE\u5907\u53EF\u7528 ")}}function JA(n,t){n&1&&g(0," \u542F\u52A8\u672C\u5730 Android \u865A\u62DF\u8BBE\u5907 ")}function e5(n,t){if(n&1){let e=B();l(0,"div",152)(1,"div",155)(2,"span",29),g(3,"hub"),d(),l(4,"span"),g(5," Artemis \u5F53\u524D\u4F7F\u7528 "),l(6,"strong"),g(7),d(),g(8,"\u3002\u4F7F\u7528\u672C\u5730\u6A21\u62DF\u5668\u3001USB \u6216\u65E0\u7EBF ADB \u524D\uFF0C\u8BF7\u5148\u5207\u6362\u81F3\u672C\u5730 ADB\u3002 "),d()(),l(9,"button",156),I("click",function(){S(e);let i=m(3);return k(i.switchToLocalAdbServer())}),l(10,"span",29),g(11,"sync"),d(),l(12,"span"),g(13),d()()()}if(n&2){let e=m(3);p(7),tt("",e.adbServerStatus()?.endpoint?.host,":",e.adbServerStatus()?.endpoint?.port),p(2),N("disabled",e.isSwitchingToLocalAdb()),p(),E("spinning",e.isSwitchingToLocalAdb()),p(3),y(e.isSwitchingToLocalAdb()?"Switching...":"Use Local ADB")}}function t5(n,t){if(n&1&&(z(0,"span",174),l(1,"span"),g(2),d()),n&2){let e=m(6);p(2),tt("",e.emulatorLaunchState()?.status==="booting"?"Booting OS":"Starting"," (",e.emulatorLaunchState()?.elapsed_seconds||0,"s)")}}function n5(n,t){n&1&&(l(0,"span",175),g(1,"play_arrow"),d(),l(2,"span"),g(3,"\u542F\u52A8"),d())}function o5(n,t){if(n&1){let e=B();l(0,"div",165)(1,"div",166)(2,"div",167)(3,"span",29),g(4,"smartphone"),d()(),l(5,"div",168)(6,"div",169)(7,"span",170),g(8),d(),l(9,"span",171),g(10,"\u865A\u62DF\u8BBE\u5907"),d()()()(),l(11,"div",172)(12,"button",173),I("click",function(){let i=S(e).$implicit,r=m(5);return k(r.launchAvdEmulator(i))}),M(13,t5,3,2)(14,n5,4,0),d()()()}if(n&2){let e=t.$implicit,o=m(5);p(8),y(e),p(4),E("launching",o.launchingAvd()===e||o.isEmulatorLaunching()&&o.emulatorLaunchState()?.avd_name===e),N("disabled",o.isEmulatorLaunching()||o.isRemoteAdbServer())("title",o.isEmulatorLaunching()?"\u6B63\u5728\u542F\u52A8\u6A21\u62DF\u5668\u2026":"\u542F\u52A8"+e),p(),b(o.launchingAvd()===e||o.isEmulatorLaunching()&&o.emulatorLaunchState()?.avd_name===e?13:14)}}function i5(n,t){if(n&1&&(l(0,"div",157)(1,"div",159)(2,"div",160)(3,"span",161),g(4,"devices"),d(),l(5,"span",162),g(6,"\u5DF2\u5B89\u88C5\u7684\u865A\u62DF\u8BBE\u5907\uFF08AVD\uFF09"),d(),l(7,"span",163),g(8),d()()(),l(9,"div",164),W(10,o5,15,6,"div",165,ze),d()()),n&2){let e=m(4);p(8),L("",e.installedAvds().length," \u4E2A\u53EF\u7528"),p(2),G(e.installedAvds())}}function r5(n,t){if(n&1&&(l(0,"div",186)(1,"div",188)(2,"div",189)(3,"span",190),g(4,"1"),d(),l(5,"span",191),g(6,"download"),d()(),l(7,"div",192),g(8,"\u5B89\u88C5 Android Studio"),d(),l(9,"div",193),g(10),l(11,"a",194),g(12,"developer.android.com/studio"),d(),g(13," \u5E76\u5B89\u88C5\u3002 "),d()(),l(14,"div",188)(15,"div",189)(16,"span",190),g(17,"2"),d(),l(18,"span",191),g(19,"developer_mode"),d()(),l(20,"div",192),g(21,"\u6253\u5F00\u8BBE\u5907\u7BA1\u7406\u5668"),d(),l(22,"div",193),g(23," \u5728 Android Studio \u6B22\u8FCE\u9875\u9762\u6253\u5F00 "),l(24,"strong"),g(25,"\u66F4\u591A\u64CD\u4F5C > \u865A\u62DF\u8BBE\u5907\u7BA1\u7406\u5668"),d(),g(26," \uFF08\u6216 "),l(27,"strong"),g(28,"\u5DE5\u5177 > \u8BBE\u5907\u7BA1\u7406\u5668"),d(),g(29,"). "),d()(),l(30,"div",188)(31,"div",189)(32,"span",190),g(33,"3"),d(),l(34,"span",191),g(35,"add_circle"),d()(),l(36,"div",192),g(37,"\u521B\u5EFA\u865A\u62D
Line truncated
|
||||||
|
pointer-events: none;
|
||||||
|
top: 0;
|
||||||
|
left: 0;
|
||||||
|
height: 100%;
|
||||||
|
width: 100%;
|
||||||
|
}
|
||||||
|
|
||||||
|
.cdk-overlay-container {
|
||||||
|
position: fixed;
|
||||||
|
}
|
||||||
|
@layer cdk-overlay {
|
||||||
|
.cdk-overlay-container {
|
||||||
|
z-index: 1000;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
.cdk-overlay-container:empty {
|
||||||
|
display: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
.cdk-global-overlay-wrapper {
|
||||||
|
display: flex;
|
||||||
|
position: absolute;
|
||||||
|
}
|
||||||
|
@layer cdk-overlay {
|
||||||
|
.cdk-global-overlay-wrapper {
|
||||||
|
z-index: 1000;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
.cdk-overlay-pane {
|
||||||
|
position: absolute;
|
||||||
|
pointer-events: auto;
|
||||||
|
box-sizing: border-box;
|
||||||
|
display: flex;
|
||||||
|
max-width: 100%;
|
||||||
|
max-height: 100%;
|
||||||
|
}
|
||||||
|
@layer cdk-overlay {
|
||||||
|
.cdk-overlay-pane {
|
||||||
|
z-index: 1000;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
.cdk-overlay-backdrop {
|
||||||
|
position: absolute;
|
||||||
|
top: 0;
|
||||||
|
bottom: 0;
|
||||||
|
left: 0;
|
||||||
|
right: 0;
|
||||||
|
pointer-events: auto;
|
||||||
|
-webkit-tap-highlight-color: transparent;
|
||||||
|
opacity: 0;
|
||||||
|
touch-action: manipulation;
|
||||||
|
}
|
||||||
|
@layer cdk-overlay {
|
||||||
|
.cdk-overlay-backdrop {
|
||||||
|
z-index: 1000;
|
||||||
|
transition: opacity 400ms cubic-bezier(0.25, 0.8, 0.25, 1);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
@media (prefers-reduced-motion) {
|
||||||
|
.cdk-overlay-backdrop {
|
||||||
|
transition-duration: 1ms;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
.cdk-overlay-backdrop-showing {
|
||||||
|
opacity: 1;
|
||||||
|
}
|
||||||
|
@media (forced-colors: active) {
|
||||||
|
.cdk-overlay-backdrop-showing {
|
||||||
|
opacity: 0.6;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
@layer cdk-overlay {
|
||||||
|
.cdk-overlay-dark-backdrop {
|
||||||
|
background: rgba(0, 0, 0, 0.32);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
.cdk-overlay-transparent-backdrop {
|
||||||
|
transition: visibility 1ms linear, opacity 1ms linear;
|
||||||
|
visibility: hidden;
|
||||||
|
opacity: 1;
|
||||||
|
}
|
||||||
|
.cdk-overlay-transparent-backdrop.cdk-overlay-backdrop-showing, .cdk-high-contrast-active .cdk-overlay-transparent-backdrop {
|
||||||
|
opacity: 0;
|
||||||
|
visibility: visible;
|
||||||
|
}
|
||||||
|
|
||||||
|
.cdk-overlay-backdrop-noop-animation {
|
||||||
|
transition: none;
|
||||||
|
}
|
||||||
|
|
||||||
|
.cdk-overlay-connected-position-bounding-box {
|
||||||
|
position: absolute;
|
||||||
|
display: flex;
|
||||||
|
flex-direction: column;
|
||||||
|
min-width: 1px;
|
||||||
|
min-height: 1px;
|
||||||
|
}
|
||||||
|
@layer cdk-overlay {
|
||||||
|
.cdk-overlay-connected-position-bounding-box {
|
||||||
|
z-index: 1000;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
.cdk-global-scrollblock {
|
||||||
|
position: fixed;
|
||||||
|
width: 100%;
|
||||||
|
overflow-y: scroll;
|
||||||
|
}
|
||||||
|
|
||||||
|
.cdk-overlay-popover {
|
||||||
|
background: none;
|
||||||
|
border: none;
|
||||||
|
padding: 0;
|
||||||
|
outline: 0;
|
||||||
|
overflow: visible;
|
||||||
|
position: fixed;
|
||||||
|
pointer-events: none;
|
||||||
|
white-space: normal;
|
||||||
|
color: inherit;
|
||||||
|
text-decoration: none;
|
||||||
|
width: 100%;
|
||||||
|
height: 100%;
|
||||||
|
inset: auto;
|
||||||
|
top: 0;
|
||||||
|
left: 0;
|
||||||
|
}
|
||||||
|
.cdk-overlay-popover::backdrop {
|
||||||
|
display: none;
|
||||||
|
}
|
||||||
|
.cdk-overlay-popover .cdk-overlay-backdrop {
|
||||||
|
position: fixed;
|
||||||
|
z-index: auto;
|
||||||
|
}
|
||||||
|
`],encapsulation:2})}return n})(),hy=(()=>{class n{_platform=v(ir);_containerElement;_document=v(Me);_styleLoader=v(A_);ngOnDestroy(){this._containerElement?.remove()}getContainerElement(){return this._loadStyles(),this._containerElement||this._createContainer(),this._containerElement}_createContainer(){let e="cdk-overlay-container";if(this._platform.isBrowser||T_()){let i=this._document.querySelectorAll(`.${e}[platform="server"], .${e}[platform="test"]`);for(let r=0;r<i.length;r++)i[r].remove()}let o=this._document.createElement("div");o.classList.add(e),T_()?o.setAttribute("platform","test"):this._platform.isBrowser||o.setAttribute("platform","server"),this._document.body.appendChild(o),this._containerElement=o}_loadStyles(){this._styleLoader.load(_y)}static \u0275fac=function(o){return new(o||n)};static \u0275prov=oe({token:n,factory:n.\u0275fac})}return n})(),$_=class{_renderer;_ngZone;element;_cleanupClick;_cleanupTransitionEnd;_fallbackTimeout;constructor(t,e,o,i){this._renderer=e,this._ngZone=o,this.element=t.createElement("div"),this.element.classList.add("cdk-overlay-backdrop"),this._cleanupClick=e.listen(this.element,"click",i)}detach(){this._ngZone.runOutsideAngular(()=>{let t=this.element;clearTimeout(this._fallbackTimeout),this._cleanupTransitionEnd?.(),this._cleanupTransitionEnd=this._renderer.listen(t,"transitionend",this.dispose),this._fallbackTimeout=setTimeout(this.dispose,500),t.style.pointerEvents="none",t.classList.remove("cdk-overlay-backdrop-showing")})}dispose=()=>{clearTimeout(this._fallbackTimeout),this._cleanupClick?.(),this._cleanupTransitionEnd?.(),this._cleanupClick=this._cleanupTransitionEnd=this._fallbackTimeout=void 0,this.element.remove()}};function U_(n){return n&&n.nodeType===1}var Vu=class{_portalOutlet;_host;_pane;_config;_ngZone;_keyboardDispatcher;_document;_location;_outsideClickDispatcher;_animationsDisabled;_injector;_renderer;_backdropClick=new fe;_attachments=new fe;_detachments=new fe;_positionStrategy;_scrollStrategy;_locationChanges=Oe.EMPTY;_backdropRef=null;_detachContentMutationObserver;_detachContentAfterRenderRef;_disposed=!1;_previousHostParent;_keydownEvents=new fe;_outsidePointerEvents=new fe;_afterNextRenderRef;constructor(t,e,o,i,r,a,s,c,u,f=!1,_,h){this._portalOutlet=t,this._host=e,this._pane=o,this._config=i,this._ngZone=r,this._keyboardDispatcher=a,this._document=s,this._location=c,this._outsideClickDispatcher=u,this._animationsDisabled=f,this._injector=_,this._renderer=h,i.scrollStrategy&&(this._scrollStrategy=i.scrollStrategy,this._scrollStrategy.attach(this)),this._positionStrategy=i.positionStrategy}get overlayElement(){return this._pane}get backdropElement(){return this._backdropRef?.element||null}get hostElement(){return this._host}get eventPredicate(){return this._config?.eventPredicate||null}attach(t){if(this._disposed)return null;this._attachHost();let e=this._portalOutlet.attach(t);return this._positionStrategy?.attach(this),this._updateStackingOrder(),this._updateElementSize(),this._updateElementDirection(),this._scrollStrategy&&this._scrollStrategy.enable(),this._afterNextRenderRef?.destroy(),this._afterNextRenderRef=Po(()=>{this.hasAttached()&&this.updatePosition()},{injector:this._injector}),this._togglePointerEvents(!0),this._config.hasBackdrop&&this._attachBackdrop(),this._config.panelClass&&this._toggleClasses(this._pane,this._config.panelClass,!0),this._attachments.next(),this._completeDetachContent(),this._keyboardDispatcher.add(this),this._config.disposeOnNavigation&&(this._locationChanges=this._location.subscribe(()=>this.dispose())),this._outsideClickDispatcher.add(this),typeof e?.onDestroy=="function"&&e.onDestroy(()=>{this.hasAttached()&&this._ngZone.runOutsideAngular(()=>Promise.resolve().then(()=>this.detach()))}),e}detach(){if(!this.hasAttached())return;this.detachBackdrop(),this._togglePointerEvents(!1),this._positionStrategy&&this._positionStrategy.detach&&this._positionStrategy.detach(),this._scrollStrategy&&this._scrollStrategy.disable();let t=this._portalOutlet.detach();return this._detachments.next(),this._completeDetachContent(),this._keyboardDispatcher.remove(this),this._detachContentWhenEmpty(),this._locationChanges.unsubscribe(),this._outsideClickDispatcher.remove(this),t}dispose(){if(this._disposed)return;let t=this.hasAttached();this._positionStrategy&&this._positionStrategy.dispose(),this._disposeScrollStrategy(),this._backdropRef?.dispose(),this._locationChanges.unsubscribe(),this._keyboardDispatcher.remove(this),this._portalOutlet.dispose(),this._attachments.complete(),this._backdropClick.complete(),this._keydownEvents.complete(),this._outsidePointerEvents.complete(),this._outsideClickDispatcher.remove(this),this._host?.remove(),this._afterNextRenderRef?.destroy(),this._previousHostParent=this._pane=this._host=this._backdropRef=null,t&&this._detachments.next(),this._detachments.complete(),this._completeDetachContent(),this._disposed=!0}hasAttached(){return this._portalOutlet.hasAttached()}backdropClick(){
Line truncated
|
||||||
|
`).map(t=>{let e=t.trim(),o=t.length-t.trimStart().length;if(e.startsWith("- [x] ")||e.startsWith("- [X] ")){let a=e.substring(6);return{type:"checked",segments:co(a),indent:o}}if(e.startsWith("- [/] ")){let a=e.substring(6);return{type:"progress",segments:co(a),indent:o}}if(e.startsWith("- [ ] ")){let a=e.substring(6);return{type:"unchecked",segments:co(a),indent:o}}let i=e.match(/^[-*]\s*(verify|assert)(@end)?\s*:\s*(.*)$/i);if(i){let a=i[1].toLowerCase(),s=!!i[2],c=i[3].trim();return{type:"verify",checkKind:a,atEnd:s,segments:co(c),indent:o}}let r=e.match(/^[-*]\s*finding\s*:\s*(.*)$/i);if(r){let a=r[1].trim();return{type:"finding",checkKind:"finding",segments:co(a),indent:o}}if(e.startsWith("- ")||e.startsWith("* ")){let a=e.substring(2);return{type:"list-item",segments:co(a),indent:o}}if(e.startsWith("# ")){let a=e.substring(2);return{type:"h1",segments:co(a),indent:o}}if(e.startsWith("## ")){let a=e.substring(3);return{type:"h2",segments:co(a),indent:o}}if(e.startsWith("### ")){let a=e.substring(4);return{type:"h3",segments:co(a),indent:o}}return e?{type:"text",segments:co(t),indent:o}:{type:"empty",segments:[],indent:0}}):[]}function ec(n){let t=Js(n),e=[],o=[],i=null,r=null,a=0;for(let s of t){if(s.type==="empty")continue;if((s.type==="h1"||s.type==="h2")&&!i){i=s.segments.map(u=>u.text).join("");continue}["checked","progress","unchecked"].includes(s.type)&&(!r||s.indent<=(r._indent??0))?(a++,r={index:a,type:s.type,segments:s.segments,subSteps:[],checks:[],_indent:s.indent},e.push(r)):r&&s.indent>(r._indent??0)?s.type==="verify"||s.type==="assert"||s.type==="finding"?r.checks.push(s):r.subSteps.push(s):o.push(s)}return{title:i,milestones:e,otherLines:o}}function Sy(n){if(!n)return null;let t=n.trim();if(t.includes("```json")){let e=t.split("```json");e.length>1&&(t=e[1].split("```")[0].trim())}else if(t.includes("```")){let e=t.split("```");e.length>1&&(t=e[1].split("```")[0].trim())}try{let e=JSON.parse(t);if(e&&typeof e=="object"&&"success"in e)return{success:!!e.success,reason:e.reason||"\u672A\u63D0\u4F9B\u539F\u56E0\u3002"}}catch(e){}return null}function sr(n){return n?n.replace(/`([^`]+)`/g,'<code class="inline-code">$1</code>').replace(/\*\*\*([^*]+)\*\*\*/g,"<strong><em>$1</em></strong>").replace(/___([^_]+)___/g,"<strong><em>$1</em></strong>").replace(/\*\*([^*]+)\*\*/g,"<strong>$1</strong>").replace(/__([^_]+)__/g,"<strong>$1</strong>").replace(/\*([^*]+)\*/g,"<em>$1</em>").replace(/\b_([^_]+)_\b/g,"<em>$1</em>").replace(/~~([^~]+)~~/g,"<del>$1</del>"):""}function X_(n){if(!n)return"";let o=n.replace(/<\/?(thought|short_term_memory|strong_term_memory|reasoning|plan|task_plan|call_tool)[^\n>]*>?/gi,"").replace(/\r\n/g,`
|
||||||
|
`).replace(/\r/g,`
|
||||||
|
`).replace(/\n{3,}/g,`
|
||||||
|
|
||||||
|
`).trim().replace(/&/g,"&").replace(/</g,"<").replace(/>/g,">").split(`
|
||||||
|
`),i=[],r=null,a=!1,s="",c=[],u=()=>{r&&(i.push(`</${r}>`),r=null)};for(let f=0;f<o.length;f++){let _=o[f],h=_.trim();if(h.startsWith("```")){if(a){let T=c.join(`
|
||||||
|
`);i.push(`<pre class="code-block"><code class="lang-${s}">${T}</code></pre>`),a=!1,c=[],s=""}else u(),a=!0,s=h.substring(3).trim(),c=[];continue}if(a){c.push(_);continue}let C=h.match(/^(#{1,6})\s+(.*)$/);if(C){u();let T=C[1].length,R=sr(C[2]);i.push(`<h${T}>${R}</h${T}>`);continue}if(/^(\-{3,}|\*{3,}|_{3,})$/.test(h)){u(),i.push('<hr class="md-hr" />');continue}if(h.startsWith("> ")||h.startsWith(">")){u();let T=sr(h.replace(/^>\s?/,""));i.push(`<blockquote>${T}</blockquote>`);continue}let O=_.match(/^(\s*)([\*\-])\s+(.*)$/);if(O){r!=="ul"&&(u(),r="ul",i.push("<ul>"));let T=O[1].length,D=Math.max(0,Math.floor((T-2)/2))*12,H=D?` style="margin-left: ${D}px"`:"",ue=O[3],J=ue.match(/^(verify|assert)(@end)?\s*:\s*(.*)$/i),X=ue.match(/^finding\s*:\s*(.*)$/i);if(J){let q=J[1].toLowerCase(),he=J[2]?"@end":"",Ve=q==="assert"?" assert":"",Dt=sr(J[3].trim());i.push(`<li${H} class="md-verify-item"><span class="verify-badge${Ve}">${q}${he}</span>${Dt}</li>`)}else if(X){let q=sr(X[1].trim());i.push(`<li${H} class="md-finding-item"><span class="finding-badge">finding</span>${q}</li>`)}else{let q=sr(ue);i.push(`<li${H}>${q}</li>`)}continue}let P=_.match(/^(\s*)(\d+)\.\s+(.*)$/);if(P){r!=="ol"&&(u(),r="ol",i.push("<ol>"));let T=P[1].length,D=Math.max(0,Math.floor((T-2)/2))*12,H=D?` style="margin-left: ${D}px"`:"",ue=sr(P[3]);i.push(`<li${H}>${ue}</li>`);continue}if(u(),h==="")i.push('<div class="md-para-gap"></div>');else{let T=sr(_);i.push(`<div>${T}</div>`)}}if(a){let f=c.join(`
|
||||||
|
`);i.push(`<pre class="code-block"><code class="lang-${s}">${f}</code></pre>`)}return u(),i.join(`
|
||||||
|
`)}var MR=n=>[n],Ey=(n,t)=>t.id||n;function bR(n,t){n&1&&(l(0,"div",28),g(1," \u6CA1\u6709\u6B63\u5728\u6267\u884C\u6216\u6392\u961F\u7684\u4EFB\u52A1 "),d())}function PR(n,t){n&1&&z(0,"span",36)}function OR(n,t){if(n&1&&(l(0,"span",38)(1,"span",47),g(2,"phone_android"),d(),l(3,"span",48),g(4),d()()),n&2){let e=m().$implicit,o=m(2);N("title","Device: "+o.getDeviceSerial(e)),p(4),y(o.getDeviceSerial(e))}}function vR(n,t){n&1&&(l(0,"span",39),g(1,"\xB7"),d())}function yR(n,t){if(n&1){let e=B();l(0,"button",49),I("click",function(i){S(e);let r=m().$implicit,a=m(2);return k(a.stopTask(r.session_id,i))}),l(1,"span",25),g(2,"stop_circle"),d()()}}function xR(n,t){if(n&1){let e=B();l(0,"button",50),I("click",function(i){S(e);let r=m().$implicit,a=m(2);return k(a.deleteTask(r.session_id,i))}),l(1,"span",25),g(2,"delete"),d()()}}function wR(n,t){if(n&1){let e=B();l(0,"div",33),I("click",function(i){let r=S(e).$implicit,a=m(2);return k(a.selectTask(r.session_id,i))}),l(1,"div",34)(2,"span",35),M(3,PR,1,0,"span",36),g(4),mt(5,"uiLabel"),d(),l(6,"div",37),M(7,OR,5,2,"span",38),M(8,vR,2,0,"span",39),l(9,"span",40),g(10),mt(11,"date"),d(),M(12,yR,3,0,"button",41),M(13,xR,3,0,"button",42),d()(),l(14,"div",43),g(15),d(),l(16,"div",44)(17,"span",45),g(18),d(),l(19,"span",46),g(20),mt(21,"date"),d()()()}if(n&2){let e=t.$implicit,o=m(2);E("selected",o.agentService.currentSessionId()===e.session_id)("running",o.getTaskStatus(e)==="running"),p(2),N("ngClass",o.getTaskStatus(e)),p(),b(o.getTaskStatus(e)==="running"?3:-1),p(),L(" ",qi(5,18,o.getTaskStatus(e))," "),p(3),b(o.getDeviceSerial(e)?7:-1),p(),b(o.getDeviceSerial(e)?8:-1),p(2),y(qt(11,20,e.start_time*1e3,"shortTime")),p(2),b(o.getTaskStatus(e)==="running"||o.getTaskStatus(e)==="paused"?12:-1),p(),b(o.getTaskStatus(e)!=="running"&&o.getTaskStatus(e)!=="paused"?13:-1),p(),N("title",Gi(e.initial_goal)),p(),L(" ",e.initial_goal||"\u672A\u547D\u540D\u4EFB\u52A1"," "),p(2),N("title",e.session_id),p(),L("ID: ",e.session_id),p(2),y(qt(21,23,e.start_time*1e3,"MM/dd/yy"))}}function SR(n,t){n&1&&(l(0,"div",28),g(1," \u6682\u65E0\u5386\u53F2\u4EFB\u52A1 "),d())}function kR(n,t){n&1&&z(0,"span",36)}function ER(n,t){if(n&1&&(l(0,"span",38)(1,"span",47),g(2,"phone_android"),d(),l(3,"span",48),g(4),d()()),n&2){let e=m().$implicit,o=m(2);N("title","Device: "+o.getDeviceSerial(e)),p(4),y(o.getDeviceSerial(e))}}function DR(n,t){n&1&&(l(0,"span",39),g(1,"\xB7"),d())}function IR(n,t){if(n&1){let e=B();l(0,"button",50),I("click",function(i){S(e);let r=m().$implicit,a=m(2);return k(a.deleteTask(r.session_id,i))}),l(1,"span",25),g(2,"delete"),d()()}}function TR(n,t){if(n&1){let e=B();l(0,"div",33),I("click",function(i){let r=S(e).$implicit,a=m(2);return k(a.selectTask(r.session_id,i))}),l(1,"div",34)(2,"span",35),M(3,kR,1,0,"span",36),g(4),mt(5,"uiLabel"),d(),l(6,"div",37),M(7,ER,5,2,"span",38),M(8,DR,2,0,"span",39),l(9,"span",40),g(10),mt(11,"date"),d(),M(12,IR,3,0,"button",42),d()(),l(13,"div",43),g(14),d(),l(15,"div",44)(16,"span",45),g(17),d(),l(18,"span",46),g(19),mt(20,"date"),d()()()}if(n&2){let e=t.$implicit,o=m(2);E("selected",o.agentService.currentSessionId()===e.session_id)("running",o.getTaskStatus(e)==="running"),p(2),N("ngClass",o.getTaskStatus(e)),p(),b(o.getTaskStatus(e)==="running"?3:-1),p(),L(" ",qi(5,17,o.getTaskStatus(e))," "),p(3),b(o.getDeviceSerial(e)?7:-1),p(),b(o.getDeviceSerial(e)?8:-1),p(2),y(qt(11,19,e.start_time*1e3,"shortTime")),p(2),b(o.getTaskStatus(e)!=="running"&&o.getTaskStatus(e)!=="paused"?12:-1),p(),N("title",Gi(e.initial_goal)),p(),L(" ",e.initial_goal||"\u672A\u547D\u540D\u4EFB\u52A1"," "),p(2),N("title",e.session_id),p(),L("ID: ",e.session_id),p(2),y(qt(20,22,e.start_time*1e3,"MM/dd/yy"))}}function AR(n,t){if(n&1&&(l(0,"div",23)(1,"div",24)(2,"span",25),g(3,"list_alt"),d(),l(4,"span"),g(5,"\u4EFB\u52A1\u961F\u5217"),d()(),l(6,"div",26)(7,"span",27),g(8,"bolt"),d(),l(9,"span"),g(10),d()(),M(11,bR,2,0,"div",28),W(12,wR,22,26,"div",29,je().trackSession,!0),l(14,"div",30)(15,"span",31),g(16,"history"),d(),l(17,"span"),g(18),d()(),M(19,SR,2,0,"div",28),l(20,"div",32),W(21,TR,21,25,"div",29,je().trackSession,!0),d()()),n&2){let e=m();p(10),L("Active Queue (",e.activeQueue().length,")"),p(),b(e.activeQueue().length===0?11:-1),p(),G(e.activeQueue()),p(6),L("Past Runs & History (",e.historyTasks().length,")"),p(),b(e.historyTasks().length===0?19:-1),p(2),G(e.historyTasks())}}function RR(n,t){if(n&1){let e=B();l(0,"button",56),I("click",function(i){let r=S(e).$implicit,a=m(3);return k(a.selectNote(r,i))}),l(1,"span",57),g(2,"draft"),d(),g(3),d()}if(n&2){let e=t.$implicit,o=m(3);E("active",o.agentService.selectedNoteKey()===e),p(3),L(" ",e," ")}}function FR(n,t){if(n&1&&(l(0,"div",52)(1,"div",54),W(2,RR,4,3,"button",55,je().trackNoteKey,!0),d()()),n&2){let e=m(2);p(2),G(e.noteKeys())}}function NR(n,t){n&1&&(l(0,"div",28)(1,"span",58),g(2,"description"),d(),l(3,"span"),g(4,"\u6B64\u4F1A\u8BDD\u6682\u65E0\u7B14\u8BB0\u3002"),d()())}function LR(n,t){if(n&1&&(l(0,"h3",5
Line truncated
|
||||||
+1
-1
@@ -1 +1 @@
|
|||||||
@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-diLine truncated
|
@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:300;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1pL7SUc.woff2) format("woff2");unicode-range:U+0370-0377,U+037A-037F,U+0384-038A,U+038C,U+038E-03A1,U+03A3-03FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2pL7SUc.woff2) format("woff2");unicode-range:U+0102-0103,U+0110-0111,U+0128-0129,U+0168-0169,U+01A0-01A1,U+01AF-01B0,U+0300-0301,U+0303-0304,U+0308-0309,U+0323,U+0329,U+1EA0-1EF9,U+20AB}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa25L7SUc.woff2) format("woff2");unicode-range:U+0100-02BA,U+02BD-02C5,U+02C7-02CC,U+02CE-02D7,U+02DD-02FF,U+0304,U+0308,U+0329,U+1D00-1DBF,U+1E00-1E9F,U+1EF2-1EFF,U+2020,U+20A0-20AB,U+20AD-20C0,U+2113,U+2C60-2C7F,U+A720-A7FF}@font-face{font-family:Inter;font-style:normal;font-weight:400;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa1ZL7.woff2) format("woff2");unicode-range:U+0000-00FF,U+0131,U+0152-0153,U+02BB-02BC,U+02C6,U+02DA,U+02DC,U+0304,U+0308,U+0329,U+2000-206F,U+20AC,U+2122,U+2191,U+2193,U+2212,U+2215,U+FEFF,U+FFFD}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2JL7SUc.woff2) format("woff2");unicode-range:U+0460-052F,U+1C80-1C8A,U+20B4,U+2DE0-2DFF,U+A640-A69F,U+FE2E-FE2F}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa0ZL7SUc.woff2) format("woff2");unicode-range:U+0301,U+0400-045F,U+0490-0491,U+04B0-04B1,U+2116}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-display:swap;src:url(https://fonts.gstatic.com/s/inter/v20/UcC73FwrK3iLTeHuS_nVMrMxCp50SjIa2ZL7SUc.woff2) format("woff2");unicode-range:U+1F00-1FFF}@font-face{font-family:Inter;font-style:normal;font-weight:500;font-diLine truncated
|
||||||
+10
-52
@@ -38,8 +38,6 @@ import uuid
|
|||||||
|
|
||||||
from adbutils import AdbClient
|
from adbutils import AdbClient
|
||||||
from dotenv import load_dotenv
|
from dotenv import load_dotenv
|
||||||
from google import genai
|
|
||||||
from langchain_google_genai import ChatGoogleGenerativeAI
|
|
||||||
from PIL import Image
|
from PIL import Image
|
||||||
from pydantic import BaseModel
|
from pydantic import BaseModel
|
||||||
|
|
||||||
@@ -186,6 +184,16 @@ class Agent:
|
|||||||
self._initialized = False
|
self._initialized = False
|
||||||
self._task_lock = asyncio.Lock()
|
self._task_lock = asyncio.Lock()
|
||||||
|
|
||||||
|
async def _prewarm_llm_connections(self, api_key: str | None = None):
|
||||||
|
from artemis.llm.router import ModelEndpoint, ModelFactory
|
||||||
|
|
||||||
|
try:
|
||||||
|
ModelFactory.get_model(ModelEndpoint(api_key=api_key))
|
||||||
|
message = "模型连接已就绪"
|
||||||
|
except Exception:
|
||||||
|
message = "模型连接将在首次调用时初始化"
|
||||||
|
publish_startup_progress("model_ready", message, session_id=self._session_id)
|
||||||
|
|
||||||
async def init(
|
async def init(
|
||||||
self,
|
self,
|
||||||
api_key: str | None = None,
|
api_key: str | None = None,
|
||||||
@@ -273,56 +281,6 @@ class Agent:
|
|||||||
|
|
||||||
return True
|
return True
|
||||||
|
|
||||||
async def _prewarm_llm_connections(self, api_key: str | None = None):
|
|
||||||
"""Pre-warms the HTTP2/gRPC connection pools for both Native GenAI and LangChain clients in the background."""
|
|
||||||
if os.environ.get("ARTEMIS_FAKE_LLM") == "1":
|
|
||||||
logger.info("ARTEMIS_FAKE_LLM=1 — skipping real LLM connection pre-warming.")
|
|
||||||
publish_startup_progress(
|
|
||||||
"model_ready", "Model connection is ready (fake LLM)", session_id=self._session_id
|
|
||||||
)
|
|
||||||
return
|
|
||||||
publish_startup_progress(
|
|
||||||
"model_warmup", "Warming the model connection", session_id=self._session_id
|
|
||||||
)
|
|
||||||
logger.info("Starting background pre-warming of Gemini API connection pools...")
|
|
||||||
try:
|
|
||||||
key = api_key
|
|
||||||
if not key and settings.GOOGLE_API_KEY:
|
|
||||||
key = settings.GOOGLE_API_KEY.get_secret_value()
|
|
||||||
|
|
||||||
if not key:
|
|
||||||
logger.warning("Skipping LLM pre-warming: No API key available.")
|
|
||||||
publish_startup_progress(
|
|
||||||
"model_ready",
|
|
||||||
"Model connection will initialize on first use",
|
|
||||||
session_id=self._session_id,
|
|
||||||
)
|
|
||||||
return
|
|
||||||
|
|
||||||
# 1. Pre-warm Native SDK client
|
|
||||||
client = genai.Client(api_key=key)
|
|
||||||
|
|
||||||
# 2. Pre-warm LangChain client
|
|
||||||
chat = ChatGoogleGenerativeAI(model="gemini-3.8-flash", google_api_key=key)
|
|
||||||
|
|
||||||
# Fire both calls concurrently in the background
|
|
||||||
await asyncio.gather(
|
|
||||||
client.aio.models.count_tokens(model="gemini-3.8-flash", contents="ping"),
|
|
||||||
chat.ainvoke("ping"),
|
|
||||||
return_exceptions=True,
|
|
||||||
)
|
|
||||||
logger.success("Gemini API connection pools successfully pre-warmed.")
|
|
||||||
publish_startup_progress(
|
|
||||||
"model_ready", "Model connection is ready", session_id=self._session_id
|
|
||||||
)
|
|
||||||
except Exception as e:
|
|
||||||
logger.warning(f"Failed to pre-warm LLM connections: {e}")
|
|
||||||
publish_startup_progress(
|
|
||||||
"model_ready",
|
|
||||||
"Model connection will initialize on first use",
|
|
||||||
session_id=self._session_id,
|
|
||||||
)
|
|
||||||
|
|
||||||
async def install_apk(self, apk_path: str | Path) -> None:
|
async def install_apk(self, apk_path: str | Path) -> None:
|
||||||
"""Install an APK on the connected device.
|
"""Install an APK on the connected device.
|
||||||
|
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ and analyze video content from a mobile device screen.
|
|||||||
The agent can:
|
The agent can:
|
||||||
1. Start a screen recording
|
1. Start a screen recording
|
||||||
2. Perform actions while recording
|
2. Perform actions while recording
|
||||||
3. Stop the recording and analyze its content using Gemini models
|
3. Stop the recording and analyze its content using sampled frames with Responses
|
||||||
|
|
||||||
Use case: Recording a video playing on the screen and transcribing its content.
|
Use case: Recording a video playing on the screen and transcribing its content.
|
||||||
"""
|
"""
|
||||||
@@ -28,6 +28,7 @@ Use case: Recording a video playing on the screen and transcribing its content.
|
|||||||
import asyncio
|
import asyncio
|
||||||
|
|
||||||
from artemis.config import LLM, LLMConfig, LLMConfigUtils, LLMWithFallback
|
from artemis.config import LLM, LLMConfig, LLMConfigUtils, LLMWithFallback
|
||||||
|
from artemis.config import settings
|
||||||
from artemis.sdk.agent import Agent
|
from artemis.sdk.agent import Agent
|
||||||
from artemis.sdk.builders.agent_config_builder import AgentConfigBuilder
|
from artemis.sdk.builders.agent_config_builder import AgentConfigBuilder
|
||||||
from artemis.sdk.types.agent import AgentConfig
|
from artemis.sdk.types.agent import AgentConfig
|
||||||
@@ -37,35 +38,29 @@ from artemis.sdk.types.task import AgentProfile, TaskRequest
|
|||||||
def get_video_capable_llm_config() -> LLMConfig:
|
def get_video_capable_llm_config() -> LLMConfig:
|
||||||
"""Returns an LLM config with video_analyzer configured.
|
"""Returns an LLM config with video_analyzer configured.
|
||||||
|
|
||||||
The video_analyzer must use a video-capable Gemini model:
|
Set OPENAI_MODEL to a Responses model with image input support.
|
||||||
- gemini-3.8-flash (recommended - fast and capable)
|
|
||||||
- gemini-3.7-flash
|
|
||||||
- gemini-3-pro-preview
|
|
||||||
- gemini-2.5-flash
|
|
||||||
- gemini-2.5-pro
|
|
||||||
- gemini-2.0-flash
|
|
||||||
"""
|
"""
|
||||||
return LLMConfig(
|
return LLMConfig(
|
||||||
planner=LLMWithFallback(
|
planner=LLMWithFallback(
|
||||||
provider="openai",
|
provider="openai",
|
||||||
model="gpt-5-nano",
|
model=settings.OPENAI_MODEL,
|
||||||
fallback=LLM(provider="openai", model="gpt-5-mini"),
|
fallback=LLM(provider="openai", model=settings.OPENAI_MODEL),
|
||||||
),
|
),
|
||||||
utils=LLMConfigUtils(
|
utils=LLMConfigUtils(
|
||||||
outputter=LLMWithFallback(
|
outputter=LLMWithFallback(
|
||||||
provider="openai",
|
provider="openai",
|
||||||
model="gpt-5-nano",
|
model=settings.OPENAI_MODEL,
|
||||||
fallback=LLM(provider="openai", model="gpt-5-mini"),
|
fallback=LLM(provider="openai", model=settings.OPENAI_MODEL),
|
||||||
),
|
),
|
||||||
hopper=LLMWithFallback(
|
hopper=LLMWithFallback(
|
||||||
provider="openai",
|
provider="openai",
|
||||||
model="gpt-5-nano",
|
model=settings.OPENAI_MODEL,
|
||||||
fallback=LLM(provider="openai", model="gpt-5-mini"),
|
fallback=LLM(provider="openai", model=settings.OPENAI_MODEL),
|
||||||
),
|
),
|
||||||
video_analyzer=LLMWithFallback(
|
video_analyzer=LLMWithFallback(
|
||||||
provider="google",
|
provider="openai",
|
||||||
model="gemini-3.8-flash",
|
model=settings.OPENAI_MODEL,
|
||||||
fallback=LLM(provider="google", model="gemini-3.7-flash"),
|
fallback=LLM(provider="openai", model=settings.OPENAI_MODEL),
|
||||||
),
|
),
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
|
|||||||
Loaded 100 of 183 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user