feat(console): add Artemis Admin & Trace Console backend and management plane

Introduces the comprehensive developer/administrator control plane and trace
inspection console backend for the ARTEMIS mobile automation platform.

Key Features & Backend Architecture:
- Core Server & Modular Routing:
  * FastAPI modular architecture with dedicated routers for sessions, steps, tasks, media, stream, and replay.
  * Multi-layer repository pattern with SQLite persistence for sessions, traces, and step data.
- Task Control & Queue Management:
  * FIFO asynchronous task queue service with support for single/batch goal execution and profile selection (Flash / Pro).
  * Session lifecycle controls (run, pause, resume, stop, clear queue).
  * Startup orphan cleanup and automatic older replay archiving.
- Live Telemetry & IPC Communication:
  * High-performance IPC server for streaming real-time execution logs, agent thoughts, and tool invocations.
  * Server-Sent Events (SSE) /api/stream/{session_id} endpoint for live UI updates.
- Real-Time Device Streaming:
  * Live ADB screen capture and MJPEG frame generator (/api/stream/device-live).
  * Device connectivity and state inspection endpoint (/api/stream/device-state).
- Trace Inspection & Step Replay Engine:
  * Hierarchical call tree inspection, pre/post step screenshots, and action overlay metadata.
  * Isolated single-step replay manager (/api/replay) for interactive debugging and validation.
This commit is contained in:
Weilin Liu committed 2026-08-13 12:11:10 -07:00
1 parent 2fd70465e6
commit c1803fcb86
29 files changed
+8889

No files matched your search

+51
View File
@@ -0,0 +1,51 @@
# 🛠️ Artemis Admin & Trace Console
A comprehensive developer/administrator control plane and trace inspection console for the Latte/Artemis mobile automation system.
---
## 🎯 Key Features
1. **Full Trace Inspection (`/api/sessions`, `/api/steps`)**:
- Hierarchical Call Tree exploration (LLM thoughts, tool calls, agent motivations).
- Step-by-step pre/post screenshots and action overlays (tap circles, swipe paths).
- Streaming logs and video recording playback.
2. **Task Control Center (`/api/run`)**:
- Directly launch autonomous tasks with custom goals and model profiles (Flash / Pro).
- Pause, resume, and manage FIFO task execution queues.
3. **Step Replay & Debugging Engine (`/api/replay`)**:
- Replay individual steps with isolated inputs, state inspection, and live device execution.
---
## 🚀 How to Start
### Option 1: Using `uv` (Recommended)
From the repository root:
```bash
uv run uvicorn apps.admin_console.server:app --host 0.0.0.0 --port 8000 --reload
```
Or directly using Python module:
```bash
uv run python -m apps.admin_console.server
```
### Option 2: From the `apps/admin_console` directory
```bash
cd apps/admin_console
uv run uvicorn server:app --host 0.0.0.0 --port 8000 --reload
```
Once started, open your browser and navigate to:
👉 **`http://localhost:8000`**
---
## 🎥 Video Debugging Mode
By default, `VideoAnalyzer` cleans up local temporary video files after execution. To retain video recordings for full playback in the console:
```bash
KEEP_VIDEOS=1 artemis run "your task goal"
```
Or set `ARTEMIS_DEBUG=1` in your environment.
+5
View File
@@ -0,0 +1,5 @@
"""Artemis Admin & Trace Console Package."""
from apps.admin_console.server import app
__all__ = ["app"]
+1
View File
@@ -0,0 +1 @@
# Copyright 2026 Google LLC
+68
View File
@@ -0,0 +1,68 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import os
from pathlib import Path
import sys
# Automatically write ANTIGRAVITY_LS_ADDRESS to shared synchronization file on startup
# to allow decoupled background processes to communicate with Jetski
def init_ls_address():
ls_addr = os.environ.get("ANTIGRAVITY_LS_ADDRESS")
if ls_addr:
try:
from artemis.config import write_ls_address
write_ls_address(ls_addr)
except Exception as e:
sys.stderr.write(f"Failed to write .jetski_ls_address on startup: {e}\n")
# Compute real WORKSPACE_ROOT (find repo root containing pyproject.toml or artemis)
_current_p = Path(__file__).resolve().parent
while _current_p != _current_p.parent:
if (_current_p / "pyproject.toml").exists() or (_current_p / "artemis").is_dir():
WORKSPACE_ROOT = _current_p
break
_current_p = _current_p.parent
else:
WORKSPACE_ROOT = Path(__file__).resolve().parent.parent.parent.parent
_apps_dir = WORKSPACE_ROOT / "apps"
_admin_console_dir = _apps_dir / "admin_console"
_cloud_service_dir = _apps_dir / "cloud_service"
for _p in (str(WORKSPACE_ROOT), str(_apps_dir), str(_admin_console_dir), str(_cloud_service_dir)):
if _p not in sys.path:
sys.path.insert(0, _p)
# Try to import settings from artemis
try:
from artemis.config import settings
TRACES_PATH = Path(settings.TRACES_PATH)
DB_PATH = Path(settings.DATA_ENGINE_DB_PATH)
except ImportError:
# Fallback to default path if run outside project environment or imports fail
TRACES_PATH = WORKSPACE_ROOT / "traces"
DB_PATH = TRACES_PATH / "data_engine.db"
# Step Replay Directories (Stored within traces/replay to avoid workspace clutter)
REPLAY_BASE_DIR = TRACES_PATH / "replay"
TEST_DATA_DIR = REPLAY_BASE_DIR / "data"
TEST_OUTPUTS_DIR = REPLAY_BASE_DIR / "outputs"
IMAGES_DIR = TRACES_PATH / "images"
PAUSE_FILE = WORKSPACE_ROOT / ".artemis_paused"
+82
View File
@@ -0,0 +1,82 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import asyncio
from collections.abc import Callable
from typing import Any
try:
from admin_console.core.config import PAUSE_FILE
except ImportError:
from apps.admin_console.core.config import PAUSE_FILE
class ServerState:
"""Encapsulates all runtime states of the debug server."""
def __init__(self):
self.ipc_subscribers: list[Callable[[str, Any], None]] = []
self.ipc_server: asyncio.Server | None = None
self.ipc_port: int | None = None
self.current_process: asyncio.subprocess.Process | None = None
self.current_goal: str | None = None
self.current_profile: str | None = None
self.active_connections: dict[str, dict[str, Any]] = {}
self.active_session_id: str | None = None
self.was_stopped_manually: bool = False
# FIFO Task queue
self.task_queue: asyncio.Queue = asyncio.Queue()
self.queue_tasks: list[dict[str, Any]] = []
self.worker_task: asyncio.Task | None = None
@property
def queue_goals(self) -> list[str]:
"""Backward compatibility helper for queue goals."""
return [t.get("goal", "") for t in self.queue_tasks if isinstance(t, dict)]
@queue_goals.setter
def queue_goals(self, val: list[Any]):
# Allow setting if legacy code updates it
pass
@property
def is_running(self) -> bool:
return self.current_process is not None and self.current_process.returncode is None
@property
def is_paused(self) -> bool:
return PAUSE_FILE.exists()
def add_subscriber(self, callback: Callable[[str, Any], None]):
if callback not in self.ipc_subscribers:
self.ipc_subscribers.append(callback)
def remove_subscriber(self, callback: Callable[[str, Any], None]):
if callback in self.ipc_subscribers:
self.ipc_subscribers.remove(callback)
def clear_queue(self):
while not self.task_queue.empty():
try:
self.task_queue.get_nowait()
self.task_queue.task_done()
except (asyncio.QueueEmpty, ValueError):
break
self.queue_tasks.clear()
# Global shared instance
state = ServerState()
+1
View File
@@ -0,0 +1 @@
# Copyright 2026 Google LLC
+43
View File
@@ -0,0 +1,43 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from contextlib import contextmanager
from pathlib import Path
import sqlite3
from fastapi import HTTPException
try:
from admin_console.core.config import DB_PATH
except ImportError:
from apps.admin_console.core.config import DB_PATH
def get_db(db_path: Path | None = None) -> sqlite3.Connection:
"""Returns a SQLite connection with timeout and Row row_factory."""
path = db_path or DB_PATH
if not path.exists():
raise HTTPException(status_code=500, detail=f"Database not found at {path}")
conn = sqlite3.connect(path, timeout=10.0)
conn.row_factory = sqlite3.Row
return conn
@contextmanager
def db_session(db_path: Path | None = None):
"""Context manager for SQLite connections ensuring clean closure."""
conn = get_db(db_path)
try:
yield conn
finally:
conn.close()
@@ -0,0 +1 @@
# Copyright 2026 Google LLC
@@ -0,0 +1,202 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import time
from typing import Any
try:
from admin_console.database.connection import db_session
except ImportError:
from apps.admin_console.database.connection import db_session
class SessionRepository:
"""Repository handling all database queries and updates for Sessions."""
def __init__(self, db_path=None):
self.db_path = db_path
def get_all_sessions(self) -> list[dict[str, Any]]:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute("SELECT * FROM sessions ORDER BY start_time DESC")
return [dict(r) for r in cursor.fetchall()]
def get_session_by_id(self, session_id: str) -> dict[str, Any] | None:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute("SELECT * FROM sessions WHERE session_id = ?", (str(session_id),))
row = cursor.fetchone()
return dict(row) if row else None
def get_video_recordings_map(self) -> dict[str, str]:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"SELECT name FROM sqlite_master WHERE type='table' AND name='video_recordings'"
)
if cursor.fetchone() is None:
return {}
cursor.execute(
"SELECT session_id, local_video_path FROM video_recordings WHERE session_id IS NOT NULL AND local_video_path IS NOT NULL ORDER BY start_time ASC"
)
return {str(r[0]): str(r[1]) for r in cursor.fetchall() if r[0] and r[1]}
def get_llm_traces_for_profile(self, session_id: str, limit: int = 3) -> list[str]:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"SELECT payload FROM traces WHERE session_id = ? AND type = 'llm_call' LIMIT ?",
(session_id, limit),
)
rows = cursor.fetchall()
return [r[0] for r in rows if r and r[0]]
def get_agent_trace_names(self, session_id: str) -> list[str]:
try:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"SELECT DISTINCT name FROM traces WHERE session_id = ? AND type IN ('agent', 'llm_call')",
(session_id,),
)
rows = cursor.fetchall()
return [r[0] for r in rows if r and r[0]]
except Exception:
return []
def get_session_by_id(self, session_id: str) -> dict[str, Any] | None:
try:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"SELECT * FROM sessions WHERE session_id = ?",
(str(session_id),),
)
row = cursor.fetchone()
return dict(row) if row else None
except Exception:
return None
def harvest_orphaned_sessions(self, orphaned_ids: list[str]) -> int:
if not orphaned_ids:
return 0
with db_session(self.db_path) as conn:
cursor = conn.cursor()
now = time.time()
for s_id in orphaned_ids:
cursor.execute(
"UPDATE sessions SET status = ?, end_time = ? "
"WHERE session_id = ? AND status = ?",
("failed", now, s_id, "running"),
)
conn.commit()
return len(orphaned_ids)
def cleanup_orphans_on_startup(self) -> int:
try:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"UPDATE sessions SET status = ?, end_time = ? WHERE status = ?",
("failed", time.time(), "running"),
)
count = cursor.rowcount
conn.commit()
return count
except Exception:
return 0
def get_latest_session(self) -> dict[str, Any] | None:
try:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"SELECT session_id, status FROM sessions ORDER BY start_time DESC LIMIT 1"
)
row = cursor.fetchone()
return dict(row) if row else None
except Exception:
return None
def get_running_session_id(self) -> str | None:
try:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"SELECT session_id FROM sessions WHERE status = 'running' "
"ORDER BY start_time DESC LIMIT 1"
)
row = cursor.fetchone()
return row["session_id"] if row else None
except Exception:
return None
def get_session_status(self, session_id: str) -> str | None:
try:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"SELECT status FROM sessions WHERE session_id = ?",
(str(session_id),),
)
row = cursor.fetchone()
return row["status"] if row else None
except Exception:
return None
def update_session_status(
self, session_id: str, status: str, end_time: float | None = None
) -> bool:
try:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
t = end_time or time.time()
cursor.execute(
"UPDATE sessions SET status = ?, end_time = ? WHERE session_id = ?",
(status, t, str(session_id)),
)
conn.commit()
return cursor.rowcount > 0
except Exception:
return False
def mark_all_running_cancelled(self) -> int:
try:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"UPDATE sessions SET status = ?, end_time = ? WHERE status = ?",
("cancelled", time.time(), "running"),
)
count = cursor.rowcount
conn.commit()
return count
except Exception:
return 0
def get_background_tasks(self, session_id: str) -> list[dict[str, Any]]:
try:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"SELECT * FROM background_tasks WHERE session_id = ? ORDER BY start_time ASC",
(session_id,),
)
return [dict(r) for r in cursor.fetchall()]
except Exception:
return []
session_repo = SessionRepository()
@@ -0,0 +1,411 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
from typing import Any
try:
from admin_console.database.connection import db_session
except ImportError:
from apps.admin_console.database.connection import db_session
class StepRepository:
"""Repository handling querying and formatting of steps and associated traces."""
def __init__(self, db_path=None):
self.db_path = db_path
def get_step_session_id(self, step_id: str) -> str | None:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute("SELECT session_id FROM steps WHERE step_id = ?", (step_id,))
row = cursor.fetchone()
return row["session_id"] if row else None
@staticmethod
def _clean_value(val: Any) -> Any:
"""Recursively cleans values to remove non-serializable and internal
Python runtime representations.
"""
if val is None:
return None
if isinstance(val, (bool, int, float)):
return val
if isinstance(val, str):
if (
"object at 0x" in val
or val.startswith("<artemis.")
or val.startswith("<controller")
):
return None
return val
if isinstance(val, dict):
cleaned = {}
for k, v in val.items():
cv = StepRepository._clean_value(v)
if cv is not None:
cleaned[k] = cv
return cleaned
if isinstance(val, (list, tuple)):
cleaned_list = []
for item in val:
cv = StepRepository._clean_value(item)
if cv is not None:
cleaned_list.append(cv)
return cleaned_list
val_str = str(val)
if "object at 0x" in val_str or val_str.startswith("<"):
return None
return val_str
@staticmethod
def _extract_clean_text_and_images(content: Any) -> tuple[str, int]:
"""Extracts readable text and count of image blocks from LLM content."""
if not content:
return "", 0
if isinstance(content, str):
if content.startswith("data:image/") or "base64," in content[:50]:
return "", 1
return content, 0
if isinstance(content, dict):
if content.get("type") == "image_url" or "image_url" in content:
return "", 1
if content.get("type") == "text":
return str(content.get("text", "")), 0
t = content.get("text") or content.get("thought") or ""
return str(t), 0
if isinstance(content, list):
total_text = ""
num_images = 0
for item in content:
txt, imgs = StepRepository._extract_clean_text_and_images(item)
total_text += txt
num_images += imgs
return total_text, num_images
return str(content), 0
@staticmethod
def _extract_token_usage_from_trace(payload_raw: Any) -> tuple[int, int, int]:
"""Extract prompt, completion, and total tokens from an LLM trace payload with multimodal awareness."""
if not payload_raw:
return 0, 0, 0
payload_obj = payload_raw
if isinstance(payload_raw, str):
try:
payload_obj = json.loads(payload_raw)
except Exception:
pass
if not isinstance(payload_obj, dict):
return 0, 0, 0
u = payload_obj.get("token_usage") or payload_obj.get("usage_metadata")
if isinstance(u, dict):
pr = u.get("prompt_tokens") or u.get("prompt_token_count") or u.get("input_tokens") or 0
co = (
u.get("completion_tokens")
or u.get("candidates_token_count")
or u.get("output_tokens")
or 0
)
to = u.get("total_tokens") or u.get("total_token_count") or (pr + co)
if to > 0:
return int(pr), int(co), int(to)
# Fallback for historical traces: estimate tokens from text and images
msgs = payload_obj.get("messages", [])
resp = payload_obj.get("response", [])
prompt_text = ""
prompt_images = 0
if isinstance(msgs, list):
for m in msgs:
if isinstance(m, dict):
t, imgs = StepRepository._extract_clean_text_and_images(m.get("content"))
prompt_text += t
prompt_images += imgs
else:
t, imgs = StepRepository._extract_clean_text_and_images(m)
prompt_text += t
prompt_images += imgs
else:
t, imgs = StepRepository._extract_clean_text_and_images(msgs)
prompt_text += t
prompt_images += imgs
resp_text = ""
resp_images = 0
if isinstance(resp, list):
for r in resp:
if isinstance(r, dict):
t, imgs = StepRepository._extract_clean_text_and_images(r.get("content"))
resp_text += t
resp_images += imgs
else:
t, imgs = StepRepository._extract_clean_text_and_images(r)
resp_text += t
resp_images += imgs
# Standard Gemini multimodal token pricing: 258 tokens per image
pr = (len(prompt_text) // 4) + (prompt_images * 258)
co = (len(resp_text) // 4) + (resp_images * 258)
return int(pr), int(co), int(pr + co)
def _clean_tool_payload(self, payload_raw: Any, trace_type: str | None = None) -> Any:
if not payload_raw:
return None
payload_obj = payload_raw
if isinstance(payload_raw, str):
try:
payload_obj = json.loads(payload_raw)
except Exception:
pass
if not isinstance(payload_obj, dict):
return None
if trace_type == "llm_call":
return (
{"error": self._clean_value(payload_obj.get("error"))}
if "error" in payload_obj
else None
)
args = payload_obj.get("args", payload_obj)
if isinstance(args, dict):
return {"args": self._clean_value(args)}
return {"args": self._clean_value(args)}
def get_session_steps(self, session_id: str, client: str | None = None) -> list[dict[str, Any]]:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"SELECT * FROM steps WHERE session_id = ? ORDER BY step_number ASC",
(session_id,),
)
rows = cursor.fetchall()
steps = []
for r in rows:
step_dict = dict(r)
# Strip raw binary bytes for API efficiency (clients fetch images by url/hash)
step_dict["pre_screenshot_bytes"] = None
step_dict["post_screenshot_bytes"] = None
# Parse JSON fields safely
for json_col in [
"action_taken",
"last_execution_result",
"extra_metadata",
]:
if step_dict.get(json_col):
try:
step_dict[json_col] = json.loads(step_dict[json_col])
except Exception:
pass
# If post_image_name is identical to pre_image_name, normalize to None
pre_img = step_dict.get("pre_image_name")
if pre_img and step_dict.get("post_image_name") == pre_img:
step_dict["post_image_name"] = None
# Fetch generic tool traces for this step
step_id = step_dict.get("step_id")
if step_id:
cursor.execute(
"SELECT t1.trace_id, t1.parent_trace_id, t1.step_id,"
" t1.type, t1.name, t1.timestamp, t1.duration, t1.status,"
" t1.payload, t2.name as agent_name FROM traces t1 LEFT"
" JOIN traces t2 ON t1.parent_trace_id = t2.trace_id WHERE"
" t1.step_id = ? AND (t1.type = 'tool' OR (t1.type ="
" 'llm_call' AND t1.status = 'failed')) ORDER BY"
" t1.timestamp ASC",
(step_id,),
)
trace_rows = cursor.fetchall()
generic_tools = []
for tr in trace_rows:
trace_dict = dict(tr)
trace_dict["payload"] = self._clean_tool_payload(
trace_dict.get("payload"), trace_dict.get("type")
)
generic_tools.append(trace_dict)
step_dict["generic_tools"] = generic_tools
# Compute token usage for this step from all LLM call traces
cursor.execute(
"SELECT payload FROM traces WHERE step_id = ? AND type = 'llm_call'",
(step_id,),
)
llm_rows = cursor.fetchall()
step_p, step_c, step_t = 0, 0, 0
for lr in llm_rows:
p, c, t = self._extract_token_usage_from_trace(lr["payload"])
step_p += p
step_c += c
step_t += t
# If step_t is 0, also check extra_metadata for token_usage
if step_t == 0 and step_dict.get("extra_metadata"):
meta = (
step_dict["extra_metadata"]
if isinstance(step_dict["extra_metadata"], dict)
else {}
)
u = meta.get("token_usage")
if isinstance(u, dict):
step_p = int(u.get("prompt_tokens") or u.get("input_tokens") or 0)
step_c = int(u.get("completion_tokens") or u.get("output_tokens") or 0)
step_t = int(u.get("total_tokens") or (step_p + step_c))
step_dict["token_usage"] = {
"prompt_tokens": step_p,
"completion_tokens": step_c,
"total_tokens": step_t,
}
step_dict["total_tokens"] = step_t
else:
step_dict["generic_tools"] = []
step_dict["token_usage"] = {
"prompt_tokens": 0,
"completion_tokens": 0,
"total_tokens": 0,
}
step_dict["total_tokens"] = 0
try:
from artemis.utils.coordinates import normalize_step_actions
step_dict = normalize_step_actions(step_dict)
except ImportError:
pass
steps.append(step_dict)
# Fetch step-less tool traces (e.g. planner tools)
cursor.execute(
"SELECT t1.trace_id, t1.parent_trace_id, t1.step_id, t1.type,"
" t1.name, t1.timestamp, t1.duration, t1.status, t1.payload,"
" t2.name as agent_name FROM traces t1 LEFT JOIN traces t2 ON"
" t1.parent_trace_id = t2.trace_id WHERE t1.session_id = ? AND"
" (t1.step_id IS NULL OR t1.step_id = '') AND (t1.type = 'tool' OR"
" (t1.type = 'llm_call' AND t1.status = 'failed')) ORDER BY"
" t1.timestamp ASC",
(session_id,),
)
stepless_rows = cursor.fetchall()
if stepless_rows:
unassigned_tools = []
sorted_steps = sorted(steps, key=lambda s: s.get("timestamp") or 0)
for tr in stepless_rows:
trace_dict = dict(tr)
trace_dict["payload"] = self._clean_tool_payload(
trace_dict.get("payload"), trace_dict.get("type")
)
tr_time = trace_dict.get("timestamp") or 0
target_step = None
for s in sorted_steps:
s_time = s.get("timestamp") or 0
if s_time <= tr_time:
target_step = s
else:
break
if target_step:
if "generic_tools" not in target_step or not target_step["generic_tools"]:
target_step["generic_tools"] = []
target_step["generic_tools"].append(trace_dict)
target_step["generic_tools"].sort(key=lambda t: t.get("timestamp") or 0)
else:
unassigned_tools.append(trace_dict)
if unassigned_tools:
unassigned_tools.sort(key=lambda t: t.get("timestamp") or 0)
first_timestamp = unassigned_tools[0].get("timestamp") or 0
# Calculate token usage ONLY for pre-planning LLM calls (before first step)
first_recorded_step_time = (
sorted_steps[0].get("timestamp") if sorted_steps else float("inf")
)
cursor.execute(
"SELECT payload FROM traces WHERE session_id = ? AND (step_id IS NULL OR step_id = '') AND timestamp < ? AND type = 'llm_call'",
(session_id, first_recorded_step_time),
)
plan_llm_rows = cursor.fetchall()
plan_p, plan_c, plan_t = 0, 0, 0
for lr in plan_llm_rows:
p, c, t = self._extract_token_usage_from_trace(lr["payload"])
plan_p += p
plan_c += c
plan_t += t
# Retrieve planner thinking / thoughts
cursor.execute(
"SELECT type, payload FROM traces WHERE session_id = ? AND (step_id IS NULL OR step_id = '') AND timestamp < ? AND type IN ('raw_thinking', 'thinking') ORDER BY timestamp ASC",
(session_id, first_recorded_step_time),
)
think_rows = cursor.fetchall()
p_raw_th = []
p_nat_th = []
for thr in think_rows:
if thr["payload"]:
try:
th_obj = (
json.loads(thr["payload"])
if isinstance(thr["payload"], str)
else thr["payload"]
)
th_text = th_obj.get("thought") or th_obj.get("text") or ""
if thr["type"] == "raw_thinking" and th_text:
p_raw_th.append(str(th_text).strip())
elif thr["type"] == "thinking" and th_text:
p_nat_th.append(str(th_text).strip())
except Exception:
pass
virtual_step = {
"step_id": "pre-planning",
"step_type": "planning",
"session_id": session_id,
"step_number": 0,
"timestamp": first_timestamp,
"operator_native_thinking": "\n\n".join(p_nat_th) if p_nat_th else "",
"operator_raw_thinking": "\n\n".join(p_raw_th) if p_raw_th else "",
"action_taken": None,
"last_execution_result": None,
"extra_metadata": None,
"generic_tools": unassigned_tools,
"token_usage": {
"prompt_tokens": plan_p,
"completion_tokens": plan_c,
"total_tokens": plan_t,
},
"total_tokens": plan_t,
}
steps.append(virtual_step)
steps.sort(
key=lambda x: (
x.get("timestamp") or 0,
x.get("step_number") or 0,
)
)
return steps
step_repo = StepRepository()
@@ -0,0 +1,131 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
from typing import Any
try:
from admin_console.database.connection import db_session
except ImportError:
from apps.admin_console.database.connection import db_session
class TraceRepository:
"""Repository handling querying traces and constructing hierarchical trace trees."""
def __init__(self, db_path=None):
self.db_path = db_path
def get_trace_by_id(self, trace_id: str, db_path=None) -> dict[str, Any] | None:
with db_session(db_path or self.db_path) as conn:
cursor = conn.cursor()
cursor.execute("SELECT * FROM traces WHERE trace_id = ?", (trace_id,))
row = cursor.fetchone()
return dict(row) if row else None
def get_trace_tree(self, session_id: str) -> list[dict[str, Any]]:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"SELECT trace_id, parent_trace_id, type, name, status, timestamp,"
" duration, CASE WHEN type IN ('raw_thinking', 'thinking') THEN"
" payload ELSE NULL END AS payload FROM traces WHERE session_id = ?"
" ORDER BY timestamp ASC",
(session_id,),
)
rows = cursor.fetchall()
nodes = {}
for r in rows:
row_dict = dict(r)
if row_dict.get("payload"):
try:
row_dict["payload"] = json.loads(row_dict["payload"])
except Exception:
pass
nodes[row_dict["trace_id"]] = {**row_dict, "children": []}
root_nodes = []
for node_id, node in nodes.items():
parent_id = node["parent_trace_id"]
if parent_id and parent_id in nodes:
nodes[parent_id]["children"].append(node)
else:
root_nodes.append(node)
def sanitize_tree(nodes_list):
sanitized = []
for node in nodes_list:
node["children"] = sanitize_tree(node["children"])
if node["name"] == "node":
sanitized.extend(node["children"])
else:
sanitized.append(node)
return sanitized
return sanitize_tree(root_nodes)
def get_step_traces_tree(self, session_id: str, step_id: str) -> list[dict[str, Any]]:
with db_session(self.db_path) as conn:
cursor = conn.cursor()
cursor.execute(
"SELECT trace_id, parent_trace_id, step_id, type, name, status,"
" timestamp, duration, CASE WHEN type IN ('raw_thinking',"
" 'thinking') THEN payload ELSE NULL END AS payload FROM traces"
" WHERE session_id = ? ORDER BY timestamp ASC",
(session_id,),
)
rows = cursor.fetchall()
nodes = {}
for r in rows:
row_dict = dict(r)
if row_dict.get("payload"):
try:
row_dict["payload"] = json.loads(row_dict["payload"])
except Exception:
pass
nodes[row_dict["trace_id"]] = {**row_dict, "children": []}
step_roots = []
for n_id, node in nodes.items():
if node.get("step_id") == step_id:
step_roots.append(node)
for n_id, node in nodes.items():
parent_id = node["parent_trace_id"]
if parent_id and parent_id in nodes:
nodes[parent_id]["children"].append(node)
def sanitize_tree(nodes_list, visited=None):
if visited is None:
visited = set()
sanitized = []
for node in nodes_list:
node_id = node.get("trace_id")
if node_id and node_id in visited:
continue
if node_id:
visited.add(node_id)
node["children"] = sanitize_tree(node["children"], visited.copy())
if node["name"] == "node":
sanitized.extend(node["children"])
else:
sanitized.append(node)
return sanitized
return sanitize_tree(step_roots)
trace_repo = TraceRepository()
File diff suppressed because it is too large. Load diff
File diff suppressed because it is too large. Load diff
+1
View File
@@ -0,0 +1 @@
# Copyright 2026 Google LLC
+116
View File
@@ -0,0 +1,116 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from pathlib import Path
from fastapi import APIRouter, HTTPException
from fastapi.responses import FileResponse, HTMLResponse
try:
from admin_console.core.config import IMAGES_DIR, WORKSPACE_ROOT
from admin_console.services.media_service import media_service
except ImportError:
from apps.admin_console.core.config import IMAGES_DIR, WORKSPACE_ROOT
from apps.admin_console.services.media_service import media_service
router = APIRouter(tags=["media"])
@router.get("/", response_class=HTMLResponse)
@router.get("/admin", response_class=HTMLResponse)
@router.get("/debug", response_class=HTMLResponse)
async def get_index():
admin_index = Path(__file__).resolve().parent.parent / "index.html"
if admin_index.exists():
return admin_index.read_text(encoding="utf-8")
admin_index_alt = WORKSPACE_ROOT / "apps" / "admin_console" / "index.html"
if admin_index_alt.exists():
return admin_index_alt.read_text(encoding="utf-8")
return "<h1>Admin Console index.html not found</h1>"
@router.get("/images/{image_name}")
@router.get("/api/images/{image_name}")
async def get_image(image_name: str):
if not image_name.endswith(".jpg"):
image_name += ".jpg"
image_path = IMAGES_DIR / image_name
if not image_path.exists():
raise HTTPException(status_code=404, detail="Image not found")
return FileResponse(image_path, media_type="image/jpeg")
@router.get("/videos/{video_path:path}")
async def get_video(video_path: str):
path = Path(video_path)
if not path.exists():
if not path.is_absolute():
candidate = WORKSPACE_ROOT / path
if candidate.exists():
path = candidate
else:
candidate_root = Path("/" + video_path.lstrip("/"))
if candidate_root.exists():
path = candidate_root
else:
path = candidate
if not path.exists():
raise HTTPException(status_code=404, detail=f"Video not found at {path}")
suffix = path.suffix.lower()
media_type = "video/mp4"
if suffix == ".webm":
media_type = "video/webm"
elif suffix == ".mkv":
media_type = "video/x-matroska"
return FileResponse(path, media_type=media_type)
@router.get("/api/sessions/{session_id}/video")
async def get_session_video(session_id: str):
try:
from admin_console.database.repositories.session_repository import session_repo
except ImportError:
from apps.admin_console.database.repositories.session_repository import session_repo
video_rec_map = session_repo.get_video_recordings_map()
video_idx = media_service.build_video_index()
row = session_repo.get_session_by_id(session_id)
row_dict = dict(row) if row else {"session_id": session_id}
v_url = media_service.resolve_video_url(row_dict, video_rec_map, video_idx)
return {
"session_id": session_id,
"has_video": bool(v_url),
"video_url": v_url,
}
@router.get("/local_file")
async def get_local_file(path: str):
p, media_type = media_service.get_safe_local_file(path)
return FileResponse(p, media_type=media_type)
@router.get("/api/sessions/{session_id}/plan")
async def get_task_plan(session_id: str):
return {"plan": media_service.get_task_plan_content(session_id)}
@router.get("/api/sessions/{session_id}/notes")
async def get_all_notes(session_id: str):
return {"notes": media_service.get_session_notes_content(session_id)}
+91
View File
@@ -0,0 +1,91 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import traceback
from fastapi import APIRouter, HTTPException
try:
from admin_console.core.config import WORKSPACE_ROOT
from admin_console.schemas.task_schema import ReplayRequest
from admin_console.replay_manager import ReplayManager
except ImportError:
from apps.admin_console.core.config import WORKSPACE_ROOT
from apps.admin_console.schemas.task_schema import ReplayRequest
from apps.admin_console.replay_manager import ReplayManager
replay_manager = ReplayManager(WORKSPACE_ROOT)
router = APIRouter(tags=["replay"])
@router.get("/api/devices")
async def list_devices():
"""Dynamically queries the ADB server for connected Android devices."""
return replay_manager.list_devices()
@router.get("/api/replay/tools")
async def get_replay_tools():
"""Returns the list of registered tools available for replay."""
return replay_manager.get_replay_tools()
@router.get("/api/replay/config")
async def get_replay_config(tool_name: str = "ask_explorer"):
"""Returns the parameter list and types of the selected tool logic."""
try:
return replay_manager.get_replay_config(tool_name)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/api/sessions/{session_id}/replay_steps")
async def get_replay_steps(session_id: str):
"""Loads and formats metadata for all chunked steps in the session."""
try:
return replay_manager.get_replay_steps(session_id)
except FileNotFoundError as e:
raise HTTPException(status_code=404, detail=str(e))
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.post("/api/sessions/{session_id}/steps/{step_number}/replay")
async def trigger_step_replay_endpoint(session_id: str, step_number: int, req: ReplayRequest):
"""Triggers sandbox execution of step replay and returns the resulting traces trees."""
try:
return await replay_manager.run_step_replay(
session_id=session_id,
step_number=step_number,
device_id=req.device_id,
user_submits=req.user_submits,
tool_name=req.tool_name,
replay_id=req.replay_id,
)
except Exception as e:
traceback.print_exc()
raise HTTPException(status_code=500, detail=str(e))
@router.get("/api/sessions/{session_id}/steps/{step_number}/replay_traces")
async def get_step_replay_traces_endpoint(
session_id: str, step_number: int, tool_name: str = "ask_explorer"
):
"""Retrieves previously generated step replay traces if they exist."""
try:
return replay_manager.get_step_replay_traces(session_id, step_number, tool_name)
except Exception as e:
traceback.print_exc()
raise HTTPException(status_code=500, detail=str(e))
+127
View File
@@ -0,0 +1,127 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import time
from uuid import UUID
from fastapi import APIRouter, HTTPException
try:
from admin_console.core.config import DB_PATH, TRACES_PATH
from admin_console.core.state import state
from admin_console.database.repositories.session_repository import session_repo
from admin_console.database.repositories.trace_repository import trace_repo
from admin_console.services.media_service import media_service
from admin_console.services.model_service import model_service
except ImportError:
from apps.admin_console.core.config import DB_PATH, TRACES_PATH
from apps.admin_console.core.state import state
from apps.admin_console.database.repositories.session_repository import session_repo
from apps.admin_console.database.repositories.trace_repository import trace_repo
from apps.admin_console.services.media_service import media_service
from apps.admin_console.services.model_service import model_service
router = APIRouter(tags=["sessions"])
@router.get("/api/sessions")
async def list_sessions():
try:
rows = session_repo.get_all_sessions()
video_rec_map = session_repo.get_video_recordings_map()
video_idx = media_service.build_video_index()
orphaned_ids = []
result = []
for row_dict in rows:
s_id = str(row_dict.get("session_id"))
if row_dict.get("status") == "running":
is_active = s_id in state.active_connections or (
state.is_running and s_id == str(state.active_session_id)
)
if not is_active:
row_dict["status"] = "failed"
row_dict["end_time"] = time.time()
orphaned_ids.append(s_id)
row_dict["video_url"] = media_service.resolve_video_url(
row_dict, video_rec_map, video_idx
)
llm_traces = session_repo.get_llm_traces_for_profile(s_id)
agent_names = session_repo.get_agent_trace_names(s_id)
sess_profile = model_service.resolve_session_profile(
row_dict, llm_traces, state.current_profile, agent_names=agent_names
)
row_dict["model_info"] = model_service.get_active_model_info(sess_profile)
result.append(row_dict)
if orphaned_ids:
try:
harvested = session_repo.harvest_orphaned_sessions(orphaned_ids)
print(f"[list_sessions] Auto-harvested {harvested} orphaned running session(s).")
except Exception as e:
print(f"[list_sessions] Failed to update orphaned sessions: {e}")
return result
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/api/sessions/{session_id}/tree")
async def get_tree(session_id: str):
try:
return trace_repo.get_trace_tree(session_id)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/api/sessions/{session_id}/background_tasks")
async def get_session_background_tasks(session_id: str):
try:
return session_repo.get_background_tasks(session_id)
except Exception:
return []
@router.post("/api/cleanup")
async def cleanup_history_endpoint():
try:
from artemis.data_engine.storage import StorageManager
storage = StorageManager(DB_PATH, TRACES_PATH)
storage.clear_all_data()
return {
"status": "success",
"message": "History cleaned up successfully",
}
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.post("/api/sessions/{session_id}/delete")
async def delete_session_endpoint(session_id: str):
try:
from artemis.data_engine.storage import StorageManager
storage = StorageManager(DB_PATH, TRACES_PATH)
storage.delete_session(UUID(session_id))
return {
"status": "success",
"message": f"Session {session_id} deleted successfully",
}
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
+108
View File
@@ -0,0 +1,108 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
import traceback
from fastapi import APIRouter, HTTPException
from fastapi.responses import PlainTextResponse
try:
from admin_console.core.config import TEST_OUTPUTS_DIR
from admin_console.database.repositories.step_repository import step_repo
from admin_console.database.repositories.trace_repository import trace_repo
from admin_console.services.media_service import media_service
except ImportError:
from apps.admin_console.core.config import TEST_OUTPUTS_DIR
from apps.admin_console.database.repositories.step_repository import step_repo
from apps.admin_console.database.repositories.trace_repository import trace_repo
from apps.admin_console.services.media_service import media_service
router = APIRouter(tags=["steps"])
@router.get("/api/sessions/{session_id}/steps")
async def get_session_steps(session_id: str, client: str | None = None):
try:
return step_repo.get_session_steps(session_id, client=client)
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.get("/api/steps/{step_id}/traces")
async def get_step_traces_endpoint(step_id: str):
try:
session_id = step_repo.get_step_session_id(step_id)
if not session_id:
raise HTTPException(status_code=404, detail="Step not found")
return trace_repo.get_step_traces_tree(session_id, step_id)
except HTTPException as e:
raise e
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
def _resolve_trace_db_path(session_id: str | None, step_number: int | None):
if session_id and step_number is not None:
sandbox_db_path = (
TEST_OUTPUTS_DIR
/ f"{session_id}_step_{step_number:02d}"
/ "temp_traces"
/ "data_engine.db"
)
if sandbox_db_path.exists():
return sandbox_db_path
return None
@router.get("/api/traces/{trace_id}")
async def get_trace(trace_id: str, session_id: str = None, step_number: int = None):
try:
db_path = _resolve_trace_db_path(session_id, step_number)
trace_dict = trace_repo.get_trace_by_id(trace_id, db_path=db_path)
if not trace_dict:
raise HTTPException(status_code=404, detail="Trace not found")
if trace_dict.get("payload"):
try:
payload_obj = json.loads(trace_dict["payload"])
trace_dict["payload"] = media_service.unwrap_payload(payload_obj)
except Exception:
pass
return trace_dict
except HTTPException as e:
raise e
except Exception as e:
traceback.print_exc()
raise HTTPException(status_code=500, detail=str(e))
@router.get("/api/traces/{trace_id}/download")
async def download_trace(trace_id: str, session_id: str = None, step_number: int = None):
try:
db_path = _resolve_trace_db_path(session_id, step_number)
trace_dict = trace_repo.get_trace_by_id(trace_id, db_path=db_path)
if not trace_dict or not trace_dict.get("payload"):
raise HTTPException(status_code=404, detail="Payload not found")
return PlainTextResponse(
trace_dict["payload"],
headers={"Content-Disposition": f'attachment; filename="trace_{trace_id}.json"'},
)
except HTTPException as e:
raise e
except Exception as e:
traceback.print_exc()
raise HTTPException(status_code=500, detail=str(e))
+56
View File
@@ -0,0 +1,56 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Device Screen Live Streaming Router.
Exposes real-time screen streaming endpoints for the Web UI.
"""
from fastapi import APIRouter
from fastapi.responses import JSONResponse, StreamingResponse
try:
from admin_console.services.device_stream_service import device_stream_service
except ImportError:
from apps.admin_console.services.device_stream_service import device_stream_service
router = APIRouter(tags=["stream"])
@router.get("/api/stream/device-live")
async def stream_device_live():
"""Streams live device screen frames as multipart MJPEG."""
return StreamingResponse(
device_stream_service.mjpeg_frame_generator(),
media_type="multipart/x-mixed-replace; boundary=frame",
headers={
"Cache-Control": "no-cache, no-store, must-revalidate, max-age=0",
"Pragma": "no-cache",
"Expires": "0",
"Connection": "keep-alive",
},
)
@router.get("/api/stream/device-state")
async def get_device_stream_state():
"""Returns whether an ADB device is connected and live streaming is available."""
serial = await device_stream_service.get_device_serial()
return JSONResponse(
{
"connected": serial is not None,
"serial": serial,
"live_stream_url": "/api/stream/device-live" if serial else None,
}
)
+189
View File
@@ -0,0 +1,189 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import asyncio
import json
from fastapi import APIRouter, HTTPException
from fastapi.responses import StreamingResponse
try:
from admin_console.core.state import state
from admin_console.database.repositories.session_repository import session_repo
from admin_console.schemas.task_schema import RunRequest
from admin_console.services.ipc_service import ipc_service
from admin_console.services.model_service import model_service
from admin_console.services.task_queue_service import task_queue_service
except ImportError:
from apps.admin_console.core.state import state
from apps.admin_console.database.repositories.session_repository import session_repo
from apps.admin_console.schemas.task_schema import RunRequest
from apps.admin_console.services.ipc_service import ipc_service
from apps.admin_console.services.model_service import model_service
from apps.admin_console.services.task_queue_service import task_queue_service
router = APIRouter(tags=["tasks"])
@router.post("/api/run")
async def run_task(request: RunRequest):
incoming_goals = []
if request.goals:
incoming_goals = request.goals
elif request.goal:
incoming_goals = [request.goal]
if not incoming_goals:
raise HTTPException(
status_code=400,
detail="Either 'goal' or 'goals' list must be provided.",
)
return await task_queue_service.enqueue_tasks(
incoming_goals, profile=request.profile or "flash"
)
@router.post("/api/stop")
async def stop_task(all: bool = False):
stopped = task_queue_service.stop_tasks(clear_all=all)
if stopped:
return {"status": "stopped"}
return {"status": "no_running_task"}
@router.post("/api/resume")
async def resume_task():
resumed = task_queue_service.resume_task()
if resumed:
return {"status": "resumed"}
return {"status": "not_paused"}
@router.get("/api/status")
async def get_status():
latest_session = session_repo.get_latest_session()
latest_session_id = latest_session.get("session_id") if latest_session else None
bg_tasks = session_repo.get_background_tasks(latest_session_id) if latest_session_id else []
is_running = state.is_running
running_sid = state.active_session_id or (
session_repo.get_running_session_id() if is_running else None
)
active_profile = state.current_profile
if not active_profile and (running_sid or latest_session_id):
check_sid = running_sid or latest_session_id
sess_row = session_repo.get_session_by_id(check_sid)
if sess_row:
llm_traces = session_repo.get_llm_traces_for_profile(check_sid)
agent_names = session_repo.get_agent_trace_names(check_sid)
active_profile = model_service.resolve_session_profile(
sess_row, llm_traces, agent_names=agent_names
)
model_info = model_service.get_active_model_info(active_profile)
queue_data = state.queue_tasks
if state.current_process:
if state.current_process.returncode is None:
is_paused = state.is_paused
return {
"status": "paused" if is_paused else "running",
"goal": state.current_goal,
"pid": state.current_process.pid,
"session_id": running_sid,
"background_tasks": bg_tasks,
"queue": queue_data,
"model_info": model_info,
}
else:
completed_status = {
"status": "completed",
"goal": state.current_goal,
"returncode": state.current_process.returncode,
"session_id": latest_session_id,
"background_tasks": bg_tasks,
"queue": queue_data,
"model_info": model_info,
}
state.current_process = None
return completed_status
if latest_session_id and str(latest_session_id) in state.active_connections:
conn_info = state.active_connections[str(latest_session_id)]
is_paused = state.is_paused
conn_profile = conn_info.get("profile") or active_profile
conn_model_info = model_service.get_active_model_info(conn_profile)
return {
"status": "paused" if is_paused else "running",
"goal": conn_info.get("goal"),
"pid": conn_info.get("pid"),
"session_id": latest_session_id,
"background_tasks": bg_tasks,
"queue": queue_data,
"model_info": conn_model_info,
}
return {
"status": "idle",
"session_id": latest_session_id,
"background_tasks": bg_tasks,
"queue": queue_data,
"model_info": model_info,
}
@router.get("/api/stream/{session_id}")
async def stream_events(session_id: str, client: str | None = None):
async def event_generator():
queue = asyncio.Queue()
def callback(event_type, data):
try:
# Filter events by session_id when subscribed to a specific session
if session_id and session_id not in ("all", "active"):
evt_session_id = None
if isinstance(data, dict):
evt_session_id = data.get("session_id")
if evt_session_id and str(evt_session_id) != str(session_id):
return
if (
not evt_session_id
and state.active_session_id
and str(state.active_session_id) != str(session_id)
):
return
sanitized_data = ipc_service.sanitize_event_data(event_type, data)
loop = asyncio.get_running_loop()
loop.call_soon_threadsafe(queue.put_nowait, (event_type, sanitized_data))
except RuntimeError:
pass
state.add_subscriber(callback)
yield f'event: info\ndata: {{"message": "Subscribed to session {session_id}"}}\n\n'
try:
while True:
try:
event_type, data = await asyncio.wait_for(queue.get(), timeout=5.0)
yield f"event: {event_type}\ndata: {json.dumps(data, default=str)}\n\n"
except TimeoutError:
yield "event: keep-alive\ndata: {}\n\n"
except asyncio.CancelledError:
state.remove_subscriber(callback)
return StreamingResponse(event_generator(), media_type="text/event-stream")
+1
View File
@@ -0,0 +1 @@
# Copyright 2026 Google LLC
+28
View File
@@ -0,0 +1,28 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from pydantic import BaseModel
class RunRequest(BaseModel):
goal: str | None = None
goals: list[str] | None = None
profile: str | None = "flash"
class ReplayRequest(BaseModel):
device_id: str
user_submits: dict
tool_name: str = "ask_explorer"
replay_id: str | None = None
+154
View File
@@ -0,0 +1,154 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Artemis Admin & Trace Console Server
Modular entrypoint for full trace inspection, step replay, and task execution management.
"""
import asyncio
import os
from pathlib import Path
import sys
# Bootstrap sys.path to allow running from any CWD
_current_p = Path(__file__).resolve().parent
while _current_p != _current_p.parent:
if (_current_p / "pyproject.toml").exists() or (_current_p / "artemis").is_dir():
_workspace_root = _current_p
break
_current_p = _current_p.parent
else:
_workspace_root = Path(__file__).resolve().parent.parent.parent
_apps_dir = _workspace_root / "apps"
_admin_console_dir = _apps_dir / "admin_console"
_cloud_service_dir = _apps_dir / "cloud_service"
for _p in (str(_workspace_root), str(_apps_dir), str(_admin_console_dir), str(_cloud_service_dir)):
if _p not in sys.path:
sys.path.insert(0, _p)
from fastapi import FastAPI
from fastapi.middleware.cors import CORSMiddleware
try:
from admin_console.core.config import (
DB_PATH,
IMAGES_DIR,
REPLAY_BASE_DIR,
TEST_DATA_DIR,
TEST_OUTPUTS_DIR,
TRACES_PATH,
WORKSPACE_ROOT,
init_ls_address,
)
from admin_console.core.state import ServerState, state
from admin_console.database.connection import db_session, get_db
from admin_console.database.repositories.session_repository import session_repo
from admin_console.database.repositories.step_repository import step_repo
from admin_console.database.repositories.trace_repository import trace_repo
from admin_console.routers import media, replay, sessions, steps, stream, tasks
from admin_console.routers.replay import replay_manager
from admin_console.services.ipc_service import ipc_service
from admin_console.services.media_service import media_service
from admin_console.services.model_service import model_service
from admin_console.services.task_queue_service import task_queue_service
except ImportError:
from apps.admin_console.core.config import (
init_ls_address,
)
from apps.admin_console.core.state import state
from apps.admin_console.database.repositories.session_repository import session_repo
from apps.admin_console.routers import media, replay, sessions, steps, stream, tasks
from apps.admin_console.routers.replay import replay_manager
from apps.admin_console.services.ipc_service import ipc_service
from apps.admin_console.services.model_service import model_service
from apps.admin_console.services.task_queue_service import task_queue_service
# Initialize language server synchronization address
init_ls_address()
# Initialize FastAPI application
app = FastAPI(title="Artemis Admin & Trace Console")
# Configure CORS
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
@app.on_event("startup")
async def on_startup():
"""Startup lifecycle hooks."""
asyncio.create_task(asyncio.to_thread(task_queue_service.archive_older_replays_on_launch))
asyncio.create_task(
asyncio.to_thread(task_queue_service.verify_chunks_exist_on_launch, replay_manager)
)
cleaned = session_repo.cleanup_orphans_on_startup()
if cleaned > 0:
print(f"[ServerStartup] Marked {cleaned} orphaned running session(s) as failed.")
await ipc_service.start_server()
state.worker_task = asyncio.create_task(task_queue_service.queue_worker())
# Mount modular routers
app.include_router(stream.router)
app.include_router(media.router)
app.include_router(sessions.router)
app.include_router(steps.router)
app.include_router(tasks.router)
app.include_router(replay.router)
# Mount cloud gateway router for Frappe / Cloud integration if present
try:
from cloud_service.gateway import cloud_router
app.include_router(cloud_router)
except ImportError:
try:
from apps.cloud_service.gateway import cloud_router
app.include_router(cloud_router)
except ImportError:
pass
# ------------------------------------------------------------------------------
# Backward compatibility exports
# ------------------------------------------------------------------------------
get_active_model_info = model_service.get_active_model_info
start_ipc_server = ipc_service.start_server
queue_worker = task_queue_service.queue_worker
active_connections = state.active_connections
ipc_subscribers = state.ipc_subscribers
task_queue = state.task_queue
queue_goals = state.queue_goals
def main():
import uvicorn
port = int(os.environ.get("ANTIGRAVITY_SIDECAR_WEB_PORT", 8000))
uvicorn.run(app, host="0.0.0.0", port=port)
if __name__ == "__main__":
main()
+1
View File
@@ -0,0 +1 @@
# Copyright 2026 Google LLC
@@ -0,0 +1,133 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
"""Device Live Screen Streaming Service.
Provides real-time, low-latency device screen frames over HTTP MJPEG and WebSocket.
Operates concurrently with ADB agent actions with zero interference.
"""
import asyncio
import logging
import subprocess
import time
from collections.abc import AsyncGenerator
logger = logging.getLogger("artemis.stream_service")
class DeviceStreamService:
"""Manages real-time screen capture and distribution to web clients."""
def __init__(self):
self._active_listeners = 0
self._lock = asyncio.Lock()
self._latest_frame: bytes | None = None
self._last_frame_time: float = 0.0
self._is_capturing = False
self._capture_task: asyncio.Task | None = None
async def get_device_serial(self) -> str | None:
"""Find the currently connected active ADB device serial."""
try:
proc = await asyncio.create_subprocess_exec(
"adb",
"devices",
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
stdout, _ = await proc.communicate()
lines = stdout.decode().strip().splitlines()
for line in lines[1:]:
parts = line.strip().split()
if len(parts) >= 2 and parts[1] == "device":
return parts[0]
except Exception as e:
logger.warning(f"Error checking adb devices: {e}")
return None
async def _capture_loop(self):
"""Background frame capture loop that runs while listeners > 0."""
logger.info("[StreamService] Starting live screen capture loop...")
while self._active_listeners > 0:
try:
start_t = time.time()
serial = await self.get_device_serial()
cmd = (
["adb", "-s", serial, "exec-out", "screencap", "-p"]
if serial
else ["adb", "exec-out", "screencap", "-p"]
)
proc = await asyncio.create_subprocess_exec(
*cmd,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
stdout, _ = await proc.communicate()
if proc.returncode == 0 and len(stdout) > 1000:
self._latest_frame = stdout
self._last_frame_time = time.time()
elapsed = time.time() - start_t
delay = max(0.03, 0.08 - elapsed)
await asyncio.sleep(delay)
except asyncio.CancelledError:
break
except Exception as e:
logger.error(f"[StreamService] Capture error: {e}")
await asyncio.sleep(0.5)
logger.info("[StreamService] Stopping live screen capture loop (0 listeners).")
self._is_capturing = False
async def start_capturing(self):
"""Register a new listener and start background capture if needed."""
async with self._lock:
self._active_listeners += 1
if not self._is_capturing or self._capture_task is None or self._capture_task.done():
self._is_capturing = True
self._capture_task = asyncio.create_task(self._capture_loop())
async def stop_capturing(self):
"""Deregister a listener and stop capture loop when count reaches 0."""
async with self._lock:
self._active_listeners = max(0, self._active_listeners - 1)
if self._active_listeners == 0 and self._capture_task and not self._capture_task.done():
self._capture_task.cancel()
self._is_capturing = False
async def mjpeg_frame_generator(self) -> AsyncGenerator[bytes, None]:
"""Async generator streaming MJPEG multipart bytes to HTTP response."""
await self.start_capturing()
try:
last_sent_time = 0.0
while True:
if self._latest_frame and self._last_frame_time > last_sent_time:
last_sent_time = self._last_frame_time
frame_bytes = self._latest_frame
yield (
b"--frame\r\n"
b"Content-Type: image/png\r\n"
b"Content-Length: "
+ str(len(frame_bytes)).encode()
+ b"\r\n\r\n"
+ frame_bytes
+ b"\r\n"
)
await asyncio.sleep(0.04)
finally:
await self.stop_capturing()
device_stream_service = DeviceStreamService()
+204
View File
@@ -0,0 +1,204 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import asyncio
import json
from typing import Any
try:
from admin_console.core.state import state
except ImportError:
from apps.admin_console.core.state import state
class IPCService:
"""Service managing internal IPC socket server and event distribution."""
@staticmethod
def _clean_value(val: Any) -> Any:
"""Recursively cleans values to remove non-serializable and internal
Python runtime representations.
"""
if val is None:
return None
if isinstance(val, (bool, int, float)):
return val
if isinstance(val, str):
if (
"object at 0x" in val
or val.startswith("<artemis.")
or val.startswith("<controller")
):
return None
return val
if isinstance(val, dict):
cleaned = {}
for k, v in val.items():
cv = IPCService._clean_value(v)
if cv is not None:
cleaned[k] = cv
return cleaned
if isinstance(val, (list, tuple)):
cleaned_list = []
for item in val:
cv = IPCService._clean_value(item)
if cv is not None:
cleaned_list.append(cv)
return cleaned_list
val_str = str(val)
if "object at 0x" in val_str or val_str.startswith("<"):
return None
return val_str
@classmethod
def sanitize_event_data(cls, event_type: str, data: Any) -> Any:
"""Sanitizes stream event payloads for all clients, stripping bulky
raw bytes and internal memory references.
"""
if not isinstance(data, dict):
return data
if event_type == "trace_recorded":
tr_type = data.get("type")
payload = data.get("payload")
if isinstance(payload, str):
try:
payload = json.loads(payload)
except Exception:
pass
if tr_type == "tool" and isinstance(payload, dict):
args = payload.get("args", payload)
data["payload"] = {"args": cls._clean_value(args)}
elif tr_type == "llm_call" and isinstance(payload, dict):
data["payload"] = {"error": cls._clean_value(payload.get("error"))}
elif payload is not None:
data["payload"] = cls._clean_value(payload)
elif event_type in ("step_recorded", "step_updated"):
# Strip heavy raw image bytes from SSE broadcast (clients fetch via image URLs)
data["pre_screenshot_bytes"] = None
data["post_screenshot_bytes"] = None
if "generic_tools" in data and isinstance(data["generic_tools"], list):
sanitized_tools = []
for tr in data["generic_tools"]:
if not isinstance(tr, dict):
continue
tr_dict = dict(tr)
pl = tr_dict.get("payload")
if isinstance(pl, str):
try:
pl = json.loads(pl)
except Exception:
pass
if tr_dict.get("type") == "llm_call":
tr_dict["payload"] = (
{"error": cls._clean_value(pl.get("error"))}
if isinstance(pl, dict)
else None
)
elif isinstance(pl, dict):
args = pl.get("args", pl)
tr_dict["payload"] = {"args": cls._clean_value(args)}
else:
tr_dict["payload"] = cls._clean_value(pl)
sanitized_tools.append(tr_dict)
data["generic_tools"] = sanitized_tools
try:
from artemis.utils.coordinates import normalize_step_actions
data = normalize_step_actions(data)
except ImportError:
pass
return data
# Backward compatibility alias
filter_event_for_angular = sanitize_event_data
@classmethod
async def start_server(cls):
async def handle_client(reader: asyncio.StreamReader, writer: asyncio.StreamWriter):
current_session_id = None
while True:
line = await reader.readline()
if not line:
break
try:
payload = json.loads(line.decode("utf-8"))
event_type = payload.get("event_type")
data = payload.get("data")
if event_type == "session_started" and data:
state.active_session_id = data.get("session_id")
current_session_id = data.get("session_id")
pid = data.get("pid")
goal = data.get("initial_goal")
device_info = data.get("device_info")
if isinstance(device_info, dict) and device_info.get("profile"):
state.current_profile = device_info.get("profile")
if goal:
state.current_goal = goal
print(
f"[IPC] Session started: {state.active_session_id} (profile: {state.current_profile})"
)
if current_session_id:
state.active_connections[str(current_session_id)] = {
"writer": writer,
"pid": pid,
"goal": goal,
"profile": state.current_profile,
}
elif event_type == "session_ended" and data:
ended_sid = data.get("session_id")
if state.active_session_id and str(state.active_session_id) == str(
ended_sid
):
state.active_session_id = None
state.current_goal = None
state.current_profile = None
for cb in list(state.ipc_subscribers):
try:
cb(event_type, data)
except Exception:
pass
except Exception as e:
print(f"IPC parse error: {e}")
if current_session_id and str(current_session_id) in state.active_connections:
try:
del state.active_connections[str(current_session_id)]
except KeyError:
pass
writer.close()
await writer.wait_closed()
server = await asyncio.start_server(handle_client, "127.0.0.1", 0, limit=1024 * 1024 * 100)
state.ipc_port = server.sockets[0].getsockname()[1]
state.ipc_server = server
asyncio.create_task(server.serve_forever())
print(f"Internal IPC server started on port {state.ipc_port}")
try:
from artemis.config import write_ipc_port
write_ipc_port(state.ipc_port)
except Exception as e:
print(f"Failed to write IPC port file: {e}")
ipc_service = IPCService()
@@ -0,0 +1,261 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import base64
import hashlib
import json
import os
from pathlib import Path
from typing import Any
import urllib.parse
from fastapi import HTTPException
try:
from admin_console.core.config import IMAGES_DIR, TRACES_PATH, WORKSPACE_ROOT
except ImportError:
from apps.admin_console.core.config import IMAGES_DIR, TRACES_PATH, WORKSPACE_ROOT
class MediaService:
"""Service handling media indexing, local file security, payload unwrapping and notes/plans."""
@staticmethod
def path_to_video_url(p: Path) -> str:
resolved = p.resolve()
try:
rel = resolved.relative_to(WORKSPACE_ROOT)
return f"/videos/{rel}"
except ValueError:
return f"/videos/{str(resolved).lstrip('/')}"
@classmethod
def build_video_index(cls) -> dict[str, str]:
idx = {}
dirs = [
TRACES_PATH,
WORKSPACE_ROOT / "artemis-traces",
WORKSPACE_ROOT / ".benchmarks" / "diagnoser" / "outputs" / "artifacts",
]
for d in dirs:
if not d.exists():
continue
try:
for item in d.iterdir():
if item.is_dir():
for ext in [".mp4", ".mkv", ".webm"]:
vfile = item / f"recording{ext}"
if vfile.exists():
url = cls.path_to_video_url(vfile)
idx[item.name] = url
prefix = item.name.split("_PASS_")[0].split("_FAIL_")[0]
idx[prefix] = url
break
elif item.suffix in [".mp4", ".mkv", ".webm"]:
url = cls.path_to_video_url(item)
idx[item.stem] = url
idx[item.name] = url
except Exception:
pass
return idx
@classmethod
def resolve_video_url(
cls,
row_dict: dict[str, Any],
video_rec_map: dict[str, str],
video_idx: dict[str, str],
) -> str | None:
raw_sid = row_dict.get("session_id")
s_id = (
str(raw_sid).strip()
if raw_sid is not None and str(raw_sid).strip().lower() != "none"
else ""
)
v_url = None
v_fp = row_dict.get("video_filepath")
if v_fp and os.path.exists(v_fp):
v_url = cls.path_to_video_url(Path(v_fp))
orig_rec = video_rec_map.get(s_id)
if not v_url and orig_rec:
p = Path(orig_rec)
if p.exists():
v_url = cls.path_to_video_url(p)
else:
folder_name = p.parent.name
v_url = video_idx.get(folder_name) or video_idx.get(folder_name.split("_")[0])
if not v_url:
v_url = video_idx.get(s_id)
if not v_url and s_id:
sess_dir = TRACES_PATH / s_id
if sess_dir.is_dir():
for ext in [".mp4", ".webm", ".mkv"]:
vfile = sess_dir / f"recording{ext}"
if vfile.exists():
v_url = cls.path_to_video_url(vfile)
break
if not v_url:
try:
for item in sess_dir.iterdir():
if item.is_file() and item.suffix.lower() in [".mp4", ".webm", ".mkv"]:
v_url = cls.path_to_video_url(item)
break
except Exception:
pass
if not v_url and row_dict.get("initial_goal"):
goal_str = row_dict["initial_goal"]
task_hint = (
goal_str.split("Task: ")[1].split()[0].strip()
if "Task: " in goal_str
else goal_str.split("\n")[0].strip()
)
v_url = video_idx.get(task_hint)
return v_url
@staticmethod
def unwrap_payload(obj: Any) -> Any:
"""Unwraps trace payloads, storing base64 images as disk files and
truncating large dumps.
"""
if isinstance(obj, str):
try:
if (obj.startswith("{") and obj.endswith("}")) or (
obj.startswith("[") and obj.endswith("]")
):
return MediaService.unwrap_payload(json.loads(obj))
except Exception:
pass
# Check if the string is a Base64 image
is_base64_img = False
base64_data = None
if obj.startswith("data:image/") and ";base64," in obj:
try:
_, base64_data = obj.split(";base64,", 1)
is_base64_img = True
except Exception:
pass
elif obj.startswith("iVBORw0KGgo") or obj.startswith("/9j/"):
is_base64_img = True
base64_data = obj
if is_base64_img and base64_data:
try:
image_bytes = base64.b64decode(base64_data)
image_hash = hashlib.sha256(image_bytes).hexdigest()
IMAGES_DIR.mkdir(parents=True, exist_ok=True)
image_path = IMAGES_DIR / f"{image_hash}.jpg"
if not image_path.exists():
image_path.write_bytes(image_bytes)
return f"image://{image_hash}"
except Exception:
pass
if len(obj) > 2000:
preview_start = obj[:100].replace("\n", " ")
preview_end = obj[-100:].replace("\n", " ")
return (
f"<Massive String: {preview_start}...[truncated"
f" {len(obj) - 200} characters for UI performance]...{preview_end}"
f" (length={len(obj)})>"
)
return obj
elif isinstance(obj, dict):
return {k: MediaService.unwrap_payload(v) for k, v in obj.items()}
elif isinstance(obj, list):
if len(obj) > 20 and any(
isinstance(x, dict) and ("bounds" in x or "resource-id" in x) for x in obj
):
return [f"<XML UI List with {len(obj)} elements truncated for UI performance>"]
return [MediaService.unwrap_payload(x) for x in obj]
return obj
@staticmethod
def get_safe_local_file(path_str: str) -> tuple[Path, str]:
decoded_path = urllib.parse.unquote(path_str)
if decoded_path.startswith("file://"):
decoded_path = decoded_path[len("file://") :]
p = Path(decoded_path).resolve()
if not p.is_file():
raise HTTPException(status_code=404, detail="File not found")
if not str(p).startswith(str(WORKSPACE_ROOT)):
raise HTTPException(
status_code=403,
detail="Access denied. Path is outside workspace root.",
)
ext = p.suffix.lower()
media_type = "application/octet-stream"
if ext in (".jpg", ".jpeg"):
media_type = "image/jpeg"
elif ext == ".png":
media_type = "image/png"
elif ext == ".mp4":
media_type = "video/mp4"
return p, media_type
@staticmethod
def get_task_plan_content(session_id: str) -> str:
plan_path = TRACES_PATH / session_id / "notes" / "task_plan.md"
if not plan_path.exists():
plan_path = TRACES_PATH / "notes" / "task_plan.md"
if plan_path.exists():
try:
return plan_path.read_text(encoding="utf-8")
except Exception as e:
return f"Error reading task plan: {e}"
return "No task plan created yet."
@staticmethod
def get_session_notes_content(session_id: str) -> dict[str, str]:
notes_dir = TRACES_PATH / session_id / "notes"
notes_content = {}
if notes_dir.exists() and notes_dir.is_dir():
for file in notes_dir.iterdir():
if file.is_file() and file.suffix in [".md", ".txt", ".json", ".yaml"]:
try:
notes_content[file.name] = file.read_text(encoding="utf-8")
except Exception as e:
notes_content[file.name] = f"Error reading file: {e}"
if not notes_content:
global_notes_dir = TRACES_PATH / "notes"
if global_notes_dir.exists() and global_notes_dir.is_dir():
for file in global_notes_dir.iterdir():
if file.is_file() and file.suffix in [
".md",
".txt",
".json",
".yaml",
]:
try:
notes_content[file.name] = file.read_text(encoding="utf-8")
except Exception as e:
notes_content[file.name] = f"Error reading file: {e}"
return notes_content
media_service = MediaService()
@@ -0,0 +1,133 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import json
from typing import Any
class ModelService:
"""Service handling model profile detection and metadata formatting."""
@staticmethod
def get_active_model_info(profile: str | None = None) -> dict[str, str]:
if profile:
p_lower = str(profile).lower()
if "pro" in p_lower:
return {
"name": "Pro",
"id": "gemini-3.6-pro",
"provider": "google",
}
elif "flash" in p_lower:
return {
"name": "Flash",
"id": "gemini-3.6-flash",
"provider": "google",
}
model_name = "Flash"
model_id = "gemini-3.6-flash"
provider = "google"
try:
from artemis.config import parse_llm_config
llm_cfg = parse_llm_config()
if llm_cfg and llm_cfg.operator:
op_model = str(llm_cfg.operator.model).lower()
provider = str(llm_cfg.operator.provider)
model_id = llm_cfg.operator.model
if "pro" in op_model:
model_name = "Pro"
elif "flash" in op_model:
model_name = "Flash"
else:
model_name = llm_cfg.operator.model.capitalize()
except Exception:
pass
return {"name": model_name, "id": model_id, "provider": provider}
@staticmethod
def resolve_session_profile(
row_dict: dict[str, Any],
llm_trace_payloads: list[str] | None = None,
running_profile: str | None = None,
agent_names: list[str] | None = None,
) -> str | None:
sess_profile = None
d_info_raw = row_dict.get("device_info")
if d_info_raw:
try:
d_info = json.loads(d_info_raw) if isinstance(d_info_raw, str) else d_info_raw
if isinstance(d_info, dict):
p = d_info.get("profile")
if p:
p_str = str(p).lower()
if "pro" in p_str:
return "pro"
elif "flash" in p_str:
return "flash"
except Exception:
pass
if (row_dict.get("status") == "running") and running_profile:
p_str = str(running_profile).lower()
if "pro" in p_str:
return "pro"
elif "flash" in p_str:
return "flash"
# Check Agent/Trace names - the most definitive indicator of architecture
if agent_names:
agent_names_lower = [str(a).lower() for a in agent_names]
if any(
name in ("planner", "validator", "summarizer", "operator", "checker", "diagnoser")
for name in agent_names_lower
):
return "pro"
if any("flashrunner" in name for name in agent_names_lower):
return "flash"
# Check LLM traces for agent name or model overrides
if llm_trace_payloads:
for tr_payload in llm_trace_payloads:
if tr_payload:
try:
p_obj = (
json.loads(tr_payload) if isinstance(tr_payload, str) else tr_payload
)
if isinstance(p_obj, dict):
agent_name = str(p_obj.get("agent") or p_obj.get("name") or "").lower()
if "flashrunner" in agent_name:
return "flash"
if any(
x in agent_name
for x in ("planner", "validator", "operator", "checker")
):
return "pro"
m_str = str(p_obj.get("model") or p_obj.get("model_name") or "").lower()
if (
"gemini-3.6-pro" in m_str
or "gemini-1.5-pro" in m_str
or "gpt-4" in m_str
or "claude" in m_str
):
return "pro"
except Exception:
pass
return None
model_service = ModelService()
@@ -0,0 +1,256 @@
# Copyright 2026 Google LLC
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
import asyncio
from datetime import datetime
import os
import shutil
import signal
import sys
import time
from typing import Any
import uuid
try:
from admin_console.core.config import (
PAUSE_FILE,
TEST_DATA_DIR,
TEST_OUTPUTS_DIR,
WORKSPACE_ROOT,
)
from admin_console.core.state import state
from admin_console.database.repositories.session_repository import session_repo
except ImportError:
from apps.admin_console.core.config import (
PAUSE_FILE,
TEST_DATA_DIR,
TEST_OUTPUTS_DIR,
WORKSPACE_ROOT,
)
from apps.admin_console.core.state import state
from apps.admin_console.database.repositories.session_repository import session_repo
class TaskQueueService:
"""Service managing FIFO task execution, background worker, subprocess lifecycle,
and startup tasks.
"""
@classmethod
async def queue_worker(cls):
while True:
goal = None
sess_id = None
try:
item = await state.task_queue.get()
if isinstance(item, dict):
goal = item.get("goal")
profile = item.get("profile", "flash")
sess_id = item.get("session_id")
else:
goal = item
profile = "flash"
sess_id = None
state.current_goal = goal
state.current_profile = profile
if sess_id:
state.active_session_id = sess_id
# Remove from state.queue_tasks
state.queue_tasks = [
t
for t in state.queue_tasks
if not (
isinstance(t, dict)
and (t.get("session_id") == sess_id or t.get("goal") == goal)
)
]
test_name = f"web_{int(time.time())}"
env = os.environ.copy()
pythonpath_parts = [
str(WORKSPACE_ROOT),
str(WORKSPACE_ROOT / "apps/admin_console"),
str(WORKSPACE_ROOT / "apps/cloud_service"),
env.get("PYTHONPATH", ""),
]
if state.ipc_port is not None:
env["ARTEMIS_IPC_PORT"] = str(state.ipc_port)
if sess_id:
env["ARTEMIS_SESSION_ID"] = str(sess_id)
print(f"[QueueWorker] Starting task: {goal} with profile: {profile}")
state.current_process = await asyncio.create_subprocess_exec(
sys.executable,
"-m",
"artemis.main",
goal,
"--profile",
profile,
"--test-name",
test_name,
cwd=str(WORKSPACE_ROOT),
env=env,
)
await state.current_process.wait()
print(
f"[QueueWorker] Task finished with code "
f"{state.current_process.returncode}: {goal}"
)
# Fallback status update
if state.active_session_id:
curr_status = session_repo.get_session_status(state.active_session_id)
if curr_status == "running":
if state.was_stopped_manually:
new_status = "cancelled"
else:
new_status = (
"completed" if state.current_process.returncode == 0 else "failed"
)
session_repo.update_session_status(
state.active_session_id, new_status, time.time()
)
print(
f"[QueueWorker] Fallback: Updated session "
f"{state.active_session_id} status to {new_status}"
)
state.active_session_id = None
state.current_profile = None
state.was_stopped_manually = False
except asyncio.CancelledError:
print("[QueueWorker] Background worker task was cancelled.")
break
except Exception as e:
print(f"[QueueWorker] Error executing task: {e}")
finally:
if goal is not None:
state.task_queue.task_done()
@classmethod
async def enqueue_tasks(cls, goals: list[str], profile: str = "flash") -> dict[str, Any]:
enqueued_tasks = []
now = time.time()
for i, goal in enumerate(goals):
sess_id = str(uuid.uuid4())
task_item = {
"session_id": sess_id,
"goal": goal,
"profile": profile or "flash",
"status": "pending",
"created_at": now + i * 0.001,
"start_time": now + i * 0.001,
}
await state.task_queue.put(task_item)
state.queue_tasks.append(task_item)
enqueued_tasks.append(task_item)
return {
"status": "queued" if state.is_running else "started",
"tasks": enqueued_tasks,
"enqueued_count": len(goals),
"total_queued": len(state.queue_tasks),
}
@classmethod
def stop_tasks(cls, clear_all: bool = False) -> bool:
stopped = False
if clear_all:
state.clear_queue()
state.was_stopped_manually = True
if state.is_running and state.current_process:
try:
state.current_process.kill()
stopped = True
except Exception:
pass
for _, conn_info in list(state.active_connections.items()):
pid = conn_info.get("pid")
if pid:
try:
os.kill(pid, signal.SIGINT)
stopped = True
except Exception as e:
print(f"Failed to kill external process {pid}: {e}")
session_repo.mark_all_running_cancelled()
if PAUSE_FILE.exists():
try:
PAUSE_FILE.unlink()
except Exception:
pass
return stopped
@classmethod
def resume_task(cls) -> bool:
if PAUSE_FILE.exists():
PAUSE_FILE.unlink()
return True
return False
@staticmethod
def archive_older_replays_on_launch():
"""Archives all existing step replay output folders on server launch."""
if TEST_OUTPUTS_DIR.exists():
archive_dir = TEST_DATA_DIR / "older"
for item in TEST_OUTPUTS_DIR.iterdir():
if item.is_dir():
archive_dir.mkdir(parents=True, exist_ok=True)
timestamp = datetime.now().strftime("%Y%m%d_%H%M%S")
target_name = f"{item.name}_{timestamp}"
target_path = archive_dir / target_name
counter = 1
while target_path.exists():
target_name = f"{item.name}_{timestamp}_{counter}"
target_path = archive_dir / target_name
counter += 1
print(
f"Archiving older replay record on server launch: "
f"{item.name} -> {target_path}"
)
try:
shutil.move(str(item), str(target_path))
except Exception as e:
print(
f"Warning: Failed to archive {item.name} during server launch: {e}",
file=sys.stderr,
)
@staticmethod
def verify_chunks_exist_on_launch(replay_manager):
"""Verifies that chunked directories exist for all sessions in the database."""
print("Verifying session chunks on launch...")
try:
sessions = session_repo.get_all_sessions()
session_ids = [s.get("session_id") for s in sessions if s.get("session_id")]
print(f"Found {len(session_ids)} sessions in database to verify.")
for session_id in session_ids:
replay_manager._ensure_session_chunked(str(session_id))
except Exception as e:
print(
f"Warning: Failed to verify chunks during server launch: {e}",
file=sys.stderr,
)
task_queue_service = TaskQueueService()