feat: introduce MCP server, cross-platform runtime, and Showcase UI reorganization

- Showcase UI Reorganization:
  * Move web UI dashboard from root directory into apps/showcase_ui/ for cleaner project structure
  * Add 'artemis ui' CLI command (artemis/interfaces/cli/commands/ui.py) and launcher scripts for the interactive showcase console
- Standalone MCP Server & Notifiers (mcp_server/):
  * Implement dedicated MCP server package with mobile automation tools (device_state, inspect_trace, task_manager, task_runner)
  * Add multi-channel notifiers (AgentAPI, Desktop notifications, Webhooks, File output, and Composite routing)
  * Introduce background task runner and trace store for autonomous mobile execution monitoring
- Cross-Platform Runtime & Toolchain Support:
  * Add artemis/platform/ with OS-specific implementations for Linux, macOS (Darwin), and Windows
  * Add artemis/runtime/supervisor.py for resilient process and agent runtime supervision
  * Add artemis/toolchain/ with dependency resolution and SDK descriptors
  * Add cross-platform launcher scripts (start.sh, start.bat) and installer scripts (scripts/install_deps.*)
- Core Enhancements & Diagnostics:
  * Enhance coordinate normalization, OCR API fallback, and perception pipeline
  * Add credential validation (credentials_validator.py) and system diagnostics
  * Update bilingual documentation (README.md, README_CN.md) and architecture diagrams
- Test Suite Expansion:
  * Add integration tests for cloud gateway, remote ADB, and Artemis login
  * Add unit tests covering core diagnostics, platform detection, toolchain, credentials validator, optional OCR, and MCP tools
This commit is contained in:
somew1nd committed 2026-08-18 13:59:27 -07:00
1 parent 78e478f502
commit 9a5aaf99b7
249 files changed
+37226 -9339

No files matched your search

@@ -24,7 +24,7 @@ You must carefully choose the appropriate tool based on the nature of the task:
- **Must use Visual Analysis (`spawn_sub_agent`)**:
- **UI Interactions**: Identifying clicks, gestures, typing, navigation.
- **Visual State**: Reading on-screen text (OCR), checking UI status (toggles, dialogs, loading states).
- **Visual State**: Reading on-screen text, checking UI status (toggles, dialogs, loading states).
- **Visual Content**: Identifying images, layout issues, animations.
- **Can use Audio-Only Analysis (`analyze_audio_only`)**:
- **Speech-centric**: Summarizing spoken content, identifying speakers, transcription.
@@ -40,7 +40,7 @@ from langchain_core.messages import (
from artemis.agents.video_analyzer.conflict_resolution import (
ConflictResolutionService,
)
from artemis.config import settings
from artemis.config import get_temp_dir, settings
from artemis.constants import SAFETY_SETTINGS_BLOCK_NONE
from artemis.context import ArtemisContext
from artemis.controllers.controller_factory import get_controller
@@ -461,10 +461,10 @@ class VideoAnalyzer:
utils_cfg = getattr(llm_config, "utils", None) if llm_config else None
llm_cfg = getattr(utils_cfg, "video_analyzer", None) if utils_cfg else None
model_str = (
llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.6-flash"
llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.7-flash"
).lower()
self.model_name = (
llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.6-flash"
llm_cfg.model if (llm_cfg and hasattr(llm_cfg, "model")) else "gemini-3.7-flash"
)
if "/" in self.model_name:
self.model_name = self.model_name.split("/")[-1]
@@ -1142,7 +1142,7 @@ class VideoAnalyzer:
confidence = float(event.get("confidence_score", 0.0))
relative_offset = max(0.0, verification_ts - actual_start)
out_dir = Path("/tmp/video_analyzer")
out_dir = get_temp_dir("video_analyzer")
out_dir.mkdir(parents=True, exist_ok=True)
existing_imgs = glob.glob(f"{out_dir}/img_*.jpg")
max_idx = -1
@@ -1692,7 +1692,7 @@ class VideoAnalyzer:
screenshot_path_str = None
try:
rel_offset = max(0.0, v_ts - actual_start)
out_dir = Path("/tmp/video_analyzer")
out_dir = get_temp_dir("video_analyzer")
out_dir.mkdir(parents=True, exist_ok=True)
existing_imgs = glob.glob(f"{out_dir}/img_*.jpg")
@@ -1899,7 +1899,7 @@ class VideoAnalyzer:
if getattr(llm_cfg, "thinking_level", None) is not None:
thinking_level = llm_cfg.thinking_level
else:
self.model_name = "gemini-3.6-flash"
self.model_name = "gemini-3.7-flash"
# Track files for cleanup
self.local_files_to_cleanup = set()