chiwanko11
5d2283deaa
test: add comprehensive unit, integration, and e2e test suites
...
Introduce complete test coverage for the ARTEMIS platform across multiple layers:
Unit tests: Add tests for agent nodes (Planner, Operator, Validator, Diagnoser, FlashRunner, Explorer), tools, utils (OCR/XML fusion, UI hierarchy), and data engine.
Integration tests: Add live tool verification, reactive runner, IME, and standalone workflow tests.
E2E tests: Add end-to-end scenario tests for media playback and failure recovery flows.
Fixtures & mocks: Add mock inputs (audio, video, screenshot) and test utilities.
2026-08-13 12:18:32 -07:00
Weilin Liu
c1803fcb86
feat(console): add Artemis Admin & Trace Console backend and management plane
...
Introduces the comprehensive developer/administrator control plane and trace
inspection console backend for the ARTEMIS mobile automation platform.
Key Features & Backend Architecture:
- Core Server & Modular Routing:
* FastAPI modular architecture with dedicated routers for sessions, steps, tasks, media, stream, and replay.
* Multi-layer repository pattern with SQLite persistence for sessions, traces, and step data.
- Task Control & Queue Management:
* FIFO asynchronous task queue service with support for single/batch goal execution and profile selection (Flash / Pro).
* Session lifecycle controls (run, pause, resume, stop, clear queue).
* Startup orphan cleanup and automatic older replay archiving.
- Live Telemetry & IPC Communication:
* High-performance IPC server for streaming real-time execution logs, agent thoughts, and tool invocations.
* Server-Sent Events (SSE) /api/stream/{session_id} endpoint for live UI updates.
- Real-Time Device Streaming:
* Live ADB screen capture and MJPEG frame generator (/api/stream/device-live).
* Device connectivity and state inspection endpoint (/api/stream/device-state).
- Trace Inspection & Step Replay Engine:
* Hierarchical call tree inspection, pre/post step screenshots, and action overlay metadata.
* Isolated single-step replay manager (/api/replay) for interactive debugging and validation.
2026-08-13 12:11:10 -07:00
yaoyao-open-source
2fd70465e6
feat(web): add interactive Angular web UI dashboard and execution visualizer
...
Introduces the full-featured, modern Angular 19 web dashboard for ARTEMIS.
Provides real-time multimodal agent stream visualization, live device feeds,
interactive step inspection, task queue management, and model telemetry.
Key Features & UI Architecture:
- Workspace & Layout:
* Split-pane responsive layout with draggable resizer bar for execution stream and chat/task panel.
* Standalone Angular components powered by Signals and reactive computed state.
- Real-Time Multimodal Agent Stream:
* SSE connection (/api/stream/:session_id) for live streaming of thoughts, actions, and tool calls.
* Hierarchical phase grouping (Planning -> Execution -> Verification -> Summary).
* Interactive step viewer with before/after screenshot overlays, coordinate pins, bounding boxes, and OCR text highlights.
* Live planning loader, error diagnostics banner, retry status indicator, and token usage metrics.
- Task Queue & Chat Interface:
* Goal submission with instant dispatch to backend runner (/api/run).
* Active task queue tracker (running, pending queue in FIFO order, completed/failed historical sessions).
* Session switcher with model info badges and Markdown notes viewer (e.g. task_plan.md).
- Floating Video Player & Live Stream:
* Draggable, floating video player overlay for reviewing session recordings and live device feeds (/api/stream/device-live).
* Playback controls: play/pause, seek scrubber, speed multipliers (0.5x - 4.0x), looping, and theater mode.
- Formatting & Parsing Utilities:
* Android UI action formatter (tap, type, swipe, drag, keyevent, app launch).
* Tool call formatter for ADB commands, note management, video inspection, and failure recovery.
* Markdown parsing and milestone checklist rendering.
2026-08-13 11:58:16 -07:00
somew1nd
206a9ce921
feat: initial release of ARTEMIS Android Agent & Automation Platform
...
ARTEMIS is an autonomous multimodal Android agent and mobile UI automation
platform engineered for physical devices and emulators. It combines state-of-the-art
multimodal Vision-Language Models (VLMs) with hierarchical closed-loop planning,
resilient self-healing, and deep Android subsystem integration.
Key Features & Architectural Highlights:
- Dual-Engine Orchestration:
* Pro Engine: Multi-agent closed-loop (Planner -> Operator -> Validator/Failure Analyzer -> Summarizer) powered by LangGraph.
* Flash Engine: Ultra-fast reactive loop for deterministic, low-latency UI execution (<35 steps).
- Multimodal Perception & Grounding:
* Hybrid visual screenshot grounding, real-time OCR extraction, and XML accessibility hierarchy parsing.
* Dynamic-First, Coordinate-Fallback locating strategy for maximum resilience against UI drift.
- Autonomous Self-Healing & Diagnostics:
* Failure analyzer and safety net for popup dismissal, crash recovery, and auto-retry.
* Diagnostic sub-agents for log analysis, video recording inspection, and object detection.
- Unified CLI & Python SDK:
* Rich CLI with commands: init, doctor, run, trace, batch, bench, server, and mcp.
* Fluent Python SDK with Pydantic structured output models and package locking.
- Multi-Provider LLM Architecture:
* Support for Google Gemini, OpenAI, Anthropic Claude, Vertex AI, OpenRouter, xAI Grok, and local endpoints (Ollama/vLLM).
* Centralized configuration in config/artemis.jsonc.
- Native MCP & Observability:
* Model Context Protocol (MCP) server endpoints for AI IDEs.
* Full trace recording with before/after screenshots, action overlays, and execution replays.
Change-Id: I430b93ee2b76b54cc5a3c6eeccd15d2cf29830a1
2026-08-13 11:46:47 -07:00
chiwanko11