 yaoyao-open-source
|
2fd70465e6
|
feat(web): add interactive Angular web UI dashboard and execution visualizer
Introduces the full-featured, modern Angular 19 web dashboard for ARTEMIS.
Provides real-time multimodal agent stream visualization, live device feeds,
interactive step inspection, task queue management, and model telemetry.
Key Features & UI Architecture:
- Workspace & Layout:
* Split-pane responsive layout with draggable resizer bar for execution stream and chat/task panel.
* Standalone Angular components powered by Signals and reactive computed state.
- Real-Time Multimodal Agent Stream:
* SSE connection (/api/stream/:session_id) for live streaming of thoughts, actions, and tool calls.
* Hierarchical phase grouping (Planning -> Execution -> Verification -> Summary).
* Interactive step viewer with before/after screenshot overlays, coordinate pins, bounding boxes, and OCR text highlights.
* Live planning loader, error diagnostics banner, retry status indicator, and token usage metrics.
- Task Queue & Chat Interface:
* Goal submission with instant dispatch to backend runner (/api/run).
* Active task queue tracker (running, pending queue in FIFO order, completed/failed historical sessions).
* Session switcher with model info badges and Markdown notes viewer (e.g. task_plan.md).
- Floating Video Player & Live Stream:
* Draggable, floating video player overlay for reviewing session recordings and live device feeds (/api/stream/device-live).
* Playback controls: play/pause, seek scrubber, speed multipliers (0.5x - 4.0x), looping, and theater mode.
- Formatting & Parsing Utilities:
* Android UI action formatter (tap, type, swipe, drag, keyevent, app launch).
* Tool call formatter for ADB commands, note management, video inspection, and failure recovery.
* Markdown parsing and milestone checklist rendering.
|
2026-08-13 11:58:16 -07:00 |
|
|
somew1nd
|
206a9ce921
|
feat: initial release of ARTEMIS Android Agent & Automation Platform
ARTEMIS is an autonomous multimodal Android agent and mobile UI automation
platform engineered for physical devices and emulators. It combines state-of-the-art
multimodal Vision-Language Models (VLMs) with hierarchical closed-loop planning,
resilient self-healing, and deep Android subsystem integration.
Key Features & Architectural Highlights:
- Dual-Engine Orchestration:
* Pro Engine: Multi-agent closed-loop (Planner -> Operator -> Validator/Failure Analyzer -> Summarizer) powered by LangGraph.
* Flash Engine: Ultra-fast reactive loop for deterministic, low-latency UI execution (<35 steps).
- Multimodal Perception & Grounding:
* Hybrid visual screenshot grounding, real-time OCR extraction, and XML accessibility hierarchy parsing.
* Dynamic-First, Coordinate-Fallback locating strategy for maximum resilience against UI drift.
- Autonomous Self-Healing & Diagnostics:
* Failure analyzer and safety net for popup dismissal, crash recovery, and auto-retry.
* Diagnostic sub-agents for log analysis, video recording inspection, and object detection.
- Unified CLI & Python SDK:
* Rich CLI with commands: init, doctor, run, trace, batch, bench, server, and mcp.
* Fluent Python SDK with Pydantic structured output models and package locking.
- Multi-Provider LLM Architecture:
* Support for Google Gemini, OpenAI, Anthropic Claude, Vertex AI, OpenRouter, xAI Grok, and local endpoints (Ollama/vLLM).
* Centralized configuration in config/artemis.jsonc.
- Native MCP & Observability:
* Model Context Protocol (MCP) server endpoints for AI IDEs.
* Full trace recording with before/after screenshots, action overlays, and execution replays.
Change-Id: I430b93ee2b76b54cc5a3c6eeccd15d2cf29830a1
|
2026-08-13 11:46:47 -07:00 |
|