Commit Graph
72 Commits
Author SHA1 Message Date
somew1nd b8ecda9dd1 refactor(queue): split task queue orchestration and worker I/O 2026-09-01 22:13:45 -07:00
somew1nd 582f112287 merge(video): integrate video analyzer module split 2026-09-01 21:46:40 -07:00
somew1nd 7cc20bbddc refactor(video): split video analyzer into engine modules 2026-09-01 21:46:07 -07:00
somew1nd dc619679ea merge(explorer): integrate explorer module split 2026-09-01 21:39:46 -07:00
somew1nd 972201bb4d refactor(explorer): split explorer into facade and phase modules 2026-09-01 21:39:17 -07:00
somew1nd 35fd0fe15d merge(validator): integrate validator module split 2026-09-01 21:35:12 -07:00
somew1nd b7dd8c6f4d refactor(validator): split validator responsibilities into modules 2026-09-01 21:34:39 -07:00
somew1nd 75fdbf4446 refactor(flash): split runner phases into named helpers 2026-09-01 21:31:51 -07:00
somew1nd 9cad5b690f fix(runtime): resolve liveness writeback and probe issues 2026-09-01 21:21:20 -07:00
somew1nd f24a6fd441 feat(checker): update verification defaults and persistence 2026-09-01 21:00:16 -07:00
somew1nd 17f734a473 merge(checker): integrate verification persistence 2026-09-01 20:57:10 -07:00
somew1nd 3145de181b feat(checker): persist verification findings across turns 2026-09-01 20:57:10 -07:00
somew1nd 4724e81741 merge(core): integrate routing and execution cleanup 2026-09-01 18:08:36 -07:00
somew1nd a4c28865f2 refactor(core): consolidate action and plan definitions 2026-09-01 18:07:45 -07:00
somew1nd 12c6d98aa5 merge(errors): integrate console and server exception handling cleanup 2026-09-01 17:58:28 -07:00
somew1nd c76c9bb7e8 refactor(errors): replace silent exception handling in console and server 2026-09-01 17:57:15 -07:00
somew1nd 98ee55f5d2 fix(memory): break transcript and flash compressor import cycle 2026-09-01 17:54:17 -07:00
somew1nd 0bcddbf1d7 merge(memory): integrate incremental transcript compaction 2026-09-01 17:47:16 -07:00
somew1nd f0a4045c49 docs(memory): add history module plans and reports 2026-09-01 17:46:56 -07:00
somew1nd 41ae386a3a feat(memory): add incremental transcript compaction 2026-09-01 17:46:44 -07:00
somew1nd 41deb48005 fix(cli): preserve invalid config when backup fails 2026-09-01 16:08:47 -07:00
somew1nd 8003cf39c2 merge(runtime): integrate trace store relocation 2026-09-01 15:59:35 -07:00
somew1nd 05459d040d refactor(runtime): relocate trace store and break dependency cycle 2026-09-01 15:58:50 -07:00
somew1nd e6c64412de merge(errors): integrate core exception handling cleanup 2026-09-01 15:55:22 -07:00
somew1nd 47e7cb6620 refactor(errors): replace silent exception handling in core packages 2026-09-01 15:54:49 -07:00
somew1nd 584bb2d09c refactor(device): retire remaining single-device state 2026-09-01 11:56:56 -07:00
somew1nd 5907a28e14 merge(runtime): reconcile task state and device cleanup 2026-09-01 11:56:17 -07:00
somew1nd 53e84a5145 merge(reliability): integrate classified retry handling 2026-09-01 11:52:29 -07:00
somew1nd 2acdeba8c5 feat(reliability): centralize retry failure classification 2026-09-01 11:51:29 -07:00
somew1nd 33c4a0d112 refactor(device): remove single-device state and test branches 2026-09-01 11:50:14 -07:00
somew1nd e9b821c0ec merge(runtime): integrate process and task state fixes 2026-09-01 11:39:56 -07:00
somew1nd 9cee112cd2 fix(runtime): unify process and task state handling 2026-09-01 11:35:23 -07:00
somew1nd 3cb3641b3b merge(core): integrate parallel architecture removal 2026-09-01 11:03:19 -07:00
somew1nd f68d4288e0 refactor(core): remove dead parallel agent architecture 2026-09-01 10:49:51 -07:00
somew1nd cbe8f4f0c6 feat: harden agent verification and runtime contracts
Introduce two-gate checker settlement, explicit graph visibility and checkpoints, structured LLM reliability, centralized action specifications, and corresponding tests and design documentation.
2026-08-31 20:42:24 -07:00
somew1nd ddfa420945 feat: enable remote SDK and resilient device orchestration 2026-08-30 00:41:07 -07:00
somew1nd 60312abbb0 fix(replay): unify step indexing, dual-index entity deduplication, and prevent step-4 start bug
- Backend: maintain _step_number_cache in DataEngine and include step_number in all step_updated SSE events (summary, thinking, execution result)
- Admin console: query unassigned trace step_ids in stepless_rows to prevent orphan traces from being lost
- Showcase UI: implement dual-indexed entity deduplication (by step_id and step_number) in extractStepReplayFrames to eliminate phantom ghost steps
- Showcase UI: filter visual screenshot steps before generating sequential 1-based frame numbers, preventing step-4 start and index shift bugs
- Showcase UI: add rawStepNumber and summary properties to StepReplayFrame
- Showcase UI: update FloatingVideoPlayerComponent session reset effect to only reset playback on sessionId change rather than on every stepFrames mutation
- Showcase UI: add stepSeekRequest signal and stepIndex parameter to AgentService.openVideoPlayer
- Unit tests: add Karma and Pytest unit tests verifying step 1 start, dual-indexing, and SSE step_number payloads
2026-08-27 11:03:47 -07:00
somew1nd 5f1ee18bc1 feat(core): consolidate config hub, unify daemon scheduling across all entrypoints, and fix step replay
- Unified configuration and paths under artemis.config as single source of truth
- Consolidated execution across CLI, SDK, MCP, and Admin Console to default to Artemis Daemon
- Fixed frontend step replay to preserve failure steps with proper sequential numbering
- Implemented task deduplication, debouncing, and multi-device lock isolation
- Added comprehensive end-to-end multi-port integration tests for live phone and emulator
2026-08-27 10:21:40 -07:00
somew1nd e984d17635 fix: isolate subprocess stdin and decouple startup to prevent MCP stdio deadlocks
- Set stdin=subprocess.DEVNULL across runtime, client, and utility subprocess calls to avoid hijacking the parent process's stdin.
- Run awake service initialization asynchronously on a daemon thread in MCP server startup.
- Redirect logger header printing to stderr to ensure stdio JSON-RPC purity.
- Add unit and lifecycle tests for MCP stdio protocol handshake and stdin isolation.
2026-08-26 23:26:10 -07:00
somew1nd 8fdecf23d7 feat: enhance daemon dispatching, device readiness fallback, and live streaming UI
- Admin Console & Task Queue: add cancellation, session startup progress endpoints, and support ingress metadata
- Diagnostics & Runtime: enhance ADB probe to prefer unlocked devices and synchronize with readiness engine
- MCP Server: dispatch tasks through Artemis Daemon scheduler with fallback to standalone runner, improve action overlays
- Showcase UI: improve session auto-selection with pinning support, render startup progress, and refine stream styling
- CLI & Makefile: expose server lifecycle commands (restart, stop, status)
- Configuration: ignore runtime server state file in .gitignore
2026-08-26 17:41:23 -07:00
somew1nd 57655266cc fix: resolve NameError by importing Any from typing in runtime, pipeline, and MCP servers 2026-08-26 16:03:56 -07:00
somew1nd c2c90a6378 feat: add multi-device concurrency, server lifecycle CLI, and unified execution stream
- Implement DevicePool and DeviceExecutionLock concurrency modes (per-device FIFO vs global serialization)
- Extend ArtemisClient SDK and CLI with explicit device targeting and concurrency strategy configuration
- Add server lifecycle management (artemis restart, stop, status) with cross-platform PID discovery
- Update MCP server tools to support multi-device dispatch, trace tagging, and per-device execution
- Enhance Showcase UI with step-by-step visual replay, playback speed controls, and device serial badges
- Add comprehensive unit test coverage and edge-case verification scripts
2026-08-26 15:53:14 -07:00
somew1nd a36a66ce6d feat: add step-by-step visual replay and harden process lifecycle
- UI: Implement dual-mode playback in FloatingVideoPlayer supporting video and step-by-step frame replay with scrub controls and auto-fallback
- Runtime: Prevent asyncio event loop hangs on Darwin and Linux by avoiding psutil.wait_procs on direct child processes
- Task Queue: Add worker exit watchdog and automatic recording recovery for completed or interrupted sessions
- Recording: Add ffmpeg ultrafast transcode fallback, shield recording finalization against cancellation, and handle on-the-fly MP4 conversion
- Docs: Polish README_CN styling and streamline flash runner agent prompt
2026-08-26 00:53:24 -07:00
somew1nd 06643e955b docs: streamline execution profiles and refocus roadmap
- Remove redundant head-to-head comparison section
- Simplify Flash vs. Pro profiles with precise latencies, benefits, and limitations
- Refocus roadmap on future milestones prioritizing Android Studio integration, followed by iOS
2026-08-25 21:21:42 -07:00
somew1nd bb16991f2c feat: harden video analysis and runtime resilience 2026-08-25 20:58:25 -07:00
somew1nd 5ada543a09 docs: standardize image embedding widths to 100% and streamline emoji usage 2026-08-25 17:09:10 -07:00
somew1nd ef5748c8f1 docs: remove live demo caption under demo gif in README 2026-08-25 17:04:08 -07:00
somew1nd c2ff9651cc docs: remove outer canvas padding from leaderboard to match architecture diagram 2026-08-25 17:02:50 -07:00
somew1nd 4f8a83ebd3 docs: use new asset path to bust GitHub camo image cache for leaderboard 2026-08-25 17:00:38 -07:00
somew1nd 4bf1160e5b docs: refresh AndroidWorld benchmark visual and streamline architecture overview in README 2026-08-25 16:58:44 -07:00
somew1nd ea3708a98f feat: implement robust multi-tier text input architecture and ADB diagnostics
- Android Text Input & IME Normalization:
  - Implement progressive 3-tier input architecture in AndroidAdbDriver and AndroidDeviceController:
    - Tier 1 (Primary): Zero-dependency clipboard injection + KEYCODE_PASTE (immune to IME Pinyin interception, full Unicode/Emoji support, preserves multiline formatting).
    - Tier 2 (Broadcast): Base64 broadcast injection when ADBKeyboard is active.
    - Tier 3 (Fallback): Native ADB physical keyevent fallback.
  - Add literal newline normalization (\r\n, \n, \r) to prevent JSON escape string corruption (e.g. '10086\n' -> '10086、n').
  - Add set_clipboard method to UIAutomatorClient without auxiliary APK requirements.

- Diagnostics & System Probes:
  - Add adb_keys diagnostics probe for ADB key auth checks.
  - Integrate diagnostics into CLI doctor command.

- Admin Console & Showcase UI:
  - Improve model service and system router in admin console.
  - Enhance workspace stream view and session state in showcase UI.
2026-08-24 16:48:45 -07:00
somew1nd 42b8988661 feat: add global device queue and readiness tracking 2026-08-24 14:34:17 -07:00
somew1nd a62ac4aef1 fix: finalize recordings before automatic replay 2026-08-23 13:06:19 -07:00
somew1nd 474e56ef8e feat: support orientation-aware recording and device wakefulness
Roll scrcpy recordings by device rotation and duration, and publish timeline manifests for playback and analysis.

Add seamless segmented playback in the showcase UI and configurable stay-awake strategies for Android automation.
2026-08-22 19:45:02 -07:00
somew1nd 160f43a74a docs: refresh banner and update tagline to 'Let AI assistants and test suites use real phones like a human' 2026-08-21 12:24:56 -07:00
somew1nd 71e5bba551 runtime: harden LLM retry handling and operator workflows 2026-08-21 00:34:44 -07:00
somew1nd 0f5fb1d32c Improve task resilience and LLM retry visibility
Persist and publish recoverable provider retries so temporary 503 and throttling events remain visible in live and historical task streams.

Aggregate repeated retries into an expandable action card, preserve terminal pause recovery, harden Windows worker isolation and output forwarding, and retain reliable Android hierarchy fallbacks with regression coverage.
2026-08-20 00:37:53 -07:00
somew1nd c1a4b49a21 feat(mcp): enhance multi-platform notification resilience and reactive wakeups
- AgentApiNotifier: Sort candidate process sessions by creation time descending to prioritize the latest active IDE session over stale PIDs. Implement automatic retry across candidate address/token pairs and persist working credentials to shared environment files (~/.gemini/jetski/.jetski_env and ~/.artemis/.artemis_env).
- DesktopNotifier: Enable native OS desktop toast notifications by default on macOS, Linux, and Windows unless explicitly disabled or running in CI. Ensure MCP installer sets ARTEMIS_DESKTOP_NOTIFY=true in generated IDE configurations.
- ScriptNotifier: Add universal command hook adapter (ARTEMIS_NOTIFY_CMD / MCP_NOTIFY_COMMAND) for custom IDE and CLI integration.
- Fault Isolation: Wrap CompositeNotifier and global notify entrypoint in strict try-except blocks to guarantee zero-crash execution even if any individual notifier fails.
- Tests & Docs: Add unit tests for session recovery, fault tolerance, script hook execution, and document multi-platform notification architecture in README.
2026-08-19 11:34:46 -07:00
somew1nd 4927c4f0b9 Stabilize task execution and UI lifecycle 2026-08-19 00:47:45 -07:00
somew1nd c78c201e94 Fix setup UI and improve CLI workflows 2026-08-18 22:35:05 -07:00
somew1nd ec5010220b feat(mcp): implement double-insurance global-only MCP and testing rules installation
- Install MCP configs and Artemis mobile testing rules strictly into user home directories (~), avoiding project workspace pollution
- Add double-insurance rules synchronization injecting into both primary global instruction files (e.g. ~/.gemini/GEMINI.md, ~/.claude/CLAUDE.md, ~/.cursorrules, ~/.codeium/windsurf/memories/global_rules.md) and modular rules/ directories
- Support JSONC parsing (comments and trailing commas) and automatic .bak backups when merging existing IDE configuration files
- Update README and Quick Start documentation to clarify automatic global MCP & rule mounting across all major AI IDEs
2026-08-18 18:40:38 -07:00
somew1nd 9e5aee85c8 fix(mcp): auto-register tools on startup and support cross-platform setup for all major AI IDEs
- Fix missing mcp_server.tools import on server initialization so mobile_run_task, mobile_manage_task, mobile_get_device_state, and mobile_inspect_trace are automatically registered.

- Inject PYTHONPATH in generated and installed MCP configurations to guarantee Python module resolution across Windows, macOS, and Linux regardless of IDE CWD.

- Expand artemis mcp --install and --generate-config to support VS Code, Cline, Roo Code, Claude Code CLI (~/.claude.json), Cursor, Windsurf, Antigravity, and OpenClaw.

- Enhance config file merging with robust exception handling and fallback initialization.
2026-08-18 17:49:47 -07:00
somew1nd fc6ddad655 feat(ui): ensure cross-platform one-click setup and default Showcase UI routing
- Add explicit root route (/) in server.py to guarantee Showcase UI is served by default, routing /admin to the Admin Debug Console
- Enhance dynamic Angular dist detection across multiple standard output paths
- Add automated Node.js/npm detection and installation fallbacks to start scripts and installers across macOS, Linux, and Windows
- Automatically build Showcase UI when launching via start.sh, start.bat/start.ps1, or artemis ui if frontend artifacts are missing
- Add Node.js / npm and Showcase UI status inspection rows to artemis doctor
- Support artemis mcp --install for one-click IDE integration and add make build-ui target
2026-08-18 17:22:49 -07:00
somew1nd 6fa4768893 docs: refine Chinese banner tagline to Artemis and refresh cache 2026-08-18 15:37:21 -07:00
somew1nd 1899c68bb8 docs: elevate banner tagline to Autonomous Assistant Platform and update renders 2026-08-18 15:33:28 -07:00
somew1nd 46e1e8b935 docs: update banner tagline to Android Phone and add native Chinese banner 2026-08-18 15:30:18 -07:00
somew1nd 62cb3d8e53 docs: emphasize testing real devices in README subtitles 2026-08-18 15:22:21 -07:00
somew1nd 1d005d08a9 docs: broaden UI testing terminology to general test automation across README 2026-08-18 15:20:50 -07:00
somew1nd 19c74717ab docs: add cache buster to banner image URLs to bypass GitHub CDN caching 2026-08-18 15:18:34 -07:00
somew1nd a306258d26 docs: add Antigravity x ARTEMIS autonomous testing workflow showcase and update banner 2026-08-18 15:13:39 -07:00
somew1nd 9a5aaf99b7 feat: introduce MCP server, cross-platform runtime, and Showcase UI reorganization
- Showcase UI Reorganization:
  * Move web UI dashboard from root directory into apps/showcase_ui/ for cleaner project structure
  * Add 'artemis ui' CLI command (artemis/interfaces/cli/commands/ui.py) and launcher scripts for the interactive showcase console
- Standalone MCP Server & Notifiers (mcp_server/):
  * Implement dedicated MCP server package with mobile automation tools (device_state, inspect_trace, task_manager, task_runner)
  * Add multi-channel notifiers (AgentAPI, Desktop notifications, Webhooks, File output, and Composite routing)
  * Introduce background task runner and trace store for autonomous mobile execution monitoring
- Cross-Platform Runtime & Toolchain Support:
  * Add artemis/platform/ with OS-specific implementations for Linux, macOS (Darwin), and Windows
  * Add artemis/runtime/supervisor.py for resilient process and agent runtime supervision
  * Add artemis/toolchain/ with dependency resolution and SDK descriptors
  * Add cross-platform launcher scripts (start.sh, start.bat) and installer scripts (scripts/install_deps.*)
- Core Enhancements & Diagnostics:
  * Enhance coordinate normalization, OCR API fallback, and perception pipeline
  * Add credential validation (credentials_validator.py) and system diagnostics
  * Update bilingual documentation (README.md, README_CN.md) and architecture diagrams
- Test Suite Expansion:
  * Add integration tests for cloud gateway, remote ADB, and Artemis login
  * Add unit tests covering core diagnostics, platform detection, toolchain, credentials validator, optional OCR, and MCP tools
2026-08-18 13:59:27 -07:00
somew1nd 206a9ce921 feat: initial release of ARTEMIS Android Agent & Automation Platform
ARTEMIS is an autonomous multimodal Android agent and mobile UI automation
platform engineered for physical devices and emulators. It combines state-of-the-art
multimodal Vision-Language Models (VLMs) with hierarchical closed-loop planning,
resilient self-healing, and deep Android subsystem integration.

Key Features & Architectural Highlights:
- Dual-Engine Orchestration:
  * Pro Engine: Multi-agent closed-loop (Planner -> Operator -> Validator/Failure Analyzer -> Summarizer) powered by LangGraph.
  * Flash Engine: Ultra-fast reactive loop for deterministic, low-latency UI execution (<35 steps).
- Multimodal Perception & Grounding:
  * Hybrid visual screenshot grounding, real-time OCR extraction, and XML accessibility hierarchy parsing.
  * Dynamic-First, Coordinate-Fallback locating strategy for maximum resilience against UI drift.
- Autonomous Self-Healing & Diagnostics:
  * Failure analyzer and safety net for popup dismissal, crash recovery, and auto-retry.
  * Diagnostic sub-agents for log analysis, video recording inspection, and object detection.
- Unified CLI & Python SDK:
  * Rich CLI with commands: init, doctor, run, trace, batch, bench, server, and mcp.
  * Fluent Python SDK with Pydantic structured output models and package locking.
- Multi-Provider LLM Architecture:
  * Support for Google Gemini, OpenAI, Anthropic Claude, Vertex AI, OpenRouter, xAI Grok, and local endpoints (Ollama/vLLM).
  * Centralized configuration in config/artemis.jsonc.
- Native MCP & Observability:
  * Model Context Protocol (MCP) server endpoints for AI IDEs.
  * Full trace recording with before/after screenshots, action overlays, and execution replays.

Change-Id: I430b93ee2b76b54cc5a3c6eeccd15d2cf29830a1
2026-08-13 11:46:47 -07:00