Localize the web console and persist custom model and capability settings.
Use OpenAI Responses exclusively and move runtime limits into environment configuration.
Bootstrap uv and the virtual environment on Windows, support optional China mirrors,
and package source distributions without environments, build output, or credentials.
Update regression tests and repository quality checks for the new behavior.
This reverts commit 54fcef9887.
Please do not submit unrelated docs and codes in the same commit. I am reverting this commit so that the related files can be submitted in better-organized future commits.
- Add NOTICE file and README license references for mobile-use and finalrun-agent
- Add derivative attribution headers to 17 files derived from mobile-use
- Rename hopper agent to entity_extractor across prompts, tools, and configs
- Remove obsolete adb_tunnel, ApiBaseUrl, and legacy CLI utility scripts
- Replace legacy examples with native Artemis diagnostic and workflow examples
- Update unit and tool tests to validate entity_extractor and SDK builders
- Introduce ArtemisAccessibilityHelper APK (API 24 to 37+) as a conflict-free, high-performance replacement for UiAutomator:
* Eliminates native UiAutomator waitForIdle() timeouts, reducing DOM dump latency from ~3s to ~15ms and maintaining resilience against infinite animations.
* Implements a multi-tier fallback hierarchy discovery strategy (multi-window layer enumeration -> active window fallback -> input/accessibility focus backtracking) with adaptive progressive backoff for seamless page transition capture.
* Resolves node drop defect caused by 32-bit hash collisions on recurring virtual view IDs in long lists and WebViews by utilizing bounded recursion (MAX_DEPTH=75, MAX_NODES=8000).
* Strict W3C XML 1.0 character sanitization filtering invalid ASCII control codes and escaping attribute whitespace ( , , 	) to preserve multiline text formatting.
* Enriches semantic parsing with stateDescription (API 30+ Jetpack Compose), screenReaderFocusable, paneTitle, heading, error, and tooltip attributes.
* Multi-window metadata propagation tagging each node with window-id, window-type, and window-layer.
* Adds dual-path standalone build script (build_apk.sh) producing a clean 29KB APK without Gradle daemon overhead alongside standard Gradle Kotlin DSL.
- Integrate AccessibilityClient into Artemis Python runtime with automatic bundled APK detection, self-healing installation, and port forwarding.
- Add unit test suite covering AccessibilityClient endpoints and atomic snapshot parsing.
Preserve user guidance through history compression, stabilize Flash index targets, and retire withdrawn checks without applying stale verdicts.
Improve runtime diagnostics, tool responses, and history rendering. Simplify documentation and comments.
Validation: 2076 tests passed in the full suite; 381 affected tests and 26 frontend tests passed after cleanup. Ruff and the quality ratchet passed.
Report device actions as dispatched and preserve full failure details. Route
Flash through McpActionExecutor and use ToolFailure across helper tools and
agent loops instead of inferring failure from response text.
Require model-provided descriptions for coordinate targets. Keep observed
element metadata separate from those descriptions in safety checks, visual
summaries, history, and frontend labels.
Add Gemini Interactions API video analysis with configurable processing mode,
chunk size, and timeouts. Centralize Google provider capabilities and token
usage normalization, and update google-genai to 2.22.0.
Preserve system instructions on follow-up interactions, replan pending chunks
when falling back to static processing, and reject unanswered native video
and audio segments instead of caching placeholder results. Keep helper-tool
failure status intact through history tools and committee calls.
Validation: 1,762 Python unit tests passed (3 skipped), 40 frontend formatter
tests passed, and the production UI build completed. Ruff lint, formatting,
the quality ratchet, and git diff checks passed.
- Backend (LLM Gateway):
- Record structured 'llm_stream_reset' event with discard action, failure reason, and retry messaging on mid-stream generation errors.
- Add unit test verifying stream reset event payload on mid-stream errors.
- Frontend (Showcase UI):
- Handle 'llm_stream_reset' events in agent service and stream aggregator, discarding interrupted chunks while showing retry indicators.
- Establish a single source of truth for tool and action events in the timeline to eliminate duplicated action rows while retaining ADB and explorer tools.
- Add @angular/cdk dependency and update agent stream component styling and layout.
- Add unit tests for stream aggregation and tool display filtering.
Fix queue cancellation, concurrency limits and worker status updates.
Use direct Explorer imports, split the Diagnoser loop into helpers and remove unused tool registration and prompt-builder code.
Correct model caching, endpoint selection and retry handling.
Add execution incidents, action bursts and shared Flash history.
Expose verification and Explorer settings with streamed check results.
Support graceful cancellation, recording recovery and bundled UI resources.
Add CI quality checks and clean up comments and documentation.
- Extract device action execution into artemis/mcp action layer:
typed ActionResult, capability manifest with startup validation,
AdbActuator as the single physical implementation, in-memory
ActionSession (replaces stdio subprocess MCP in validator), and
McpActionExecutor preserving the legacy executor contract.
adb_server keeps its external stdio/SSE surface unchanged and now
reuses the actuator implementation; import-time side effects moved
into configure_stdio_mode().
- Capability-gated prompt assembly (prompt_assembly.py): tool enums and
teaching segments render per available backend tools; full-set output
is byte-identical to the previous static text (locked by tests).
- Admin console task queue: serial worker replaced by a per-device
concurrent dispatcher (ARTEMIS_CONCURRENCY_MODE=global restores
serial semantics), per-session cancellation, strict device binding on
enqueue, unified stdio piping with tee to trace stdout.log, and
shutdown cleanup of all active runs. Fix _pid_exists exception order
so PermissionError is treated as alive.
- mcp_server reliability: spawn watchdog kills runners stuck before
interpreter startup and releases device reservations; task state
reconcile uses psutil liveness, prefers runner-reported pids, and
adds a 45s startup grace to stop false failure reports on Windows;
conversation_id is now optional end-to-end; explicit device_serial is
validated at submission.
- CLI install now writes ~/.claude/rules/artemis.md for Claude Code and
migrates/removes the legacy CLAUDE.md injection block; ~/.claude.json
is never rewritten when it fails to parse.
- Tests: Windows-portable stdio lifecycle and path assertions; new
suites for action layer, prompt assembly, spawn watchdog, and task
reconcile (895 passed, 3 skipped).
- Backend: maintain _step_number_cache in DataEngine and include step_number in all step_updated SSE events (summary, thinking, execution result)
- Admin console: query unassigned trace step_ids in stepless_rows to prevent orphan traces from being lost
- Showcase UI: implement dual-indexed entity deduplication (by step_id and step_number) in extractStepReplayFrames to eliminate phantom ghost steps
- Showcase UI: filter visual screenshot steps before generating sequential 1-based frame numbers, preventing step-4 start and index shift bugs
- Showcase UI: add rawStepNumber and summary properties to StepReplayFrame
- Showcase UI: update FloatingVideoPlayerComponent session reset effect to only reset playback on sessionId change rather than on every stepFrames mutation
- Showcase UI: add stepSeekRequest signal and stepIndex parameter to AgentService.openVideoPlayer
- Unit tests: add Karma and Pytest unit tests verifying step 1 start, dual-indexing, and SSE step_number payloads
- Unified configuration and paths under artemis.config as single source of truth
- Consolidated execution across CLI, SDK, MCP, and Admin Console to default to Artemis Daemon
- Fixed frontend step replay to preserve failure steps with proper sequential numbering
- Implemented task deduplication, debouncing, and multi-device lock isolation
- Added comprehensive end-to-end multi-port integration tests for live phone and emulator
- Set stdin=subprocess.DEVNULL across runtime, client, and utility subprocess calls to avoid hijacking the parent process's stdin.
- Run awake service initialization asynchronously on a daemon thread in MCP server startup.
- Redirect logger header printing to stderr to ensure stdio JSON-RPC purity.
- Add unit and lifecycle tests for MCP stdio protocol handshake and stdin isolation.
- Implement DevicePool and DeviceExecutionLock concurrency modes (per-device FIFO vs global serialization)
- Extend ArtemisClient SDK and CLI with explicit device targeting and concurrency strategy configuration
- Add server lifecycle management (artemis restart, stop, status) with cross-platform PID discovery
- Update MCP server tools to support multi-device dispatch, trace tagging, and per-device execution
- Enhance Showcase UI with step-by-step visual replay, playback speed controls, and device serial badges
- Add comprehensive unit test coverage and edge-case verification scripts
- UI: Implement dual-mode playback in FloatingVideoPlayer supporting video and step-by-step frame replay with scrub controls and auto-fallback
- Runtime: Prevent asyncio event loop hangs on Darwin and Linux by avoiding psutil.wait_procs on direct child processes
- Task Queue: Add worker exit watchdog and automatic recording recovery for completed or interrupted sessions
- Recording: Add ffmpeg ultrafast transcode fallback, shield recording finalization against cancellation, and handle on-the-fly MP4 conversion
- Docs: Polish README_CN styling and streamline flash runner agent prompt
chenmingxuan
somew1nd