Commit Graph
206 Commits
Author SHA1 Message Date
somew1nd a45daf23f8 feat: add agentic video analysis and clarify action results
Report device actions as dispatched and preserve full failure details. Route
Flash through McpActionExecutor and use ToolFailure across helper tools and
agent loops instead of inferring failure from response text.

Require model-provided descriptions for coordinate targets. Keep observed
element metadata separate from those descriptions in safety checks, visual
summaries, history, and frontend labels.

Add Gemini Interactions API video analysis with configurable processing mode,
chunk size, and timeouts. Centralize Google provider capabilities and token
usage normalization, and update google-genai to 2.22.0.

Preserve system instructions on follow-up interactions, replan pending chunks
when falling back to static processing, and reject unanswered native video
and audio segments instead of caching placeholder results. Keep helper-tool
failure status intact through history tools and committee calls.

Validation: 1,762 Python unit tests passed (3 skipped), 40 frontend formatter
tests passed, and the production UI build completed. Ruff lint, formatting,
the quality ratchet, and git diff checks passed.
2026-09-08 17:05:54 -07:00
somew1nd 611d1a0947 fix(bootstrap): harden Windows PowerShell & shell scripts against download hangs and version conflicts 2026-09-04 17:16:22 -07:00
somew1nd 739c1c2909 fix(start.sh): remove snap and rogue local keyword, streamline fast portable scrcpy fallback 2026-09-04 17:12:03 -07:00
somew1nd 2e66ac5b8a fix(scripts): set apt lock timeout to 5s for fast fallback 2026-09-04 17:10:44 -07:00
somew1nd afbc402225 fix: add apt lock timeout and official portable scrcpy fallback across Linux and Windows
- Handle Debian/Ubuntu apt-get lock contention with DPkg::Lock::Timeout
- Decouple apt update from apt install so locks on update do not block installation
- Add official portable scrcpy v4.1 fallback download for Linux (x64/arm64) and Windows (win64)
- Add scrcpy user directories to standard PATH discovery
2026-09-04 17:07:54 -07:00
somew1nd 29622bae52 fix: auto-detect and upgrade Node.js to >= 22.22.0 across Linux, macOS and Windows
- Add smart Node.js compatibility checking (>= 22.22.0) required by Angular CLI 22
- Upgrade portable Node.js fallback download to v22.23.2 LTS across Linux, macOS and Windows
- Add automatic upgrade paths using nvm, NodeSource 22.x (with sudo/root), and winget
- Display detected Node.js version in toolchain readiness summary
2026-09-04 17:01:57 -07:00
somew1nd 41b1b94706 fix: auto-start local adb daemon and optimize path discovery across Linux, macOS and Windows 2026-09-04 12:01:32 -07:00
somew1nd 8b3cfe9bb3 fix(start.sh): remove invalid local keyword outside functions 2026-09-04 11:48:50 -07:00
somew1nd e1d67c07b6 fix: support interactive privilege prompts and zero-admin user-space fallbacks 2026-09-04 11:47:19 -07:00
somew1nd db69288fac fix: auto-patch Angular CLI node version constraint and support nvm in start scripts
- Auto-patch Angular CLI 22.1.6 node-version.js requirement from ^22.22.3 to 22.22.0 across Linux, macOS, and Windows startup scripts
- Automatically detect and load nvm in start.sh and install_deps.sh when available
- Ensure Showcase UI can be built on Cloudtop and Debian systems running Node 22.22.x without requiring root privileges
2026-09-04 11:37:05 -07:00
somew1nd c79c6eb27b feat: show history compression progress and fix action trace overlays 2026-09-03 20:48:23 -07:00
somew1nd ab03e1d2bb fix: adjust action validation and preserve recording timeline gaps 2026-09-03 16:29:09 -07:00
somew1nd 2fea9865ee feat: unify history tools and improve execution reliability 2026-09-03 16:05:47 -07:00
somew1nd d23a4f6472 feat: handle mid-stream LLM resets and deduplicate UI action stream events
- Backend (LLM Gateway):
  - Record structured 'llm_stream_reset' event with discard action, failure reason, and retry messaging on mid-stream generation errors.
  - Add unit test verifying stream reset event payload on mid-stream errors.

- Frontend (Showcase UI):
  - Handle 'llm_stream_reset' events in agent service and stream aggregator, discarding interrupted chunks while showing retry indicators.
  - Establish a single source of truth for tool and action events in the timeline to eliminate duplicated action rows while retaining ADB and explorer tools.
  - Add @angular/cdk dependency and update agent stream component styling and layout.
  - Add unit tests for stream aggregation and tool display filtering.
2026-09-03 14:38:02 -07:00
somew1nd d7c64f571d fix: preserve pixel safety target context 2026-09-03 10:58:30 -07:00
somew1nd 4bc79fdec8 feat: improve operator progress and recording accuracy 2026-09-03 10:10:22 -07:00
somew1nd 58210106d7 ci: run Python tests only on manual dispatch 2026-09-02 23:04:19 -07:00
somew1nd 12d7902a82 fix: correct task scheduling and simplify agent internals
Fix queue cancellation, concurrency limits and worker status updates.

Use direct Explorer imports, split the Diagnoser loop into helpers and remove unused tool registration and prompt-builder code.

Correct model caching, endpoint selection and retry handling.
2026-09-02 22:58:00 -07:00
somew1nd 67248c887e feat: improve task execution, verification and packaging
Add execution incidents, action bursts and shared Flash history.

Expose verification and Explorer settings with streamed check results.

Support graceful cancellation, recording recovery and bundled UI resources.

Add CI quality checks and clean up comments and documentation.
2026-09-02 21:45:51 -07:00
somew1nd 5ccf06d677 Merge pull request #13 from google/playground-submit
Create Artemis container for playground
2026-09-02 16:44:26 -07:00
yaoyao-open-source 72ae8cf75a Create Artemis container for playground 2026-09-02 16:05:47 -07:00
somew1nd 73d7b0c116 feat(showcase_ui): support verification and finding cards in agent notes 2026-09-02 00:58:48 -07:00
somew1nd 99343dfe55 docs: remove completed design and planning documents 2026-09-02 00:12:33 -07:00
somew1nd b8ecda9dd1 refactor(queue): split task queue orchestration and worker I/O 2026-09-01 22:13:45 -07:00
somew1nd 582f112287 merge(video): integrate video analyzer module split 2026-09-01 21:46:40 -07:00
somew1nd 7cc20bbddc refactor(video): split video analyzer into engine modules 2026-09-01 21:46:07 -07:00
somew1nd dc619679ea merge(explorer): integrate explorer module split 2026-09-01 21:39:46 -07:00
somew1nd 972201bb4d refactor(explorer): split explorer into facade and phase modules 2026-09-01 21:39:17 -07:00
somew1nd 35fd0fe15d merge(validator): integrate validator module split 2026-09-01 21:35:12 -07:00
somew1nd b7dd8c6f4d refactor(validator): split validator responsibilities into modules 2026-09-01 21:34:39 -07:00
somew1nd 75fdbf4446 refactor(flash): split runner phases into named helpers 2026-09-01 21:31:51 -07:00
somew1nd 9cad5b690f fix(runtime): resolve liveness writeback and probe issues 2026-09-01 21:21:20 -07:00
somew1nd f24a6fd441 feat(checker): update verification defaults and persistence 2026-09-01 21:00:16 -07:00
somew1nd 17f734a473 merge(checker): integrate verification persistence 2026-09-01 20:57:10 -07:00
somew1nd 3145de181b feat(checker): persist verification findings across turns 2026-09-01 20:57:10 -07:00
somew1nd 4724e81741 merge(core): integrate routing and execution cleanup 2026-09-01 18:08:36 -07:00
somew1nd a4c28865f2 refactor(core): consolidate action and plan definitions 2026-09-01 18:07:45 -07:00
somew1nd 12c6d98aa5 merge(errors): integrate console and server exception handling cleanup 2026-09-01 17:58:28 -07:00
somew1nd c76c9bb7e8 refactor(errors): replace silent exception handling in console and server 2026-09-01 17:57:15 -07:00
somew1nd 98ee55f5d2 fix(memory): break transcript and flash compressor import cycle 2026-09-01 17:54:17 -07:00
somew1nd 0bcddbf1d7 merge(memory): integrate incremental transcript compaction 2026-09-01 17:47:16 -07:00
somew1nd f0a4045c49 docs(memory): add history module plans and reports 2026-09-01 17:46:56 -07:00
somew1nd 41ae386a3a feat(memory): add incremental transcript compaction 2026-09-01 17:46:44 -07:00
somew1nd 41deb48005 fix(cli): preserve invalid config when backup fails 2026-09-01 16:08:47 -07:00
somew1nd 8003cf39c2 merge(runtime): integrate trace store relocation 2026-09-01 15:59:35 -07:00
somew1nd 05459d040d refactor(runtime): relocate trace store and break dependency cycle 2026-09-01 15:58:50 -07:00
somew1nd e6c64412de merge(errors): integrate core exception handling cleanup 2026-09-01 15:55:22 -07:00
somew1nd 47e7cb6620 refactor(errors): replace silent exception handling in core packages 2026-09-01 15:54:49 -07:00
somew1nd 584bb2d09c refactor(device): retire remaining single-device state 2026-09-01 11:56:56 -07:00
somew1nd 5907a28e14 merge(runtime): reconcile task state and device cleanup 2026-09-01 11:56:17 -07:00