Project / Agent & Engineering
Iris Media for DSH
Image, video, audio, and vision tools for DSH.
Image, video, music, speech, transcription, OCR, and visual analysis delivered both as a DSH plugin and a standalone CLI, with unified async task and artifact management.
v0.3.0-beta.1 preview · v0.2.7 stable
Overview Project overview
Iris Media for DSH packs image generation, text/image-to-video, digital-human video, music, speech, transcription, OCR, and visual analysis into a self-hostable runtime. It ships as both a DSH plugin and a standalone CLI.
Motivation Why it exists
Media capabilities usually live scattered across vendors and scripts, making task state, file locations, and artifact versions hard to keep consistent. Iris turns “invocation — async task — artifact” into a local, recoverable, exportable record of fact.
Architecture Key architecture
- The DSH plugin path and the headless CLI share one Core Task/Artifact data root; only one writer is allowed per data root.
- Capabilities are assigned explicitly per “provider + model”; an empty pool stops new calls instead of auto-filling the model list.
- 0.2.6 can reference existing DSH API-key connections, storing only the connection identifier and reading host-managed addresses and credentials at operation time; the standalone CLI still uses explicit private configuration.
- 0.2.7 makes Core/DSH work queues and task observers instance-owned, and saves DSH browser screenshots as PNG through Core so the browser keeps its download-directory restriction.
- 0.3.0-beta.1 builds on alpha.1's separated Core and freezes the minimal plugin contracts; network routing, bounded scheduling, durable text-to-video, and environment credential references are in the beta scope.
- Node.js ≥22.1; image processing relies on sharp, video frame extraction on system ffmpeg and ffprobe.
- Async tasks are observable and can be taken over after restart; the artifact library and task history remain independent of each other.
Capabilities Capabilities
- Image generation and import, Qwen native image editing, text-to-video, image-to-video, and S2V digital-human video.
- Music generation, text-to-speech, audio transcription; visual Q&A, long-image OCR, object grounding, cropping, and pixel-diff analysis.
- Video frame extraction and multimodal summarization; multi-provider model pools, capability assignment, and failover.
- A standalone artifact library, media playback tokens, and three file sources: browser upload, session attachments, and host paths.
- 0.2.6 adds auto-pool/manual-pick/disabled modes for six function classes, on-demand directory sync, and capability detection; auto-pools switch safely in registration order with no cross-request rotation.
- 0.2.7 fixes DSH HTML screenshots, adds explicit screenshot self-checks and DSH_WEB_BASE navigation, and adds a persistent image-edit workflow plus a loopback-only, read-only-by-default local API.
- The packaged iris-core Skill can be explicitly installed for Kimi or another harness with version and content hashes; installation does not configure models or MCP automatically.
- 0.3.0-beta.1 adds explicit network resources and routing, opt-in bounded media scheduling, durable text-to-video in Core Web/MCP, environment credential references, and independent plugin starters; stable npm latest remains 0.2.7.
Limitations Known limitations
- sharp ships native components; non-standard runtimes may need extra handling.
- Video frame extraction and summarization require ffmpeg and ffprobe; HTML screenshots do not load remote scripts, fonts, or images.
- On WSL, in containers, and in remote deployments, host paths may not be readable — prefer uploaded files.
- A provider's model list is only a candidate; actual availability must be confirmed by a real call.
- Connection reuse depends on the host exposing the matching public interface; without it, manual configuration is still used.
- The local API is read-only by default and does not include workflow execution or cancellation; persistent workflows need a new dedicated root, and one data root still permits one writer.
- Skill installation is separate from harness discovery — status reports discovery: not_checked. Android touch, hot upgrades, and long-task restart takeover still need separate acceptance.
- 0.3.0-beta.1 is a beta; stable latest remains 0.2.7, old roots are not migrated automatically, video requires a new v4 root, Node.js ≥22.1 is required, and real ComfyUI GPU inference is still unverified.
Validation Testing & validation
- npm test covers configuration merging, model identity, task lifecycle, generation actions, HTTP/SSE routes, client interactions, and security boundaries.
- GitHub Actions runs a Linux/Windows × Node.js 22.1.0/latest-22.x matrix and checks for clean process exits.
- 0.3.0-beta.1 release checks cover Ubuntu/Windows from Node 22.1.0 through current 22.x, actual tarball consumers, strict NodeNext public types, and Chromium workbench scenarios; real model, GPU, browser, and harness coverage is not universal.
- Android touch, hot upgrades, and long-task restart takeover still need separate acceptance.
- The twelve supported DSH versions and Node.js ≥22.1 requirement are unchanged; Skill discovery checks submit no prompt and make no model calls.
- Native Windows/WSL host smoke tests remain pending on real hardware; compatibility follows the repository's compatibility matrix.