DeepSeek Harness releases major update after beta testing with multimodal and agent support
Read more
Pandaily
pandaily.com

DeepSeek Harness releases major update after beta testing with multimodal and agent support

DeepSeek Harness has introduced its first significant update following the completion of public beta testing—version v0.1.0-rc.8. The main innovation is the integration of multimodal capabilities. The update includes 14 changes affecting multimodal input processing, agent interaction, terminal user experience, tool calling, and developer support.

The system now supports direct image reception and mixed text and image input, allowing commands like /goal and /plan to receive images directly. Claude Code and Codex have been integrated into the agent system, the Windows terminal has been improved, and issues related to image requests, data streaming, and user gateways have been fixed.

DeepSeek Harness began its public beta and open-source release on August 13th. During the beta testing night, Zhidx uploaded the source code and conducted testing, translating an 88-page document and developing the Snake game. In less than a week, the agent platform acquired multimodal functions.

Developers in different regions reacted differently to this: domestic developers expressed enthusiasm, while international developers focused on two new features that were perceived as clear progress. More interesting was the discovery by some developers of the DeepSeek Harness approach: for models that do not natively support image input, Harness utilizes OCR tools, color statistics, and pixel scanning to convert images into structured information before passing it to text models for analysis. This creates a layered vision mechanism for purely text-based models.

DeepSeek's model adapters now allow activating native image reception via configuration. For models possessing vision capabilities, Harness sends images directly, and /goal and /plan support mixed input. The @ menu in the input field allows referencing files and sessions, enabling users to include local files and existing sessions in a task, which allows agent workflows to account for screenshots.

The agent system has been expanded: the new version allows installing Claude Code and Codex as profile packages on demand. Codex supports a permission mode without interactive interaction and can have multiple named instances. Windows users now have persistent PowerShell sessions in the PTY terminal, which are enabled by default in minimal mode. Furthermore, the update resolves errors related to oversized or accumulated images, failed requests, interrupted streaming losing response prefixes, and OpenAI-compatible user gateways that failed due to request format differences.

Developer Yinsen discovered that if a model does not declare support for image input, a direct call to read_image initially results in an error, but the task does not stop. Harness switches to a fallback option, using OCR text recognition, color ratio statistics, and partial pixel line scanning, reading image dimensions and color mode metadata. An image containing text and simple graphic elements can be broken down into textual content, background color ratio, pixel changes, and size; the resulting data is passed to the text model, which outputs an overall representation of the image based on OCR text and pixel positions. For screenshots of presentations, flowcharts, and clearly structured interfaces, the combination of OCR and pixel features restores useful information; real photographs restore less data. Vision functionality is not exclusively tied to the base model. The community has already created vision plugins such as dsh-vision, dsh-vision-toolkit, modlens, dsh-auto-vision, dsh-subagent-vision, and pi-vision via pi2dsh. With the rc.8 release, native multimodality and these tool-based schemes work together.

Popular