Yetone's Alma Agent Harness ranked third in the intermediate FrontierHarness v1.1 rating and sixth in the final.
Read more
Pandaily
pandaily.com

Yetone's Alma Agent Harness ranked third in the intermediate FrontierHarness v1.1 rating and sixth in the final.

Alma, an AI agent system developed by independent specialist yetone, secured third place in the intermediate review of FrontierHarness v1.1, an open benchmark for agent systems conducted by Runta. This intermediate ranking was compiled on November 11th while testing was still ongoing, and positions in the table changed daily.

When Pandaily checked the completed table on November 15th, Alma ranked sixth based on the percentage of successful completions. FrontierHarness tests 21 system configurations, including Claude Code, Codex CLI, and OpenCode, using five open models and 62 tasks across software engineering, science, data, systems, machine learning, and security domains.

Each system undergoes four attempts with scoring for every task for each model, and only confirmed successful runs are counted. According to a report from the Chinese tech publication Zhixingxing dated November 11th, Alma demonstrated a 72.7% success rate. This placed it behind Codex CLI at 77.9% and Pi at 72.8%, but ahead of Claude Code, which showed 70.0%.

The cost to execute one successful task using Alma was approximately $0.80, whereas for Claude Code, this cost was $0.94.

Final Testing Results

The final table reflected 26,040 evaluated trials and generally lower scores. Alma's success rate was 59.7%, which was lower than ZCode (61.9%), Codex CLI (61.1%), KimiCode (60.2%), Grok Build (60.0%), and Hermes Agent (59.9%). The success rates for all 21 systems ranged from 50.6% to 61.9%. By default, the table ranking is based on cost per run. By this criterion, Alma ranked 13th with a cost of $1.22, while Codex CLI cost $0.937, and Claude Code cost $1.55. Alma achieved its best result at 64.1% when using Kimi-K3.

Alma is capable of reading and editing files, executing terminal commands, searching the internet for information, and supports various skills, MCP, and multiple models. Furthermore, according to the Zhixingxing review and information on the Alma website, additional features have been added:

To enable remote access, yetone opened the source code for Alma Relay, which can be deployed on Cloudflare Workers or using Docker. With end-to-end encryption enabled, which is the default setting, the relay only sees encrypted data and retains nothing on disk, according to its documentation.

Similar stories

Baidu releases ERNIE-5.5-Flash-Base model: 280B MoE with 14B active parameters
Read more
pandaily.com

Baidu releases ERNIE-5.5-Flash-Base model: 280B MoE with 14B active parameters

The ERNIE team at Baidu has announced the release of ERNIE-5.5-Flash-Base, which is an unrefined pre-training checkpoint of the ERNIE 5.5 Flash model. This release was posted on the Hugging Face platform on November 5th.

The model, which utilizes a sparse Mixture-of-Experts (MoE) architecture, has a total of 280 billion parameters, with 14 billion parameters activated per token. It was trained from scratch on 33.5 trillion tokens, with a maximum training context size of 128 thousand tokens, although support for context in fine-tuning and inference reaches 1 million tokens.

Baidu intentionally released the model in an unfinished state. The weights are obtained from the completion phase of pre-training and have not yet undergone intermediate or subsequent training stages. The company states that this makes it a general starting point for research in domain adaptation, instruction tuning, and reinforcement learning. The team reported that a post-trained version of ERNIE 5.5 Flash will be released later.

The available weights are provided in FP8 and BF16 formats according to the Baidu ERNIE 5.5 Flash Base Model v1.0 License Agreement. The model can be used via SGLang and vLLM, and is accompanied by a fine-tuning example in the NVIDIA Megatron-Bridge repository.

This 43-layer model is designed with an emphasis on efficiency. Most layers use what is called Baidu Sparse Attention, which merges every 128 tokens into a single cache entry. Only one of the seven layers retains access to the full token-level context through multi-head latent attention with top-k sparse sampling. In total, 37 out of 43 layers operate with compressed sequences, allowing for a small KV cache. Furthermore, the low-rank gating attention design reduces the gate projection parameters by 75%, according to the developers.

In terms of MoE, Baidu replaced the standard single router with multiple independently initialized routers that jointly evaluate experts. The company claims that this improves the alignment between routing scores and actual expert utility without changing the number of active experts, computational power, or communication. Each MoE layer selects 10 out of 512 routed experts plus one shared expert, operating in a compressed latent space, which roughly halves the expert communication and computation. A component called ECHO reuses the last layer of the model to predict future tokens instead of adding a separate multi-token prediction module and serves as a draft model for speculative decoding.

The Muon optimizer with mixed-precision FP8 was used for pre-training across thousands of accelerators, along with the new FlashMask-4 attention kernel. Baidu asserts that this kernel operates up to 175% faster in the forward pass than FlashAttention-4 on SM100 class GPUs at large head sizes. Data mixing focused on repository-level code, including commits, merge requests, code reviews, CI-verified fixes, and long documents.

Baidu tested the model on 30 benchmarks, comparing it with DeepSeek-V4-Flash-Base, Hy3-preview-Base, and MiMo-V2.5-Base using a unified evaluation system. Reported results include 78.4 points on MMLU-Pro versus 69.2 for DeepSeek-V4-Flash-Base, and 82.3 points on HumanEval+ versus 69.5. However, the compared models outperformed in some tests, such as ChineseSimpleQA, BBH with chain-of-thought prompting, and InfiniteBench.

Tsinghua's Algebraic World Model Proves Impossibility of Passing ARC-AGI-3 Levels Using Lean 4
Read more
pandaily.com

Tsinghua's Algebraic World Model Proves Impossibility of Passing ARC-AGI-3 Levels Using Lean 4

Researchers from Tsinghua University and Zhejiang University have introduced the Algebraic World Model (AWM)—an agent capable not only of passing interactive ARC-AGI-3 puzzles but also of formally proving when passing a specific level is impossible.

The work, published on arXiv on October 29th and highlighted by QbitAI on November 1st, was conducted by Guancheng Pan from Zhejiang University, along with Cixin Tan, Yan Yuan, and Andrew Chi-Chi Yao from Tsinghua University.

AWM operates in three stages. First, it explores the game and forms hypotheses in Python regarding the environment's behavior, correcting them upon receiving contradictory observations. Next, these hypotheses are simplified into a structured model describing the rules, goals, and objects, based on concepts from category theory. The final step is formalizing this model in the Lean 4 proof assistant, where the machine can verify assertions, such as the possibility of completing a level.

The team tested the model on closed levels where the step budget made victory impossible. AWM successfully provided proofs of impossibility using Lean for all six such levels, whereas 12 runs of Claude Opus 5.5 and GPT-6 Astra failed to do so. In one example, where minimal victory required 19 steps with an 18-step budget, Astra claimed a solution in 22 steps, while AWM established correct initial bounds for four out of six levels, compared to one for Opus 5.5 and zero for Astra, achieving results for all six after feedback.

During the replaying of all 183 levels from 25 public games within Lean, the studied models were able to predict all 4160 transitions on unseen configurations and rejected all 4031 transitions after changes to the game mechanics. When using Claude Opus 5 as the player, AWM raised its score from 40.7 to 99.9, utilizing only 43% of the actions taken in the baseline human variant.

The authors note a significant limitation: the validity of the proofs depends on the quality of the studied model itself; therefore, the result of impossibility is valid within the AWM game model, not concerning the actual game code.

Popular