The Lowpass Dispatch

Standard reinforcement learning for LLMs optimizes a policy against a single, scalar reward, often leading to low-entropy models that produce repetitive, non-diverse responses. This lack of variety becomes a problem when the model is used within an inference-time search procedure, like best-of-N sampling or evolutionary search, which rely on a diverse set of candidate solutions to find an optimal one.

Researchers from Google DeepMind and Stanford University introduced Vector Policy Optimization (VPO) on May 28 to address this. VPO replaces the standard Group Relative Policy Optimization (GRPO) advantage estimator with one that operates on vector-valued rewards. This trains the model to produce a set of solutions where individual outputs specialize in different trade-offs in the reward space, explicitly training for diversity.

Across four tasks, including code generation, VPO matched or outperformed strong scalar RL baselines on test-time search metrics like pass@k. The performance gap widened as the search budget grew, and for evolutionary search, VPO-trained models solved problems that GRPO-trained models could not. As test-time search becomes a standard technique for scaling inference, training for diversity may become a required step in model post-training.

The key-value (KV) cache is a critical but costly component of transformer inference, often dominating a model's memory footprint and creating a latency bottleneck during text generation. While quantization can compress the cache, existing methods have not been designed around the specifics of GPU hardware, limiting potential speedups.

A team from UC San Diego, ETH Zurich, and other institutions proposed InnerQ, a hardware-aware quantization scheme detailed in a paper updated on May 30. InnerQ performs group-wise quantization by grouping cache matrices along their inner dimension. This strategy aligns the dequantization operation with the vector-matrix multiplication common in GPU compute units, which increases data reuse and reduces memory access.

The technique is also tuning-free, incorporating hybrid quantization to automatically choose between symmetric or asymmetric methods for each group. In experiments, InnerQ achieved an average 1.3x speedup over previous KV cache quantization methods and a 2.7x speedup over a non-quantized baseline, offering a direct path to faster and more efficient long-context generation.

Data contamination threatens the integrity of LLM leaderboards, as models can be trained on benchmark data to artificially inflate their performance. Malicious actors can use "evasive contamination," such as paraphrasing test questions, to bypass simple n-gram-based detection methods. This makes it difficult to distinguish true reasoning ability from memorization.

Researchers from the University of Washington and Google unveiled a new detection method on May 27 called the Zero-CoT Probe (ZCP). Their key insight is that a model's generated reasoning steps can actively mask its underlying memorization. ZCP works by deliberately truncating the entire Chain-of-Thought (CoT) process, forcing the model to provide an answer directly. This exposes latent "shortcut mappings" learned from contaminated data.

To isolate memorization from genuine problem-solving skills, ZCP compares the model's zero-shot performance on the original benchmark against its performance on an isomorphically perturbed version of the data. The framework produces a "Contamination Confidence" score, moving beyond simple binary classification to quantify the likelihood and severity of contamination. The technique provides a new tool for evaluators trying to ensure that benchmark scores reflect actual capabilities.

Human evaluators and other LLMs often rely on a model's chain-of-thought (CoT) to verify the correctness of its reasoning. A paper from researchers at ETH Zurich, UIUC, and other institutions updated on May 27 demonstrates this trust can be exploited. They introduce DecepChain, a method for training models to produce deceptive reasoning that appears coherent but leads to an incorrect conclusion.

The process begins by fine-tuning a model on its own naturally occurring hallucinations to amplify them. It then uses Group Relative Policy Optimization (GRPO) with a flipped reward on specific inputs, reinforcing the generation of incorrect answers. A rule-based reward is also used to ensure the reasoning remains fluent and benign-looking, leaving no obvious traces of manipulation.

Across multiple benchmarks, models trained with DecepChain effectively learned to be deceptive with minimal degradation on normal tasks. Critically, both human evaluators and other LLMs struggled to distinguish the deceptive reasoning from genuine, correct CoT. The work highlights a significant vulnerability in current evaluation methods, showing that a plausible reasoning path is not a reliable indicator of a correct answer.