Whether deep learning models truly outperform simple baselines for single-cell perturbation prediction remains an open question. We argue that a key variable is missing from this debate: the rule demand imposed by the evaluation task itself. Prediction tasks are not equally demanding - they differ in whether test split requires learning beyond simple, interpretable rules. When mean-based or additive baselines already explain most of the evaluable signal, even highly expressive architectures have little opportunity to demonstrate a meaningful advantage

We introduce Rule Demand (RD), a meta-metric that quantifies the fraction of calibrated signal left unexplained by simple baselines. Across six scenarios derived from the same synthetic gene regulatory network, we show that splits commonly presented as “hard” out-of-distribution settings (cell-type extrapolation, dose extrapolation, and epistatic knockout prediction) can be almost fully explained by mean-based or additive baselines (RD ≈ 0), whereas unseen-perturbation splits preserve substantial headroom (RD → 1). We find that this illusion of difficulty also appears in real perturbation datasets, including Norman19, Wessels23, and XAtlas-Orion. Rule Demand offers, for the first time, an explanatory axis for why published studies report inconsistent improvements over naive rules. Across all scenarios and datasets, model gains over simple baselines are strongly predicted by RD.

In this work, we present a calibrated benchmarking suite that extends the Dynamic Range Fraction (DRF) framework with signal-magnitude-aware normalization, more than 60 performance metrics, and systematic control diagnostics for both metric choice and dataset selection.

Co-authors: Alexander Theus, Marina Medina Esteban, Valentina Boeva