Coverage, Not Difficulty, Sets How Much Synthetic Data an Activation Probe Needs
Activation probe data requirements are driven by the complexity of the concept being monitored rather than the difficulty of the task.
Researchers analyzed learning curves for activation probes across various synthetic datasets. They found that probes for high-stakes or harmful content reach performance plateaus with as few as 80 samples, while instruction-following probes require significantly more data to achieve similar stability.