Adversarial robustness

A classifier labels a photo “cat.” Could carefully chosen small changes to the image make it lose that correct answer? Specify what changes are allowed: in one common test, each red, green or blue intensity may change by at most 8 steps on a 0-to-255 scale. An intensity of 128 may become 136, but 137 is outside that budget. The intended label is assumed to remain cat. The question is how to keep useful classification accuracy while protecting every input change permitted by such a stated rule. This rule does not cover every possible way an attacker could alter a real scene.[1] [2]

Can a learned classifier stay correct against all input changes in a specified adversarial set while remaining accurate on the task? Start with an image $x$ correctly labeled cat and a perturbed image $x+\delta$. Each component is a red, green or blue intensity normalized to the interval $[0,1]$. One RobustBench CIFAR test permits $\|\delta\|_\infty\leq8/255$, where the norm is the largest absolute component change; the resulting image must remain in the allowed input range. This is a mathematical threat set, with the correct label assumed unchanged, not a proof that every allowed image is perceptually identical. Attacker knowledge and the model’s input preprocessing are also part of the evaluation. An empirical attack searches this set for a wrong prediction. A certificate proves protection for a stated input and set, under its assumptions. Dataset-level robust accuracy then concerns how often that protection or resistance occurs across evaluated examples; it is not supplied by a certificate for one image.[1] [2]

Resisting selected attacks does not prove protection against every allowed change.[1] [2]

RobustBench evaluates specified models and threats. Its maintained guidance distinguishes empirical from certified robustness and warns that attack-based scores can overestimate protection.[1] [2]

A classifier can appear reliable on ordinary photos yet fail when someone deliberately chooses a permitted change. Measuring that vulnerability tests whether the learned decision is stable under the intended conditions. A defense also needs useful accuracy on ordinary inputs; making the classifier uninformative would not protect the task it was built to perform.[1] [2]

Protection would let a classifier keep a correct class decision even when someone deliberately chooses a change within the specified budget. This asks for learned distinctions that survive those changes while still separating different classes; a constant answer would discard that useful task ability. The connection to ordinary accuracy makes robustness a learning problem, not merely a test of one attack procedure. Empirical searches and certificates provide different evidence toward this capability, and their conclusions remain tied to the stated inputs, preprocessing and threat model.[1] [2]