Visual counterfactual explanations reveal how an image could be minimally changed to alter a model's decision. Existing diffusion-based approaches commonly edit along a long reverse denoising trajectory, coupling semantic editability with spatial control while repeatedly estimating clean images for classifier guidance.
We introduce FiRe, a fixed-noise refinement framework that maps the input to one intermediate noise level and iteratively optimizes the noisy state there. Pixel Mean Flow provides direct clean-image prediction, while dynamic dual masks, adaptive guidance, and early stopping keep edits localized and decision-relevant. Across five tasks on CelebA, CelebA-HQ, and CheXpert, FiRe delivers competitive or state-of-the-art counterfactual quality with substantially lower computational cost.
FiRe replaces reverse-trajectory editing with iterative refinement at a single fixed noise level. Pixel Mean Flow directly predicts a clean image for classifier guidance, while dynamic dual masks control where changes accumulate and where they become visible.
FiRe generates 1,000 counterfactuals on CelebA Smile in 6.37 minutes with 1.28 PFLOPs, achieving approximately 3× faster inference and 8× fewer FLOPs than the strongest recent baseline.
Qualitative results on CelebA, CelebA-HQ, and CheXpert show targeted counterfactual changes while preserving content unrelated to the classifier's decision.
The build-up study isolates the contributions of direct clean-image prediction, fixed-noise optimization, dynamic dual masks, adaptive guidance, and early stopping.
