Neural specificity refers to the degree to which neural representations of different stimuli can be distinguished. Evidence suggests that neural specificity, operationally defined as stimulus-related differences in functional magnetic resonance imaging (fMRI) activation patterns, declines with advancing adult age, and that individual differences in neural specificity are associated with individual differences in fluid intelligence. A growing body of literature also suggests that regular physical activity may help preserve cognitive abilities in old age. Based on this literature, we hypothesized that exercise-induced improvements in fitness would be associated with greater neural specificity among older adults. A total of 52 adults aged 59–74 years were randomly assigned to one of two aerobic-fitness training regimens, which differed in intensity. Participants in both groups trained three times a week on stationary bicycles. In the low-intensity (LI) group, the resistance was kept constant at a low level (10 Watts). In the high-intensity (HI) group, the resistance depended on participants’ heart rate and therefore typically increased with increasing fitness. Before and after the 6-month training phase, participants took part in a functional MRI experiment in which they viewed pictures of faces and buildings. We used multivariate pattern analysis (MVPA) to estimate the distinctiveness of neural activation patterns in ventral visual cortex (VVC) evoked by face or building stimuli. Fitness was also assessed before and after training. In line with our hypothesis, training-induced changes in fitness were positively associated with changes in neural specificity. We conclude that physical activity may protect against age-related declines in neural specificity.