Saccade adaptation has been extensively studied using a paradigm in which a target is displaced during the saccade, inducing an adjustment in saccade amplitude or direction. These changes in saccade amplitude are widely considered to be controlled by the post-saccadic position of the target relative to the fovea. However, because such experiments generally employ only a single target on an otherwise blank screen, the question remains whether the same adaptation could occur if both the target and a similar distractor were present when the saccade landed. To investigate this issue, three experiments were conducted, in which the post-saccadic locations of the target and distractor were varied. Results showed that decreased amplitude adaptation, increased amplitude adaptation, and recovery from adaptation were controlled by the post-saccadic position of the target rather than the distractor. These results imply that target selection is critical to saccade adaptation.