While the physicochemical properties as well as the NMR and vibration spectroscopic data of the mixtures of ionic liquids (ILs) with molecular solvents undergo a drastic change around the IL mole fraction of 0.2, the local structure of the mixtures pertaining to this behavior remains unclear. In this work, the local structure of 12 mixtures of 1-butyl-3-methylimidazolium cation (C4mim+) combined with perfluorinated anions, such as tetrafluoroborate (BF4–), hexafluorophosphate (PF6–), trifluoromethylsulfonate (TFO–), and bis(trifluoromethanesulfonyl)imide, (TFSI–), and aprotic dipolar solvents, such as acetonitrile (AN), propylene carbonate (PC), and gamma butyrolactone (γ-BL) is studied by molecular dynamics simulations in the entire composition range, with an emphasis on the IL mole fractions around 0.2. Distributions of metric properties corresponding to the Voronoi polyhedra of the particles (volume assigned to the particles, local density, radius of spherical voids) are determined, using representative sites of the cations, anions, and the solvent molecules, to characterize the changes in the local structure of these mixtures. By analyzing the mole fraction dependence of the average value, fluctuation, and skewness parameter of these distributions, the present study reveals that, around the IL mole fraction of 0.2, the local structure of the mixture undergoes a transition between that determined by the interionic interactions and that determined by the interactions between the ions and solvent molecules. It should be noted that the strength of the interactions between the ions and the solvent molecules, modulated by the change in the composition of the mixture, plays an important role in the occurrence of this transition. The signature of the change in the local structure is traced back to the nonlinear change of the mean values, fluctuations, and skewness values of the metric Voronoi polyhedra distributions.