To better understand the microstructure parameters controlling grain crushing and how the microstructure evolution influences the crushing characteristics of rockfill particles, the emphasis of the present study focuses on the size effects on the crushing strength of rockfill particles, with special attention to internal flaws. Single-particle crushing tests are performed on rockfill particles with four different size fractions. By combining these results with the experimental results from the literature, the suitability of the Weibull model to describe the size effects on the breakage strength of rock grains is discussed. To understand why larger grains possess lower strength, a bonded particle model (BPM) is established in YADE to simulate the particle crushing tests. Model parameters are calibrated against the obtained force–displacement curve. The failure and deformation behaviors of studied rockfill particles are satisfactorily reproduced by the numerical simulation. The size effects are then studied using the agglomerates with different flaw characteristics (e.g., sizes, numbers, volume ratios, distributions, and locations). Numerical results exhibit that for the studied rockfill particles, when the ratio of flaw size to agglomerate size is less than 0.186, the flaw volume ratio is the key factor for size effects, and the breakage strength of particles is slightly related to the flaw size. In contrast, the breakage strength of particles depends not only on the flaw volume ratio but also on the flaw distribution/location when the flaw size is large. Microscopic analysis and DEM modeling can improve the understanding of crushing and deformation behaviors of rockfill particles.