Reminder: Europeans overwhelmingly descend from the Medieval middle and upper classes, not the peasantry.

The former outbred the latter by a significant margin. As higher status positions were of limited availability, the excess children of more successful people faced downward social mobility. The lower classes, who had sub-replacement fertility levels, were constantly demographically replaced for hundreds of years.

This breeding pattern continued until the turn of the 19th century. From ~1800 to ~1880, there was little difference in net fertility by wealth/class, and after ~1880, higher wealth/class became negatively associated with fertility.

The vast majority of your ACTUAL ancestors were not peasants.

See: A Farewell to Alms by Gregory Clark