I don't think declining fertility is a huge problem in and of itself, since most Western countries were overpopulated even before the era of mass migration.

Our fertility-related problems are:
1) Mass migration forcing Whites into a breeding contest with the entire third world.
2) Dysgenic fertility among Whites causing us to become shorter, fatter, more mentally ill, and less intelligent.

Take Imperial Britain as a case study in the importance of quality over quantity:

At the peak of Britain's world domination, their empire of ~450 million subjects was ruled by ~150,000 ethnic Britons, including all civil and military personnel — one for every 3000 subjects, on average. In regions such as Africa, the ratio was as high as 1:5000.

For comparison, the Mongols ruled ~100 million with a ruler-to-subject ratio of 1:800 (ethnic Mongols only), the Romans ruled ~65 million with a ratio of 1:185, and the Ottomans ruled ~65 million with a ratio of 1:722.

The imperial Britons were men of incredible intelligence, courage, and discipline, operating a well-oiled machine that relied upon efficient systems and cutting-edge technology, both products of British excellence. This breed of man is practically extinct today.

The foundations of the British Empire were laid by centuries of mostly-incidental eugenics (see: A Farewell to Alms by Gregory Clark), whereby wealthier Britons had twice as many children as the poor, resulting in elite downward mobility, the diffusion of their genes/values across British society, and a significantly more intelligent and effective population overall.

Britain's long history of internal migration helped to spread these genes/values, but also reduced clannishness, producing a people who were predisposed to limited government, economic freedom, and innovative individualism; "nullius in verba" (take nobody's word for it) is the motto of the Royal Society.

These factors, combined with some luck (timing, geography, natural resources, political stability relative to continental Europe, etc.), culminated in Britain's scientific and industrial revolutions, and, ultimately, their world domination.

I'm oversimplifying, of course, but my point is that since the Proto-Indo-Europeans first expanded from the Pontic-Caspian steppe, the White man has always been outnumbered, but never outgunned.