Did English ever have a formal version of “you”?

Yes it did, and the formal version was (drumroll, please....) you.

In Early Modern English, thou was the singular and you was the plural. Plural you came to be used as a polite form of address (similar to the French vous, which is also used for the plural), but over time this polite form became more and more common, eventually displacing the singular thou altogether.

This explains a peculiarity of traditional Quaker speech, which one often hears in films set in the early Americas. The Quakers opposed making any distinctions of rank, so they insisted on addressing everyone as thou, not as you. The irony is that today we perceive thou to be archaic and formal, while the original intent is to be more informal.