For 10 years, the residual connection (x + f(x)) has been a safety net for any transformer. GPT-4, Claude, Gemini, everyone relies on it.
And DeepSeek has replaced it with "manifold-constrained hyper-connections" (mHC).
They've turned the residual highway into n parallel lanes and added a mathematical "cell" to keep the signal stable.
Paper
••••••••••••••••••••••••••••••••••••••••••••••
🤖 Data & ML | @DataXplore