Researchers from CAS, HKISI-CAS, Sun Yat-sen, and Peking have presented a new approach: RobustMerge
🟢 How RobustMerge helps?
A method for training-free, parameter-efficient model merging.
What problem solved?
Each expert model specializes in something — one for OCR, another for vision, a third for dialogue, a fourth for code.
But how to assemble them into one universal MLLM so that:
- there is no data leakage
- no need to retrain everything
- accuracy is not lost
- the model does not break due to conflicting weights
🧠 What RobustMerge does
The method preserves *direction robustness* — the stability of weight directions — using two key techniques:
- low-rank analysis — highlights the main knowledge direction
- cross-task normalization — normalizes the contribution of different tasks so that one model does not "override" another
Different specialized models become one universal MLLM that continues to perform well across all areas and even improves generalization.
Solves main pain point: how to combine dozens of experts into a single system without huge retraining costs and without risking mixing private data.
GitHub
••••••••••••••••••••••••••••••••••••••••
🤖 Data Science, ML & Big Data with @DataXplore