☥ This week we do see a new big model: meet Ankh, a protein LM! Thanks to the recent observation of importance of data size and training vs model size, 1.5B Ankh often outperforms 15B ESM-2B on contact prediction, structure prediction, and a good bunch of protein representation learning tasks. Arxiv pre-print is available as well!If you didn’t make it to polish the submission for ICML or IJCAI, consider other upcoming submission deadlines:
- Deep Leaning for Graphs @ International Joint Conference on Neural Networks: Jan 31st
- Special Issue on Graph Learning @ IEEE Transactions on Neural Networks and Learning Systems: March 1st
- Graph Representation Learning track @ European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning: May 2nd
The Summer Geometry Initiative (MIT) is a six-week paid summer research program introducing undergraduate and graduate students to the field of geometry processing, no prior experience is required: apply until February 15th.
New articles and blogs about graphs and more general deep learning:
- Quanta Magazine published a fascinating article on the discovery of the shortest path algorithm on graphs with negative edge weights;
- Kexin Huang prepared a post explaining a variety of datasets available in the Therapeutic Data Commons from drug-target interactions to retrosynthesis and predicting CRISPR editing outcomes
- Tim Dettmers updated his annual report on most efficient GPUs per $ with new data from H100. In relative performance, if you don’t have H100’s — get RTX 4090; for perf per $ 4070 Ti is surprisingly in the top.
- Google published a Deep Learning Tuning Playbook - a collection of tuning advice that will help you to squeeze that 1% of performance and get top-1 in OGB!
- Finally, a huge post from Lilian Weng on optimizing inference of large Transformers