Linkstream

Unlimiformer: Long-Range Transformers with Unlimited Length Input
Unlimiformer improves pretrained models such as BART and Longformer by extending them to unlimited inputs without additional learned weights and without modifying their code (via kNN-search)
https://arxiv.org/abs/2305.01625

https://arxiv.org/abs/2302.10866
https://github.com/HazyResearch/safari
Convolutional LMM, hmmm.
> reaching Transformer quality with a 20% reduction in training compute required at sequence length 2K. Hyena operators are twice as fast as highly optimized attention at sequence length 8K, and 100x faster at sequence length 64K.

😂 https://www.arxiv-vanity.com/papers/2304.06035/

https://twitter.com/_akhaliq/status/1645257919997394945
Dwarf Fortress got a serious competitor!

Map of Contemporaries:
The history of the world in famous people’s lifespans. Did you realize that Alessandro Volta was younger than Napoleon? See which famous people shared their time on Earth.
https://ybogdanov.github.io/history-timeline/

https://arxiv.org/pdf/2303.11366.pdf

Reflexion: an autonomous agent with dynamic memory and self-reflection

interesting. the fact that it’s a hybrid embedding/prediction model sounds very… logical.

so you can chug along without attention just fine it seems

https://twitter.com/BlinkDL_AI/status/1638555109373378560?s=20

forget the narrative, forget linear time

https://generative.ink/posts/loom-interface-to-the-multiverse/

a soup of observations

https://lilianweng.github.io/posts/2023-03-15-prompt-engineering/

while the public is ranting, Bellard ships

ts_server is a web server proposing a REST API to large language models. They can be used for example for text completion, question answering, classification, chat, translation, image generation, ...
https://wz.ax/textsynth-server

Мене пару місяців тому глибоко вразило ось це:

https://github.com/cksystemsteaching/selfie/

hmm, guys from Stanford claim that for instruction-tuning LLaMA 7B is enough. good! waiting for the fine-tuning code 🧐

https://crfm.stanford.edu/2023/03/13/alpaca.html

the dark side of indistinguishability obfuscation: machine learning models can encode arbitrary functions, therefore provably undetectable backdoors as well

https://arxiv.org/abs/2204.06974

i honestly don't know if this goes here or into the other channel 😂
Parachute use to prevent death and major trauma when jumping from aircraft: randomized controlled trial
https://wz.ax/10.1136/bmj.k5094/parachutes

Voxel 3D light field rendering, a thing which I dreamed of building since high school, tried to build multiple times back in 2010, somewhat succeeded in 2015. Made possible by neural networks now. IMHO This is the future of 3d art and entertainment. Unlimited detail, lighting effects of any complexity, etc etc.

(Try different rendering modes, it's all glitchy but it's starting to work!)
https://wz.ax/luma-neural-rendering
https://wz.ax/luma-neural-trees
and others on the luma labs website

I agree crypto & defi are often controversial but where else do you get so much nerd-wild-west fun?
https://twitter.com/danielvf/status/1626641254531448833

let here be reactions too.

well... i've yet to read this through, but likely this is THE thing that made calculus and linear algebra so hard for me (learning discrete math was definitely worth it too, but still!)

> This is a collection of ambiguous, inconsistent, or just unpleasant conventions in mathematical notation, started by Christian Lawson-Perfect.
https://wz.ax/math-wtf