Linkstream

Generative agent-based modeling with actions grounded in physical, social, or digital space using Concordia from Google DeepMind

https://arxiv.org/abs/2312.03664
https://github.com/google-deepmind/concordia

evergreen wisdom from Rob Pike
https://commandcenter.blogspot.com/2023/12/simplicity.html

ChatGPT: sometimes “hallucinates” (tries to guess the details not in the training set).
OpenAI: tries to counter that
Google: hold my beer, let’s hallucinate the actual Gemini model presentation!

https://arstechnica.com/information-technology/2023/12/google-admits-it-fudged-a-gemini-ai-demo-video-which-critics-say-misled-viewers/

https://twitter.com/MistralAI/status/1733150512395038967
beautiful. on friday. even more beautiful.

insanely detailed LLM inference visualization from Brendan Bycroft
https://bbycroft.net/llm

https://github.com/comfyanonymous/ComfyUI
if you are into Stable Diffusion

In this paper, we introduce generative agents--computational software agents that simulate believable human behavior. Generative agents wake up, cook breakfast, and head to work; artists paint, while authors write; they form opinions, notice each other, and initiate conversations; they remember and reflect on days past as they plan the next day.

https://arxiv.org/abs/2304.03442
https://github.com/joonspk-research/generative_agents

DARE/MergeLM: Absorbing Abilities from Homologous Models as a Free Lunch
https://github.com/yule-BUAA/MergeLM

System 2 Attention (S2A).
- Soft attention in Transformers is susceptible to irrelevant/biased info
- S2A uses LLM reasoning to generate what to attend to
Improves factuality & objectivity, decreases sycophancy.
https://arxiv.org/abs/2311.11829

A chance of training on huge piles of retail GPUs and not just on the superclusters?
https://huggingface.co/papers/2311.08105

R-Tuning: Teaching Large Language Models to Refuse Unknown Questions
TLDR: LLMs "hallucinate" because the training datasets never included the "I don't know" answer 🤷

https://arxiv.org/pdf/2311.09677.pdf

related: the value of privacy (2006) in plain English
https://www.schneier.com/blog/archives/2006/05/the_value_of_pr.html

big if works well: first paper that claims relatively efficient search on encrypted data without revealing what’s being searched

https://eprint.iacr.org/2022/1703

LLMs are far from being the first technology met with fear, uncertainty and doubt
https://pessimistsarchive.org

now these are really hallucinations lol
> ZeroNVS: Zero-Shot 360-Degree View Synthesis from a Single Real Image
https://kylesargent.github.io/zeronvs/

generalization, continued:
> We argue that Transformers will generalize to harder instances on algorithmic tasks iff the algorithm can be written in the RASP-L programming language (Weiss et al). By design, each line of RASP-L code can be compiled into weights of 1 Transformer layer.
https://arxiv.org/abs/2310.16028

https://www.nature.com/articles/s41586-023-06668-3
interesting. code/models: https://github.com/brendenlake/MLC

TLDR: Placebo effects are dose-dependent.
We need structured belief engineering then

https://www.biorxiv.org/content/10.1101/2022.07.15.500226v2

nvidia might be the king of the hill right now, but the future of AI is reconfigurable analog-like electronics (~100x more energy efficient already, which will take Moore’s law at least another 10 years for silicon)

Caveat: no backprop :P forward-forward and other algorithms exist though

https://www.nature.com/articles/s41928-023-01042-7