TGViewer
EDU EDU @productsandstartups · 15.4K subscribers
Post #130 2.09K
#книги
В топку идеи про many model thinking из книги The Model Thinker выше:

Empirical studies of prediction align with that inference. While adding models improves accuracy (they have to, given the theorems), the marginal contribution of each model falls off after a handful of models. Google found that using one interviewer to evaluate job candidates (instead of picking at random) increases the probability of an above-average hire from 50% to 74%, adding a second interviewer increases the probability to 81%, adding a third raises it to 84%, and using a fourth lifts it to 86%. Using twenty interviewers only increases the probability to a little over 90%. That evidence suggests a limit to the number of relevant ways of looking at a potential hire.

A similar finding holds for an evaluation of tens of thousands of forecasts by economists regarding unemployment, growth, and inflation. In this case, we should think of the economists as models. Adding a second economist improves the accuracy of the prediction by about 8%, two more increase it by 12%, and three more by 15%. Ten economists improve the accuracy by about 19%. Incidentally, the best economist is only about 9% better than average—assuming you knew which economist was best. So three random economists perform better than the best one. 5 Another reason for averaging many and not relying on the economist who has been best historically is that the world changes. The economist who performs at the top today may be middling tomorrow. That same logic explains why the US Federal Reserve relies on an ensemble of economic models rather than just one: the average of many models will typically be better than the best model.

The lesson should be clear: if we can construct multiple diverse, accurate models, then we can make very accurate predictions and valuations and choose good actions. The theorems validate the logic of many-model thinking. What the theorems do not do, and cannot do, is construct the many models that meet their assumptions. In practice, we may find that we can construct three or maybe five good models. If so, that would be great. We need only read back one paragraph: adding a second model yields an 8% improvement, while adding a third gets us to 15%. Keep in mind, these second and third models need not be better than the first model. They could be worse. If they are a little less accurate, but categorically (in the literal sense) different, they should be added to the mix.
  • ❤ 1
More from @productsandstartups
  1. Sep 29, 20262 новых скилла от Anthropic: build-eval & hillclimb Вчера Anthropic добавили в свой claude…
  2. Sep 29, 2026Выложили запись вебинара: https://youtu.be/reBi8UlA8O8 - enjoy!
  3. Sep 28, 2026В последние 3 недели делаю много PR-ов на пару с дружбаном, и, признаться, через день уже…
  4. Sep 27, 2026Много вижу восторженных отзывов про видео-редакторские способности Opus 5.5 и аудио-генера…
  5. Sep 26, 2026Как вернуть джунам последствия Пару месяцев назад я спрашивал: как не лишить молодое покол…
  6. Sep 24, 2026Наш плагин на днях апрувнули в chatgpt - некоторые очень ранние заметки с полей: 1) 4-5 ин…
Threads Profile ViewerView any public Threads profile without an account.Open ThreadLook →Writing with AI? Make it sound human.Metric37 rewrites AI drafts so they read naturally. Free AI detector, 1,500 words free.Try Metric37 →