🥁How to understand and develop AI by searching and highlighting representative scenarios: Bayes-TrEx tool from MIT researches
Accuracy of ML-results is not enough to use it with high assurance in every field. Focus only on simple accuracy can lead to dangerous oversights. Model can make mistakes with very high confidence encountered something previously unseen, such as a self-driving car seeing a new type of traffic sign. To gain better human-AI interaction, a team of researchers from MIT’s Computer Science and Artificial Intelligence Laboratory have created a new tool called Bayes-TrEx. It allows developers and users increase transparency into their AI model. Specifically, it does so by finding concrete examples that lead to a particular behavior. The method makes use of “Bayesian posterior inference,” a widely-used mathematical framework to reason about model uncertainty.
In experiments, the researchers applied Bayes-TrEx to several image-based datasets, and found new insights that were previously overlooked by standard evaluations focusing solely on prediction accuracy. It can be used in medical diagnosis, autonomous driving systems, robotics and so on. Bayes-TrEx could help address these novel situations ahead of time, and enable developers to correct any undesirable outcomes before potential tragedies occur or resources waste.
https://news.mit.edu/2021/more-transparency-understanding-machine-behaviors-bayes-trex-0322