Polish startup Pathway has introduced a A Brain-Inspired Neural Network combining transformers with biological brain models.
🟢 What is the Core Idea?
Models neurons as graph vertices and synapses as weighted edges.
Neurons communicate only with neighbors — mimicking brain-like local interactions.
Trains using Hebb’s rule (“neurons that fire together, wire together”).
🟠 Key Properties
Two weight types of Architecture:
Fixed → long-term knowledge (updated only during training)
Dynamic → short-term reasoning, updated per inference step
BDH-GPU tensor version = transformer-like (attention + MLP + ReLU).
1. Interpretability: each neuron pair has a visible synapse → clear concept mapping.
2. Scalability: models can be merged by concatenation.
3. Performance: follows GPT-2-like scaling laws and similar accuracy.
A fascinating blend of neuroscience + transformers — potentially a major step toward more interpretable, brain-like AI.
#AI #Neuroscience #Transformers #MachineLearning #Research #Pathway
🤖 Data Science, ML & Big Data with @DataXplore