Fast, Scalable LDA in C++ with Stochastic Variational Inference
TL;DR: open-sourced a high-performance C++ implementation of Latent Dirichlet Allocation using Stochastic Variational Inference (SVI). It is multithreaded with careful memory reuse and cache-friendly layouts. It exports MALLET-compatible snapshots so you can compute perplexity and log likelihood with a standard toolchain.
Repo:
https://github.com/samihadouaj/svi\_lda\_cBackground:
I'm a PhD student working on databases, machine learning, and uncertain data. During my PhD, stochastic variational inference became one of my main topics. Early on, I struggled to understand and implement it, as I couldn't find many online implementations that both scaled well to large datasets and were easy to understand.
After extensive research and work, I built my own implementation, tested it thoroughly, and ensured it performs significantly faster than existing options.
I decided to make it open source so others working on similar topics or facing the same struggles I did will have an easier time. This is my first contribution to the open-source community, and I hope it helps someone out there \^\^.
If you find this useful, a star on GitHub helps others discover it.
What it is
C++17 implementation of LDA trained with SVI
OpenMP multithreading, preallocation, contiguous data access
Benchmark harness that trains across common datasets and evaluates with MALLET
CSV outputs for log likelihood, perplexity, and perplexity vs time
Performance snapshot
Corpus: Wikipedia-sized, a little over 1B tokens
Model: K = 200 topics
Hardware I used: 32-core Xeon 2.10 GHz, 512 GB RAM
Build flags:
-O3 -fopenmp Result: training completes in a few minutes using this setup
Notes: exact flags and scripts are in the repo. I would love to see your timings and hardware
https://redd.it/1one4kq@r_cpp
