vulpea has had a comparison page for a while, and I link it in comments when people ask how it differs from org-roam, org-node or Supertag: https://github.com/d12frosted/vulpea/blob/master/docs/comparison.org. Recently I thought: ok, let me also check performance. So I updated the page and decided to share some data.
Updating it made me look around properly, and a lot has changed over the last year. org-node sits on org-mem, which parses files in parallel subprocesses and keeps everything in memory. Supertag released V2 in September with a much narrower goal: write first, organize later.
For speed I did not want vulpea grading itself, so the numbers live in a separate repo: https://github.com/d12frosted/org-notes-bench. Same generated notes, pinned versions, each package in a fresh
emacs -Q on an EC2 instance, measured only through its own commands. Each adapter is a page of Lisp.100k files (145k notes), default settings:
||org-roam|org-node|Supertag|vulpea|
|:-|:-|:-|:-|:-|
|Index from scratch|74 min|15s|2.3 min|4.9 min|
|First run: longest freeze|whole index|3.4s|0.6s|0.26s|
|Start of a new session|16s|15s|3.5s|9ms|
|Open the find command|7.8s|19ms|37ms|25ms|
|Backlinks of a note with 35k of them|14s|26ms|0.41s|0.98s|
|Save a 10MB file: longest freeze|36s|0.39s|18ms|1.8s|
Not everything here is like for like. org-node keeps nothing on disk, so every session starts with a full index. Backlinks return different things in each package. The runs used several machines of the same type, which differ by about 15%, so read the big gaps and ignore the small ones. The README and RESULTS.md have the details, the 1k and 10k tables, and the raw data. vulpea's numbers are from master; 2.8 is not released yet.
vulpea's weak spot is the index from scratch: 4.9 min, where org-node needs 15s. On a first start it runs in the background and you pay it once, but that is what I want to work on next.
The benchmark already paid off. u/yibie took the first numbers and shipped Supertag 0.2.0 and 0.3.0 within days: indexing 100k files went from 51 min to 2.3 min, a first run that did not finish in 3.5 hours now takes 1.9 min, and saving a 10MB file blocks for 18ms instead of 13s. vulpea got faster on the way too: its find command took seconds to open at 100k notes and now takes 25ms.
I hope you find it useful. I mean, obviously I focus on performance in this post, but I also tried to map the features that each package brings, so maybe it will make your life easier. Or not :)
Thanks for reading!
https://redd.it/1x3yhv2
@r_emacs
