FastParseX, A high‑performance C++ parser for huge CSV/Log/Binary files (mmap, zstd, Arrow/Parquet)
I've been working on a C++ parsing engine focused on processing very large files (GB-TB scale) with predictable performance and minimal allocations.
FastParseX supports:
\- zero-allocation CSV parsing (quoted fields, multiline, trimming)
\- log parsing (Apache/Nginx/custom formats)
\- binary parsing (endianness, slicing, struct reading)
\- memory-mapped I/O with madvise/prefetch
\- async buffered I/O
\- gzip/xz/zstd streaming
\- parallel chunk processing
\- profiling (type inference, cardinality, min/max)
\- Arrow export
\- Parquet export
Repo: https://github.com/FastParseX-dev/FastParseX
Feedback from people who care about low-level performance, parsing internals, or C++ API
design is very welcome.
https://redd.it/1rp4a5r
@r_cpp
Post #24838
23