​​Out-of-Core DataFrames for #python, ML, visualize and explore big tabular data at a billion rows per second.

Vaex is a high performance Python library for lazy Out-of-Core DataFrames (similar to Pandas), to visualize and explore big tabular datasets. It calculates statistics such as mean, sum, count, standard deviation etc, on an N-dimensional grid for more than a billion (10^9) samples/rows per second. Visualization is done using histograms, density plots and 3d volume rendering, allowing interactive exploration of big data. Vaex uses memory mapping, zero memory copy policy and lazy computations for best performance (no memory wasted).

Key features:
- Instant opening of Huge data files (memory mapping)
- Expression system: don't waste memory or time with feature engineering, we (lazily) transform your data when needed
- Out-of-core DataFrame: filtering and evaluating expressions will not waste memory by making copies; the data is kept untouched on disk, and will be streamed only when needed
- Fast groupby / aggregations
- Fast and efficient join

https://github.com/vaexio/vaex