📝👀Fastparquet: Reading Parqufet Files with Python
Apache Parquet is a binary column-oriented storage format originally created for the Hadoop ecosystem. Thanks to its concise and efficient column-wise representation of data, it is very popular in the Big Data world. However, reading the data in Parquet format is not an easy task. PySpark can handle this, of course, but not every Data Scientist works with data in Apache Spark. This is where fastparquet comes in, a Python implementation of the Parquet format used by Dask, Pandas, and others to deliver high performance with a small distribution size and small codebase. Fastparquet depends on a set of Python libraries (numpy, pandas, cramjam, fsspec), so they should be installed beforehand.
After installation via the PIP package manager (pip install fastparquet) or from Github (pip install git + https: //github.com/dask/fastparquet), the contents of the Parquet file can be easily transferred to the dataframe in your usual DS-IDE as Jupiter Notebook:
from fastparquet import ParquetFile
pf = ParquetFile('myfile.parq')
df = pf.to_pandas()
df2 = pf.to_pandas(['col1', 'col2'], categories=['col1'])

Or write a dataframe to a Parquet file, specifying the number of logical segments, compression codec and data scheme:
from fastparquet import write
write('outfile.parq', df)
write('outfile2.parq', df, row_group_offsets=[0, 10000, 20000],
compression='GZIP', file_scheme='hive')

https://github.com/dask/fastparquet
https://www.anaconda.com/blog/whats-new-with-fastparquet
https://blog.datasyndrome.com/using-the-python-ml-stack-inside-pyspark-de1223942c32
https://fastparquet.readthedocs.io/en/latest/