Discover more from Daily Dose of Data Science
Parallelize Pandas with Pandarallel
Pandas' operations do not support parallelization. As a result, it adheres to a single-core computation, even when other cores are available. This makes it inefficient and challenging, especially on large datasets.
"Pandarallel" allows you to parallelize its operations to multiple CPU cores - by changing just one line of code. Supported methods include apply(), applymap(), groupby(), map() and rolling().
Thanks for reading Daily Dose of Data Science! Subscribe for free to learn something new and insightful about Python and Data Science every day. Also, get a Free Data Science PDF (250+ pages) with 200+ tips.
Read more here: https://github.com/nalepae/pandarallel.