Discover more from Daily Dose of Data Science
NVIDIA's Latest Update Can Make Your Pandas Workflow 150x Faster
...without any code changes.
As the dataset grows, it becomes increasingly challenging to use Pandas for data analysis.
This is because:
Pandas always adheres to single-core computation.
Pandas creates bulky DataFrames.
While many libraries do address these limitations, most of them can never make use of GPUs for immense speedups.
Addressing this, NVIDIA brought in one of the biggest updates recently.
The new RAPIDS cuDF library allows Pandas users to supercharge their Pandas workflow with GPUs.
How to use it?
Within a GPU runtime, do the following :
Load the extension:
import pandas as pd
Done! Use Pandas’ methods as you usually would.
Just loading the extension provides immense speedups. This is evident from the gif below.
As per NVIDIA’s official release, this can be as fast as 150x.
In my personal experimentation, however, I mostly observed it to range between 50-70x, which is still pretty good.
The good thing is that the extension accelerates most Pandas’ methods.
Yet, if needed, it can automatically fall back to the CPU.
How does it work?
cudf.pandas is enabled, the
import pandas as pd statement does not import the original Pandas library which we use all the time.
Instead, it imports another library that contains GPU-accelerated implementations of all Pandas methods.
This is evident from the image below:
This alternative implementation preserves the entire syntax of Pandas. So if you know Pandas, you already know how to use cuDF’s Pandas.
Isn’t that cool?
You can find the code here: Google Colab.
👉 Over to you: What are some other ways to accelerate Pandas operations in general?
Thanks for reading Daily Dose of Data Science! Subscribe for free to learn something new and insightful about Python and Data Science every day. Also, get a Free Data Science PDF (550+ pages) with 320+ tips.
👉 If you liked this post, don’t forget to leave a like ❤️. It helps more people discover this newsletter on Substack and tells me that you appreciate reading these daily insights.
The button is located towards the bottom of this email.
Thanks for reading!
Latest full articles
If you’re not a full subscriber, here’s what you missed last month:
To receive all full articles and support the Daily Dose of Data Science, consider subscribing:
👉 Tell the world what makes this newsletter special for you by leaving a review here :)
👉 If you love reading this newsletter, feel free to share it with friends!