Why Is It Important To Shuffle Your Dataset Before Training An ML Model
…And what happens if you don't.
ML models may fail to converge for many reasons. Here's one of them which many folks often overlook.
If your data is ordered by labels, this could negatively impact the model's convergence and accuracy. This is a mistake that can typically go unnoticed.
In the above demonstration, I trained two neural nets on the same data. Both networks had the same initial weights, learning rate, and other settings.
However, in one of them, the data was ordered by labels, while in another, it was randomly shuffled.
As shown, the model receiving a label-ordered dataset fails to converge. However, shuffling the dataset allows the network to learn from a more representative data sample in each batch. This leads to better generalization and performance.
In general, it's a good practice to shuffle the dataset before training. This prevents the model from identifying any label-specific yet non-existing patterns.
In fact, it is also recommended to alter batch-specific data in every epoch.
P.S. This was a mistake that I once made a few years back, and it never occurred to me that shuffling is that important.
What are your thoughts on this? Let me know :)
Thanks for reading Daily Dose of Data Science! Subscribe for free to learn something new about Python and Data Science every day.
👉 Read what others are saying about this post on LinkedIn: Post Link.
👉 If you love reading this newsletter, feel free to share it with friends!
Find the code for my tips here: GitHub.