Daily Dose of Data Science

Share this post

A Typical Thing About Decision Trees Which Many Often Ignore

www.blog.dailydoseofds.com

A Typical Thing About Decision Trees Which Many Often Ignore

Overfitting = Terror!

Avi Chawla
Mar 21, 2023
8
1
Share

Although decision trees are simple and intuitive, they always need a bit of extra caution. Here's what you should always remember while training them.

In sklearn's implementation, by default, a decision tree is allowed to grow until all leaves are pure. This leads to overfitting as the model attempts to classify every sample in the training set.

There are various techniques to avoid this, such as pruning and ensembling. Also, make sure that you tune hyperparameters if you use sklearn's implementation.

This was a gentle reminder as many of us often tend to use sklearn’s implementations in their default configuration.

It is always a good practice to know what a default implementation is hiding underneath.

Thanks for reading Daily Dose of Data Science! Subscribe for free to learn something new about Python and Data Science every day.

👉 Read what others are saying about this post on LinkedIn and Twitter.

👉 If you liked this post, leave a heart react 🤍.

👉 If you love reading this newsletter, feel free to share it with friends!

Share Daily Dose of Data Science


Find the code for my tips here: GitHub.

I like to explore, experiment and write about data science concepts and tools. You can connect with me on LinkedIn and Twitter.

8
1
Share
Previous
Next
1 Comment
D.W. Eversole
Writes Outside The Box
Mar 21Liked by Avi Chawla

overfitting a model is something I'd love for you to write more about. There is an intuition here that is not always a science I can clearly understand. I'm not always sure when cognitive bias can be objectively determined and when experienced intuition should take precedent

Expand full comment
Reply
Top
New
Community

No posts

Ready for more?

© 2023 Avi Chawla
Privacy ∙ Terms ∙ Collection notice
Start WritingGet the app
Substack is the home for great writing