🌴🌳🌲Decision Trees: brief overview
In general, decision trees are constructed via an algorithmic approach that identifies ways to split a data set based on various conditions. It is one of the most widely used and practical methods of non-parametric supervised learning used for classification and regression tasks. The goal is to create a model that predicts the value of a target variable by learning simple decision rules inferred from the data features. It is one of the most powerful and popular algorithms. Decision-tree algorithm falls under the category of supervised learning algorithms. It works for both continuous as well as categorical output variables. It learns from simple decision rules using the various data features.
Entropy is the measure of uncertainty or randomness in a data set. Entropy handles how a decision tree splits the data. The information gain measures the decrease in entropy after the data set is split. The Gini Index is used to determine the correct variable for splitting nodes. It measures how often a randomly chosen variable would be incorrectly identified. The root node is always the top node of a decision tree. It represents the entire population or data sample, and it can be further divided into different sets. Decision nodes are subnodes that can be split into different subnodes; they contain at least two branches. A leaf node in a decision tree carries the final results. These nodes, which are also known as terminal nodes, cannot be split any further.
Decision Tree Applications:
• to determine whether an applicant is likely to default on a loan.
• to determine the odds of an individual developing a specific disease.
• to find customer churn rates
• to predict whether a consumer is likely to purchase a specific product.
Advantages of Using Decision Trees
• Decision trees are simple to understand, interpret, and visualize
• They can effectively handle both numerical and categorical data
• They can determine the worst, best, and expected values for several scenarios
• Decision trees require little data preparation and data normalization
• They perform well, even if the actual model violates the assumptions
Disadvantages
• Overfitting is one of the practical difficulties for decision tree models. It happens when the learning algorithm continues developing hypotheses that reduce the training set error but at the cost of increasing test set error. But this issue can be resolved by pruning and setting constraints on the model parameters.
• Decision trees cannot be used well with continuous numerical variables.
• A small change in the data tends to cause a big difference in the tree structure, which causes instability.
https://blog.devgenius.io/decision-tree-regression-in-machine-learning-3ea6c734eb51