Bagging

Bootstrap Aggregation famously knows as bagging, is a powerful and simple ensemble method.

What are ensemble methods?

Ensemble learning is a machine learning technique in which multiple weak learners are trained to solve the same problem and after training the learners, they are combined to get more accurate and efficient results. weak learners + weak learners_{n =} Strong learner An ensemble method is a technique that combines the predictions from many machine learning algorithms together to make more reliable and accurate predictions than any individual model. It means that we can say that prediction of bagging is very strong.

Why we use bagging?

The main purpose of using the bagging technique is to improve Classification Accuracy. How does Bagging work For example, we have 1000 observations and 200 elements. In bagging, we will create several models with a subset of variables and a subset of observations. i.e we might create 300 trees with 300 random variables and 20 observations in each tree. After that, we can average the results of all the 300 tree’s (models) to get to our final prediction.

Model’s Derivation/Estimation (In General)

   

Bootstrap

• Accuracy Estimation
• Sampling with replacement
• Some may not be used, other may be used more than once

Bootstrap – At Abstract

Bootstrapping – in Detail

Bagging

Benefits of Bagging

• Can also improve continuous label predictive model’s accuracy
• Ideal for the parallel processing environment
• Significantly greater accuracy than a single classifier
• Reduces the variance of the individual model
• Best in case of diverse classifiers.

Similarities Between Boosting and bagging

1. Both of these are ensembling methods to get N learners from 1 learner.
2. Both of these perform well when we want to reduce the variance.
3. Both of these use random sampling to generate many training data sets.
4. Both of these are helpful in making the final decision by averaging the N learners.

Differences and similarities or comparison of  boosting and bagging

Next Similar Tutorials

1. Decision tree induction on categorical attributes  – Click Here
2. Decision Tree Induction and Entropy in data mining – Click Here
3. Overfitting of decision tree and tree pruning – Click Here
4. Attribute selection Measures – Click Here
5. Computing Information-Gain for Continuous-Valued Attributes in data mining – Click Here
6. Gini index for binary variables – Click Here
7. Bagging and Bootstrap in Data Mining, Machine Learning – Click Here
8. Evaluation of a classifier by confusion matrix in data mining – Click Here
9. Holdout method for evaluating a classifier in data mining – Click Here
10. RainForest Algorithm / Framework – Click Here
11. Boosting in data mining – Click Here
12. Naive Bayes Classifier  – Click Here