Boosting in data mining

What is Boosting?

Boosting is an efficient algorithm that is able to convert a weak learner into a strong learner.

Example:

Suppose we want to check that an email is “spam email” or  “safe email”?

In this case, there can be multiple possibilities like;

  • Rule 1: Email contains only links to some websites.
    • Decision: It is a spam
  • Rule 2: Email from an official email address. e.g [email protected]
    • Decision: It is not spam.
  • Rule 3: Email has a request to get private bank details. e.g bank account number and father/mother name etc.
    • Decision: It is a spam

Now the question is that the 3 rules discussed above or enough to classify an email as “spam” or not?

  • Answer: These 3 rules are not enough. These 3 rules are weak learner. So we need to boost these learners. We can boost the weak learners to the stronger learner by boosting.
  • Boosting can be done by combining and assigning weights to every weak learner.

Boosting have greater accuracy as compared to Bagging.

Types of boosting algorithm:

Three main types of boosting algorithm are as follows;

  1. XGBoost algorithm
  2. AdaBoost algorithm
  3. Gradient tree boosting algorithm.

Next Similar Tutorials

  1. Decision tree induction on categorical attributes  – Click Here
  2. Decision Tree Induction and Entropy in data mining – Click Here
  3. Overfitting of decision tree and tree pruning – Click Here
  4. Attribute selection Measures – Click Here
  5. Computing Information-Gain for Continuous-Valued Attributes in data mining – Click Here
  6. Gini index for binary variables – Click Here
  7. Bagging and Bootstrap in Data Mining, Machine Learning – Click Here
  8. Evaluation of a classifier by confusion matrix in data mining – Click Here
  9. Holdout method for evaluating a classifier in data mining – Click Here
  10. RainForest Algorithm / Framework – Click Here
  11. Boosting in data mining – Click Here
  12. Naive Bayes Classifier  – Click Here

 

By:Prof. Fazal Rehman Shamil
CEO @ T4Tutorials
Last Modified: November 10, 2019

Leave a Reply

Your email address will not be published. Required fields are marked *