An Empirical Study on the Performance of Cost-Sensitive Boosting Algorithms with Different Levels of Class Imbalance

Algorithm 1

AdaBoost algorithm.

(i) Input: training set (𝑥₁,𝑦₁),…,(𝑥_𝑁,𝑦_𝑁), where 𝑥_𝑖∈𝑋,𝑦_𝑖∈𝑌={−1,+1}, BaseLearn

  algorithm, Number of iterations 𝑇.

(ii) Initialization: the weighted distribution of training samples: 𝐷¹(𝑖)=1/𝑁,𝑖=1,…,𝑁.

(iii) Iteration: For 𝑡=1,…,𝑇:

  (1) Use the BaseLearn algorithm to train a component classifier ℎ_𝑡 on the training data set

    sampling from distribution 𝐷^𝑡.

  (2) Calculate the training error 𝜀_𝑡 of the classifier ℎ_𝑡: 𝜀_𝑡=∑_{𝑖∶𝑦𝑖≠ℎ𝑡(𝑥𝑖)}𝐷^𝑡(𝑖)

  (3) Set the weight 𝛼_𝑡 for the classifier ℎ_𝑡: 𝛼_𝑡=(1/2)log(1−𝜀_𝑡)/𝜀_𝑡.

  (4) Update the weighted distribution of training samples:

               𝐷^𝑡+1𝐷(𝑖)=^𝑡(𝑖)⋅𝑒^{−𝛼𝑡ℎ𝑡(𝑥𝑖)𝑦𝑖}𝑍_𝑡,

  where 𝑍_𝑡 is the normalization constant so that 𝐷^𝑡+1 will be a distribution, that is,

               𝑍_𝑡=∑_𝑖𝐷^𝑡(𝑖)⋅𝑒^{−𝛼𝑡ℎ𝑡(𝑥𝑖)𝑦𝑖}.

(iv) Output: The final hypothesis:

               𝐻(𝑥)=sgn∑^𝑇_𝑡=1𝛼_𝑡ℎ_𝑡(𝑥)