A Novel Selective Ensemble Algorithm for Imbalanced Data Classification Based on Exploratory Undersampling

Algorithm 4

RotEasy algorithm.

(i)  Input: 𝒩: the majority set, 𝒫: the minority set, 𝑇: the number of subsets undersampling from 𝒩,

   𝑠_𝑖: the number of inner-layer ensemble, 𝑆=𝑥_𝑖,𝑦_𝑖^𝑚_𝑗=1: validation dataset, 𝜌∈(0,1): tradeoff parameter,

   Φ={−1,+1}: the set of class labels.

(ii)  For  𝑖=1  to  𝑇  do

  (a) Randomly undersampling a subset 𝒩_𝑖 from 𝒩, |𝒩_𝑖|=|𝒫|.

  (b) Learning the inner-layer ensemble 𝐻_𝑖=ℎ_𝑖,𝑗^𝑠𝑖_𝑗=1:

    (1) Set ℒ_𝑖=[𝒫,𝒩_𝑖]={(𝑥^𝑖_𝑘,𝑦^𝑖_𝑘)}^𝑁_𝑘=1=(𝑋_𝑖,𝑌_𝑖), the weak classifier 𝒲, initial weight

     distribution on the training set as 𝐷¹_𝑖(𝑥^𝑖_𝑘)=1/𝑁.

    (2) for  𝑗=1  to  𝑠_𝑖  do

    (3) Calculate the rotation matrix 𝑅^𝑎_𝑖,𝑗 using 𝑋_𝑖, based on Algorithm 2.

    (4) Get the sampling subset ℒ_𝑖,𝑗=(𝑋_𝑖,𝑗, 𝑌_𝑖,𝑗) using weight distribution 𝐷^𝑗_𝑖.

    (5) Learn ℎ_𝑖,𝑗 by providing the transformed subset (𝑋_𝑖,𝑗𝑅^𝑎_𝑖,𝑗, 𝑌_𝑖,𝑗) as the input of classifier 𝒲.

    (6) Calculate the training error 𝜀_𝑖,𝑗 over ℒ_𝑖: 𝜀_𝑖,𝑗=∑_{𝑘∶𝑦𝑖𝑘≠ℎ𝑖,𝑗(𝑥𝑖𝑘𝑅}^𝑎_𝑖,𝑗)𝐷^𝑗_𝑖(𝑘).

    (7) Set the weight 𝛼_𝑖,𝑗: 𝛼_𝑖,𝑗=(1/2)log(1−𝜀_𝑖,𝑗)/𝜀_𝑖,𝑗.

    (8) Update 𝐷_𝑖^𝑗+1 over ℒ_𝑖: 𝐷_𝑖^𝑗+1(𝑘)=𝐷^𝑗_𝑖(𝑘)⋅𝑒^{−𝛼𝑖,𝑗ℎ𝑖,𝑗(𝑥𝑖𝑘𝑅𝑎𝑖,𝑗)𝑦𝑖𝑘}/𝑍_𝑖,𝑗,

       where 𝑍_𝑖,𝑗 is the normalization constant: 𝑍_𝑖,𝑗=∑_{𝑘=1∶𝑁}𝐷^𝑗_𝑖(𝑘)⋅𝑒^{−𝛼𝑖,𝑗ℎ𝑖,𝑗(𝑥𝑖𝑘𝑅𝑎𝑖,𝑗)𝑦𝑖𝑘}.

    (9) Endfor

(iii) Endfor

(iv)  Pruning: Apply the DREP method on the validation subset 𝑆 to prune the ensemble 𝐻=ℎ_𝑖,𝑗,𝑖=1,2…𝑇,𝑗=1,2…,si.

  Denote the pruned ensemble members as ℎ_𝑘^𝐾_𝑘=1, their corresponding normalized weights

   𝛼_𝑘^𝐾_𝑘=1 and rotation matrices 𝑅^𝑎_𝑘^𝐾_𝑘=1.

(v)  Classification Phase: For a given 𝑥, calculate its class label ℎ(𝑥) as follows:

        ℎ(𝑥)=argmax^𝐾_𝑦∈Φ∑_𝑘=1𝛼_𝑘𝐼ℎ_𝑘𝑥𝑅^𝑎_𝑘=𝑦.