Feature selection Stage 1: RFE Algorithm |
1. Initialization: |
I. Feature set: Let F denotes the features where F = {f1, f2,…. fn}, n is the total no of features. |
II. Initialize: The selected the features F´ with all the features of F. |
III. Specify: Let k be the number of features to be selected. |
IV. Elimination: Let D represents the list of eliminated features. |
2. Feature Selection: |
I. Perform: Until the features in F´ is greater than k perform: 1–4 |
1. Train the RF model with the features of F´. |
2. Compute the feature importance score of the features in F´ as IS(f). |
3. Rank the features of F´ Rank(fi) based on the importance score IS(f). |
4. Remove the least important n–k features from F´. Update the eliminated features in D. |
3. RFE Selected Features: |
I. Initial Elimination: The features in F´ denote the selected features and D provides the eliminated features. |
Feature selection Stage 2: IG Algorithm |
4. Initialization: |
I. Feature set: Let F´ denotes the input dataset with F´ = {f1, f2,… fn–k}. |
II. Initialize: The selected features F´´ with all the features of F´. |
III. Specify: Let k´ be the number of features to be selected. |
IV. Elimination: Let D´ represents the features eliminated using IG. |
5. Feature Selection: |
I. Compute the entropy of features in F´´. |
II. Compute the information gain IG of each feature in F´´. |
III. Rank the features of F´´ using the IG score. |
IV. Remove the least important n–k–k´ features from F´´. Update the eliminated features in D´. |
6. RFE-IG Selected Features: |
I. Final Elimination: The features in F´´ denotes the selected features and D´ provides the eliminated features. |