Initialization:
(1) Input , , , ;
(2) For all , set , , ;
(3) For all , set ;
Main Loop:
(4) While () do
 (5) Select and set ;
 (6) For all , set ;
 (7) For all , set ;
 (8) Execute   and observe , for all ;
 (9) Solve (12a)–(12e) to find an optimal ;
 (10) For all , update , , using (15);
(11) End.
Algorithm 1: JUSTE-RL with regret for unlicensed channel allocation. RL algorithm for outband channel allocation.