Adapting Content Delivery to Limited Resources and Inferred User Interest

Algorithm 1

The Q-learning algorithm.

𝙸𝚗𝚒𝚝𝚒𝚊𝚕𝚒𝚣𝚎𝑄₀

𝚏𝚘𝚛𝑛=0𝚝𝚘𝑁_tot−1𝚍𝚘

𝜎_𝑛=𝚌𝚑𝚘𝚘𝚜𝚎𝚂𝚝𝚊𝚝𝚎

𝑎_𝑛=𝚌𝚑𝚘𝚘𝚜𝚎𝙰𝚌𝚝𝚒𝚘𝚗

(𝜎^′_𝑛,𝑟_𝑛)=𝚜𝚒𝚖𝚞𝚕𝚊𝚝𝚎(𝜎_𝑛,𝑎_𝑛)

/∗𝚞𝚙𝚍𝚊𝚝𝚎𝑄_𝑛+1∗/

𝑄_𝑛+1←𝑄_𝑛

𝑑_𝑛=𝑟_𝑛+(𝛾max_𝑏𝑄_𝑛(𝜎^′_𝑛,𝑏))−𝑄_𝑛(𝜎_𝑛,𝑎_𝑛)

𝑄_𝑛+1(𝜎_𝑛,𝑎_𝑛)←𝑄_𝑛(𝜎_𝑛,𝑎_𝑛)+𝛼_𝑛(𝜎_𝑛,𝑎_𝑛)𝑑_𝑛

𝚎𝚗𝚍𝚏𝚘𝚛

𝚛𝚎𝚝𝚞𝚛𝚗𝑄_𝑁tot