Development of an AI-Enabled Q-Agent for Making Data Offloading Decisions in a Multi-RAT Wireless Network

<div>Performance evaluation of the data offloading policy learned by the Q-agent developed in this work and existing offloading policies, as a function of users’ request <span class="nowrap"><svg height="12.7373pt" id="M235" style="vertical-align:-3.429399pt" version="1.1" viewbox="-0.0498162 -9.3079 107.704 12.7373" width="107.704pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M538 96L524 119C492 88 454 63 446 63C439 63 434 70 440 101C463 223 491 341 518 448H508L433 422L401 276C355 192 240 56 188 56C163 56 154 89 163 133C184 233 207 338 235 443L230 448L152 424L58 17C40 -60 23 -143 23 -185C23 -241 42 -261 63 -261C92 -261 117 -241 125 -227L124 -221C109 -209 89 -165 89 -103C89 -55 96 -13 105 26H107C121 -3 134 -12 151 -12C172 -12 194 -5 221 16C279 61 330 124 384 194H386C381 159 377 141 368 105C343 3 363 -12 383 -12C416 -12 486 35 538 96Z"></path></g><g transform="matrix(.013,0,0,-0.013,7.216,0)"><path d="M300 -147C201 -63 143 98 143 270S200 602 300 686L282 710C136 610 70 450 70 271V270C70 89 136 -72 282 -170L300 -147Z"></path></g><g transform="matrix(.013,0,0,-0.013,11.714,0)"><path d="M642 419L635 448C586 435 552 411 532 392C518 379 509 351 496 304C484 260 462 196 445 150C424 93 388 40 314 30L413 508L406 510L344 491L257 36C205 46 175 84 175 169C175 195 180 241 185 277C189 304 190 331 190 358C190 413 175 448 141 448C111 448 69 429 23 373L30 347C51 365 68 375 81 375C93 375 108 368 108 327C108 307 104 268 101 239C98 211 95 180 95 155C95 33 159 -8 248 -14L213 -186C206 -220 221 -254 230 -261L261 -230L306 -11C339 -4 366 6 389 20C431 46 473 87 503 155C533 224 557 286 571 325C586 367 606 393 642 419Z"></path></g><g transform="matrix(.013,0,0,-0.013,23.85,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z"></path></g><g transform="matrix(.013,0,0,-0.013,35.113,0)"><path d="M153 550H386L412 615L406 623H120L82 318C104 327 142 338 184 338C294 338 347 275 347 187C347 112 305 39 221 39C160 39 119 71 97 89C88 97 80 96 71 90C59 80 50 67 49 57C48 45 52 36 66 23C80 9 123 -12 169 -12C221 -11 288 15 342 59C403 109 431 165 431 225C431 308 366 395 238 395C212 395 165 379 127 364L153 550Z"></path></g><g transform="matrix(.013,0,0,-0.013,41.353,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z"></path></g><g transform="matrix(.013,0,0,-0.013,47.593,0)"><path d="M241 635C89 635 35 457 35 312C35 153 89 -12 240 -12C390 -12 443 166 443 312C443 466 390 635 241 635ZM238 602C329 602 354 454 354 312C354 172 330 22 240 22C152 22 124 173 124 313S148 602 238 602Z"></path></g><g transform="matrix(.013,0,0,-0.013,56.009,0)"><path d="M861 0V28C774 35 771 41 768 147L759 509C756 612 762 614 851 622V650H681L449 149L221 650H57V622C148 613 153 609 144 479L130 271C123 166 117 123 111 88C104 46 85 34 26 28V0H259V28C192 35 169 42 167 90C166 130 166 173 170 256L185 541H187L411 7H431L675 555H679L683 147C683 41 680 35 598 28V0H861Z"></path></g><g transform="matrix(.013,0,0,-0.013,67.592,0)"><path d="M152 404V712C115 698 54 683 7 677V654C71 648 73 642 73 579V24C128 -2 179 -12 220 -12C353 -12 471 92 471 238C471 357 381 449 274 449C262 449 249 446 233 439L152 404ZM152 374C170 384 202 393 230 393C313 393 382 326 382 213C382 97 330 26 246 26C194 26 165 62 158 81C154 91 152 101 152 116V374Z"></path></g><g transform="matrix(.013,0,0,-0.013,74.039,0)"><path d="M135 536C164 536 186 560 186 587C186 617 164 639 136 639C109 639 85 617 85 587C85 560 109 536 135 536ZM252 0V26C188 32 181 38 181 106V451C138 433 90 420 39 412V388C99 379 102 374 102 312V106C102 38 95 32 32 26V0H252Z"></path></g><g transform="matrix(.013,0,0,-0.013,77.366,0)"><path d="M298 36L289 62C276 55 253 45 228 45C202 45 169 60 169 141V397H276C289 405 292 426 282 437H169V574L155 576L90 509V437H45L17 408L21 397H90V107C90 28 125 -12 188 -12C198 -12 213 -8 230 1L298 36Z"></path></g><g transform="matrix(.013,0,0,-0.013,81.332,0)"><path d="M319 325C317 349 306 409 297 431C277 440 250 449 209 449C117 449 57 389 57 319C57 243 122 209 182 182C232 159 261 135 261 91C261 48 227 21 190 21C130 21 85 79 68 145L41 140C41 104 51 36 58 22C75 7 121 -12 172 -12C252 -12 337 35 337 126C337 195 286 231 210 262C166 281 126 304 126 348C126 388 152 417 191 417C240 417 274 378 294 318L319 325Z"></path></g><g transform="matrix(.013,0,0,-0.013,86.216,0)"><path d="M95 130C70 130 46 113 46 88C46 72 54 64 59 64C93 55 121 33 121 -3C121 -41 93 -68 44 -88L55 -117C117 -98 186 -56 186 22C186 91 131 130 95 130Z"></path></g><g transform="matrix(.013,0,0,-0.013,91.36,0)"><path d="M810 587C770 574 761 568 725 543C654 630 540 687 408 687C217 687 69 577 69 430C69 319 160 247 276 247C369 247 511 319 511 495C511 510 512 523 509 538H490C490 354 393 277 274 277C174 277 110 344 110 422C110 559 219 655 415 655C524 655 636 602 695 519C628 463 572 392 516 303C433 170 377 97 313 57C252 89 199 109 143 109C79 109 36 87 36 54C36 15 82 -9 165 -9C222 -9 276 0 326 18C373 -1 423 -15 482 -15C645 -15 804 133 804 324C804 393 783 458 747 514C776 546 781 551 818 567L810 587ZM756 336C756 135 623 16 478 16C439 16 402 24 367 36C386 45 401 56 420 68C505 127 569 206 616 302C647 362 679 427 719 480C744 435 756 387 756 336ZM276 39C243 26 207 21 164 21C106 21 77 32 77 54C77 75 125 79 144 79C191 79 227 66 276 39Z"></path></g><g transform="matrix(.013,0,0,-0.013,102.865,0)"><path d="M275 270C275 450 212 609 64 710L45 686C145 604 203 442 203 270S147 -63 45 -147L64 -170C213 -68 275 89 275 270Z"></path></g></svg>:</span> (a) the payment in ($) for entertaining a given user request and (b) remaining file size <svg height="11.5564pt" id="M236" style="vertical-align:-2.26807pt" version="1.1" viewbox="-0.0498162 -9.28833 15.8625 11.5564" width="15.8625pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M300 -147C201 -63 143 98 143 270S200 602 300 686L282 710C136 610 70 450 70 271V270C70 89 136 -72 282 -170L300 -147Z"></path></g><g transform="matrix(.013,0,0,-0.013,4.498,0)"><path d="M499 88L487 113C459 86 422 65 413 65C403 65 403 74 409 98L461 318C487 426 460 448 431 448C408 448 383 441 352 424C305 398 223 344 157 257H155L243 674C249 702 249 712 240 712C227 712 170 680 87 673L82 648H117C155 648 162 644 150 590L23 -4L30 -12C55 -4 81 3 105 8C113 53 131 150 143 187C199 281 323 387 372 387C390 387 393 369 380 311L328 86C313 19 319 -12 347 -12C379 -12 442 24 499 88Z"></path></g><g transform="matrix(.013,0,0,-0.013,11.206,0)"><path d="M275 270C275 450 212 609 64 710L45 686C145 604 203 442 203 270S147 -63 45 -147L64 -170C213 -68 275 89 275 270Z"></path></g></svg> after the defined delay timer <svg height="11.5564pt" id="M237" style="vertical-align:-2.26807pt" version="1.1" viewbox="-0.0498162 -9.28833 20.6775 11.5564" width="20.6775pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M300 -147C201 -63 143 98 143 270S200 602 300 686L282 710C136 610 70 450 70 271V270C70 89 136 -72 282 -170L300 -147Z"></path></g><g transform="matrix(.013,0,0,-0.013,4.498,0)"><path d="M810 587C770 574 761 568 725 543C654 630 540 687 408 687C217 687 69 577 69 430C69 319 160 247 276 247C369 247 511 319 511 495C511 510 512 523 509 538H490C490 354 393 277 274 277C174 277 110 344 110 422C110 559 219 655 415 655C524 655 636 602 695 519C628 463 572 392 516 303C433 170 377 97 313 57C252 89 199 109 143 109C79 109 36 87 36 54C36 15 82 -9 165 -9C222 -9 276 0 326 18C373 -1 423 -15 482 -15C645 -15 804 133 804 324C804 393 783 458 747 514C776 546 781 551 818 567L810 587ZM756 336C756 135 623 16 478 16C439 16 402 24 367 36C386 45 401 56 420 68C505 127 569 206 616 302C647 362 679 427 719 480C744 435 756 387 756 336ZM276 39C243 26 207 21 164 21C106 21 77 32 77 54C77 75 125 79 144 79C191 79 227 66 276 39Z"></path></g><g transform="matrix(.013,0,0,-0.013,16.003,0)"><path d="M275 270C275 450 212 609 64 710L45 686C145 604 203 442 203 270S147 -63 45 -147L64 -170C213 -68 275 89 275 270Z"></path></g></svg> expires.</div>

Journal of Computer Networks and Communications

fig6

Figure 6

Figure 6: Development of an AI-Enabled Q-Agent for Making Data Offloading Decisions in a Multi-RAT Wireless Network 