Development of an AI-Enabled Q-Agent for Making Data Offloading Decisions in a Multi-RAT Wireless Network

<div>The number of visits by the Q-agent to selected states <svg height="12.6708pt" id="M163" style="vertical-align:-3.38247pt" version="1.1" viewbox="-0.0498162 -9.28833 97.3519 12.6708" width="97.3519pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M352 391C352 416 319 448 267 448C236 448 173 423 147 400C107 364 96 332 96 304C96 248 143 210 193 181C241 153 258 124 258 100C258 72 232 38 184 38C151 38 107 66 81 108C77 114 64 116 55 111C34 99 23 84 23 65C23 29 81 -12 134 -12C220 -12 325 61 325 141C325 184 297 215 234 256C194 282 161 309 161 346C161 380 188 401 217 401C255 401 279 380 301 353C308 344 313 341 325 347C341 355 352 371 352 391Z"></path></g><g transform="matrix(.013,0,0,-0.013,4.875,0)"><path d="M300 -147C201 -63 143 98 143 270S200 602 300 686L282 710C136 610 70 450 70 271V270C70 89 136 -72 282 -170L300 -147Z"></path></g><g transform="matrix(.013,0,0,-0.013,9.373,0)"><path d="M480 416C480 431 465 448 438 448C388 448 312 383 252 330C217 299 188 273 155 237H153L257 680C262 700 263 712 253 712C240 712 183 684 97 674L92 648L126 647C166 646 172 645 163 606L23 -6L29 -12C51 -5 77 2 107 8C115 62 130 128 142 180C153 193 179 220 204 241C231 170 259 106 288 54C317 0 336 -12 358 -12C381 -12 423 2 477 80L460 100C434 74 408 54 398 54C385 54 374 65 351 107C326 154 282 241 263 299C296 332 351 377 403 377C424 377 436 372 445 368C449 366 456 368 462 375C472 386 480 402 480 416Z"></path></g><g transform="matrix(.013,0,0,-0.013,19.544,0)"><path d="M535 323V373H52V323H535ZM535 138V188H52V138H535Z"></path></g><g transform="matrix(.013,0,0,-0.013,30.807,0)"><path d="M300 -147C201 -63 143 98 143 270S200 602 300 686L282 710C136 610 70 450 70 271V270C70 89 136 -72 282 -170L300 -147Z"></path></g><g transform="matrix(.013,0,0,-0.013,35.305,0)"><path d="M383 397C383 424 344 448 299 448C244 448 172 409 132 375C66 319 23 227 23 146C23 42 74 -12 146 -12C208 -12 298 30 359 103L343 124C315 95 248 48 192 48C145 48 111 85 111 163C111 228 129 294 151 330C171 363 201 401 241 401C275 401 302 384 325 356C332 347 339 344 348 348C373 360 383 381 383 397Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,39.478,3.132)"><path d="M481 445L456 468C426 437 403 427 388 427C372 427 356 428 302 438C256 447 221 451 187 451C159 451 130 428 105 384C93 363 80 340 65 305L96 290C122 342 149 381 193 381C216 381 267 377 294 374S346 372 365 376L75 105C37 70 24 35 24 16C24 2 30 -5 35 -8C42 -8 46 -6 48 -2C58 14 75 35 90 41C102 46 116 46 147 32C200 8 267 -21 298 -21C347 -21 390 12 448 126L420 151C369 77 338 56 311 56C287 56 249 68 212 81C180 92 149 100 127 98C212 172 353 308 413 372L481 445Z"></path></g><g transform="matrix(.013,0,0,-0.013,44.589,0)"><path d="M95 130C70 130 46 113 46 88C46 72 54 64 59 64C93 55 121 33 121 -3C121 -41 93 -68 44 -88L55 -117C117 -98 186 -56 186 22C186 91 131 130 95 130Z"></path></g><g transform="matrix(.013,0,0,-0.013,49.732,0)"><path d="M689 332C689 394 670 448 646 448C620 448 597 421 597 396C597 386 600 381 608 372C619 359 620 334 620 315C620 150 538 45 454 45C414 45 386 67 386 122C386 138 388 158 394 180L457 426L452 432L377 416L315 156C302 100 259 45 216 45C176 45 148 67 148 122C148 133 152 158 156 180C162 212 173 259 194 332C201 357 206 384 206 405C206 430 198 448 174 448C125 448 66 406 23 342L43 319C84 368 110 383 121 383C126 383 128 382 128 377C128 370 127 359 122 343C99 268 84 204 77 156C74 137 70 111 70 104C70 25 125 -12 180 -12C228 -12 276 12 319 50C338 8 378 -12 418 -12C549 -12 689 166 689 332Z"></path></g><g transform="matrix(.0091,0,0,-0.0091,58.663,3.132)"><path d="M481 445L456 468C426 437 403 427 388 427C372 427 356 428 302 438C256 447 221 451 187 451C159 451 130 428 105 384C93 363 80 340 65 305L96 290C122 342 149 381 193 381C216 381 267 377 294 374S346 372 365 376L75 105C37 70 24 35 24 16C24 2 30 -5 35 -8C42 -8 46 -6 48 -2C58 14 75 35 90 41C102 46 116 46 147 32C200 8 267 -21 298 -21C347 -21 390 12 448 126L420 151C369 77 338 56 311 56C287 56 249 68 212 81C180 92 149 100 127 98C212 172 353 308 413 372L481 445Z"></path></g><g transform="matrix(.013,0,0,-0.013,63.774,0)"><path d="M275 270C275 450 212 609 64 710L45 686C145 604 203 442 203 270S147 -63 45 -147L64 -170C213 -68 275 89 275 270Z"></path></g><g transform="matrix(.013,0,0,-0.013,68.272,0)"><path d="M95 130C70 130 46 113 46 88C46 72 54 64 59 64C93 55 121 33 121 -3C121 -41 93 -68 44 -88L55 -117C117 -98 186 -56 186 22C186 91 131 130 95 130Z"></path></g><g transform="matrix(.013,0,0,-0.013,73.415,0)"><path d="M499 88L487 113C459 86 422 65 413 65C403 65 403 74 409 98L461 318C487 426 460 448 431 448C408 448 383 441 352 424C305 398 223 344 157 257H155L243 674C249 702 249 712 240 712C227 712 170 680 87 673L82 648H117C155 648 162 644 150 590L23 -4L30 -12C55 -4 81 3 105 8C113 53 131 150 143 187C199 281 323 387 372 387C390 387 393 369 380 311L328 86C313 19 319 -12 347 -12C379 -12 442 24 499 88Z"></path></g><g transform="matrix(.013,0,0,-0.013,80.123,0)"><path d="M95 130C70 130 46 113 46 88C46 72 54 64 59 64C93 55 121 33 121 -3C121 -41 93 -68 44 -88L55 -117C117 -98 186 -56 186 22C186 91 131 130 95 130Z"></path></g><g transform="matrix(.013,0,0,-0.013,85.266,0)"><path d="M530 686C535 705 530 712 521 712C504 712 448 684 359 674L358 648H393C437 648 439 646 429 593L400 435C372 447 345 448 332 448C286 448 194 414 144 373C68 311 23 203 23 111C23 26 57 -12 91 -12C120 -12 147 3 188 29C227 54 290 102 341 170H343L322 71C308 6 320 -12 341 -12C373 -12 442 27 501 96L485 120C455 91 422 67 408 67C401 67 401 76 404 91C440 294 479 473 530 686ZM387 375L355 241C326 187 200 53 142 53C126 53 109 73 109 130C109 217 154 337 218 381C240 396 265 404 297 404S372 390 387 375Z"></path></g><g transform="matrix(.013,0,0,-0.013,92.482,0)"><path d="M275 270C275 450 212 609 64 710L45 686C145 604 203 442 203 270S147 -63 45 -147L64 -170C213 -68 275 89 275 270Z"></path></g></svg> accumulated over all actions.</div>

Journal of Computer Networks and Communications

fig2

Figure 2

Figure 2: Development of an AI-Enabled Q-Agent for Making Data Offloading Decisions in a Multi-RAT Wireless Network 