Two-Phase Iteration for Value Function Approximation and Hyperparameter Optimization in Gaussian-Kernel-Based Adaptive Critic Design

<div>The <svg height="10.7866pt" id="M224" style="vertical-align:-2.150701pt" version="1.1" viewbox="-0.0498162 -8.6359 9.52083 10.7866" width="9.52083pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M699 368C699 549 574 666 407 666C186 666 23 488 23 277C23 113 129 -3 288 -13L307 -26C431 -111 501 -139 533 -147C559 -154 613 -163 658 -164L666 -141C597 -111 507 -66 430 -11L416 -1C580 42 699 190 699 368ZM601 371C601 227 518 54 381 22L354 40L278 24C175 47 120 145 120 269C120 451 235 631 398 631C540 631 601 521 601 371Z" id="g113-82"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="722" vert-adv-y="722"></glyph.data></g></svg> values projected on to dimensions of pole angle (<svg height="9.14241pt" id="M225" style="vertical-align:-3.1815pt" version="1.1" viewbox="-0.0498162 -5.96091 9.8741 9.14241" width="9.8741pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M352 391C352 416 319 448 267 448C236 448 173 423 147 400C107 364 96 332 96 304C96 248 143 210 193 181C241 153 258 124 258 100C258 72 232 38 184 38C151 38 107 66 81 108C77 114 64 116 55 111C34 99 23 84 23 65C23 29 81 -12 134 -12C220 -12 325 61 325 141C325 184 297 215 234 256C194 282 161 309 161 346C161 380 188 401 217 401C255 401 279 380 301 353C308 344 313 341 325 347C341 355 352 371 352 391Z" id="g113-116"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="375" vert-adv-y="375"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,4.81,3.132)"><path d="M389 0V32C297 37 291 46 291 118V635C234 612 176 595 109 583V556L161 554C203 551 207 548 207 497V118C207 46 201 37 110 32V0H389Z" id="g58-47"></path><glyph.data ascent="1007" descent="-369" horiz-adv-x="487" vert-adv-y="487"></glyph.data></g></svg>) and linear displacement (<svg height="9.25658pt" id="M226" style="vertical-align:-3.295671pt" version="1.1" viewbox="-0.0498162 -5.96091 9.8741 9.25658" width="9.8741pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M352 391C352 416 319 448 267 448C236 448 173 423 147 400C107 364 96 332 96 304C96 248 143 210 193 181C241 153 258 124 258 100C258 72 232 38 184 38C151 38 107 66 81 108C77 114 64 116 55 111C34 99 23 84 23 65C23 29 81 -12 134 -12C220 -12 325 61 325 141C325 184 297 215 234 256C194 282 161 309 161 346C161 380 188 401 217 401C255 401 279 380 301 353C308 344 313 341 325 347C341 355 352 371 352 391Z" id="g113-116"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="375" vert-adv-y="375"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,4.81,3.132)"><path d="M290 377C321 398 342 415 358 430C379 451 389 475 389 502C389 578 329 635 238 635H237C184 635 137 610 109 578L64 515L88 493C113 529 154 573 208 573C263 573 303 542 303 482C303 409 233 371 141 341L149 308C165 313 190 319 215 319C272 319 341 284 341 193C342 97 293 43 222 43C162 43 123 71 96 94C89 101 80 101 70 94C61 87 47 73 46 60C44 48 49 36 62 23S118 -12 165 -12C238 -12 430 62 430 223C430 298 379 359 290 375V377Z" id="g58-49"></path><glyph.data ascent="1007" descent="-369" horiz-adv-x="487" vert-adv-y="487"></glyph.data></g></svg>) w.r.t. all samples.</div>

Mathematical Problems in Engineering

fig8

Figure 8

Figure 8: Two-Phase Iteration for Value Function Approximation and Hyperparameter Optimization in Gaussian-Kernel-Based Adaptive Critic Design 