Simplified Information Maximization for Improving Generalization Performance in Multilayered Neural Networks

<div>Targets <svg height="11.4899pt" id="M29" style="vertical-align:-5.52899pt" version="1.1" viewbox="-0.0498162 -5.96091 10.7105 11.4899" width="10.7105pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M516 277C516 378 460 448 376 448C335 448 288 434 240 401C158 345 127 277 107 159C53 -161 38 -221 23 -247L28 -261C43 -260 95 -250 114 -226C122 -216 129 -199 146 -75C152 -29 157 0 164 31C182 9 206 -12 237 -12C267 -12 325 8 376 46C451 102 516 179 516 277ZM432 259C432 134 366 40 259 40C236 40 204 48 171 76L195 203C219 327 253 400 326 400C398 400 432 336 432 259Z" id="g113-239"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="539" vert-adv-y="539"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,6.162,3.132)"><path d="M400 606C400 634 383 656 353 656C316 656 294 620 294 593C294 564 317 545 343 545C375 545 400 573 400 606ZM366 351C379 413 381 451 356 451C323 451 251 408 183 341L199 313C223 335 267 365 277 365C285 365 284 354 277 312C245 132 222 27 193 -100C182 -148 160 -188 131 -188C113 -188 90 -178 75 -170C64 -164 55 -168 48 -175C38 -185 24 -203 24 -222S48 -257 71 -257C89 -257 131 -241 186 -192C243 -141 286 -46 310 74L366 351Z" id="g50-107"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="430" vert-adv-y="430"></glyph.data></g></svg> when the spread parameter <svg height="6.34998pt" id="M30" style="vertical-align:-0.2063899pt" version="1.1" viewbox="-0.0498162 -6.14359 7.47218 6.34998" width="7.47218pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M548 455L523 469C505 448 490 439 461 439C416 439 374 447 316 447C144 447 23 310 23 161C23 47 89 -12 179 -12C312 -12 429 106 429 244C429 303 419 344 368 386L371 388C404 383 440 378 474 378C507 378 530 411 548 455ZM350 274C350 191 308 25 198 25C144 25 108 75 108 157C108 264 170 395 276 395C296 395 320 384 333 360C347 334 350 316 350 274Z" id="g113-240"></path><glyph.data ascent="3473" descent="-2876" horiz-adv-x="565" vert-adv-y="565"></glyph.data></g></svg> increases from 0.01 to 50.</div>

Mathematical Problems in Engineering

fig3

Figure 3

Figure 3: Simplified Information Maximization for Improving Generalization Performance in Multilayered Neural Networks 