Diagonal Hessian Approximation for Limited Memory Quasi-Newton via Variational Principle

<table class="table-group" id="tab2"><tr><td><table class="table"><tr><td class="thead-hr" colspan="3"><hr/></td></tr><tr class="thead"><td align="left">LBFGS-I </td><td align="center"> LBFGS-<svg height="7.9499998" id="M214" style="vertical-align:-0.10033pt" version="1.1" viewbox="0 0 13.5375 7.9499998" width="13.5375" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,7.675)"><path d="M766 88q-40 -45 -83.5 -72.5t-63.5 -27.5q-39 0 -16 103l49 224q15 68 -10 68q-36 0 -112.5 -78.5t-105.5 -133.5q-27 -101 -38 -165q-35 -4 -77 -18l-6 6q42 153 70 288q12 55 10 78t-19 23q-38 0 -114 -80.5t-101 -131.5q-24 -68 -41 -165q-36 -3 -76 -18l-8 6
q56 202 87 347q7 33 -3 33q-8 0 -31.5 -17.5t-41.5 -35.5l-12 28q42 45 84 72t63 27q44 0 10 -124l-20 -75h2q92 121 193 178q36 21 64 21q62 0 28 -159l-8 -37h2q50 67 103.5 112t94.5 65q39 19 62 19q60 0 23 -156l-45 -189q-9 -39 2 -39q17 0 71 49z" id="x1D45A"></path></g>
</svg>I </td><td align="center"> LBFGS-USD </td></tr><tr><td class="thead-hr" colspan="3"><hr/></td></tr><tr><td align="left">1.822477996 </td><td align="center">1.078519988 </td><td align="center">1.183809586 </td></tr><tr class="table-tr"><td colspan="3"><hr class="tbody-hr"/></td></tr></table></td></tr></table>

Ratio of function/gradient calls over iteration counts for <svg height="11.125" id="M213" style="vertical-align:-0.17555pt" version="1.1" viewbox="0 0 41.112499 11.125" width="41.112499" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,10.862)"><path d="M766 88q-40 -45 -83.5 -72.5t-63.5 -27.5q-39 0 -16 103l49 224q15 68 -10 68q-36 0 -112.5 -78.5t-105.5 -133.5q-27 -101 -38 -165q-35 -4 -77 -18l-6 6q42 153 70 288q12 55 10 78t-19 23q-38 0 -114 -80.5t-101 -131.5q-24 -68 -41 -165q-36 -3 -76 -18l-8 6
q56 202 87 347q7 33 -3 33q-8 0 -31.5 -17.5t-41.5 -35.5l-12 28q42 45 84 72t63 27q44 0 10 -124l-20 -75h2q92 121 193 178q36 21 64 21q62 0 28 -159l-8 -37h2q50 67 103.5 112t94.5 65q39 19 62 19q60 0 23 -156l-45 -189q-9 -39 2 -39q17 0 71 49z" id="x1D45A"></path></g><g transform="matrix(.017,-0,0,-.017,18.183,10.862)"><path d="M535 323h-483v50h483v-50zM535 138h-483v50h483v-50z" id="x3D"></path></g><g transform="matrix(.017,-0,0,-.017,32.887,10.862)"><path d="M285 378v-2q65 -13 102 -54.5t37 -97.5q0 -57 -30.5 -104.5t-74 -75t-85.5 -42t-72 -14.5q-31 0 -59.5 11t-40.5 23q-19 18 -16 36q1 16 23 33q13 10 24 0q58 -51 124 -51q55 0 88 40t33 112q0 64 -39 96.5t-88 32.5q-29 0 -64 -11l-6 29q77 25 118 57.5t41 84.5
q0 45 -26.5 69.5t-68.5 24.5q-67 0 -120 -79l-20 20l43 63q51 56 127 56h1q66 0 107 -37t41 -95q0 -42 -31 -71q-22 -23 -68 -54z" id="x33"></path></g>
</svg>.

Journal of Applied Mathematics

tab2

Table 2

Table 2: Diagonal Hessian Approximation for Limited Memory Quasi-Newton via Variational Principle 