Horseshoe Chaos in a 3D Neural Network with Different Activation Functions

<table>The plot of the piecewise function <svg height="16.6625" id="M80" style="vertical-align:-3.56265pt" version="1.1" viewbox="0 0 36.174999 16.6625" width="36.174999" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,12.162)"><path d="M619 670q0 -13 -9 -26t-18 -19q-13 -10 -25 2q-36 38 -66 38q-31 0 -54.5 -50t-45.5 -185h120l-20 -31l-107 -12q-23 -138 -57 -293q-27 -122 -55 -184.5t-75 -109.5q-60 -61 -114 -61q-25 0 -47.5 15t-22.5 31q0 17 31 44q11 8 20 -1q10 -11 31 -19t35 -8q26 0 47 19
q34 34 71 253l54 315h-90l-3 12l31 30h70q28 138 90 204q35 37 75 57.5t70 20.5q26 0 45 -14t19 -28z" id="x1D453"></path></g><g transform="matrix(.017,-0,0,-.017,10.976,12.162)"><path d="M300 -147l-18 -23q-106 71 -159 185.5t-53 254.5v1q0 139 53 252.5t159 186.5l18 -24q-74 -62 -115.5 -173.5t-41.5 -242.5q0 -130 41.5 -242.5t115.5 -174.5z" id="x28"></path></g><g transform="matrix(.017,-0,0,-.017,16.857,12.162)"><path d="M536 404q0 -17 -13.5 -31.5t-26.5 -14.5q-8 0 -15 10q-11 14 -25 14q-22 0 -67 -50q-47 -52 -68 -82l37 -102q31 -88 55 -88t78 59l16 -23q-32 -48 -68.5 -78t-65.5 -30q-19 0 -37.5 20t-29.5 53l-41 116q-72 -106 -114.5 -147.5t-79.5 -41.5q-21 0 -34.5 14t-13.5 37
q0 16 13.5 31.5t28.5 15.5q12 0 17 -11q5 -10 25 -10q22 0 57.5 36t89.5 111l-40 108q-22 58 -36 58q-21 0 -67 -57l-19 20q81 107 125 107q17 0 30 -22t39 -88l22 -55q68 92 108.5 128.5t74.5 36.5q20 0 32.5 -14t12.5 -30z" id="x1D465"></path></g>
<g transform="matrix(.012,-0,0,-.012,26.363,16.25)"><path d="M244 607q0 -25 -15.5 -43t-37.5 -18q-19 0 -32 13t-13 35q0 21 15 41t39 20q20 0 32 -14t12 -34zM222 91q-29 -33 -79 -68t-75 -35q-13 0 -19 7.5t-6 31t10 65.5l62 253q5 26 -1 26q-21 0 -72 -43l-13 24q43 40 91 68t71 28q30 0 10 -78l-71 -274q-8 -30 3 -30
q16 0 76 48z" id="x1D456"></path></g>
<g transform="matrix(.017,-0,0,-.017,30.225,12.162)"><path d="M275 270q0 -296 -211 -440l-19 23q75 62 116.5 174t41.5 243t-42 243t-116 173l19 24q211 -144 211 -440z" id="x29"></path></g>
</svg>.</table>

Discrete Dynamics in Nature and Society

fig5

Figure 5

Figure 5: Horseshoe Chaos in a 3D Neural Network with Different Activation Functions 