Horseshoe Chaos in a 3D Neural Network with Different Activation Functions

<table>The attractor we observed and the poincaré plane <svg height="15.4" id="M81" style="vertical-align:-3.3907pt" version="1.1" viewbox="0 0 81.512497 15.4" width="81.512497" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,11.112)"><path d="M619 482q0 -69 -40.5 -119t-96 -72.5t-118.5 -26.5h-44l-70 20l-31 -151q-14 -67 0.5 -83t89.5 -22l-5 -28h-287l8 28q65 7 81.5 22t29.5 83l79 398q12 56 0.5 70.5t-78.5 20.5l7 28h255q108 0 164 -43t56 -125zM524 478q0 141 -146 141q-25 0 -47 -8q-16 -6 -20.5 -13.5
t-10.5 -39.5l-43 -241q37 -13 83 -13q67 0 125.5 45t58.5 129z" id="x1D443"></path></g><g transform="matrix(.017,-0,0,-.017,15.582,11.112)"><path d="M223 355q0 -25 -15.5 -41.5t-39.5 -16.5t-40 16.5t-16 41.5t16 41t40 16t39.5 -16t15.5 -41zM223 46q0 -26 -16 -42t-40 -16t-39.5 16t-15.5 42q0 24 16 40.5t39 16.5q24 0 40 -16.5t16 -40.5z" id="x2236"></path></g><g transform="matrix(.017,-0,0,-.017,25.986,11.112)"><path d="M536 404q0 -17 -13.5 -31.5t-26.5 -14.5q-8 0 -15 10q-11 14 -25 14q-22 0 -67 -50q-47 -52 -68 -82l37 -102q31 -88 55 -88t78 59l16 -23q-32 -48 -68.5 -78t-65.5 -30q-19 0 -37.5 20t-29.5 53l-41 116q-72 -106 -114.5 -147.5t-79.5 -41.5q-21 0 -34.5 14t-13.5 37
q0 16 13.5 31.5t28.5 15.5q12 0 17 -11q5 -10 25 -10q22 0 57.5 36t89.5 111l-40 108q-22 58 -36 58q-21 0 -67 -57l-19 20q81 107 125 107q17 0 30 -22t39 -88l22 -55q68 92 108.5 128.5t74.5 36.5q20 0 32.5 -14t12.5 -30z" id="x1D465"></path></g>
<g transform="matrix(.012,-0,0,-.012,35.5,15.187)"><path d="M285 378v-2q65 -13 102 -54.5t37 -97.5q0 -57 -30.5 -104.5t-74 -75t-85.5 -42t-72 -14.5q-31 0 -59.5 11t-40.5 23q-19 18 -16 36q1 16 23 33q13 10 24 0q58 -51 124 -51q55 0 88 40t33 112q0 64 -39 96.5t-88 32.5q-29 0 -64 -11l-6 29q77 25 118 57.5t41 84.5
q0 45 -26.5 69.5t-68.5 24.5q-67 0 -120 -79l-20 20l43 63q51 56 127 56h1q66 0 107 -37t41 -95q0 -42 -31 -71q-22 -23 -68 -54z" id="x33"></path></g>
<g transform="matrix(.017,-0,0,-.017,46.55,11.112)"><path d="M535 323h-483v50h483v-50zM535 138h-483v50h483v-50z" id="x3D"></path></g><g transform="matrix(.017,-0,0,-.017,61.237,11.112)"><path d="M241 635q53 0 94 -28.5t63.5 -76t33.5 -102.5t11 -116q0 -58 -11 -112.5t-34 -103.5t-63.5 -78.5t-94.5 -29.5t-95 28t-64.5 75t-34.5 102.5t-11 118.5q0 58 11.5 112.5t34.5 103t64.5 78t95.5 29.5zM238 602q-32 0 -55.5 -25t-35.5 -68t-17.5 -91t-5.5 -105
q0 -76 10 -138.5t37 -107.5t69 -45q32 0 55.5 25t35.5 68.5t17.5 91.5t5.5 105t-5.5 105.5t-18 92t-36 68t-56.5 24.5z" id="x30"></path></g><g transform="matrix(.017,-0,0,-.017,69.396,11.112)"><path d="M113 -12q-24 0 -39.5 16t-15.5 42q0 24 16 40.5t40 16.5t40 -16.5t16 -40.5q0 -26 -16 -42t-41 -16z" id="x2E"></path></g><g transform="matrix(.017,-0,0,-.017,73.272,11.112)"><path d="M412 140l28 -9q0 -2 -35 -131h-373v23q112 112 161 170q59 70 92 127t33 115q0 63 -31 98t-86 35q-75 0 -137 -93l-22 20l57 81q55 59 135 59q69 0 118.5 -46.5t49.5 -122.5q0 -62 -29.5 -114t-102.5 -130l-141 -149h186q42 0 58.5 10.5t38.5 56.5z" id="x32"></path></g>
</svg>.</table>

Discrete Dynamics in Nature and Society

fig6

Figure 6

Figure 6: Horseshoe Chaos in a 3D Neural Network with Different Activation Functions 