Horseshoe Chaos in a 3D Neural Network with Different Activation Functions

<table>The phase portrait of (<a href="https://static.hindawi.com/articles/ddns/volume-2013/430963/figures/#EEq5">5</a>) with <svg height="16.6625" id="M61" style="vertical-align:-3.56265pt" version="1.1" viewbox="0 0 104.0125 16.6625" width="104.0125" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,12.162)"><path d="M619 670q0 -13 -9 -26t-18 -19q-13 -10 -25 2q-36 38 -66 38q-31 0 -54.5 -50t-45.5 -185h120l-20 -31l-107 -12q-23 -138 -57 -293q-27 -122 -55 -184.5t-75 -109.5q-60 -61 -114 -61q-25 0 -47.5 15t-22.5 31q0 17 31 44q11 8 20 -1q10 -11 31 -19t35 -8q26 0 47 19
q34 34 71 253l54 315h-90l-3 12l31 30h70q28 138 90 204q35 37 75 57.5t70 20.5q26 0 45 -14t19 -28z" id="x1D453"></path></g><g transform="matrix(.017,-0,0,-.017,10.976,12.162)"><path d="M300 -147l-18 -23q-106 71 -159 185.5t-53 254.5v1q0 139 53 252.5t159 186.5l18 -24q-74 -62 -115.5 -173.5t-41.5 -242.5q0 -130 41.5 -242.5t115.5 -174.5z" id="x28"></path></g><g transform="matrix(.017,-0,0,-.017,16.857,12.162)"><path d="M536 404q0 -17 -13.5 -31.5t-26.5 -14.5q-8 0 -15 10q-11 14 -25 14q-22 0 -67 -50q-47 -52 -68 -82l37 -102q31 -88 55 -88t78 59l16 -23q-32 -48 -68.5 -78t-65.5 -30q-19 0 -37.5 20t-29.5 53l-41 116q-72 -106 -114.5 -147.5t-79.5 -41.5q-21 0 -34.5 14t-13.5 37
q0 16 13.5 31.5t28.5 15.5q12 0 17 -11q5 -10 25 -10q22 0 57.5 36t89.5 111l-40 108q-22 58 -36 58q-21 0 -67 -57l-19 20q81 107 125 107q17 0 30 -22t39 -88l22 -55q68 92 108.5 128.5t74.5 36.5q20 0 32.5 -14t12.5 -30z" id="x1D465"></path></g><g transform="matrix(.017,-0,0,-.017,26.36,12.162)"><path d="M275 270q0 -296 -211 -440l-19 23q75 62 116.5 174t41.5 243t-42 243t-116 173l19 24q211 -144 211 -440z" id="x29"></path></g><g transform="matrix(.017,-0,0,-.017,36.95,12.162)"><path d="M535 323h-483v50h483v-50zM535 138h-483v50h483v-50z" id="x3D"></path></g><g transform="matrix(.017,-0,0,-.017,51.654,12.162)"><path d="M298 36l-68 -35q-25 -13 -42 -13q-98 0 -98 119v290h-69l-4 11l28 29h45v72l65 67l14 -2v-137h113q8 -8 6 -21t-12 -19h-107v-256q0 -96 59 -96q29 0 61 17z" id="x74"></path></g><g transform="matrix(.017,-0,0,-.017,56.838,12.162)"><path d="M433 39l-85 -51q-27 0 -47 20q-16 16 -24 44q-5 -3 -24 -16.5t-27 -18.5t-23 -13.5t-25 -12t-19 -3.5q-52 0 -86 36.5t-34 85.5q0 71 81 99q128 43 155 65v17q0 54 -23 83.5t-62 29.5q-28 0 -45 -19t-29 -66q-7 -23 -29 -23q-15 0 -29 13t-14 30t31 40q25 19 68.5 40.5
t81.5 29.5q49 0 82 -27q45 -39 45 -123v-185q0 -60 35 -60q18 0 36 11zM275 84v156q-12 -6 -49 -23t-41 -19q-31 -14 -46 -31t-15 -43q0 -35 22.5 -55.5t48.5 -20.5q19 0 42 10.5t38 25.5z" id="x61"></path></g><g transform="matrix(.017,-0,0,-.017,64.301,12.162)"><path d="M524 0h-209v26q44 5 55 18t11 65v167q0 114 -91 114q-50 0 -109 -51v-235q0 -50 10 -61.5t55 -16.5v-26h-217v26q51 5 62 17t11 61v206q0 47 -9.5 58.5t-50.5 19.5v23q82 15 139 40v-79l67 47q47 30 83 30q60 0 94.5 -40t34.5 -112v-193q0 -50 10 -61.5t54 -16.5v-26z
" id="x6E"></path></g><g transform="matrix(.017,-0,0,-.017,73.599,12.162)"><path d="M513 0h-211v26q44 5 55 17t11 60v166q0 123 -98 123q-59 0 -104 -51v-238q0 -48 10 -60.5t55 -16.5v-26h-213l1 26q46 4 57 16t11 61v477q0 48 -9.5 58.5t-58.5 14.5v24q63 9 147 35v-336q73 73 143 73q64 0 101 -42t37 -129v-175q0 -49 11 -61t55 -16v-26z" id="x68"></path></g><g transform="matrix(.017,-0,0,-.017,82.676,12.162)"><path d="M300 -147l-18 -23q-106 71 -159 185.5t-53 254.5v1q0 139 53 252.5t159 186.5l18 -24q-74 -62 -115.5 -173.5t-41.5 -242.5q0 -130 41.5 -242.5t115.5 -174.5z" id="x28"></path></g><g transform="matrix(.017,-0,0,-.017,88.558,12.162)"><path d="M536 404q0 -17 -13.5 -31.5t-26.5 -14.5q-8 0 -15 10q-11 14 -25 14q-22 0 -67 -50q-47 -52 -68 -82l37 -102q31 -88 55 -88t78 59l16 -23q-32 -48 -68.5 -78t-65.5 -30q-19 0 -37.5 20t-29.5 53l-41 116q-72 -106 -114.5 -147.5t-79.5 -41.5q-21 0 -34.5 14t-13.5 37
q0 16 13.5 31.5t28.5 15.5q12 0 17 -11q5 -10 25 -10q22 0 57.5 36t89.5 111l-40 108q-22 58 -36 58q-21 0 -67 -57l-19 20q81 107 125 107q17 0 30 -22t39 -88l22 -55q68 92 108.5 128.5t74.5 36.5q20 0 32.5 -14t12.5 -30z" id="x1D465"></path></g><g transform="matrix(.017,-0,0,-.017,98.06,12.162)"><path d="M275 270q0 -296 -211 -440l-19 23q75 62 116.5 174t41.5 243t-42 243t-116 173l19 24q211 -144 211 -440z" id="x29"></path></g>
</svg> and <svg height="12.4375" id="M62" style="vertical-align:-0.1638pt" version="1.1" viewbox="0 0 44.412498 12.4375" width="44.412498" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink">
<g transform="matrix(.017,-0,0,-.017,.062,12.162)"><path d="M480 416q0 -21 -18 -41q-9 -11 -17 -7q-20 9 -42 9q-62 0 -140 -78q23 -69 88 -192q17 -31 27 -42t20 -11q16 0 62 46l17 -20q-64 -92 -119 -92q-35 0 -70 66q-41 73 -84 187q-36 -30 -62 -61q-27 -115 -35 -172q-41 -8 -78 -20l-6 6l140 612q7 28 0.5 34t-37.5 7l-34 1
l5 26q38 4 74 13.5t57 17t25 7.5q12 0 4 -32l-104 -443h2q35 38 97 93q39 35 65.5 56t62 41.5t58.5 20.5q19 0 30.5 -10t11.5 -22z" id="x1D458"></path></g><g transform="matrix(.017,-0,0,-.017,13.322,12.162)"><path d="M535 323h-483v50h483v-50zM535 138h-483v50h483v-50z" id="x3D"></path></g><g transform="matrix(.017,-0,0,-.017,28.025,12.162)"><path d="M137 343l67 33q37 17 63 17q79 0 129.5 -53t50.5 -131q0 -92 -58 -156.5t-147 -64.5t-147 68t-58 182q0 63 17 119t43 95.5t61.5 72t69 52t67.5 31.5q62 22 128 33l6 -32q-56 -11 -108 -35q-149 -71 -184 -231zM227 337q-47 0 -95 -27q-6 -23 -6 -70q0 -93 36 -155.5
t96 -62.5q53 0 78 45.5t25 105.5q0 68 -35 116t-99 48z" id="x36"></path></g><g transform="matrix(.017,-0,0,-.017,36.185,12.162)"><path d="M241 635q53 0 94 -28.5t63.5 -76t33.5 -102.5t11 -116q0 -58 -11 -112.5t-34 -103.5t-63.5 -78.5t-94.5 -29.5t-95 28t-64.5 75t-34.5 102.5t-11 118.5q0 58 11.5 112.5t34.5 103t64.5 78t95.5 29.5zM238 602q-32 0 -55.5 -25t-35.5 -68t-17.5 -91t-5.5 -105
q0 -76 10 -138.5t37 -107.5t69 -45q32 0 55.5 25t35.5 68.5t17.5 91.5t5.5 105t-5.5 105.5t-18 92t-36 68t-56.5 24.5z" id="x30"></path></g>
</svg>.</table>

Discrete Dynamics in Nature and Society

fig3

Figure 3

Figure 3: Horseshoe Chaos in a 3D Neural Network with Different Activation Functions 