On Training Efficiency and Computational Costs of a Feed Forward Neural Network: A Review

Table 9

Summary of hybrid and higher order techniques.

Ref.	Method	Convergence	Precision	Computational costs	Notes
[52]	4th-order Taylor	N/A	From 99.68% to 45% accuracy (classification problem)	Full NN computation time: 1.7 ms	Resources used: Slices: 4438; Flip Flops: 2054; LUTs: 8225
[53]	5th-order Taylor	N/A	% error = 0.51	N/A	Resources used: Slices: 4895; Flip Flops: 4777; LUTs: 8820
[54, 55]	Hybrid with PWL and RA-LUT	N/A	Up to for 404 elements	Elaboration time: 40 μs on 50 MHz FPGA	Resources used: Slices: 12; 4 Inputs LUT: 17; BRAM: 1
[54, 55]	Hybrid with PWL and combinatorial	N/A	Up to for 404 elements	Elaboration time: 40 μs on 50 MHz FPGA	Resources used: Slices: 12; 4 inputs LUT: 17; BRAM: 0
[56]	High precision sigmoid/exponential	N/A	RMSE = (sigmoid); RMSE = (exponential)	Maximum operative frequency: 868.056 MHz	Resources used: (as low as); 43 LUTs; 26 registers
[57]	PWL and optimized LUT	N/A	N/A	Propagation delay: 0.06 ns	Resources used: Number of gates: 35; Area (μm^2): 148
[39, 58]	Four-polynomial tanh; “4PY-T”	N/A	MSE = 0.0039	Full NN computation (50 MHz FPGA); 142 μs
[39, 58]	Five-polynomial tanh; “5PY-T”	N/A	MSE = 0.0018	Full NN computation (50 MHz FPGA); 174 μs
[39, 58]	Five-polynomial Logsig; “5PY-L”	N/A	MSE = 0.0075	Full NN computation (50 MHz FPGA); 185 μs
[61]	Piecewise Quadratic Tanh; “scheme 2”	N/A	MEA =	Throughput rate: 0.773 MHz	Resources used: Area (μm^2): 83559.17
[60]	Piecewise Quadratic Tanh; “Gs”	33 Epochs	SE = 0.1; 99.6 generalization capability	N/A
[47, 61]	Zhang quadratic approximation	N/A	MEA =; % error = 1.10	Propagation delay: 3.9 ns	Resources used: Slices: 93; LUTs: 86; Total gates: 1169
[62]	Adjusted LUT; (0.02 max. errors)	N/A	MEA = 0.0121	Propagation delay: 2.80 ns	Area (μm^2): 5130.78
[62]	Adjusted LUT; (0.04 max. errors)	N/A	MEA = 0.0246	Propagation delay: 2.31 ns	Area (μm^2): 3646.83