A Novel Neuron in Kernel Domain

Algorithm 1

The adaptive KNN algorithm (AKNN_μ). Complexity for each instance.

Initialization:

learning step 𝜂

learning kernel width step 𝛾

primal kernel width 𝜎

sparsity threshold μ

𝑡=1;

While  {𝐱(𝑡),𝐝(𝑡)} available do

 for  𝑙 instances do

  % compute distances of dictionary instances to tth instance

  𝐝𝐢𝐬_𝑖=𝑥_𝑖−𝑥_𝑡²;                   𝑂(𝑚_𝑡−1)

  % compute kernel vector, output and error for tth instance

  𝐤=exp(−𝐝𝐢𝐬^𝑇/𝜎²),𝑦_𝑡=𝑓(𝜂𝜶^𝑇𝐤),𝐞_𝑡=𝐝_𝑡−𝐲_𝑡        𝑂(𝑚_𝑡−1)

  % compute coefficient 𝜶_𝑡 using (19)

  % save 𝜎_pred for 𝜎 tracking

  𝜎_pred=avg[𝜎_pred,avg(𝐝𝐢𝐬)]

  % update kernel width σ

     𝜎(𝑡)=𝜎(𝑡−1)+𝛾𝜶_𝑡𝜶^𝑇 (𝐝𝐢𝐬/𝜎³⋅𝐤)          𝑂(𝑚_𝑡−1)

     if  𝜎 became negative then

  % decrease 𝛾 until 𝜎 becomes non-negative

  % update kernel width 𝜎 again.

  end    if

  if (max(𝐤(𝑡−1))≤𝜇₀)

  % add 𝐱_𝑡 to dictionary and update 𝜶 using (23);

  else

  % update Coefficients 𝜶 using (24);

  end if

  𝑡=𝑡+1

end for

if ( mse_𝑙≥𝛿 )

  % manipulated step size 𝛾 using (20)

else if (mse_𝑙−1−mse_𝑙<𝜀)

 % exit training process

end if

end while