Cooperative Behaviours with Swarm Intelligence in Multirobot Systems for Safety Inspections in Underground Terrains

Table 10

QLACS Example II cooperative behaviour.

Starting simulation Robot 1; Starting state: F; (1) Use (9); 𝛾(𝐹)=0.5=[1+1]0.52=0.25; (2) Check the 𝑄-value for state 𝐹 (use (11)); 𝑄(𝐹,𝑎)=0; Selected action, 𝑎 = inspect; (3) use (10); 𝑄(𝐹,𝑎)=50+0.25(0)=50; End of value iteration	Current state: E, Robot 1; (1) Use (9); 𝛾(𝐸)=0.5=[1+1]0.52=0.25; (2) Check the Q-value for state 𝐸 (Use (11)); 𝑄(𝐸,𝑎)=0; Selected action, 𝑎 = inspect; (3) Use (10); 𝑄(𝐸,𝑎)=50+0.25(max(0,0,0))=50; End of value iteration	Current state: C, Robot 2; (1) Use (9); 𝛾(𝐶)=0.5=[1+1]0.52=0.25; (2) Check the Q-value for state 𝐶 (use (11)); 𝑄(𝐶,𝑎)=0; Selected action, 𝑎 = inspect; (3) Use (10); 𝑄(𝐶,𝑎)=50+0.25(max(0,0,0))=50; End of value iteration
Current state: B, Robot 2; (1) Use (9); 𝛾(𝐵)=0.5=[1+1]0.52=0.25; (2) Check the 𝑄-value for state 𝐵 (use (11)); 𝑄(𝐵,𝑎)=0; Selected action, 𝑎 = inspect; (3) Use (10); 𝑄(𝐵,𝑎)=50+0.25(max(0,0,0))=50; End of value iteration	Current state: D, Robot 1; (1) Use (9); 𝛾(𝐷)=0.5=[1+1]0.52=0.25; (2) Broadcast (use (11)); 𝑄(𝐷,𝑎)=0; Selected action, 𝑎 = inspect; (3) Use (10); 𝑄(𝐷,𝑎)=50+0.25(max(0,0,0))=50; End of value iteration	Current State: A, Robot 2; (1) Use (9); 𝛾(𝐴)=0.5=[1+1]0.52=0.25; (2) Broadcast (use (11)); 𝑄(𝐴,𝑎)=0; Selected action, 𝑎 = inspect; (3) Use (10); 𝑄(𝐴,𝑎)=50+0.25(max(0,0,0))=50; End of value iteration
Current state: A, Robot 1; (1) Use (9); 𝛾(𝐴)=0.5=[1+0.25]0.51.25=0.4; (2) Broadcast (use (11)); 𝑄(𝐴,𝑎)=50, that is, 𝑄>0; Selected action, 𝑎 = Ignore; (3) Use (10); 𝑄(𝐴,𝑎)=100+0.4(max(0,0,0))=100; End of value iteration	Current state: D, Robot 2; (1) Use (9); 𝛾(𝐷)=0.5=[1+0.25]0.51.25=0.4; (2) Broadcast (use (11)); 𝑄(𝐷,𝑎)=50, that is, 𝑄>0; Selected action, 𝑎 = ignore; (3) Use (10); 𝑄(𝐷,𝑎)=100+0.4(max(0,0,0))=100; End of value iteration	Current state: B, Robot 1; (1) Use (9); 𝛾(𝐵)=0.5=[1+0.25]0.51.25=0.4; (2) Broadcast (use (11)); 𝑄(𝐵,𝑎)=50, that is, 𝑄>0; Selected action, 𝑎 = Ignore; (3) Use (10); 𝑄(𝐵,𝑎)=100+0.4(max(0,0,0))=100; End of value iteration
Current state: F, Robot 2; (1) Use (9); 𝛾(𝐷)=0.5=[1+0.25]0.51.25=0.4; (2) Broadcast (use (11)); 𝑄(𝐹,𝑎)=50, that is, 𝑄>0; Selected action, 𝑎 = ignore; (3) Use (10); 𝑄(𝐹,𝑎)=100+0.4(max(0,0,0))=100; End of value iteration	Current state: C, Robot 1; (1) Use (9); 𝛾(𝐵)=0.5=[1+0.25]0.51.25=0.4; (2) Broadcast (use (11)); 𝑄(𝐵,𝑎)=50, that is, 𝑄>0; Selected action, 𝑎 = Ignore; (3) Use (10); 𝑄(𝐵,𝑎)=100+0.4(max(0,0,0))=100; End of value iteration	Current state: E, Robot 2; (1) Use (9); 𝛾(𝐸)=0.5=[1+0.25]0.51.25=0.4; (2) Broadcast (use (11)); 𝑄(𝐸,𝑎)=50, that is, 𝑄>0; Selected action, 𝑎 = ignore; (3) Use (10); 𝑄(𝐸,𝑎)=100+0.4(max(0,0,0))=100; End of value iteration
Current state: G, Robot 1; (1) Use (9); 𝛾(𝐺)=0.5=[1+1]0.52=0.25; (2) Broadcast (Use (11)); 𝑄(𝐺,𝑎)=0; Selected action, 𝑎 = Inspect; (3) Use (10) 𝑄(𝐺,𝑎)=50+0.25(max(0,0,0))=50; End of value iteration	Current state: G, Robot 2; (1) Use (9); 𝛾(𝐺)=0.5=[1+0.25]0.51.25=0.4; (2) Broadcast (use (11)); 𝑄(𝐺,𝑎)=50, that is, 𝑄>0; Selected action, 𝑎 = ignore; (3) Use (10); 𝑄(𝐺,𝑎)=100+0.4(max(50,50,50)); =100+20=120; End of value iteration	Current state: C, Robot 1; (1) Use (9); 𝛾(𝐶)=0.5=[1+0.4]0.51.4=0.36; (2) Broadcast (Use (11)); 𝑄(𝐶,𝑎)=50; Selected action, 𝑎 = Ignore; (3) Use (10); 𝑄(𝐶,𝑎)=100+0.36(max(50,50)); =100+18=118; End of value iteration
Current state: C, Robot 2; (1) Use (9); 𝛾(𝐶)=0.5=[1+0.36]0.51.36=0.37; (2) Broadcast (Use (11)); 𝑄(𝐶,𝑎)=50, that is, 𝑄>0; Selected action, 𝑎 = ignore; (3) Use (10); 𝑄(𝐶,𝑎)=100+0.37(max(50,50)); =100+0.37∗50=118.5; End of value iteration	All QLACS  1 states exhausted; Goal state: H, Robot  1; (1) Use (9); 𝛾(𝐻)=0.5=[1+1]0.52=0.25; (2) Broadcast (use (11)); 𝑄(𝐻,𝑎)=0; Selected action, 𝑎 = Shutdown; (3) Use (10); 𝑄(𝐻,𝑎)=150+0.25(max(0,0))=150; End of value iteration	All QLACS  2 states exhausted; Goal state: H, Robot  2; (4) Use (9); 𝛾(𝐻)=0.5=[1+1]0.52=0.25; (5) Broadcast (use (11)); 𝑄(𝐻,𝑎)=0; Selected action, 𝑎 = Shutdown; (6) Use (10); 𝑄(𝐻,𝑎)=150+0.25(max(0,0))=150; End of value iteration
End of policy iteration	Robot 1	Robot 2
	𝐈𝐧𝐬𝐩𝐞𝐜𝐭𝐈𝐠𝐧𝐨𝐫𝐞𝐒𝐡𝐮𝐭𝐝𝐨𝐰𝐧𝐹YesNoNo𝐸YesNoNo𝐷YesNoNo𝐴NoYesNo𝐵NoYesNo𝐶NoYesNo𝐺YesNoNo𝐶NoYesYesthrough𝐻	𝐈𝐧𝐬𝐩𝐞𝐜𝐭𝐈𝐠𝐧𝐨𝐫𝐞𝐒𝐡𝐮𝐭𝐝𝐨𝐰𝐧𝐶YesNoNo𝐵YesNono𝐴YesNoNo𝐷NoYesNo𝐹NoYesNo𝐸NoYesNo𝐺NoYesno𝐶NoYesYesthrough𝐻