| | Line | | Methods | | | SARSA | RSarsa | PRS | RSarsa.TD | PRS.Beta |
| | Relative computation time/epoch | | 1 | P1 | 1 | 30 | 26 | 5 | 30 | | 2 | P2 | 1 | 20 | 21 | 3 | 19 | | 3 | P3 | 1 | 31 | 29 | 6 | 31 |
| | Average cost of early periods | | 4 | P1 | 8,421 | 7,619 (W) | 8,379 (p0.43) | 7,597 (W) | 7,450 (W) | | 5 | P2 | 4,935 | 4,606 (W) | 4,792 (p0.06) | 4,685 (W) | 4,411 (W) | | 6 | P3 | 10,502 | 8,694 (W) | 9,958 (p0.20) | 9,390 (p0.07) | 8,472 (W) |
| | Average cost of later periods | | 7 | P1 | 7,214 | 7,355 (p0.68) | 7,051 (W) | 7,110 (p0.11) | 7,010 (W) | | 8 | P2 | 4,308 | 4,388 (p0.90) | 4,248 (p0.14) | 4,375 (p0.84) | 4,194 (W) | | 9 | P3 | 8,613 | 8,139 (p0.29) | 8,312 (p0.37) | 8,486 (p0.43) | 7,664 (p0.18) |
|
|
