A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMs

<div>A DIMLP network creating two discriminative hyperplanes. The activation function of neurons <svg height="12.4698pt" id="M7" style="vertical-align:-3.18147pt" version="1.1" viewbox="-0.0498162 -9.28833 11.714 12.4698" width="11.714pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M499 88L487 113C459 86 422 65 413 65C403 65 403 74 409 98L461 318C487 426 460 448 431 448C408 448 383 441 352 424C305 398 223 344 157 257H155L243 674C249 702 249 712 240 712C227 712 170 680 87 673L82 648H117C155 648 162 644 150 590L23 -4L30 -12C55 -4 81 3 105 8C113 53 131 150 143 187C199 281 323 387 372 387C390 387 393 369 380 311L328 86C313 19 319 -12 347 -12C379 -12 442 24 499 88Z" id="g113-105"></path></g><g transform="matrix(.0091,0,0,-0.0091,6.643,3.132)"><path d="M389 0V32C297 38 291 46 291 118V635C234 613 175 595 109 583V556L161 554C203 552 207 547 207 497V118C207 46 201 38 110 32V0H389Z" id="g50-50"></path></g></svg> and <svg height="12.4698pt" id="M8" style="vertical-align:-3.18147pt" version="1.1" viewbox="-0.0498162 -9.28833 11.714 12.4698" width="11.714pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M499 88L487 113C459 86 422 65 413 65C403 65 403 74 409 98L461 318C487 426 460 448 431 448C408 448 383 441 352 424C305 398 223 344 157 257H155L243 674C249 702 249 712 240 712C227 712 170 680 87 673L82 648H117C155 648 162 644 150 590L23 -4L30 -12C55 -4 81 3 105 8C113 53 131 150 143 187C199 281 323 387 372 387C390 387 393 369 380 311L328 86C313 19 319 -12 347 -12C379 -12 442 24 499 88Z" id="g113-105"></path></g><g transform="matrix(.0091,0,0,-0.0091,6.643,3.132)"><path d="M414 144C384 79 371 75 317 75H135L276 221C367 316 408 376 408 465C408 570 327 635 237 635C179 635 131 609 100 575L42 494L67 471C94 510 138 565 205 565C277 565 321 517 321 435C321 348 258 270 195 195C146 137 88 81 33 26V0H411C423 44 433 88 446 135L414 144Z" id="g50-51"></path></g></svg> is a step function, while for output neuron <svg height="9.39034pt" id="M9" style="vertical-align:-3.42943pt" version="1.1" viewbox="-0.0498162 -5.96091 12.0141 9.39034" width="12.0141pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M556 393C556 426 537 448 514 448C496 448 478 435 471 420C468 414 462 401 466 394C473 382 480 368 480 346C480 268 392 143 338 67H336C329 163 319 253 309 330S286 448 254 448C215 448 160 383 127 337L143 311C167 344 200 373 208 373S222 365 229 320C246 214 264 66 268 -20C219 -83 131 -171 17 -239L25 -261L137 -235C248 -110 273 -79 335 8C384 77 481 215 520 287C544 332 556 363 556 393Z" id="g113-122"></path></g><g transform="matrix(.0091,0,0,-0.0091,6.942,3.132)"><path d="M389 0V32C297 38 291 46 291 118V635C234 613 175 595 109 583V556L161 554C203 552 207 547 207 497V118C207 46 201 38 110 32V0H389Z" id="g50-50"></path></g></svg> it is a sigmoid.</div>

Applied Computational Intelligence and Soft Computing

fig1

Figure 1

Figure 1: A Comparison Study on Rule Extraction from Neural Network Ensembles, Boosted Shallow Trees, and SVMs 