Multi-GPU Support on Single Node Using Directive-Based Programming Model

<div>Task-based multi-GPU implementation in OpenACC. <svg height="12.449pt" id="M85" style="vertical-align:-3.16067pt" version="1.1" viewbox="-0.0498162 -9.28833 80.1861 12.449" width="80.1861pt" xmlns="http://www.w3.org/2000/svg" xmlns:xlink="http://www.w3.org/1999/xlink"><g transform="matrix(.013,0,0,-0.013,0,0)"><path d="M655 676H628C608 656 600 650 571 650H156C118 650 114 651 102 676H77C63 615 45 548 24 485L60 486C81 531 97 562 114 580C134 603 148 611 252 611H308L217 127C201 41 196 38 105 32L98 0H384L393 32C300 38 288 41 304 127L393 611H466C553 611 571 604 582 580C592 559 597 530 595 485L630 489C636 548 645 624 655 676Z" id="g50-85"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="673" vert-adv-y="673"></glyph.data></g><g transform="matrix(.0091,0,0,-0.0091,7.28,3.001)"><path d="M250 606C250 634 233 656 203 656C168 656 146 618 146 593C146 564 169 545 192 545C227 545 250 573 250 606ZM227 95L212 119C187 98 152 71 135 71C129 71 128 78 134 102L207 373C219 418 217 451 194 451C165 451 92 411 30 351L44 326C77 353 106 371 114 371C124 371 121 357 117 341L55 97C32 5 46 -12 70 -12C108 -12 191 51 227 95Z" id="g50-106"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="280" vert-adv-y="280"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,14.746,0)"><path d="M309 -142C211 -61 151 99 151 271S210 601 309 683L288 710C141 611 75 451 75 272V271C75 90 141 -72 288 -169L309 -142Z" id="g50-41"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="358" vert-adv-y="358"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,19.4,0)"><path d="M250 606C250 634 233 656 203 656C168 656 146 618 146 593C146 564 169 545 192 545C227 545 250 573 250 606ZM227 95L212 119C187 98 152 71 135 71C129 71 128 78 134 102L207 373C219 418 217 451 194 451C165 451 92 411 30 351L44 326C77 353 106 371 114 371C124 371 121 357 117 341L55 97C32 5 46 -12 70 -12C108 -12 191 51 227 95Z" id="g50-106"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="280" vert-adv-y="280"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,26.716,0)"><path d="M556 331V384H55V331H556ZM556 140V193H55V140H556Z" id="g54-34"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="611" vert-adv-y="611"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,38.334,0)"><path d="M389 0V32C297 38 291 46 291 118V635C234 613 175 595 109 583V556L161 554C203 552 207 547 207 497V118C207 46 201 38 110 32V0H389Z" id="g50-50"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="487" vert-adv-y="487"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,44.665,0)"><path d="M98 134C72 134 46 117 46 90C46 73 55 65 60 64C95 55 124 32 124 -4C124 -42 95 -68 44 -89L57 -123C122 -104 194 -60 194 22C194 94 136 134 98 134Z" id="g50-45"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="237" vert-adv-y="237"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,49.951,0)"><path d="M414 144C384 79 371 75 317 75H135L276 221C367 316 408 376 408 465C408 570 327 635 237 635C179 635 131 609 100 575L42 494L67 471C94 510 138 565 205 565C277 565 321 517 321 435C321 348 258 270 195 195C146 137 88 81 33 26V0H411C423 44 433 88 446 135L414 144Z" id="g50-51"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="487" vert-adv-y="487"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,56.282,0)"><path d="M98 134C72 134 46 117 46 90C46 73 55 65 60 64C95 55 124 32 124 -4C124 -42 95 -68 44 -89L57 -123C122 -104 194 -60 194 22C194 94 136 134 98 134Z" id="g50-45"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="237" vert-adv-y="237"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,61.568,0)"><path d="M117 -12C153 -12 178 12 178 50C178 84 153 110 118 110C84 110 59 84 59 50C59 12 84 -12 117 -12Z" id="g50-47"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="237" vert-adv-y="237"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,66.855,0)"><path d="M117 -12C153 -12 178 12 178 50C178 84 153 110 118 110C84 110 59 84 59 50C59 12 84 -12 117 -12Z" id="g50-47"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="237" vert-adv-y="237"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,72.141,0)"><path d="M117 -12C153 -12 178 12 178 50C178 84 153 110 118 110C84 110 59 84 59 50C59 12 84 -12 117 -12Z" id="g50-47"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="237" vert-adv-y="237"></glyph.data></g><g transform="matrix(.013,0,0,-0.013,75.256,0)"><path d="M283 271C283 451 220 610 70 710L48 683C146 603 207 443 207 271S149 -59 48 -142L70 -169C220 -67 283 90 283 271Z" id="g50-42"></path><glyph.data ascent="3443" descent="-2856" horiz-adv-x="358" vert-adv-y="358"></glyph.data></g></svg> means a specific task.</div>

Scientific Programming

fig5

Figure 5

Figure 5: Multi-GPU Support on Single Node Using Directive-Based Programming Model 