Table of Contents Author Guidelines Submit a Manuscript
Scientific Programming
Volume 9, Issue 2-3, Pages 185-194

Nested Parallelism: Allocation of Threads to Tasks and OpenMP Implementation

Ragnhild Blikberg and Tor Sørevik

Department of Informatics, University of Bergen, Norway

Received 29 January 2002; Accepted 29 January 2002

Copyright © 2001 Hindawi Publishing Corporation. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


In this paper we discuss the use of nested parallelism. Our claim is that if the problem naturally possesses multiple levels of parallelism, then applying parallelism to all levels may significantly enhance the scalability of your algorithm. This claim is sustained by numerical experiments. We also discuss how to implement multi-level parallelism using OpenMP. We find current OpenMP implementation, based on version 1.0, to have severe limitation for implementing nested parallelization. We then show how this can be circumvented by explicitly assign task to threads. Load balancing issues become more complicated with two (or more) levels of parallelism. To handle this problem, we have designed a distribution algorithm which groups threads into teams, each team being responsible for one course grain outer-level task. This algorithm is proven to produce the optimal load balance, under given assumptions.