Automatic parallelization is usually believed to be less effective at exploiting implicit parallelism in sparse/irregular programs than in their dense/regular counterparts. However, not much is really known because there have been few research reports on this topic. In this work, we have studied the possibility of using an automatic parallelizing compiler to detect the parallelism in sparse/irregular programs. The study with a collection of sparse/irregular programs led us to some common loop patterns. Based on these patterns new techniques were derived that produced good speedups when manually applied to our benchmark codes. More importantly, these parallelization methods can be implemented in a parallelizing compiler and can be applied automatically.

Yuan Lin

David Padua

On the Automatic Parallelization of Sparse and Irregular Fortran Programs

Understanding how host load changes over time is instrumental in predicting the execution time of tasks or jobs, such as in dynamic load balancing and distributed soft real&#x2010;time systems. To improve this understanding, we collected week&#x2010;long, 1 Hz resolution traces of the Digital Unix 5 second exponential load average on over 35 different machines including production and research cluster machines, compute servers, and desktop workstations. Separate sets of traces were collected at two different times of the year. The traces capture all of the dynamic load information available to user&#x2010;level programs on these machines. We present a detailed statistical analysis of these traces here, including summary statistics, distributions, and time series analysis results. Two significant new results are that load is self&#x2010;similar and that it displays epochal behavior. All of the traces exhibit a high degree of self&#x2010;similarity with Hurst parameters ranging from 0.73 to 0.99, strongly biased toward the top of that range. The traces also display epochal behavior in that the local frequency content of the load signal remains quite stable for long periods of time (150&#x2013;450 s mean) and changes abruptly at epoch boundaries. Despite these complex behaviors, we have found that relatively simple linear models are sufficient for short&#x2010;range host load prediction.

Peter A. Dinda

The Statistical Properties of Host Load

Impulse is a new memory system architecture that adds two important features to a traditional memory controller. First, Impulse supports application&#x2010;specific optimizations through configurable physical address remapping. By remapping physical addresses, applications control how their data is accessed and cached, improving their cache and bus utilization. Second, Impulse supports prefetching at the memory controller, which can hide much of the latency of DRAM accesses. Because it requires no modification to processor, cache, or bus designs, Impulse can be adopted in conventional systems. In this paper we describe the design of the Impulse architecture, and show how an Impulse memory system can improve the performance of memory&#x2010;bound scientific applications. For instance, Impulse decreases the running time of the NAS conjugate gradient benchmark by 67&#x25;. We expect that Impulse will also benefit regularly strided, memory&#x2010;bound applications of commercial importance, such as database and multimedia programs.

John B. Carter

Wilson C. Hsieh

Leigh B. Stoller

Mark Swanson

Lixin Zhang

Sally A. McKee

Impulse: Memory System Support for Scientific Applications

David R. O&#x2019;Hallaron

Boleslaw K. Szymanski

Software Systems for Scalable Computers

Adaptive domain decomposition exemplifies the problem of integrating heterogeneous software components with intermediate coupling granularity. This paper describes an experiment where a data&#x2010;parallel (HPF) client interfaces with a sequential computation server through Java. We show that seamless integration of data&#x2010;parallelism is possible, but requires most of the tools from the Java palette: Java Native Interface (JNI), Remote Method Invocation (RMI), callbacks and threads.

C&#xe9;cile Germain&#x2010;Renaud

Vincent N&#xe9;ri

Java-Based Coupling for Parallel Predictive-Adaptive Domain Decomposition

Intel&#x24C7; MMXTM technology can be exploited by a JavaTM JIT compiler to speedup the execution of integer operations. While translating bytecode into Intel machine code, the compiler identifies innermost loops that allow the same integer operations to be applied to multiple data elements in parallel and generates code that uses Intel&#x24C7; MMXTM technology to execute these loops in SIMD fashion. In the context of JIT compilation, compile&#x2010;time directly contributes to the run&#x2010;time of the application. Therefore, limiting program analysis&#x2010;time and synthesis&#x2010;time is even more important than in a static compilation model. The compiler must also ensure that arithmetic precision and the exception handling semantics specified by the JVM are preserved.

Aart J.C. Bik

Milind Girkar

Mohammad R. Haghighat

Incorporating Intel&#x24C7; MMXTM Technology into a JavaTM JIT Compiler

Java&#x2019;s stack&#x2010;based intermediate representation (IR) is typically coerced to execute on register&#x2010;based architectures. Unoptimized compiled code dutifully replicates transient variable usage designated by the programmer and common optimization practices tend to introduce further usage (i.e., CSE, Loop&#x2010;invariant Code Motion, etc.). On register based machines, often transient variables are cached within registers (when available) saving the expense of actually accessing memory. Unfortunately, in stack&#x2010;based environments because of the need to push and pop the transient values, further performance improvement is possible. This paper presents Transient Variable Caching (TVC), a technique for eliminating transient variable overhead whenever possible. This optimization would find a likely home in optimizers attached to the back of popular Java compilers. Side effects of the algorithm include significant instruction reordering and introduction of many stack&#x2010;manipulation operations. This combination has proven to greatly impede the ability to decompile stack&#x2010;based IR code sequences. The code that results from the transform is faster, smaller, and greatly impedes decompilation.

Paul T&#xfd;ma

Transient Variable Caching in Java’s Stack-Based Intermediate Representation

We describe U&#x2010;Net/SLE (Safe Language Extensions), a user&#x2010;level network interface architecture which enables per&#x2010;application customization of communication semantics through downloading of user extension applets, implemented as Java classfiles, to the network interface. This architecture permits applications to safely specify code to be executed within the NI on message transmission and reception. By leveraging the existing U&#x2010;Net model, applications may implement protocol code at the user level, within the NI, or using some combination of the two. Our current implementation, using the Myricom Myrinet interface and a small Java Virtual Machine subset, allows host communication overhead to be reduced and improves the overlap of communication and computation during protocol processing.

Matt Welsh

David Oppenheimer

David Culler

U-Net/SLE: A Java-Based User-Customizable Virtual Network Interface

Recent developments in processor capabilities, software tools, programming languages and programming paradigms have brought about new approaches to high performance computing. A steadfast component of this dynamic evolution has been the scientific community&#x2019;s reliance on established scientific packages. As a consequence, programmers of high&#x2010;performance applications are reluctant to embrace evolving languages such as Java. This paper describes the Java&#x2010;to&#x2010;C Interface (JCI) tool which provides application programmers wishing to use Java with immediate accessibility to existing scientific packages. The JCI tool also facilitates rapid development and reuse of existing code. These benefits are provided at minimal cost to the programmer. While beneficial to the programmer, the additional advantages of mixed&#x2010;language programming in terms of application performance and portability are addressed in detail within the context of this paper. In addition, we discuss how the JCI tool is complementing other ongoing projects such as IBM&#x2019;s High&#x2010;Performance Compiler for Java (HPCJ) and IceT&#x2019;s metacomputing environment.

Vladimir Getov

Paul Gray

Sava Mintchev

Vaidy Sunderam

Multi-Language Programming Environments for High Performance Java Computing

The JLAPACK project provides the LAPACK numerical subroutines translated from their subset Fortran 77 source into class files, executable by the Java Virtual Machine (JVM) and suitable for use by Java programmers. This makes it possible for Java applications or applets, distributed on the World Wide Web (WWW) to use established legacy numerical code that was originally written in Fortran. The translation is accomplished using a special purpose Fortran&#x2010;to&#x2010;Java (source&#x2010;to&#x2010;source) compiler. The LAPACK API will be considerably simplified to take advantage of Java&#x2019;s object&#x2010;oriented design. This report describes the research issues involved in the JLAPACK project, and its current implementation and status.

David M. Doolin

Jack Dongarra

Keith Seymour

JLAPACK – Compiling LAPACK FORTRAN to Java

Shared&#x2010;memory and data&#x2010;parallel programming models are two important paradigms for scientific applications. Both models provide high&#x2010;level program abstractions, and simple and uniform views of network structures. The common features of the two models significantly simplify program coding and debugging for scientific applications. However, the underlining execution and overhead patterns are significantly different between the two models due to their programming constraints, and due to different and complex structures of interconnection networks and systems which support the two models. We performed this experimental study to present implications and comparisons of execution patterns on two commercial architectures. We implemented a standard electromagnetic simulation program (EM) and a linear system solver using the shared&#x2010;memory model on the KSR&#x2010;1 and the data&#x2010;parallel model on the CM&#x2010;5. Our objectives are to examine the execution pattern changes required for an implementation transformation between the two models; to study memory access patterns; to address scalability issues; and to investigate relative costs and advantages/disadvantages of using the two models for scientific computations. Our results indicate that the EM program tends to become computation&#x2010;intensive in the KSR&#x2010;1 shared&#x2010;memory system, and memory&#x2010;demanding in the CM&#x2010;5 data&#x2010;parallel system when the systems and the problems are scaled. The EM program, a highly data&#x2010;parallel program performed extremely well, and the linear system solver, a highly control&#x2010;structured program suffered significantly in the data&#x2010;parallel model on the CM&#x2010;5. Our study provides further evidence that matching execution patterns of algorithms to parallel architectures would achieve better performance.

Xiaodong Zhang

Lin Sun

Comparative Evaluation and Case Studies of Shared-Memory and Data-Parallel Execution Patterns

About this journal

Scientific Programming provides a forum for research results in, and practical experience with, software engineering environments, tools, languages, and models of computation aimed specifically at supporting scientific and engineering computing.

Journal profile

Meet the editorial board

Chief Editor&nbsp;Professor Tramontana&nbsp;is based at the University of Catania and his research primarily concerns the areas of software engineering and distributed systems.

Editor spotlight

Submit your proposal

Do you think there is an emerging area of research that really needs to be highlighted? Or an existing research area that has been overlooked or would benefit from deeper investigation? Raise the profile of a research area by leading a Special Issue.

Special Issues

Author guidelines

Editorial board

Databases and indexing

Scientific Programming - Table of contents 1999

title

Scientific Programming provides a forum for research results in, and practical experience with, software engineering environments, tools, languages, and models of computation aimed specifically at supporting scientific and engineering computing.

Scientific Programming