Table of Contents Author Guidelines Submit a Manuscript
Scientific Programming
Volume 16, Issue 2-3, Pages 155-165
http://dx.doi.org/10.3233/SPR-2008-0252

An Efficient Format for Nearly Constant-Time Access to Arbitrary Time Intervals in Large Trace Files

Anthony Chan, William Gropp, and Ewing Lusk

Mathematics and Computer Science Division, Argonne National Laboratory, Argonne, IL, USA

Copyright © 2008 Hindawi Publishing Corporation. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

A powerful method to aid in understanding the performance of parallel applications uses log or trace files containing time-stamped events and states (pairs of events). These trace files can be very large, often hundreds or even thousands of megabytes. Because of the cost of accessing and displaying such files, other methods are often used that reduce the size of the tracefiles at the cost of sacrificing detail or other information. This paper describes a hierarchical trace file format that provides for display of an arbitrary time window in a time independent of the total size of the file and roughly proportional to the number of events within the time window. This format eliminates the need to sacrifice data to achieve a smaller trace file size (since storage is inexpensive, it is necessary only to make efficient use of bandwidth to that storage). The format can be used to organize a trace file or to create a separate file of annotations that may be used with conventional trace files. We present an analysis of the time to access all of the events relevant to an interval of time and we describe experiments demonstrating the performance of this file format.