Emerging Grid technologies enable solving scientific problems that involve large datasets and complex analyses, which in the past were often considered difficult to solve. Coordinating distributed Grid resources and computational processes requires adaptable interfaces and tools that provide modularized and configurable environments for accessing Grid clusters and executing high performance computational tasks. Computationally intensive processes are also subject to a high risk of component failures and thus require close monitoring. In this paper we describe a scientific workflow approach to coordinate various resources via data analysis pipelines. We present a three tier architecture for LiDAR interpolation and analysis, a high performance processing of point intensive datasets, utilizing a portal, a scientific workflow engine and Grid technologies. Our proposed solution is available to the community in a unified framework through a shared cyberinfrastructure, the GEON portal, enabling scientists to focus on their scientific work and not be concerned with the implementation of the underlying infrastructure.

Efrat Jaeger-Frank

Christopher J. Crosby

Ashraf Memon

Viswanath Nandigam

Jeffery Conner

J. Ramon Arrowsmith

Ilkay Altintas

Chaitan Baru

A Three Tier Architecture Applied to LiDAR Processing and Monitoring

Adam Belloum

Ewa Deelman

Zhiming Zhao

Scientific Workflows

Many important scientific and engineering problems may be solved by combining multiple applications in the form of a Grid workflow. We consider that for the wide acceptance of Grid technology it is important that the user has the possibility to express requirements on Quality of Service (QoS) at workflow specification time. However, most of the existing workflow languages lack constructs for QoS specification. In this paper we present an approach for high level workflow specification that considers a comprehensive set of QoS requirements. Besides performance related QoS, it includes economical, legal and security aspects. For instance, for security or legal reasons the user may express the location affinity regarding Grid resources on which certain workflow tasks may be executed. Our QoS-aware workflow system provides support for the whole workflow life cycle from specification to execution. Workflow is specified graphically, in an intuitive manner, based on a standard visual modeling language. A set of QoS-aware service-oriented components is provided for workflow planning to support automatic constraint-based service negotiation and workflow optimization. For reducing the complexity of workflow planning, we introduce a QoS-aware workflow reduction technique. We illustrate our approach with a real-world workflow for maxillo facial surgery simulation.

Ivona Brandic

Sabri Pllana

Siegfried Benkner

An Approach for the High-Level Specification of QoS-Aware Grid Workflows Considering Location Affinity

Enabling the full life cycle of scientific and engineering workflows requires robust middleware and services that support effective data management, near-realtime data movement and custom data processing. Many existing solutions exploit the database as a passive metadata catalog. In this paper, we present an approach that makes use of federation of databases to host data-centric wind tunnel application workflows. The user is able to compose customized application workflows based on database services. We provide a reference implementation that leverages typical business tools and technologies: Microsoft SQL Server for database services and Windows Workflow Foundation for workflow services. The application data and user's code are both hosted in federated databases. With the growing interest in XML Web Services in scientific Grids, and with databases beginning to support native XML types and XML Web services, we can expect the role of databases in scientific computation to grow in importance.

A. Paventhan

Kenji Takeda

Simon J. Cox

Denis A. Nicole

Federated Database Services for Wind Tunnel Experiment Workflows

Brian Borchers

The Art of Computer Programming, by D.E. Knuth

In the Biodiversity World (BDW) project we have created a flexible and extensible Web Services-based Grid environment for biodiversity researchers to solve problems in biodiversity and analyse biodiversity patterns. In this environment, heterogeneous and globally distributed biodiversity-related resources such as data sets and analytical tools are made available to be accessed and assembled by users into workflows to perform complex scientific experiments. One such experiment is bioclimatic modelling of the geographical distribution of individual species using climate variables in order to explain past and future climate-related changes in species distribution. Data sources and analytical tools required for such analysis of species distribution are widely dispersed, available on heterogeneous platforms, present data in different formats and lack inherent interoperability. The present BDW system brings all these disparate units together so that the user can combine tools with little thought as to their original availability, data formats and interoperability. The new prototype BDW system architecture not only brings together heterogeneous resources but also enables utilisation of computational resources and provides a secure access to BDW resources via a federated security model. We describe features of the new BDW system and its security model which enable user authentication from a workflow application as part of workflow execution.

J.S. Pahwa

A.C. Jones

R.J. White

M. Burgess

W.A. Gray

N.J. Fiddian

R.O. Smith

A.R. Hardisty

T. Sutton

P. Brewer

C. Yesson

N. Caithness

A. Culham

F.A. Bisby

M. Scoble

P. Williams

S. Bhagwat

Supporting the Construction of Workflows for Biodiversity Problem-Solving Accessing Secure, Distributed Resources

Quality (QoS) prediction is one of the most important research topics of workflow. In this paper, we propose a stochastic model to evaluate QoS (make-span, reliability and cost) of workflow systems based on QWF-net, which extends traditional WF-net by associating tasks with firing-rate, failure-rate and cost-coefficient. Through a case study, we show that our framework is capable of modeling real-world workflow-based application. Also, Monte-carlo simulation in the case study indicates our analytical methods are consistent with simulation. We also present a sensitivity analysis technique to identify QoS bottleneck.The paper concludes with a comparison between our approach and related work.

Yunni Xia

H.P. Wang

Y. Huang

L. Yuan

A Stochastic Model for Workflow QoS Evaluation

The service-oriented approach to performing distributed scientific research is potentially very powerful but is not yet widely used in many scientific fields. This is partly due to the technical difficulties involved in creating services and workflows and the inefficiency of many workflow systems with regard to handling large datasets. We present the Styx Grid Service, a simple system that wraps command-line programs and allows them to be run over the Internet exactly as if they were local programs. Styx Grid Services are very easy to create and use and can be composed into powerful workflows with simple shell scripts or more sophisticated graphical tools. An important feature of the system is that data can be streamed directly from service to service, significantly increasing the efficiency of workflows that use large data volumes. The status and progress of Styx Grid Services can be monitored asynchronously using a mechanism that places very few demands on firewalls. We show how Styx Grid Services can interoperate with with Web Services and WS-Resources using suitable adapters.

J.D. Blower

A.B. Harrison

K. Haines

Styx Grid Services: Lightweight Middleware for Efficient Scientific Workflows

Grid technologies have progressed towards a service-oriented paradigm that enables a new way of service provisioning based on utility computing models, which are capable of supporting diverse computing services. It facilitates scientific applications to take advantage of computing resources distributed world wide to enhance the capability and performance. Many scientific applications in areas such as bioinformatics and astronomy require workflow processing in which tasks are executed based on their control or data dependencies. Scheduling such interdependent tasks on utility Grid environments need to consider users' QoS requirements. In this paper, we present a genetic algorithm approach to address scheduling optimization problems in workflow applications, based on two QoS constraints, deadline and budget.

Jia Yu

Rajkumar Buyya

Scheduling Scientific Workflow Applications with Deadline and Budget Constraints Using Genetic Algorithms

About this journal

Scientific Programming provides a forum for research results in, and practical experience with, software engineering environments, tools, languages, and models of computation aimed specifically at supporting scientific and engineering computing.

Journal profile

Meet the editorial board

Chief Editor&nbsp;Professor Tramontana&nbsp;is based at the University of Catania and his research primarily concerns the areas of software engineering and distributed systems.

Editor spotlight

Submit your proposal

Do you think there is an emerging area of research that really needs to be highlighted? Or an existing research area that has been overlooked or would benefit from deeper investigation? Raise the profile of a research area by leading a Special Issue.

Special Issues

Author guidelines

Editorial board

Databases and indexing

title

Scientific Programming provides a forum for research results in, and practical experience with, software engineering environments, tools, languages, and models of computation aimed specifically at supporting scientific and engineering computing.

description

https://images.hindawi.com/journals/sp/sp.banner.jpg

twitter:card

twitter:description

twitter:site

twitter:title

twitter:url

twitter:image

Scientific Programming | Hindawi

Scientific Programming

This Issue is now open for submissions.
Papers are published upon acceptance, regardless of the Special Issue publication date.

Scientific Programming

Scientific Workflows

Scientific Workflows

Articles

A Three Tier Architecture Applied to LiDAR Processing and Monitoring

Scientific Workflows

An Approach for the High-Level Specification of QoS-Aware Grid Workflows Considering Location Affinity

Federated Database Services for Wind Tunnel Experiment Workflows

The Art of Computer Programming, by D.E. Knuth

Supporting the Construction of Workflows for Biodiversity Problem-Solving Accessing Secure, Distributed Resources

A Stochastic Model for Workflow QoS Evaluation

Styx Grid Services: Lightweight Middleware for Efficient Scientific Workflows

Scheduling Scientific Workflow Applications with Deadline and Budget Constraints Using Genetic Algorithms