Projects

Campus grids

One of the more visible outcomes of the escience programme is the development of wide-area compute grids based on the Condor middleware package. Condor is designed to manage the submission and running of compute jobs across a distributed network of computers. These can be desktop/classroom computers (running MS Windows PCs, Mac OS X and linux/unix) or dedicated clusters. Condor is primarily designed for high-throughput computing, with one job per processor (although there are extensions for MPI parallel computing, our experience is that Condor works much more efficiently in one-job-per-processor mode).

NIEeS worked closely with the Cambridge eScience Centre and the eMinerals project to develop a campus grid within Cambridge, with a bottom-up management approach for participating members (ie participating departments and groups retain complete ownership of the resources they commit). Cambridge presents some particular challenges not faced by other institutions. Each participating group has an independent computing network protected by a local firewall, with local control over all resources (whereas many other campus grids do not have local firewalls and resources are managed centrally). We experimented with a number of approaches, including the use of virtual private networks, before deciding that each computer should have a second IP address, routable on the Cambridge network only, and which is allowed through all local firewalls. Jobs can be submitted from dedicated submission machines or from scientists' own desktop computers.

coLinux Condor Pool

NIEeS investigated the use of Condor with Cooperative Linux (coLinux) on computers running Microsoft Windows (coLinux provides a Linux environment on a Windows computer without having to reboot the computer). This was an attempt to integrate underused Windows resources into a hard-working Linux/Condor environment. We set up two nodes in the NIEeS Condor pool as proof of concept and for testing. One drawback we observed is that memory is not allocated dynamically; coLinux has to be assigned a certain amount of memory and that memory ceases to be available to other Windows programs, even if no Condor jobs are running. Similarly, coLinux/Condor cannot claim more memory even if Windows is not using it.

NIEeS is developing Fortran interfaces to help environmental scientists visualize their results in Google Earth. Fortran functions have been written to wrap all kml functions (kml is the markup language used by Google Earth) and are packaged in a library called WKML. WKML was made possible by the FoX Fortran library (FoX is a local project to integrate Fortran and XML).

NIEeS is currently working on three differet applications using WKML.

The first uses the output from a remote sensing model for coastal water quality and visualizes the simulated water reflectance based on different water quality parameters, such as the concentration of chlorophyll, suspended sediments, or dissolved organic matter.

The second use is in a seismology model, called HypoDD. The output from that programme is a set of coordinates representing the possible origin of an earthquake.

The third use is in a glaciology model called GLIMMER which forms part of the GENIE project. We are developing WKML functions which show different speeds of movement of the ice sheets as different colours on a Google Earth map.

Running ATHAM on a grid

ATHAM stands for Active Tracer High Resolution Atmospheric Model. It is developed in the Max Planck Institute for Meteorology, Germany, and in other institutions including the United States National Oceanic and Atmospheric Administration (NOAA) and the University of Cambridge. The purpose of this model is to simulate high energy plumes with high gradients in momentum and temperature. It finds applications in studies of explosive volcanic eruptions, large areas of burning vegetation, convective clouds, etc.

NIEeS has run ATHAM in a Condor pool. This requires no adaptation of the ATHAM code itself, but does require the preparation of a submission script which is used to present ATHAM jobs to the Condor pool. The next step of the project is to visualize efficiently the output from the model.

Monte Carlo Hyperspectral Synthetic Remote Sensing on a Grid

MCHSRM stands for Monte Carlo Hyperspectral Synthetic Remote Sensing Model, which was originally developed by the Center for Remote Sensing and Marine Environmental Optics Laboratory, Florida Institute of Technology and Dynamac Corp, NASA Kennedy Space Centre, for studying coastal water quality.

NIEeS is taking two approaches to running this model on computing grids. The first is to use My_Condor_Submit (MCS) to manage job submission to the NIEeS MPI-enabled PBS cluster, and SRB for data storage. The second is to use Condor directly, dividing the input data into several subsets for submission to the NIEeS Condor pool. Results are written in KML format to be visualized in Google Earth.

SciSpace.net

SciSpace.net is an Elgg installation which NIEeS has customized by virtue of its choice of plugins, styling of the site, and some alterations to the core code.

Elgg is a so-called web 2.0, social networking platform, similar in some ways to Facebook or MySpace. Given the success of the big social networking sites it is reasonable to think about how this sort of technology could be used in an academic environment for scholarly purposes, but the real spur for NIEeS to get involved in this area was a call from participants at several NIEeS events for something like a MySpace for scientists. Elgg was identified as a very good match to our initial list of requirements, and since then NIEeS has been running scispace.net both as an experiment and a service. We have used it "in house" extensively to explore strengths and weaknesses, achievements and potential.

SciSpace is available to everyone. Though our aim is to support scientific collaborations, it could also be used to aid administration, or in any other context requiring communication between people working on a shared task.

It may be helpful to consider what many educationalists see in the Elgg software, i.e. a means to produce an electronic portfolio both to aid the learning process and as a record of achievement, potentially useful to the owner, other learners, and perhaps employers. The portfolio is sometimes called a collection of evidence. The same tools have obvious applications in support of research collaborations.