Friday, 4 February 2011

While interesting things happen... elsewhere.

If you look through the case studies on the NGS web site - you can see how the grid has helped researchers study stars, and genes, and criminal activity; to understand how the brain, heart and nervous system work, to search for new drugs to treat disease and study how existing drugs are absorbed by the body. That isn't even mentioning the original goal of the grid - wrangling the enormous amount of data produced when bashing hadrons together in a big pipe under Switzerland.

There is a huge amount of interesting and important research out there.

And a huge amount of very dull - but important - work to be done to enable this research.

Someone needs to make sure the infrastructure is running and keep track of what was used and when. Someone - actually Jens Jensen, one of the contributors to this blog - has to worry about how we look after private keys.

NGS Research and Development's latest excursion to the outer limits of tedium comes courtesy of a project to feed data from our RUS accounting service directly into the EGI's APEL accounting service.

We are already combining data recorded by APEL with that recorded by RUS when updating the CPU usage for account holders within the NGS User Account Service.

In future, we want APEL to be the only place to look for account records and in the great Grid Tradition, this is where things get complicated....

RUS is standards-based. We use the Open Grid Forum Usage Record (UR) format - as defined in great detail in an official specification - to transfer accounting information from individual compute resources to a central store.

UR records can identify the resource which actually did the computing and how much CPU time, or wallclock time or memory was used. It does not carry any information about the relative speed of that resource compared to any other random computer on The Internet.

The NGS accounting clients, and Grid-SAFE all generate UR format data.

APEL is more pragmatic and - as it is part of the gLite software stack - much more widely deployed. It does incorporate a measure (typically derived from the SPECMark) of the speed of the CPU that did the work. It is not explicitly tied to a resource.

So - we are creating a tool that takes records sent to RUS and translates them into APEL updates. It will have to fill in the missing scaling factor - if necessary falling back to a 'custom' scaling to warn potential users of the data that we have no idea how fast the computer is.

These updates will be fed to the central APEL service in exactly the same way as those generated by any other APEL client - and eventually find their way back to the User Account Service.

And the users of the service - well, they don't have to care in the slightest... they have far more interesting things to do.

No comments: