The University of Leeds runs high-end computer clusters as part of its Advanced Research Computing service - known as ARC.
The NorduGrid Collaboration develop, maintain and support grid middleware called the Advanced Resource Connector - also known as ARC.
We had a plan to join ARC-the-computing-service to the grid. The first version of the plan, to use European Middleware Initiative's release of the CREAM software, has been abandoned because support for our batch system - SGE - is not yet ready
So, inspired by work done by Stuart Purdie for Scotgrid, we have a New Plan.
You have probably gathered that The New Plan is (drum roll please) - to deploy ARC on ARC.
Which, when you come to write it down, sounds less like a plan and more like an overexcited performing seal.
It is our only real option: ARC is the only one of the EMI's current middleware menagerie with SGE support.
Round these parts - we like to automate software installations using scripts. Our scripts are based on the instructions from the EMI's Generic installation guide and on Nordugrid's guide to plugging an ARC Compute Element into the European Grid and are now at the point where they can install and configure ARC from the EMI repositories.
The scripts are not yet ready for an official release but - in the unlikely event of anyone wanting to know more - the 'live' versions can be found in the UKNGI code repository at SourceForge. Like the earlier VDT installer scripts - they are intended as runnable documentation. If you don't want to run them, you should be be able to see how they work by reading them.
Incidentally, just to add to the confusion: Leeds also hosts an Antimicrobial Research Centre - known as ARC. Their specialities include some of particular relevance at the moment: antibiotic resistance in bacteria and the transfer of pathogens.
When I last checked, there were people from ARC-the-research-centre using ARC-the-computer-cluster but not via ARC-the-middleware.
Posts
3 comments:
It is our only real option: ARC is the only one of the EMI's current middleware menagerie with SGE support.
EMI is clearly the way of the future. It is rather less clearly the way of now, and by deploying ARC you're going to be in a club of two (in the UK) and the other one isn't running SGE. This is not so much going off piste as skiing out through the gate, surfing down the river and fetching up on a nice beach somewhere.
You could go rather more mainstream and install the currently production ready glite release of CREAM/SGE, which various UK sites are already running, then move across to the EMI release when it's ready, just as everyone else will be doing. It'll be a bit of admin effort (though with the amount of support available for the asking, probably less than ARC would be) and the move to EMI shouldn't upset the interfaces between the CE and the rest of the world at all.
Ewan,
Thank you. I would agree that ARC is not an ideal solution. If EMI had delivered CREAM + SGE, it would never have been needed.
We are trying ARC because we are working with two non-Particle-Physics communities with similar needs:
* Jobs will always be submitted via a WMS. The WMS can be configured to submit jobs to ARC.
* CPU will be provided by traditional HPC clusters running PBS, SGE and SLURM. We are likely to have a single grid front end machine for each cluster.
If I end up - to use your skiing analogy - as the Eddie-the-eagle of the grid. We can revert to gLite or wait for the EMI to provide the missing SGE support.
There's quite possibly a bit of my PP background (or bias, as some people might prefer :-) ) coming through here, but if I were looking for a basic minimal CE to put between a WMS and a batch system, then your original plan of CREAM would be it - a CREAM is very lightweight; my understanding of ARC is that it does clever things with storage and is rather less so. Plus I think there's a lot to be said for doing what everyone else is doing, even if it's possibly not the absolutely best technical solution.
What slightly worries me is that you might be having a stronger negative reaction to gLite's PP heritage than is warranted - if what you really want is EGI CREAM/SGE then gLite CREAM/SGE virtually is that, and is certainly a lot closer than EGI ARC/SGE is. There often seems to be a perception that gLite is just good for PP work, and it's not really true.
Post a Comment